WhisperX on an AMD Radeon 890M: GPU Transcription with ROCm in Docker

Published on August 13, 2026 by Dominic Böttger (today) · 15 min read

The most tedious part of editing video is the searching. Where exactly does the sentence I want to keep begin? Where is the spot where someone starts over three times? From which point on is the other person talking?

That is exactly what I use WhisperX for. It doesn’t just produce a transcript, it produces word-level timestamps: Whisper for speech recognition, a wav2vec2 model that afterwards pins every single word to the audio track down to the millisecond, and pyannote for speaker diarization. What comes out is essentially a searchable edit list: Ctrl+F in the transcript, and you have your timecode. Speaker changes are in there as markers, and subtitles fall out as SRT along the way.

More importantly, all of it comes out as JSON rather than prose — a transcript in which every single word knows its timecode and its speaker. That makes it machine-readable, and that is precisely what turns it into the raw material for AI-assisted editing. It’s also why json is the default output format of my CLI further down.

On the CPU it works perfectly well, it just takes its time: nearly four minutes for a four-and-a-half-minute recording. Roughly real time, with the fan at full speed. For an hour-long interview you sit and wait accordingly, before you can even start cutting.

Meanwhile my Framework laptop has an AMD Ryzen AI 9 HX 370 with an integrated Radeon 890M in it. A GPU with 16 compute units, sitting idle the entire time. The obvious thought: move the transcription there.

The obvious thought was also the one I failed at first. Building a ROCm-capable CTranslate2 is only half the job — the pipeline still computed on the CPU every single time afterwards. No error message, no warning. It simply behaved as if no GPU existed.

What followed was a rather instructive debugging session. The end result is a setup that brings the full pipeline down to 1:13 min — plus three insights you won’t find documented anywhere, because all of them fail silently.

The Result First

A permanently running Docker container and a Bash script in $PATH. That is the entire user interface:

whisperx recording.m4a                   # German, large-v3, diarization -> JSON
whisperx *.wav --output_format all       # your own flags always win
whisperx video.mkv --language en --min_speakers 2 --max_speakers 4
whisperx --no-diarize interview.wav      # without speaker detection
whisperx --cpu file.wav                  # CPU fallback, no ROCm

Measured on a 4:40 min German recording with large-v3:

Over 3× faster across the whole pipeline. The interesting part is where the gain comes from: not primarily from transcription (just under 2×), but from diarization — speaker detection gets roughly 10× faster. It had been the real bottleneck all along, which you never notice on CPU because you see the total time and not its breakdown.

The Architecture

None of this is installed on the host. No ROCm, no PyTorch, no Python environment:

• Base image: rocm/dev-ubuntu-24.04:7.1-complete
• PyTorch/torchaudio/Triton: the official AMD ROCm 7.1 wheels from repo.radeon.com
• CTranslate2: built from source with the HIP backend
• WhisperX: pinned to a fixed commit
• GPU access through /dev/kfd and /dev/dri, without --privileged

The container runs permanently in the background with restart: unless-stopped and sleep infinity. The CLI script calls docker exec against it. No container start per invocation, no model reloading, no waiting.

The heart of the thing is CTranslate2 — the inference engine behind faster-whisper and therefore behind WhisperX. For NVIDIA there are ready-made wheels. For AMD there aren’t. It has to be built from source, and that build is where all three pitfalls live.

Pitfall 1: The Wrong GPU Architecture

CTranslate2 is compiled for one specific GPU architecture, and which one that is has to be stated explicitly at build time:

-DCMAKE_HIP_ARCHITECTURES=gfx1150

You determine your own architecture like this:

rocminfo | grep gfx

The 890M is Strix Point and reports itself as gfx1150. Put anything else in that CMake line and the resulting binary simply contains no code that runs on this hardware.

The trap is how innocuous a wrong line looks. Practically every guide you find on ROCm for AMD iGPUs is written for the Radeon 780M — that is, for gfx1103. It’s the previous generation (Phoenix), it sits in a great many laptops, and its build instructions are correspondingly widespread. You carry the line over while assembling your own Dockerfile, with no particular reason to question it.

The genuinely fatal part, though, is what happens next: nothing. At startup CTranslate2 notices that no runnable code exists for the present device and silently takes the CPU path. --device cuda is still accepted, transcription completes, the output is correct — just computed on the CPU.

Pitfall 2: No Conv1D Without cuDNN

The second finding was the subtlest — a missing option:

-DWITH_CUDNN=ON

CTranslate2 can compute Conv1D layers on the GPU only through cuDNN, or its ROCm counterpart MIOpen. Without this option the build contains no GPU kernel for convolutions at all.

And the Whisper encoder begins with exactly two Conv1D layers. They sit at the very front of the pipeline, right behind the mel spectrogram. If the kernel for them is missing, the entire encoder falls back to the CPU — and with it, effectively, the whole transcription.

The name is the real obstacle when debugging this. On an AMD system you don’t go looking for an option called CUDNN. In CTranslate2 it is simply the switch for “use the vendor’s convolution library” — which, in a HIP build, is MIOpen.

Pitfall 3: A CUDA Wheel Overwrites Your Own Build

The third was the most infuriating, because it undoes everything you did correctly beforehand.

You build CTranslate2 cleanly against ROCm, install it, all is well. Then the Dockerfile does:

pip install faster-whisper

faster-whisper has ctranslate2 as a dependency. pip sees the name, pulls the wheel from PyPI — the CUDA build with a bundled libcudnn — and overwrites the laboriously built ROCm version with it.

The error message that eventually shows up:

CUDA driver version is insufficient for CUDA runtime version

An NVIDIA error message on a system with no NVIDIA hardware in it. So you go hunting for driver problems, for ROCm version conflicts, you consider a downgrade — and the actual problem is one pip install line that quietly replaced a file.

The fix has two parts. First: the ROCm CTranslate2 is installed as the very last step in the Dockerfile, after everything else. Second, and more importantly, the build verifies that this actually held:

RUN python3 -m pip uninstall -y ctranslate2 && \
    python3 -m pip install --no-deps /opt/wheels/*.whl && \
    ldconfig && \
    python3 - <<'PY'
import ctranslate2, glob, os, subprocess, sys
ext = glob.glob(os.path.join(os.path.dirname(ctranslate2.__file__), "*.so"))[0]
ldd = subprocess.run(["ldd", ext], capture_output=True, text=True).stdout
if "libamdhip64" not in ldd and "libctranslate2" not in ldd:
    sys.exit("ERROR: ctranslate2 is not linked against HIP")
if "cudnn" in ldd or "cudart" in ldd:
    sys.exit("ERROR: the PyPI CUDA wheel overwrote the ROCm build")
PY

An ldd on the compiled extension, and if CUDA libraries turn up there, the build aborts. That is my favourite line in the whole Dockerfile. It turns a failure that only surfaces weeks later as “why is this so slow, actually?” into a build that goes red immediately.

That is the pattern behind all three pitfalls: the GPU path doesn’t fail, it disappears. There is no crash, no warning, no log line. The only trace is a runtime you can’t recognise as suspicious without a baseline to compare against. Anyone building a setup like this should include a check from day one that actively proves the GPU is doing the work — rather than trusting that a failure would announce itself.

The gfx1100 Trick

With that, transcription ran on the GPU. Diarization did not — pyannote aborted with:

cannot compile inline asm

MIOpen cannot compile its BatchNorm kernels for gfx1150. That kills voice activity detection and speaker diarization, i.e. everything except plain transcription.

The fix is an old ROCm classic: you lie to the runtime about the hardware.

environment:
  HSA_OVERRIDE_GFX_VERSION: "11.0.0"

The 890M now presents itself as gfx1100 — an RDNA 3 desktop card. And because CTranslate2 is built for both architectures in the Dockerfile (GPU_ARCH=gfx1100;gfx1150), matching code exists. MIOpen compiles its kernels, pyannote runs.

The surprising part: it doesn’t just run, it runs considerably faster.

Almost twice as fast, purely because the card claims to be a different one. The reason is unglamorous: ROCm ships hand-tuned rocBLAS and MIOpen kernels for gfx1100, because that is a widely sold desktop architecture. For gfx1150 those don’t exist (yet), so generic code runs instead. The hardware is closely enough related that the 1100 kernels compute correctly — they are simply tuned.

Once ROCm supports gfx1150 properly, the override can go away entirely. The build is already prepared for that.

The Duplicate OpenMP Runtime

The last finding falls squarely into the “I would never have guessed that” category.

PyTorch ships its own OpenMP runtime (GCC’s libgomp). CTranslate2 links against LLVM’s libomp. Both end up in the same Python process, both believe they have the machine to themselves, and both spin up a thread pool the size of the core count. Result: twice as many active threads as cores, all evicting each other from the CPU.

The measured effect on a 20-second clip in CPU inference:

6.7 s → 93.5 s. Fourteen times slower.

And it happens in every single WhisperX run, because WhisperX always imports torch before CTranslate2. The fix is one line:

LD_PRELOAD: /usr/lib/x86_64-linux-gnu/libomp.so.5

Both libraries are forced onto the same runtime, and there is only one thread pool again.

Two dead ends along the way, in case anyone is facing the same problem:

• OMP_NUM_THREADS, KMP_BLOCKTIME and OMP_WAIT_POLICY only partially contain it — 48 s at best, instead of 93.5 s. You reduce the oversubscription, but you don’t remove it.
• -DOPENMP_RUNTIME=NONE in the CTranslate2 build makes it worse (91 s). CTranslate2 needs OpenMP for its own CPU kernels; take it away and you lose more than you gain.

Version Pinning Is Not Optional Here

The complete Python stack lives in a requirements-lock.txt with exact versions, frozen at a state that demonstrably works. WhisperX itself is pinned to a fixed commit too:

RUN git clone https://github.com/m-bain/whisperX.git && \
    cd whisperX && git checkout d32ec3e && \
    python3 -m pip install --no-deps -e .

That isn’t caution as a reflex, it’s experience. Left unpinned, pip pulls in for example lightning from the 2.6 line, which then fails while loading the pyannote VAD model on torch.load(weights_only=True) — a security tightening in PyTorch that older model checkpoints no longer match.

The fixed WhisperX commit has a second reason: sentence segmentation changes between versions. If you want transcripts to stay comparable across months, you don’t want a docker build shifting your segment boundaries in the background.

The Docker Setup

A few decisions in docker-compose.yml that shape daily use:

services:
  whisperx:
    image: whisperx-gpu:latest
    container_name: whisperx-gpu
    restart: unless-stopped
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    security_opt:
      - seccomp:unconfined
    ipc: host
    user: "1000:1000"
    environment:
      HSA_OVERRIDE_GFX_VERSION: "11.0.0"
      LD_PRELOAD: /usr/lib/x86_64-linux-gnu/libomp.so.5
      MIOPEN_FIND_MODE: FAST
      MIOPEN_USER_DB_PATH: /home/you/.cache/miopen
    volumes:
      - /home:/home
      - /mnt:/mnt
      - /run/media:/run/media
      - /srv:/srv
      - /tmp:/tmp
    command: ["sleep", "infinity"]

• devices instead of privileged: ROCm only needs /dev/kfd (the kernel fusion driver) and /dev/dri. A privileged container is not required.
• user: "1000:1000": the container writes its results as my own user. No chown afterwards, no root-owned JSON in my home directory.
• Identical paths host ↔ container: /home, /mnt, /run/media, /srv and /tmp are mounted 1:1. That lets the CLI accept any path at all without copying files anywhere first. The container sees them at exactly the same address.
• MIOPEN_FIND_MODE=FAST plus a persistent MIOPEN_USER_DB_PATH: without this, MIOpen spends minutes searching for optimal kernel configurations on every start. With a persistent database that happens exactly once.
• The Hugging Face model cache lives in ~/.cache/huggingface on the host and is shared. A rebuild does not re-download large-v3.

The CLI

The wrapper script is deliberately thin. It sets defaults, resolves paths and passes everything else through to WhisperX untouched:

defaults=(--model "$MODEL" --language "$LANGUAGE" --batch_size "$BATCH_SIZE"
          --device "$DEVICE" --compute_type "$COMPUTE" --output_format "$FORMAT")
[ "$DIARIZE" = 1 ] && defaults+=(--diarize --speaker_embeddings)

The defaults are placed before the user’s arguments. Python’s argparse lets the later occurrence win — which means any flag you set yourself automatically overrides the default, without the script having to parse a single argument. On top of that, every default can be set through an environment variable (WHISPERX_MODEL, WHISPERX_LANGUAGE, …).

One detail that mattered to me: the Hugging Face token for the gated pyannote models is not in the script. It comes from $HF_TOKEN or ~/.config/whisperx/hf_token. A token inside the script makes the script unshareable — you can’t show it to anyone or put it in a repository without redacting something first. And that is exactly the step you eventually forget.

And because GPU setups do occasionally break, there’s an honest way out:

whisperx --cpu file.wav

Computes with int8 on the CPU, no ROCm involved. Slower, but independent of whether a kernel update just rearranged the driver world.

Where the Speedup Actually Comes From

The most interesting set of numbers is the one that isolates just the ASR stage:

Row three is the important one: sequentially, the GPU is slower than the CPU. 1.73× versus 2.09×. Had I measured only that, the whole project would have gone into a drawer marked “failed”.

The reason is that an integrated GPU shares LPDDR5X memory with the CPU. It has no fast VRAM of its own. That makes it bandwidth-bound, not compute-bound: for every forward pass, the model weights travel across the same memory bus the CPU uses. Process one audio segment after another and the GPU spends most of its time waiting for data.

Batching is what flips this. Push 8 to 16 segments through at once and the weights, loaded once, get used repeatedly — and the compute capacity starts to matter.

So the total gain is made of three factors, none of which is “the GPU as such”:

1. Batching — without it, the iGPU loses to the CPU.
2. The gfx1100 override — nearly a factor of 2 from tuned kernels.
3. A single OpenMP runtime — otherwise thread contention eats everything.

Two things that did not help: int8 on the GPU is slower than float16 — quantisation costs more than it saves in bandwidth. And batches beyond 16 stop producing reliable gains, because the APU thermally throttles under sustained load. The spread between two identical runs (6× to 8.5×) is larger than the effect of batch size. The 1:13 min above applies to a cold APU; after half an hour of sustained load it’s closer to 1:55 min.

Does the GPU Compute the Same Thing?

A fair question: different hardware, different kernels, different precision (float16 instead of int8) — does the same text come out at the end?

I compared the GPU output against a CPU run of the same recording:

• 99.04 % identical word sequence
• 99.8 % of words with identical timestamps, median deviation: 0.000 s

The remaining differences are hyphenation variants (e-mail versus e mail) and ordinary beam-search noise. No content errors, no dropped sentences, no shifted speaker turns.

As an aside: a ROCm downgrade to 6.4.3, which I initially assumed was the solution, was never necessary. ROCm 7.1 drives this hardware just fine. Every fault was in the build, not in the stack.

Takeaways

For a setup like this the rule is: GPU acceleration doesn’t fail loudly, it fails quietly. A missing CMake flag, an architecture number that’s one model off, a pip package overwriting another — none of it produces an error. All of it produces a number you only recognise as wrong once you have something to compare it to.

That’s why the most important line in the whole Dockerfile isn’t the one that builds CTranslate2, but the one that afterwards uses ldd to check what it got linked against. And the most important habit is to actually measure every optimisation — instead of assuming it took effect because it should have.

The result is a laptop that turns an hour-long interview into a searchable edit list in roughly fifteen minutes — with word-level timestamps, speaker attribution and finished subtitles, without a single byte of it leaving the device. On an integrated graphics unit that was really only meant for moving windows around.

That is exactly what my actual editing workflow builds on, with an AI working over this JSON. But that is an article of its own.

Video editing is merely the occasion I built it for. A transcript that knows word boundaries and speakers is just as much the first stage of a tutorial made from a screen recording, or of a knowledge base article you simply talked through instead of typing out. None of that changes the setup — only what happens to the JSON afterwards.

Written by Dominic Böttger