Skip to content

    Development documentation — this describes the current development branch. Documented behavior may not exist in the latest release.

    Container deployment

    Container deployment

    Section titled “Container deployment”

    The single-container path is the simplest way to run the server for local testing or a single-service deployment.

    Prerequisites

    Section titled “Prerequisites”
    • Docker with the NVIDIA Container Toolkit for GPU inference on an x86_64 NVIDIA GPU. A CPU-only run works with reduced throughput.
    • For Jetson Orin, use the Jetson image and NVIDIA runtime — see Hardware profiles.

    Images

    Section titled “Images”
    HardwareImageBaked model
    x86_64 NVIDIA (CUDA cu128)ghcr.io/forgeguard-ai/faster-whisper-server:latest (alias -cu128)large-v3 + tiny
    NVIDIA Jetson Orin (JetPack 6)ghcr.io/forgeguard-ai/faster-whisper-server-jetson:latestsmall + tiny

    Both images expose port 8000, run as a non-root user (uid 1000), and declare a /data volume. Pin a release tag (for example :1.1.0) for stable deployments; :latest tracks the newest release.

    Run it

    Section titled “Run it”
    Terminal window
    docker run -d --name whisper --gpus all -p 8000:8000 \
      -v whisper-data:/data \
      ghcr.io/forgeguard-ai/faster-whisper-server:latest

    Verify with curl http://localhost:8000/health and curl http://localhost:8000/ready. See Health and readiness.

    Volumes

    Section titled “Volumes”
    MountPurposeNotes
    /data (DATA_DIR)TLS cert/key and the persisted active-model choiceMount to keep TLS certs and model selection across restarts
    /home/appuser/.cache/huggingfaceHugging Face download cacheMount when using a model not baked into the image

    Do not mount a volume over /app/models. That directory holds the baked weights; mounting over it shadows them and forces a download. See Model provisioning.

    Common environment

    Section titled “Common environment”
    Terminal window
    docker run -d --name whisper --gpus all -p 8000:8000 \
      -v whisper-data:/data \
      -e API_KEY=change-me \
      -e MODEL_SIZE=large-v3 \
      ghcr.io/forgeguard-ai/faster-whisper-server:latest

    The full list is in Environment variables. For authentication and TLS, see Security hardening.

    CPU-only

    Section titled “CPU-only”

    For a machine without a GPU (for example a quick functional test), drop --gpus and set the device:

    Terminal window
    docker run -d --name whisper -p 8000:8000 \
      -e DEVICE=cpu -e MODEL_SIZE=tiny \
      ghcr.io/forgeguard-ai/faster-whisper-server:latest

    COMPUTE_TYPE auto-selects int8 on CPU. Expect substantially lower throughput.

    Next steps

    Section titled “Next steps”