ForgeGuard Faster Whisper Server

Overview

What it solves.

ForgeGuard Faster Whisper Server exposes OpenAI-style /v1/audio/transcriptions and /v1/audio/translations endpoints backed by Whisper large-v3 running on CTranslate2. It packages the operational details self-hosted transcription needs: baked-in model weights, warmup-aware health and readiness, optional bearer authentication, built-in HTTPS, upload limits, request queuing with load shedding, and live GPU telemetry.

Transcripts are treated as sensitive data by default: input text is never logged unless explicitly enabled, and uploaded audio is not persisted beyond the request.

Verified capabilities

What it does.

• OpenAI-compatible transcription and translation/v1/audio/transcriptions and /v1/audio/translations accept multipart uploads with json, text, and verbose_json responses; any OpenAI SDK works unchanged.
• Word-level timestampsverbose_json returns per-segment detail plus a words array with start/end times and probabilities when timestamp_granularities[]=word is requested.
• Voice-activity detectionVAD filtering (on by default) skips non-speech so silence is not transcribed.
• Runtime model pickerSwitch the resident Whisper model (tiny through large-v3, large-v3-turbo, distil-*) at runtime via the auth-guarded /api/model/activate; the choice persists across restarts.
• Warmup-aware health contractConnections are accepted immediately while the model warms in the background; /health, /ready, and Retry-After semantics keep probes accurate, and failed warmup exits non-zero.
• Optional bearer authentication and built-in HTTPSA single API_KEY gates API routes; TLS_ENABLED=true serves HTTPS directly with an auto-generated, persisted self-signed certificate by default.
• Bounded concurrency with load sheddingMAX_CONCURRENCY and QUEUE_SIZE bound in-flight work; excess requests shed with 503 instead of piling up, and /system reports the live queue.
• Privacy-first defaultsLOG_INPUT_TEXT defaults off because transcripts are PII; PERSIST_AUDIO defaults off so uploads never outlive the request.

Operational facts

The operating contract.

Evidence

Console and request contract.

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F 'file=@audio.mp3' \
  -F 'response_format=text'

Architecture

Data flow.

Clients speak the OpenAI API to a FastAPI interface layer that handles auth, upload limits, and the bounded work queue; inference runs on local GPU compute through CTranslate2 with telemetry surfaced alongside.

Deployment

Deployment paths.

docker run -d --name whisper --gpus all -p 8000:8000 \
  ghcr.io/forgeguard-ai/faster-whisper-server:latest

docker run -d --name whisper --runtime nvidia -p 8000:8000 \
  ghcr.io/forgeguard-ai/faster-whisper-server-jetson:latest

docker compose -f deploy/docker-compose.local.yml up -d

helm install whisper oci://ghcr.io/forgeguard-ai/charts/faster-whisper-server --version 1.1.0

Compatibility

Support matrix.

Lineage & attribution

Where this work comes from.

An independent ForgeGuard AI server, originally derived from SYSTRAN/faster-whisper. Licensed MIT with required attributions in NOTICE.

• SYSTRAN/faster-whisperMIT

  The inference library this server builds on.

• OpenAI WhisperMIT

  Model weights, as converted to CTranslate2 format by Systran.

• CTranslate2MIT

  The inference engine.

Status & limitations

Known limitations and roadmap.

• CTranslate2 cannot run float16 on CPU; the server auto-downgrades to int8 on CPU-only hosts.
• The 8 GB Jetson Orin shares memory between CPU and GPU; the Jetson image defaults to the small model and larger models need confirmed headroom.
• Models not baked into the image download from Hugging Face on first activation and need a cache volume to persist.
• AMD (ROCm) and Intel images are planned but not yet published.

Roadmap

• Real-time streaming transcriptionIncremental partial transcripts over persistent connections (WebSocket/SSE) with barge-in-friendly cancellation, alongside the existing request/response endpoints.
• More inference backendsAMD (ROCm) and Intel images as hardware becomes available to validate on.

Full history in the changelog; planned work is tracked in the repository roadmap.