ForgeGuard Faster Whisper Server

ForgeGuard Faster Whisper Server

A container-native, OpenAI-compatible speech-to-text server built on faster-whisper and CTranslate2. Point any OpenAI audio SDK at it and transcribe or translate on hardware you control — nothing about a request leaves the container.

What it is

• An HTTP service exposing /v1/audio/transcriptions and /v1/audio/translations, plus a small set of operational and admin routes.
• Distributed as container images and a Helm chart only — there is no supported bare-metal install path.
• Published for x86_64 NVIDIA GPUs (CUDA cu128) and NVIDIA Jetson Orin (JetPack 6), each baking a default Whisper model so there are no downloads at container start.
• Conservative with sensitive data by default: transcripts stay out of the logs, and uploaded audio lives only in a per-request temporary file.

Where to start

Run it

• Quickstart — run a container, verify health and readiness, and make your first transcription request.
• Container deployment — the single-container path with volumes, GPU flags, and environment.

Integrate it

• Transcription and translation — how the endpoints, formats, and options behave.
• OpenAI-compatible API reference — endpoints, parameters, and response schemas with examples.
• Compatibility — what “OpenAI-compatible” means here, and where it differs from the OpenAI service.

Operate it safely

• Health and readiness — the liveness and readiness contract for orchestrators.
• Observability and queues — the /system telemetry and the admission queue.
• Security hardening — authentication, TLS, upload limits, and container hardening.
• Privacy and responsible use — how audio and transcripts are handled, and your obligations as an operator.

Understand models and data handling

• Model selection — choose, provision, activate, and persist a Whisper model.
• Architecture overview — components, request lifecycle, and model lifecycle.

Deployment methods

Method	Intended use	Guide
Container	Local testing and single-service deployment	Container
Docker Compose	Durable single-host operation	Compose
Portainer	Managed remote Docker environments	Portainer
Kubernetes (Helm)	Cluster and production deployment	Kubernetes
Jetson Orin	Edge / on-device deployment	Hardware profiles

Project status

Current release 1.1.0. Versioned releases and immutable container tags are recommended for persistent deployments; :latest tracks the newest stable release. Real-time streaming and additional inference backends (AMD ROCm, Intel) are planned but not yet available. See Upgrades for the release and tagging policy.