A warmup-aware health contract for local model servers

Model servers have a startup problem that ordinary web services do not: the process can accept TCP connections seconds — sometimes minutes, on edge devices — before it can actually do its job. A health check that answers “the process is up” invites an orchestrator to route traffic into a server that cannot serve it; a health check that answers “the model is loaded” invites the same orchestrator to kill a perfectly healthy server that is still warming.

The contract both ForgeGuard inference servers ship separates the two questions explicitly:

Endpoint	While warming	Ready	Failed warmup
GET /health (liveness)	200 with status: "warming"	200 with status: "healthy"	503, then the process exits non-zero
GET /ready (readiness)	503 + Retry-After	200	503
Inference route (POST /v1/...)	503 + Retry-After: 10, typed error	normal	503, typed error

Three details carry most of the value:

1. Liveness stays green during warmup. /health returns 200 from the first moment, with an honest "warming" status in the body. Restart policies leave the pod alone.
2. Readiness gates traffic, with Retry-After. /ready returns 503 until the model can actually synthesize or transcribe, so Kubernetes readinessProbes and load balancers hold traffic back — and well-behaved clients know when to retry.
3. Permanent failure exits the process. A warmup that cannot succeed (missing weights, incompatible hardware) does not linger as a 503-forever zombie; the container exits non-zero so restart policies and operators see a crash, not a mystery.

The Helm charts wire this contract straight into probes: startupProbe and readinessProbe on /ready, livenessProbe on /health. Pods receive traffic only after warmup and are never restarted because of it.

Where this applies beyond speech

Nothing in the contract is specific to text-to-speech or transcription. Any server that fronts an expensive-to-load artifact — an LLM, an embedding model, a large index — faces the same three questions: is the process alive, can it serve, and is the current failure permanent? Encoding the answers as distinct endpoints with honest status bodies is cheap at implementation time and pays for itself in the first incident that does not happen.

Limitations

This note documents shipped behavior, not a benchmark. Warmup durations quoted in the project READMEs (“seconds on datacenter GPUs, longer on edge devices”) are qualitative; they vary by hardware, model size, and storage.