Skip to content

    Health and readiness

    Health and readiness

    Section titled “Health and readiness”

    The server binds 0.0.0.0:8880 and accepts connections immediately; the model loads and warms in a background task. This keeps orchestrator health polls happy during warmup while a strict readiness endpoint gates real traffic.

    Endpoints

    Section titled “Endpoints”

    All three endpoints are unauthenticated so orchestrators and Kubernetes probes can reach them before any API key is known.

    EndpointWhile warmingReadyFailed warmup
    GET /health (liveness)200 {"status":"warming","model_loaded":false}200 {"status":"healthy","model_loaded":true}503 {"status":"failed","error":"..."} (then the process exits)
    GET /ready (readiness)503 + Retry-After: 10200 {"status":"ready"}503
    POST /v1/audio/speech503 + Retry-After: 10, error model_warmingnormal503, error model_failed

    model_loaded means “warmed”, not “currently resident”: POST /dev/unload frees the model from VRAM but the flag stays true, and the next request lazily reloads.

    Warmup failure exits the container

    Section titled “Warmup failure exits the container”

    A warmup that fails permanently (for example, missing or shadowed weights) sets the status to failed and then exits the process with a non-zero code. This is deliberate: an orchestrator should see a dead container it can restart or surface, not a healthy-looking server that cannot synthesize. The /health 503 window is therefore brief.

    Lazy loading

    Section titled “Lazy loading”

    Set WARMUP_ON_START=false to skip eager loading. The model then loads on the first synthesis request instead of at startup. With warmup disabled, /ready reports not-ready until that first request has loaded the model.

    Using the contract

    Section titled “Using the contract”

    Kubernetes (wired by the Helm chart):

    startupProbe:   { httpGet: { path: /ready,  port: 8880 }, periodSeconds: 5, failureThreshold: 60 }
    readinessProbe: { httpGet: { path: /ready,  port: 8880 }, periodSeconds: 10 }
    livenessProbe:  { httpGet: { path: /health, port: 8880 }, periodSeconds: 30 }

    The generous startupProbe.failureThreshold allows a long cold-load window (up to ~300s) without the liveness probe restarting the pod mid-warmup.

    Docker / Compose — poll /health for liveness and /ready before sending traffic:

    Terminal window
    until curl -fsS http://localhost:8880/ready >/dev/null; do sleep 2; done
    echo "ready"

    See also

    Section titled “See also”