Observability

AI-in-a-Box has three observability paths:

Path	Purpose
First-party observability service	Generation events, tokens, TTFT, latency, cost, prices, budgets, and admin API views.
Langfuse	LLM trace/generation logging for model-call debugging.
Audit service	Security-relevant platform events, turn envelopes, receipts, and proof exports.

Access URLs

Surface	Dev URL
Gateway health	http://localhost:8080/healthz
Langfuse	http://localhost:13000 when using deploy/docker-compose.dev.yml
Observability API	http://localhost:8080/v1/admin/observability/*
Optional Grafana/Prometheus/Jaeger	Start with make up-observability or make up-all

Production compose closes most direct service ports. Use the gateway and your deployment's ingress instead.

Check Generation Usage

The inference router emits generation records to OBSERVABILITY_URL when configured:

curl "http://localhost:8080/v1/admin/observability/usage?tenant_id=default" \
  -H "Authorization: Bearer $TOKEN"

Usage includes model, provider, token counts, latency, time-to-first-token, and cost estimates when pricing is known.

Inspect Traces

curl "http://localhost:8080/v1/admin/observability/traces?limit=20" \
  -H "Authorization: Bearer $TOKEN"

For Langfuse, open http://localhost:13000 in dev mode and inspect model generations emitted by the inference router.

Refresh Model Prices

The observability service can refresh OpenRouter pricing metadata:

curl -X POST "http://localhost:8080/v1/admin/observability/prices/refresh" \
  -H "Authorization: Bearer $TOKEN"

For air-gapped deployments, set OBS_LOCAL_PRICE_JSON with local model price overrides and point OPENROUTER_MODELS_URL at an internal mirror if needed.

Audit and Receipts

Audit queries require tenant and admin context:

curl "http://localhost:8080/v1/admin/audit?tenant_id=default&limit=10" \
  -H "Authorization: Bearer $TOKEN"

curl "http://localhost:8080/v1/admin/audit/verify?tenant_id=default" \
  -H "Authorization: Bearer $TOKEN"

Use /v1/receipts/{turn_id} and /v1/receipts/{turn_id}/proof for turn-level attestations. See Audit Trail.

Configuration

Variable	Purpose
OBSERVABILITY_URL	Inference-router destination for first-party generation events.
OPENROUTER_MODELS_URL	Source for OpenRouter pricing metadata.
OBS_LOCAL_PRICE_JSON	Local price overrides for self-hosted models.
LANGFUSE_HOST	Langfuse API endpoint used by model logging.
LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY	Langfuse project credentials.
AUDIT_SIGNING_KEY, AUDIT_KEY_VERSION	Audit and turn-envelope signing.

Troubleshooting

Symptom	Check
No generation usage	OBSERVABILITY_URL must be set in the inference-router container.
Langfuse has no generations	Check LANGFUSE_HOST, LANGFUSE_PUBLIC_KEY, and LANGFUSE_SECRET_KEY.
Cost is zero or missing	Refresh prices or configure OBS_LOCAL_PRICE_JSON.
Audit verify fails	Inspect audit service logs and database restore history. Partial restores can invalidate the chain.