OpenAI-compatible gateway in front of OpenAI, Anthropic and Groq with fallback, per-tenant
token budgets and a semantic cache. Send the same question twice (or reword it) and the
second one should come back from the cache.
Source on GitHub | API docs
Running in mock mode (no provider keys configured), so answers are placeholders. Routing, caching and cost tracking still work the same way.
Model:
| Prompt | Cache | Model | Latency | Cost | Saved |
|---|---|---|---|---|---|
| No requests yet. | |||||
Refreshes every 5 seconds. Stats only include requests made through this demo page.