Skip to main content
Go-based LLM gateway. Unifies multiple AWS Bedrock accounts and a few other providers behind OpenAI-, Anthropic-, and native-shaped HTTP APIs with virtual keys, routing rules, and request logs. Replaces LiteLLM. Single container on i3, SQLite-backed.

Endpoints

  • Admin UI: bifrost.augustin.ai — Basic auth with BIFROST_USER/BIFROST_PASSWORD
  • Inference (VK auth — send x-bf-vk: sk-bf-... or Authorization: Bearer sk-bf-...):
    • OpenAI dialect: bifrost.augustin.ai/openai/v1/...
    • Anthropic dialect: bifrost.augustin.ai/anthropic/v1/...
    • Native: bifrost.augustin.ai/v1/...
  • Health: bifrost.augustin.ai/health
Model strings use {provider}/{model_id} (e.g. cecelia/us.anthropic.claude-haiku-4-5-20251001-v1:0). Bare model strings without a provider prefix get resolved by the model catalog at request time, but explicit prefixes are deterministic.

State

FilePurpose
bifrost/config.jsonVersioned config: store paths, admin auth, routing rules. No providers, no VKs. In git.
data/config.dbRuntime config: providers, keys, virtual keys. Authoritative for everything not in config.json.
data/logs.dbRequest logs with 365-day retention.
Three layers interact:
  1. config.json is mounted read-only into the container.
  2. config.db is the SQLite runtime store.
  3. The Web UI and Admin API both read and write config.db.
On every restart Bifrost reconciles config.json against config.db per-entity. Entities only in DB are preserved. Entities only in the file are re-seeded. Entities in both: DB wins unless the file’s hash changed. For this setup, providers and VKs live exclusively in DB so they are never re-seeded — the only things that come from config.json are the three store blocks, the auth config, and the routing rules.

Providers

ProviderAuthRegionCostNotes
bedrock(built-in, no keys)Built-in to Bifrost binary, not deletable. Has no keys; any traffic 403s. Stale clients sending bedrock/* get rewritten to cecelia by a routing rule.
ceceliarotating STSus-west-2Free, work-fundedMac launchd job pushes fresh STS creds via PUT /api/providers/cecelia/keys/<uuid> on a 5-minute timer. Streaming requires User-Agent: claude-cli/... set in the provider’s network_config.extra_headers because Cecelia’s IAM role gates streaming on a UA Condition.
isengard-runtimelong-term bearerus-east-1Personal AWS, paid74 models. Anthropic profile IDs + non-Anthropic foundation IDs. Budget exhausted for the current month — minimize traffic.
isengard-mantle-openailong-term bearerus-east-1Free, work-funded38 OpenAI-shape Mantle models (gpt-oss, qwen, kimi, etc.) at bedrock-mantle.us-east-1.api.aws.
isengard-mantle-openai-east2long-term bearer (same as -openai)us-east-2Free, work-fundedOpenAI’s proprietary GPT models on Bedrock: openai.gpt-5.4, openai.gpt-5.5. Responses API only, at bedrock-mantle.us-east-2.api.aws/openai. Separate provider because base_url/region is provider-level — GPT-5.x is Ohio-only, the -openai provider is us-east-1. Reached only by the client-scoped work-laptop-codex VK (passthrough verified end-to-end 2026-06-06).
isengard-mantle-anthropiclong-term bearer (in extra_headers)us-east-1Free, work-funded2 Anthropic-shape models at bedrock-mantle.us-east-1.api.aws/anthropic. Keyless because env. interpolation only fires on config_keys.value, not on extra_headers.
kiroOAuth via aiclient2apin/aFreeOpenAI-compatible proxy at http://aiclient2api:3000/claude-kiro-oauth (aiclient2api runs in ns infra, same as Bifrost, so the bare name resolves). 5 Claude variants. Needs a seeded Kiro OAuth token — see cluster/infra/aiclient2api/AGENTS.md. OAuth wedges roughly monthly.
Anthropic model ID rule: cecelia and isengard-runtime need us.* or global.* cross-region inference profile IDs. Bare anthropic.* IDs reject. Mantle is the opposite — uses bare IDs only. Other model families accept either.

Routing

Two active global routing rules (both in config.json under governance.routing_rules, also live in DB):
  1. bedrock-to-cecelia-redirect (priority 0): rewrites any request with provider == "bedrock" to cecelia.
  2. work-laptop-anthropic-to-cecelia (priority 10): for the work-laptop VK, routes any model starting with us.anthropic. or global.anthropic. to cecelia, regardless of which provider/ prefix the client sent.
Together these route all work-laptop Claude Code traffic through cecelia (work-funded) instead of isengard-runtime (paid, exhausted) or bedrock (broken). Rules execute before governance provider selection. Model-catalog default-provider resolution runs only as a last resort when no rule matched and the request had no provider/ prefix.

Virtual keys

Nine VKs. Six legacy machine-named, flat-scoped across the five usable providers (augustin-default, i3, mastra, n8n-classifier, pentium, work-laptop; the dead built-in bedrock is granted to none). Three client-scoped (the reshape, in progress), each scoped via explicit key_ids in the create request to sidestep the allow_all_keys POST bug:
  • work-laptop-codexisengard-mantle-openai-east2 only (openai.gpt-5.4/5.5).
  • i3-t3codececelia only (Claude profile IDs) — for t3code on i3.
  • i3-temporalcecelia only (Claude profile IDs) — for Temporal workers on i3.
Per-client VK selection on a shared host (Claude Code clients): ~/.claude/bin/api-key-helper.sh is env-aware — if a client exports ANTHROPIC_AUTH_TOKEN, the helper echoes that VK; otherwise it falls back to the machine VK (BIFROST_VK from bifrost/.env, i.e. i3). So multiple Claude clients on one box (native CLI, t3code, a Temporal worker) map to distinct VKs without separate HOMEs — the client just sets ANTHROPIC_AUTH_TOKEN to its VK. Native Claude sets nothing, so it stays on i3. VK values are bearer tokens; they’re stored plaintext in governance_virtual_keys.value.

Why not LiteLLM

LiteLLM bundles a postgres requirement, an 800MB+ Python image, and uneven provider support. Bifrost is a single Go binary with sqlite, faster cold start, real OAuth-style governance primitives, and a coherent provider abstraction. Tradeoff: some governance features (audit logs, clustering, advanced load balancing) are enterprise-gated, but the OSS core covers our needs.