BLOCK 4 · AI GATEWAY (PRODUCTION) 3:43 – 3:58 · 15 min · 4L / 11HO · Lab

TOPIC 4.6 / 10 · HO #5 · ★ LOAD-BEARING

Routing rule

The workshop's "OSS earns its keep" moment. One rule, one gateway, real production-grade cost-and-failover infrastructure. The 3+3 provider split landing in your terminal is the evidence.

★ LOAD-BEARING — DO NOT CUT

4.6

Routing Rule — provider split + fallback

15 min 4L / 11HO 3:43 – 3:58 ★ LOAD-BEARING

Slide 1 / 3 · Routing primitives

What Bifrost OSS gives you for routing

CEL-based conditions — built via Rule Builder (AND/OR + Add Rule), CEL emitted as a read-only preview
Weighted targets — each target carries Provider + Model + Weight (both Provider and Model optional; empty = use incoming request value)
Fallback chains — Provider required, Model optional per fallback
Priority ordering — lower number = higher priority (0 is highest)
Chain Rule (advanced) — re-evaluates rules with the resolved provider/model as new context, e.g., normalize a model alias first, then route

Slide 2 / 3 · Three production uses

Same primitive, different jobs

Routing rule targets can pin provider, model, both, or neither (leave empty → use the incoming request's value). That flexibility is the whole point — one primitive, many jobs.

Production use	How it maps to a rule
A/B test two models on two providers	Each target pins Provider + Model + Weight (e.g., 50% Anthropic/claude-sonnet · 50% OpenAI/gpt-4o-mini)
Provider failover for the same model	Target pins provider only (model = incoming); Fallback list does provider failover
Tier-based routing	Rule Builder: `headers.x-tier == "premium"` → target a premium-provider

VK provider_configs (allowed_models + weights inside one key) is the other routing lever — used when you want a single key to constrain models per-provider. Routing rules operate at the gateway; VK provider_configs operate at the key. Both compose.

Aliases swap a model. Routing rules swap a provider. Different lever, same goal.

Slide 3 / 3 · Five steps to routing rule

HO #5 — Routing rule (11 min)

Sidebar → Models → Routing Rules → Create New Routing Rule
Name: workshop-route-1 · Description: optional · Priority: 0 (highest)
Enable Rule toggle ON · Rule Builder empty (rule applies to all requests)
Target 1: Provider anthropic · Model claude-sonnet-4-6 · Weight 0.5
Add Target → Target 2: Provider openai · Model gpt-4o-mini · Weight 0.5
Fallback: Provider anthropic (model empty → uses incoming) · Save Rule

Speaker notes

Slide 1: "Bifrost OSS gives you five routing primitives. CEL-based conditions — built via the Rule Builder, AND/OR plus Add Rule; the CEL itself is read-only output, not freeform-typed. Weighted targets — each target carries Provider, Model, and Weight, with Provider and Model both optional. Leave them empty and the rule uses whatever's in the incoming request. Specify them and the rule overrides. Fallback chains — provider required, model optional. Priority ordering — lower number wins, zero is highest. And Chain Rule — advanced — re-runs the routing pipeline using the resolved provider/model as new context. We won't demo Chain Rule; bring it up if anyone asks about composing rules."

Slide 2 — same primitive, three uses: "Routing rule targets are flexible. Each target can pin provider, pin model, pin both, or leave both empty. That flexibility lets one primitive do three different production jobs. A/B test two models on two providers — pin both fields per target. Today's HO is this shape: half the traffic to Anthropic+Claude, half to OpenAI+gpt-4o-mini. Provider failover for the same model — pin provider per target but leave model empty; the fallback list handles when the primary errors. Tier-based routing — Rule Builder catches a header like x-tier == premium and routes that traffic to a premium provider. And — keep in mind — VK provider_configs is the other routing lever. Routing rules are gateway-wide; VK provider_configs constrain a single key. Both compose."

Slide 3 — HO: "Eleven minutes. Routing Rules → Create New Routing Rule. Name workshop-route-1. Priority zero, Enable Rule on. Leave the Rule Builder empty — that means apply to every request. Add two targets: Anthropic with Claude Sonnet at weight 0.5, OpenAI with gpt-4o-mini at weight 0.5. Add a fallback: Anthropic, model empty — if both targets fail, fall back to Anthropic and use whatever model the request asked for. Save. The sanity-check below will confirm the rule fires and the split lands — that 50/50 split landing in your terminal is the workshop's 'OSS earns its keep' moment. One rule, one gateway, real production-grade cost-and-failover infrastructure. Free."

Demo piece — Attendees

Click sequence:

Sidebar → Models → Routing Rules → Create New Routing Rule
Rule Name workshop-route-1 · Description (optional)
Enable Rule toggle ON · Priority 0 · Scope Global · Rule Builder left empty (rule applies to every request)
Routing Targets (click Add Target for the second):
- Target 1: Provider Anthropic · Model claude-sonnet-4-6 · Weight 0.5
- Target 2: Provider OpenAI · Model gpt-4o-mini · Weight 0.5
Fallbacks: Provider Anthropic, Model empty (uses incoming request value)
Save Rule

Form notes (for the instructor walking attendees through it):

Rule Builder uses AND/OR + Add Rule. Leaving it empty is intentional — that's how you say "apply to all." Watch the CEL Expression Preview below; it'll say "No rules defined yet."
Provider and Model dropdowns on each target show "Incoming (optional)" as the default — that's the placeholder, meaning "use the value the request sent." We're overriding both today.
Weights must sum to 1 — the form will reject save if you fat-finger.

Instructor mirror on projector throughout.

Sanity check after save — confirm the rule fires and the split lands (instructor only):

for i in {1..6}; do
  curl -s -X POST http://localhost:8080/v1/chat/completions \
    -H "Authorization: Bearer $WORKSHOP_VK" \
    -H "Content-Type: application/json" \
    -d '{"model":"claude-sonnet-4-6","messages":[{"role":"user","content":"ping"}]}' \
  | jq -r '.model'
done

Note: the request sends claude-sonnet-4-6 without a provider prefix — that's intentional so the routing rule is free to pick the provider. Each target then overrides the model with whatever it pinned. Expected output: roughly 3 lines saying claude-sonnet-4-6 (Anthropic-served) and 3 saying gpt-4o-mini (OpenAI-served). Visceral.

Optional fallback demo (instructor projector, ~60s if time): disable the Anthropic provider in the dashboard (toggle off), re-run the loop. Requests that the rule routes to Anthropic now fail, the fallback kicks in, and all 6 lines should return gpt-4o-mini via OpenAI. Re-enable Anthropic before moving on.