Block 4 / Topic 4.1 — Gateway landscape + Boot Bifrost

4.1.1

Opening hook

2 min Lecture 2:45 – 2:47

Slide 1 / 2 · Title

Block 4 — AI Gateway (Production)

The infrastructure altitude.

135 min · 50% hands-on · Q&A floor at 4:45 PM

Slide 2 / 2 · From skill to fleet

From "your skill works" to "your fleet works"

This morning: one skill, benchmarked and optimized
After lunch: how engineers compose skills daily (Sam)
Now: how fleets of skills survive production
The gateway is where infrastructure concerns live

Speaker notes

Open hard: "Your skill works. Your team's skills work. Now you ship them to ten thousand users. The Vue side calls one endpoint. Behind that endpoint is the gateway — the layer that handles routing, failover, budgets, caching, governance, and observability. We're going to spend the next 135 minutes setting up a real one and walking what production teams configure inside it."

Slide 2 — close the altitude loop: "This morning Martin gave you the prompt and the skill altitudes. After lunch Sam gave you the developer-workflow altitude. This block is infrastructure. Same discipline — constraint engineering — applied to where concerns live. Vue should not hold provider keys. Prompts should not be tied to deploy cycles. Skills should not hold provider state. The gateway is the answer."

4.1.2

The gateway landscape

5 min Lecture 2:47 – 2:52

Slide 1 / 3 · Pre-emptive framing

Bifrost is not the answer

"Bifrost is the concrete instantiation. The pattern transfers."

Workshop teaches the pattern. Pick the gateway that fits your stack.

Slide 2 / 3 · The landscape

Where you might land

Gateway	Shape	When to reach for it
Bifrost	Self-hosted · Go · Open-core (Apache 2.0 → Enterprise)	Today's tool. OSS teaches the model; production teams run Enterprise (clustering, SSO, Vault, RBAC, in-VPC).
PortKey	Self-hosted or SaaS · Open-core	Strong A/B + conditional routing; lots of plugins.
LiteLLM	Self-hosted · Python · OSS	Python-shop default; provider-shape unification.
OpenRouter	SaaS	Cheap multi-provider access; less governance focus.
AWS Bedrock	Managed cloud, multi-model	One AWS config → Claude, Llama, Mistral, Cohere, Nova. Enterprise AWS shops land here.

Slide 3 / 3 · How to choose

Decision questions

Is your stack Python? → LiteLLM is the path of least resistance
Already AWS-heavy? → Bedrock fits without new infra
Need rich governance + SaaS option? → PortKey
Cost-driven, low-governance? → OpenRouter
Self-host + open-core + want OSS depth? → Bifrost (today)

Speaker notes

Slide 1 — pre-emptive framing: "Before we touch Bifrost — Bifrost is not 'the answer.' It's the gateway we're using today because its OSS feature set lets us teach the most without paywalls. The pattern transfers to every alternative. Workshop teaches the pattern. Pick the gateway that fits your stack."

Slide 2 — walk the five briefly: "Five names you'll see in any RFC about gateway choice. Bifrost — what we're using. Open-core: Apache 2.0 OSS is what we'll touch today; Enterprise tier is what real production deployments run on. Hold that distinction — we'll come back to it in a few beats. PortKey — strong A/B and conditional routing; lots of plugins; also open-core. LiteLLM — Python default. OpenRouter — cheap multi-provider, less governance. AWS Bedrock — the managed-multi-model entry point. One AWS config gives you Claude, Llama, Mistral, Cohere, Nova. Most enterprise AWS shops just use Bedrock and call it a gateway."

Slide 3 — decision tree: "Five questions. Python shop, AWS shop, governance-heavy, cost-driven, self-host. Different paths. None of these are wrong — they're optimized for different concerns. We chose Bifrost for the workshop because its OSS surface is the broadest, so we can show the most without paywall hand-waving. The Enterprise tier is where you actually deploy — but every concept you'll learn today applies to both layers."

4.1.3

Why a gateway? + Boot Bifrost

3 min 2L / 1HO 2:52 – 2:55 ★ LOAD-BEARING (boot moment)

Slide 1 / 2 · Five concerns

What the gateway handles

Concern	Why it doesn't belong in Vue
Multi-LLM routing	Your app shouldn't know whether it's Anthropic or OpenAI
Failover + retries	Providers go down; the gateway routes around
Cost control	Budgets enforced at the policy layer, not in app code
Observability	Logs, dashboards, attribution — once, in one place
Safety (guardrails, RBAC, audit)	Production hardening, not sprinkled through your app

Slide 2 / 2 · Boot the gateway

One command. Two open tabs.

"Let's set one up."

# from the workshop repo root
npm run bifrost

Wraps npx -y @maximhq/bifrost --app-dir ./bifrost
--app-dir tells Bifrost to store all state (config.json, config.db, logs.db) in ./bifrost/ — workshop-local, not in ~/.config/bifrost/
Bifrost boots on http://localhost:8080 — API + dashboard, same URL
Leave the terminal running · open a browser tab on localhost:8080
Eighty minutes of HO from here. You leave with a configured Bifrost and a Vue app calling it.

Speaker notes

Slide 1 — walk the five concerns fast (~25s each): "Routing — your Vue app shouldn't know whether the request goes to Anthropic or OpenAI. Failover — providers go down; the gateway routes around. Cost control — budgets enforced at the policy layer, not in your application code. Observability — logs and dashboards in one place, attributed by team or customer. Safety — guardrails, access control, audit. Production hardening lives at the gateway, not sprinkled through every endpoint."

Slide 2 — boot it together: "Let's set one up. From the workshop repo root, run npm run bifrost. This wraps npx -y @maximhq/bifrost --app-dir ./bifrost. The --app-dir flag is doing real work — it tells Bifrost to store everything it persists (config.json, the SQLite database, the logs) inside the ./bifrost/ folder of the workshop repo. Workshop-local state. That means no conflicts with any Bifrost you might already have installed, and we can wipe and restart cleanly with npm run bifrost:fresh. The gateway starts on localhost:8080 — API and dashboard, same URL. Curl it for inference, browser it for the UI. While it's booting, open a browser tab on localhost:8080 — give the dashboard a few seconds to come up. We'll click through the sidebar for the next eighty minutes. Hard transition: dashboard tab open, terminal in the corner, here we go."

Stage direction: boot Bifrost on the projector terminal alongside your speaker-note narration. The terminal output landing — "Bifrost gateway started on http://localhost:8080" — is the visual cue that we've left lecture mode and entered HO mode. Don't wait for every attendee's terminal to land; just announce "if yours isn't up in 10 seconds, raise a hand."

Demo piece — Boot the gateway

Instructor runs on stage, attendees follow in parallel.

# from the workshop repo root
npm run bifrost

Wraps:

npx -y @maximhq/bifrost --app-dir ./bifrost

Bifrost boots on http://localhost:8080. Dashboard is at the same URL — open a browser tab and leave it there for the rest of the block. The --app-dir ./bifrost flag is load-bearing: it tells Bifrost to persist its state (config.json, config.db, logs.db) inside the workshop repo at ./bifrost/, not in the default global location. Workshop-local state = clean reset via npm run bifrost:fresh.

Instructor mirror:

Project the terminal output until "Bifrost gateway started on http://localhost:8080" lands.
Project the dashboard tab — sidebar visible — and hold for ~5s. "This is the gateway we're about to configure."
Hard transition into Beat 4.2.

Common gotchas:

Port 8080 in use → lsof -i :8080 finds the process; kill it; re-run. Or start Bifrost on a different port via CLI flag and update BIFROST_URL in .env.
npx download stalls → corporate proxy / VPN. Hop to a hotspot. If the binary was pre-cached during Prerequisites, this should be instant.
Dashboard tab 404s for a few seconds → give it 5–10s after the terminal says "started" for the static assets to load. Don't refresh frantically.
No npm run bifrost script → attendee skipped Prerequisites step that installs the workshop repo. Fall back: cd <workshop-repo>; npx -y @maximhq/bifrost --app-dir ./bifrost directly.