The Model Router layer, explained.
The single most misunderstood component in an AI OS. Get it right and the firm's compliance officer sleeps at night. Get it wrong and every model choice is a vendor lock-in.
A policy-aware component inside an AI OS that decides, per request, which model handles the query — considering data sensitivity, latency, cost, capability and jurisdiction — and enforces guardrails on the way in and out.
The Model Router is the layer that stops the phrase 'we use GPT' or 'we use Claude' from meaning anything at the firm level. In a serious AI OS, no application ever hard-codes a model. Every call goes through a router that decides — based on the data, the task, the jurisdiction and the cost — which model actually runs it. That decision is a policy, not a preference.
What the router actually decides
- Sensitivity: is this a public-web summary or is it a client's full portfolio? Public gets a frontier API; portfolio-level context stays on private-hosted open weights.
- Capability: does the task need a top-tier reasoning model, or is a small local model good enough? Small-model-first saves eighty per cent of the cost with no loss on the tasks it handles.
- Jurisdiction: does the caller's data need to stay in the UK, EU, or a specific region? The router picks a route that satisfies it.
- Latency and cost budget: a real-time deal-room briefing routes differently from an overnight batch report.
- Fallback and quorum: for high-stakes calls, two models run and disagreements escalate.
Why 'we use one model' is a strategic error
Vendors will pitch you their model as the answer to everything. It never is. Frontier models are extraordinary on the top ten per cent of tasks and often overkill on the other ninety. Open-weight models on private infra are the right choice for the majority of routine work — cheaper, private by default, and immune to the vendor's next pricing move. A firm without a router is a firm exposed to whichever model got sold hardest to the CTO that quarter.
A model is a choice. A router is a strategy.
How we build them
Every KJ Capital OS ships a Model Router with the same shape.
- A private-hosted open-weight tier — one general model, one code model, one embeddings model — running on the firm's infra or a private-VPC provider.
- A frontier tier — the two best current APIs, gated behind data-classification rules.
- A policy layer — a small, fast classifier decides sensitivity and jurisdiction before the request goes anywhere.
- An observability layer — every call logged with model version, latency, cost, tokens and the guardrails it hit.
- A swap contract — every model behind the router is replaceable in a day. No skill, agent or workflow hard-codes a vendor.
Where to start
Every KJ Capital engagement begins with a two-week £15k AI Diagnostic. We map your existing data estate, model your firm's OS across the six layers above, cost the build, name the first three skills to ship, and hand you a plan your board can act on. It is deliberately priced to be a rounding error against the value it unlocks.
FAQ
Isn't this just LangChain or LiteLLM?
Those are libraries. The router is a policy layer built on top of a library. The important part is the policy — which model runs which query, and why — not the plumbing that makes the call.
How much does this cost to run?
For a mid-sized firm, a private-hosted open-weight tier plus frontier fallback typically comes in at 30–60% of the equivalent all-frontier bill, with better latency on routine tasks.
Can we still use ChatGPT Enterprise if we want to?
Yes. It becomes one route inside the router — used where its capability is worth the sensitivity trade. It is no longer the whole stack.
Start with the £15k AI Diagnostic
Two weeks, one costed OS blueprint, three named skills to ship first.