Embedded Routing Stack: Give your customers control over model routing

Today we’re launching Embedded Routing Stack in Merge Gateway. It gives your customers dedicated LLM routing controls to manage their own keys, budgets, usage data and policies like fallback rules, model allowlists and more, all through a single API. 

Users and companies increasingly want full control over the AI models running in your product. They know their own workloads better than anyone else, and want to optimize every dollar of token spend. 

Embedded Routing Stack is built to solve this. 

Why we built Embedded Routing Stack

Model choice is now table stakes for any AI product. Open-source models keep closing the gap on the frontier, and our benchmarking shows them matching frontier models on correctness at up to 65% less cost.  With no single model clearly ahead, customers expect to pick what fits, bring their own, and pay less doing it.

Enterprises feel this most. They want to own the model layer for cost, security, scale, and the freedom to fine-tune models to fit their use cases. That pushes them to self-host, and to buy products that let them run AI on their own terms: to pick their own models and routing, bring their own API keys, and get billed for the tokens they actually used.

Products that fight their customers' model strategy will lose to products that fit it. Embedded Routing Stack lets yours fit, with a single set of APIs.

How it works

Embedded Routing Stack gives each customer their own setup in your Gateway account. You create it from your backend with your production key, then manage it all via API or in the dashboard.

This includes:

• Routing policies: set model routing for a customer, or let them choose their own models and criteria

• BYOK: let customers bring their own LLM keys. Merge Gateway tries the customer’s key first, then yours, and then a Merge-managed key if you allow it 

• Budgets: set hard or soft budget limits per customer and enforce them from your backend

• Usage reporting: read each customer's usage and cost by model, provider, and key tier, then send it straight to billing

Start giving customers their own AI controls

If you’re building AI into a product other companies pay for, this is the layer that gives them control without rebuilding your backend around it.

You can talk to our team about Embedded Routing Stack by scheduling a demo, or you can read our docs to get started.