Gateway's Build Your Own Router: define exactly how Gateway picks your LLMs

Every product team building with LLMs has different needs. The best model for a legal document summarizer isn't the best model for a real-time coding assistant. And the tradeoffs you care about (cost, latency, accuracy) aren't the same as the next team's.

Today, we're launching Build Your Own Router in Merge Gateway to give you full control of your routing logic.

It lets you define what "best" means for your product. Merge then uses that definition to pick the right model on every request automatically, at scale, without any routing code to maintain.

How Gateway routing works today

Gateway already handles routing decisions for you. You pick a goal (cost, performance, or capability) and Gateway routes each request accordingly.

This works well for most use cases. But the more deeply your product depends on LLMs, the more that general routing logic stops fitting your requirements.

Why we built Build Your Own Router

Your routing requirements are yours: the benchmarks that matter, the tradeoffs you're willing to make, the models you trust. These are decisions that can make or break your product.

So instead of Merge deciding which model wins, Build Your Own Router lets you set the criteria. You define the benchmarks, assign the weights, and Merge scores every available model against your configuration on every request.

How it works

You’ll simply create a routing config scoped to your organization. In that config, you pick which benchmarks matter and weigh them. When a request comes in, Merge scores every available model against your weighted benchmarks and picks the winner.

The benchmark library

We maintain a library of ML benchmarks, industry leaderboards, and third-party scores:

• Standard ML benchmarks: HumanEval (code), MMLU (knowledge), GPQA (reasoning), GSM8K (math), IFEval (instruction)

• Industry leaderboards: LMSYS Chatbot Arena Elo, Aider's code editing benchmark, Contra's Human Creativity benchmark

• Custom benchmarks: Your own scores, uploaded via UI, JSON, or API

Merge keeps curated benchmark scores for each provider up to date and will always have the latest models and their benchmarks added. You can add dataset imports for custom scoring when Merge doesn't cover your use case yet and for extra control.  If there are any benchmarks you want to see, you can request that directly from Merge.

Presets

Not every team wants to start from scratch. It adds the extra burden of needing to have benchmark knowledge and additional engineering efforts. Presets give you a working configuration out of the box. Pick one and adjust from there, or use it as-is.

Available presets include:

• Balanced: Equal weight across all benchmarks

• Code-heavy: Prioritizes code generation (HumanEval, Aider)

• Reasoning-heavy: Prioritizes logic and chain-of-thought (GPQA, GSM8K)

• Knowledge-heavy: Prioritizes factual breadth (MMLU)

• Instruction-following: Prioritizes precise adherence to complex prompts
  

Pick a preset and Merge populates the weights. Then you can either adjust the configuration until it meets your needs, or start from scratch.

Why Build Your Own Router is the future of LLM routing

Full control over routing logic

You can use your benchmarks, your weights, and  your definition of "best." If Merge's general-purpose scoring doesn't match your use case, you can change it.

You can run your own evals and upload scores per model, and then feed them into Merge. The routing logic then lives in your config, not in Merge's ML model.

Transparent routing decisions

Every routing decision now includes an explanation:

• Which model was chosen
• Why it won (the dominant factor)
• Where each score came from (Merge curated, customer data, default handling)

You can audit any decision. No guessing, no trusting the system to get it right. You can verify it yourself.

Auditability for security and compliance teams

When your security or compliance team asks how your system is selecting models, you have a concrete answer: a versioned, readable config that shows exactly which criteria are being applied. 

This matters for teams in regulated industries or enterprises with AI governance requirements. It turns "we're using AI to pick models" into "here is the documented policy that governs every routing decision."

Faster iteration without extra engineering

When product velocity matters, you can rapidly launch without dedicating engineering resources to building a whole new AI service. It becomes even easier to run evals on your routing config and iterate as your requirements change.

The supporting layer: observability, security, control

Build Your Own Router handles model selection. These three features handle everything around it:

• Policy-based routing. Route requests by customer, geography, compliance boundary, or any custom rule. Make sure the right model always runs in the right context, for the right user

• Data loss prevention (DLP). We scan every request in real time and ensure that sensitive data, like PII, credentials, and secrets get blocked or redacted before they leave your infrastructure

• Prompt injection protection.  Code is scanned for injection vulnerabilities. You’ll always stay in control  of your agents

Together, they give you visibility and control over your LLM layer. Build Your Own Router picks the right model. These features make sure it's safe.

What's next

You can create configs via the Gateway dashboard or the API. Presets are also ready to use.

If there's a benchmark you care about that's not in the library yet, please request it. We'll add the ones that matter most to the community. 

We’ll continue to deliver additional customizability within routers, more cost controls, and more governance. 

Sign up before June 30th for zero fees for 12 months

Zero fees means zero fees: pay by bank or ACH for unlimited zero-fee spend, or pay by credit card for zero fees on your first $40,000 in annual Gateway spend.

This is the easiest time to build your routing infrastructure. Don't wait until your LLM costs and model choices are already a problem.

Create a Merge Gateway account and start building your router in seconds here.

_{*Bank or ACH gives unlimited zero-fee spend. The ~$40,000 credit card figure is derived from $100/month in fee coverage ($1,200 over 12 months) at standard credit card rates (approx 3%); fees beyond $100/month will be billed at standard credit card rates. Limited to 1 account per company, verified by email domain. Subject to verification and offer terms. See merge.dev/offer-terms}