Back to changelog
Gateway
October 9, 2026
https://www.merge.dev/changelog/october-9-2026-gateway

Self-hosted models and performance-based routing

Gateway routes to your self-hosted and fine-tuned models

Add any OpenAI-compatible endpoint, such as vLLM, TGI, SGLang, or Baseten, and route to its models alongside the catalog.

  • Add an endpoint under Configure > Self-hosted models with its URL, API key, and per-model settings
  • Self-hosted models join Priority routing policies from the Self-hosted group in the model picker
  • Traffic to your endpoint carries no Merge charges, and Gateway encrypts the API key and never shows it again
  • Organizations with zero data retention can use them, and failover to curated vendors stays ZDR-compliant

Read more in the Gateway self-hosted models docs.

Route to Claude Haiku 5.5, and to Claude Sonnet 5.5 through deepinfra

Claude Haiku 5.5 by Anthropic is in the catalog, and deepinfra is a new model vendor serving Claude Sonnet 5.5 with zero data retention. Read more in the Gateway model catalog docs.

Priority routing policies filter and rank models by measured throughput and latency

A Performance section in the policy editor and the API sets thresholds on each Priority policy.

  • A minimum throughput and a maximum time to first token
  • Model ordering by measured latency or throughput
  • Per-request min_throughput and max_latency overrides
  • The routing reason explains how each threshold applied

Read more in the Gateway product page (docs link TBD - link to product page for now).

X-Merge-Customer-Origin-Id names the customer on each request with your own ID

Send your own customer ID in the header, and Gateway creates the customer on first use. You can also create or update customers in batches, mint customer API keys by origin ID, and read per-request customer usage records. Read more in the Gateway customer API keys docs.

Improvements

  • Audio transcriptions - Usage reports cover SRT and VTT formats and GPT-4o transcribe models
  • Billing - Bring-your-own-key web searches and enterprise auto-pay invoices updated to bill once, at the expected rate
  • Embeddings endpoints - Key model allowlists, organization blocklists, and vendor access rules extend to the OpenAI, LangChain, and AI SDK compatible embeddings endpoints
  • Gemini - Text-to-speech accepts an explicit audio format
  • Logprobs - Requests with logprobs complete on vendors that do not support them
  • Management API keys - /v1/keys adds expiry dates, spend limits, project scoping, and list filters
  • Management keys - Scopes cap at the permissions of the user who created the key
  • MiniMax
    • Voice settings such as emotion, language boost, and pronunciation dictionaries reach the provider
    • Adds native image generation and image-to-video from first frame, last frame, and reference images
  • Public models API - Each model adds measured latency, throughput, uptime, and a popularity rank, with a performance history endpoint
  • Routing policies
    • Fail over to the next model when a vendor safety filter blocks a response
    • Requests combining tools and JSON schema output reach every vendor that supports structured outputs
    • Failover covers models a vendor retires
  • Token counting - Compression and the token count endpoints estimate documents, files, audio, and images
  • Refined tool-call handling, prompt caching, and batch request parameters across Amazon Bedrock, Anthropic, DeepSeek, and Gemini.

‍