Zero data retention gateway: overview, benefits, and implementation steps

As you build products that use LLMs, you’ll likely have customers with relatively strict privacy requirements. As a result, you may not be able to route their requests through LLM providers that store prompt and response data by default.

To provide these customers your LLM-powered features, you can use an LLM gateway that offers zero data retention (ZDR). 

We'll help you start using this type of gateway by breaking down how ZDR works, why it’s important, and what you can do to implement it today.

What is a zero data retention gateway?

It's an LLM gateway that ensures users' prompts and outputs aren't stored by the providers processing them.

Regardless of the ZDR gateway you use, they’ll come with the following components:

• Credential/auth layer: The gateway handles API key management and token refresh for each provider without storing those credentials or the payloads they protect

• In-flight encryption: All data moves over TLS, meaning prompts and responses are encrypted between the user and the gateway, and again between the gateway and the provider. This ensures that even during transit, data can't be read or intercepted

• Metadata-only logging: Requests can be recorded with high-level information, like timestamp, model, token count, and latency, without logging the prompt or completion content

Examples of using zero data retention gateways

ZDR gateways can be applied to virtually any workflow where the prompt contains sensitive, regulated, or competitively valuable content, and where retention by a third party creates legal, contractual, or reputational exposure.

Here are a few real-world scenarios that require ZDR gateways:

• ‍Healthcare SaaS: If your product helps clinicians document visits, summarize charts, or draft care plans, your users are passing PHI through nearly every LLM call. ZDR lets you offer AI features without becoming a HIPAA liability for your customers

• Financial and fintech applications: Imagine your product surfaces portfolio data, generates client reports, or assists with underwriting. The prompts will often carry regulated financial information, so ZDR is often a prerequisite for enterprise deals in this space

• HR and people management platforms: Say your product handles performance reviews, compensation workflows, or headcount planning. AI features mean employee PII and salary data are in the prompt. Customers won't turn those features on without a retention guarantee

• Security and compliance tools: If your product analyzes logs, triages alerts, or generates incident reports, feeding that data into an LLM without ZDR means vulnerability details and attack patterns could be retained externally

Related: Examples of LLM routing

Benefits of using zero data retention gateways

Here are just a few benefits of using a ZDR gateway:

• Regulatory compliance out of the box: Meets HIPAA, GDPR, SOC 2, and other frameworks that prohibit retaining sensitive data on third-party systems, without requiring custom legal agreements per vendor

• Reduced breach surface: Data that isn't stored can't be leaked. If the gateway is compromised, there's no prompt or completion history to expose

• Vendor trust without vendor dependency: You get the capability of frontier models without handing those vendors a copy of your users' data or your proprietary context  

• Faster enterprise sales: Security reviews and procurement questionnaires move faster when you can answer "data is not retained at any point in the chain" rather than negotiating DPAs with each model provider

• Auditability without exposure: Metadata logging (timestamps, token counts, latency) still gives you operational visibility and cost tracking without the liability of storing the actual content

• Enables AI features in regulated products. If you’re operating in an industry like healthcare, finance, HR, and legal, you can still ship AI-powered features you otherwise couldn't

How to implement a zero data retention gateway

You can use an LLM gateway solution like Merge Gateway to implement ZDR in a matter of clicks.

Start by navigating to the Configure tab. 

Hover to “Vendor.”

Then toggle on “Zero data retention.” 

That's it!

Now, every request Merge Gateway makes to a model provider is sent with a header or parameter that instructs the provider not to log or store the payload. 

For providers like OpenAI and AWS Bedrock, this is a specific API-level flag they support under their ZDR agreements. The prompt leaves your system, passes through Merge Gateway in memory only, hits the provider, and the response travels back the same way. 

Nothing is written to disk at any point in the chain. The provider processes the request and discards it, and what remains afterward is only the metadata (which model was called, at what cost, how long it took, etc.).

{{this-blog-only-cta}}