The Gateway is model-agnostic. Your app talks to the Gateway; the Gateway talks to the model. Because policy and redaction live in the Gateway, you can choose, combine, or swap models without changing a single rule — the reason to run a model-agnostic AI privacy gateway instead of wiring protection into one provider's SDK.
Available models
| Provider | Models | Notes |
|---|---|---|
| OpenAI | GPT-4o, GPT-4o mini, and newer GPT models | General-purpose reasoning and tool use. |
| Anthropic | Claude (Opus, Sonnet, Haiku tiers) | Strong long-context and careful instruction-following. |
| Azure OpenAI | GPT models in your Azure tenant | Data stays in your Azure region; billed through Azure. |
| Private / on-prem | Open-weight models (e.g. Llama-class) you host | Nothing leaves your network; the Gateway proxies locally. |
You supply the provider credentials once per project; the Gateway stores them as secrets and calls the model on your behalf. Add or change providers in AI Gateway → Models.
Choosing per project — and per request
Set a default model on the project, and optionally override it per request:
await gateway.chat({
project: "support-assistant",
model: "claude-sonnet", // override the project default for this call
messages: [...],
});
Common patterns:
- Default + override — a cheap model for routine calls, a stronger one for hard ones.
- Task routing — send summaries to one model and code to another.
- Failover — if the primary provider errors or rate-limits, the Gateway retries on a configured backup.
Same policy, any model
Switching from GPT to Claude — or to a private model — does not change what gets redacted. The policy is enforced by the Gateway before the model call, so protection is identical no matter which model answers.
Agents and tools
For agentic workflows, the Gateway sits in front of each model call the agent makes, so multi-step tools stay protected end to end:
- Tool inputs and outputs pass through the Gateway, so data a tool returns is redacted before the next model step sees it.
- Retrieval (RAG) context is sanitized on the way into the prompt — your vector store can hold real data while the model only sees safe context.
- Response inspection runs on every hop, not just the final answer.
Private and self-hosted models
When policy or residency requires that nothing leaves your network, run the Gateway and a self-hosted model together. The Gateway proxies to the local endpoint, applies the same detection and policy, and keeps the audit trail on your side. This is the strongest privacy posture — no third-party model is ever involved. See Privacy & compliance.
Cost and usage
The Gateway records tokens and calls per project and per model, so you can see spend by team or client and set alerts. Because you can route routine work to smaller models and reserve large models for hard prompts, model choice becomes a cost lever you control centrally.