Do You Have to Self-Host LiteLLM? A Hosted LLM Gateway Compared (2026)

LiteLLM is a solid LLM gateway, but running the proxy yourself has a cost nobody puts in the comparison table. What self-hosting actually buys you, when a hosted gateway is the better trade, and what each option costs per million tokens.

If you are searching for an LLM gateway, you have probably already found LiteLLM. It is the default answer, and for good reason. The question this article answers is narrower and more useful: do you actually need to run the proxy yourself?

This is not a takedown. LiteLLM is a good project. But every comparison table I have read prices the software at zero and stops there, which is the one number that is never in dispute. The costs that decide this are somewhere else.

What an LLM gateway gives you

Whether self-hosted or hosted, a gateway sits between your application and the model vendors and gives you:

• One endpoint and one request format instead of one SDK per vendor
• Model routing — swap a model without touching application code
• Failover — when one provider degrades, requests continue somewhere else
• Key management — issue per-service or per-agent keys with their own limits
• Spend visibility — cost per key, per model, per team, in one ledger

Those five capabilities are the product. Everything below is about who operates them.

The part that is not in the comparison table

Self-hosting a proxy means you own the following, permanently:

None of these are hard problems individually. Together they are a standing assignment for somebody on your team, and that assignment does not end.

When self-hosting is clearly right

Be honest about this, because sometimes it is:

• Data cannot leave your network. If your compliance position is that inference traffic must stay inside your own infrastructure, a hosted gateway is not a candidate and the discussion ends here.
• You need to modify routing logic in code. Custom fallback chains, bespoke cost accounting, in-house model servers — that is what an open-source proxy is for.
• You already run this class of infrastructure. If you operate service meshes and API gateways today, one more proxy is a marginal cost, not a new capability.

If two or more of those describe you, self-host. You will not get a better answer from this page.

When a hosted gateway is the better trade

• You want the five capabilities, not the operational assignment. Most teams building a product want routing and failover to exist, not to be theirs to maintain.
• You are one or two engineers. The proxy will be the thing you are debugging on the day you wanted to ship a feature.
• You need per-agent keys today. Issuing a capped key per agent or per customer is a feature, and building the key lifecycle yourself is a project of its own.

You can keep your own vendor keys

The usual objection to a hosted gateway is that you lose your direct vendor relationship, your negotiated rate or your existing credits. That is a real concern and it has a real answer: bring your own key.

With BYOK you register your own vendor API key, and traffic for the models that key covers is billed by your vendor, on your account, at your rate. What you get from the gateway is the routing, the failover, the per-key limits and the unified usage log — not a markup on inference you already pay for.

This is the configuration most teams actually want when they say they are evaluating a proxy: keep the vendor relationship, drop the operational assignment.

What it costs

Model prices on BazaarLink, USD per million tokens, read from the live catalogue on 2026-09-14:

There is no platform fee and no seat licence. You pay for tokens.

Usage rebate, stated honestly

Paid usage on three model families earns balance credit back. It is not a percentage discount — it is a set of fixed bonuses that pay out when your monthly eligible spend crosses each threshold. Every threshold you reach pays, and they add up.

What that works out to is not a flat number, so here is the shape of it. You get the full 10% only when you land on a threshold. Between thresholds it is diluted:

So: above $500 it settles at a steady 10%, and below $20 there is none at all. If your monthly spend is in the single digits, ignore the rebate and compare unit prices — that is where your money is.

Free-tier usage, BYOK usage and usage paid with subscription points do not count toward the thresholds. Settlement is by UTC calendar month and resets every month — it does not carry over. Full rules and the current model list are on the usage rebate page.

Trying it without committing

There is a free tier: Qwen3.7 Flash at no cost, 10 requests per minute and 50 per day, no credit card. That is enough to port your code and confirm the request format works end to end before you decide anything. It is not a credit grant — it is free access to a specific model under a rate limit.

Switching is a base URL

The API is OpenAI-compatible, so in most codebases this is two environment variables:

export OPENAI_API_KEY=sk-bl-YOUR_KEY
export OPENAI_BASE_URL=https://api.bazaarlink.ai/v1

from openai import OpenAI

client = OpenAI(base_url='https://api.bazaarlink.ai/v1', api_key='sk-bl-YOUR_KEY')
resp = client.chat.completions.create(
    model='claude-sonnet-5',
    messages=[{'role': 'user', 'content': 'Hello'}],
)

The same two variables are what LangChain, CrewAI and most agent frameworks read, which is why the migration is usually smaller than the evaluation.

How to decide

Ask one question: is operating a proxy something your team wants to own?

If yes — because of compliance, because you need custom routing code, or because you already run this kind of infrastructure — self-host, and LiteLLM is a reasonable place to start.

If no, you are not choosing between software packages. You are choosing whether to take on a permanent operational assignment in exchange for control you may not need. A hosted gateway with BYOK gives you the routing, failover and key management without the assignment, and lets you keep the vendor relationship you already have.

Related reading: AI gateway for engineering teams and how to tell whether a relay is trustworthy.

台幣計費・統一發票・主流 AI 模型・OpenAI 相容 API