What is LiteLLM?
LiteLLM is an open-source AI gateway and LLM proxy that unifies access to over 140 providers and 1,800+ models behind a single OpenAI-compatible API for platform teams and developers. Its primary differentiator is the ability to self-host the entire infrastructure, enabling organizations to manage spend, routing, and governance for LLMs, agents, and MCP servers without relying on external SaaS dependencies.
Key Features
- Unified OpenAI-Compatible API: LiteLLM exposes a single interface that supports 140+ providers and 1,892 models, allowing developers to swap backend models via configuration updates without modifying application code.
- Spend Tracking and Budgeting: The platform provides virtual keys with specific budget caps, enabling teams to monitor real-time spend per team, project, or app and prevent cost overruns before they occur.
- Self-Hosted Deployment: LiteLLM is designed to be self-hosted anywhere, including air-gapped environments, giving organizations full control over their AI infrastructure and data security.
- SSO and Scoped Access: The gateway integrates with existing secret managers and supports Single Sign-On (SSO), allowing platform teams to grant scoped access to models, agents, and MCP servers based on team or project roles.
- Day-Zero Model Support: LiteLLM offers immediate support for newly released models, ensuring developers can access the latest AI capabilities typically within a day of release.
- Agent and MCP Gateway: Beyond standard LLMs, LiteLLM routes requests to agents and Model Context Protocol (MCP) servers through the same gateway, consolidating the entire AI stack under one key.
- Routing and Optimization: The system routes requests to the appropriate model based on defined criteria, helping organizations optimize performance and reduce costs for specific tasks like coding.
Use Cases
- Enterprise Platform Teams: Organizations can provide the entire company with access to every model and agent through a single login, eliminating the need for individual procurement cycles or repetitive security reviews for each new model.
- Cost-Conscious Engineering Teams: Teams can implement LiteLLM to route advanced AI tasks, such as coding, to more efficient models, resulting in significant cost reductions, as demonstrated by a 56% decrease in coding costs at AT&T.
- Internal Model Integration: Companies can bring their own internal, fine-tuned, or self-hosted models behind the same gateway key, allowing developers to interact with proprietary models using the same standard interface as public LLMs.
- Rapid Model Adoption: Development teams can integrate newly released LLM models into their applications quickly, saving months of work by avoiding the need to rewrite integration code for each new provider.
Who It Is For
LiteLLM is designed for platform engineers, DevOps teams, and software developers who need to manage a complex AI stack involving multiple providers, models, and agents. It solves the problem of fragmented API management by consolidating access, security, and spend tracking into a single, self-hosted gateway, allowing organizations to scale AI usage without becoming a bottleneck for developer access.
Pricing
LiteLLM is an open-source project that can be self-hosted, with an enterprise version available for organizations requiring additional support or features. The source material does not specify public pricing figures for the enterprise tier.
Frequently Asked Questions
What is LiteLLM used for?
LiteLLM is used as an AI gateway to unify access to multiple LLM providers, agents, and MCP servers behind a single OpenAI-compatible API. It enables organizations to track spend, enforce budgets, and route requests to the optimal model without changing application code.
Is LiteLLM free to use?
LiteLLM is open-source and can be self-hosted, allowing users to deploy the gateway on their own infrastructure. While the core software is open-source, an enterprise version is available for those requiring specific commercial support or features.
Does LiteLLM support self-hosting?
Yes, LiteLLM is designed to be self-hosted anywhere, including air-gapped environments. This allows organizations to maintain full control over their AI data and infrastructure while still accessing a wide range of external models through the gateway.
How does LiteLLM handle model switching?
LiteLLM allows users to switch backend models through simple configuration updates in the gateway. This process requires no code changes, procurement cycles, or repetitive security reviews, enabling rapid adaptation to new or different models.
What types of AI endpoints does LiteLLM support?
LiteLLM supports standard LLMs, agents, and Model Context Protocol (MCP) servers. It provides a unified interface for these different types of AI endpoints, allowing them to be accessed and managed through the same gateway and key management system.