What is Comet?
Comet is an AI observability and machine learning platform that combines LLM tracing, evaluation and experiment management. It is made by Comet ML and its Opik product covers AI application tracing, test suites and prompt optimisation, while the Comet MLOps tools track experiments, artifacts and models. The problem it solves is visibility across the AI lifecycle: teams can debug an agent in production, compare prompts and models against datasets, and keep experiment history in one place.
Key features
- LLM tracing. Opik records prompts, model calls, tool calls and agent steps with inputs, outputs, latency and token usage.
- Test suites and assertions. Teams create pass or fail tests for agents and run them over datasets.
- Evaluation metrics. More than 30 built-in metrics, custom LLM-as-a-judge metrics and code-based checks are available.
- Experiment tracking. Comet records runs, parameters, metrics and artifacts for machine learning experiments.
- Model registry. Models can be versioned and registered for deployment and production monitoring.
- Prompt optimisation. Opik includes an agent optimizer with several algorithms for improving prompts and tool use.
- MCP and integrations. An MCP server and integrations with common AI frameworks and model providers connect existing stacks.
Who is Comet for?
AI engineering teams building LLM applications and agents, and ML teams that need experiment tracking alongside model management. It suits teams moving from manual prompt testing to structured evaluation, organisations that want tracing and experiment history in one platform, and companies that need self-hosted or enterprise deployment options. It is also used by research teams comparing models and training runs.
Use cases
- Tracing an agent's tool calls to find a failing step.
- Running regression tests on prompts before a release.
- Comparing model providers on cost, latency and quality.
- Tracking training experiments and model artifacts.
- Optimising prompts and tool definitions automatically.
Pricing
Opik open source is free to self-host with the full observability and agent testing feature set. Opik Cloud has a Free plan at $0 with up to 10 team members, 25,000 spans per month and 60-day retention. Pro Cloud costs $19 per month with up to 50 team members, 100,000 spans per month and 60-day retention, plus customizable span limits and retention. Additional spans cost $5 per 100,000 on Pro. Enterprise pricing is custom and adds SSO, RBAC, flexible deployment, compliance and support. See the official Comet pricing page.
Getting started
- Create a Comet or Opik account, or self-host Opik.
- Install the Opik SDK and configure the API key.
- Wrap a model or agent call in a trace.
- Add a dataset and assertions to turn the trace into a repeatable test.
pip install opik
opik configureAlternatives
- Braintrust is an AI evaluation and observability platform.
- LangSmith traces, evaluates and monitors LLM applications.
- Arize Phoenix is an open source AI observability tool.
- Langfuse is an open source LLM observability platform.
FAQ
Is Comet free?
Opik open source is free to self-host. Opik Cloud has a Free plan with 25,000 spans per month and up to 10 team members, and Pro costs $19 per month.
What is Comet used for?
Comet is used for AI observability and machine learning experiment management. Opik traces and evaluates LLM applications and agents, while Comet tracks experiments, artifacts and models.
Comet vs Braintrust?
Both provide LLM tracing and evaluation. Comet pairs Opik with a broader MLOps platform for experiment tracking and model management, while Braintrust focuses on evaluation and observability for AI applications. The better fit depends on whether you also need ML experiment tracking.
Can I self-host Comet?
Opik can be self-hosted for free. Enterprise Comet plans add flexible deployment options, SSO, RBAC and compliance support.
Does Comet support agents?
Yes. Opik traces agent execution graphs, tool calls and multi-turn sessions, and can evaluate individual steps or the full conversation.
