Eval and benchmark tools for monitoring agents in production
Evaluation tools that score agent outputs, catch regressions and compare prompts and models. Of the 48 tools on our monitoring agents in production page, 15 fit this narrower job. By pricing: 6 open source, 1 source-available, 7 free or freemium, 1 paid, usage-based or enterprise. The most-starred open repos here are MLflow (28,418 stars), Opik (22,700 stars), Arize Phoenix (11,828 stars). Most come from the evals & observability (14), security & governance (1) categories.
Updated
MLflow
Open-source platform for tracing, evaluating and managing agents, LLM apps and ML models.
Added
Opik
Comet's open-source platform for tracing, evaluating and monitoring LLM apps, RAG systems and agent workflows, self-hosted or in the cloud.
Added
Arize Phoenix
Open-source tracing and evaluation for LLM apps.
Added
Agenta
Open-source LLMOps platform for prompt management, evaluation and observability.
Added
Future AGI
Open-source platform for evaluating, tracing, simulating and guarding LLM and AI agent applications, self-hostable or cloud.
Added
OpenAPPA
OpenAPPA is a deterministic guardrail that sits between an AI agent and its tools and tracks how data flows across tool calls, instead of asking a second model to judge each call. Policies decide whether untrusted or sensitive data may reach a given action, which blocks data exfiltration caused by prompt injection or hallucination while letting agents finish their tasks. It plugs into Claude Code, Pi, OpenCode, kAgent and Archestra, comes with reusable policy "batteries", validation, observability and reporting, and is backed by a NeurIPS workshop paper with published benchmarks. It is in preview as an open RFC.
Added
Arize AI
Agent observability and evaluation platform with tracing, evals and experiments for improving AI agents.
Added
Braintrust
Evals and observability platform for AI products.
Added
Confident AI
Enterprise platform for AI evals and observability that gives teams one standard for measuring and monitoring AI quality.
Added
Kore.ai Autoloop
Optimization engine in Kore.ai Agent Platform Artemis that evaluates agents against business goals, diagnoses failures from traces, applies targeted fixes, and re-verifies; GA Oct 2026.
Added
Langfuse
Open-source LLM observability, tracing and evals.
Added
LangSmith
Tracing, evals and monitoring for LLM apps and agents.
Added
Openlayer MCP
Openlayer's MCP server, letting coding agents pull evaluation results, failing tests and monitoring data from Openlayer so they can debug and improve AI systems from the IDE.
Added
Respan
Observability and evaluation platform for LLM apps and agents.
Added
W&B Weave
Weights & Biases toolkit for tracing and evaluating agents.
Added
GitHub repos (6)
| Repo | GitHub stars | Licence | Last push |
|---|---|---|---|
| MLflow ↗ | 28,418 | Apache License 2.0 | 2026-10-22 |
| Opik ↗ | 22,700 | Apache License 2.0 | 2026-10-22 |
| Arize Phoenix ↗ | 11,828 | Elastic License 2.0 (source-available) | 2026-10-22 |
| Agenta ↗ | 4,840 | MIT License, except ee/ directories (separate licence) | 2026-10-21 |
| Future AGI ↗ | 2,174 | — | 2026-10-22 |
| OpenAPPA ↗ | 1,596 | — | 2026-10-22 |
6 repos. Stars and licence come from each repo, checked when the listing was last verified.
FAQ
What are the best AI tools for monitoring agents in production focused on evals and benchmarks?
We list 15. The first three by GitHub stars and featured status are MLflow: Open-source platform for tracing, evaluating and managing agents, LLM apps and ML models.; Opik: Comet's open-source platform for tracing, evaluating and monitoring LLM apps, RAG systems and agent workflows, self-hosted or in the cloud.; Arize Phoenix: Open-source tracing and evaluation for LLM apps.
Are there free or open-source options?
Yes: 6 open source, 1 source-available, 7 free or freemium, 1 paid, usage-based or enterprise. Pricing is as each maker states it.
More from monitoring agents in production
Tools are grouped by what each says it does. Missing one? Submit it free.