Open-source LLM observability and eval tools (2026)
Open-source tracing, evaluation and monitoring tools for LLM apps and agents that you can self-host. Here are all 14 from the AgentScout directory.
How we picked and ranked: tools are matched by category, the pricing each maker states and what each tool says it does. They are ranked by GitHub stars as recorded on each listing (link checks 2026-10-18); tools without a tracked repo follow, by listing maturity. No paid placements.
1. MLflow
Open-source platform for tracing, evaluating and managing agents, LLM apps and ML models.
Pricing (as stated): Open source + cloud · Mature
2. Arize Phoenix
Open-source tracing and evaluation for LLM apps.
Pricing (as stated): Open source · Mature
3. Guardrails AI
Open-source framework for validating LLM inputs and outputs with configurable guardrails.
Pricing (as stated): Open source + cloud · Mature
4. AI-Infra-Guard
Tencent's AI red-teaming platform that scans agents, skills and MCP servers for risks.
Pricing (as stated): Open source · Mature
5. Agent Governance Toolkit
Microsoft toolkit for policy enforcement, identity and sandboxing for AI agents.
Pricing (as stated): Open source · Mature
6. Failproof AI
Observability and policy enforcement for AI agent harnesses.
Pricing (as stated): Open source · Mature
7. Agenta
Open-source LLMOps platform for prompt management, evaluation and observability.
Pricing (as stated): Open source + cloud · Active
8. TruLens
Evaluation and tracking for LLM experiments and AI agents.
Pricing (as stated): Open source · Active
9. Snyk Agent Scan
Snyk's scanner for security risks in MCP servers and agent skills.
Pricing (as stated): Open source · Active
10. OpenLIT
Open-source platform to trace, evaluate and guard agents and LLM apps.
Pricing (as stated): Open source · Active
11. Agent Memory Leaderboard
Public leaderboard and open evaluation protocol, run by a group of universities and labs, for comparing long-term memory systems for agents in recurring submission rounds.
Pricing (as stated): Open source · Active
12. Galileo Agent Leaderboard
Galileo's leaderboard that ranks how well LLMs perform as agents on tool use and multi-step tasks.
Pricing (as stated): Open source · Active
13. Helicone
evals & observabilityOpen source + cloudOpen-source LLM observability and gateway.
Pricing (as stated): Open source + paid cloud · Mature
14. Langfuse
evals & observabilityOpen source + cloudOpen-source LLM observability, tracing and evals.
Pricing (as stated): Open source + paid cloud · Mature
FAQ
What are the best open-source LLM observability?
Ranked by tracked GitHub stars, the top open-source LLM observability in our directory are MLflow, Arize Phoenix, Guardrails AI, AI-Infra-Guard, Agent Governance Toolkit. Stars measure popularity, not fit, so compare a few against your own task.
How many open-source LLM observability does AgentScout list?
14 tools match. This page shows the top 14. Pricing mix on this page: Open source (9), Open source + cloud (5).
How much do these tools cost?
Pricing is what each maker states on their site. Open-source tools are free to self-host but may need a paid model API key or offer paid cloud plans; freemium tools have a free plan with limits. Check each official site.
Where can I see every tool for monitoring agents in production?
Our task page lists all 16 tools we match to monitoring agents in production.
Related: All tools for monitoring agents in production · Best AI Agent Observability & Eval Tools · evals & observability (14)
By role: For AI engineers · For DevOps and SRE engineers
All popular searches · AI agents by role · Free tools
Facts come from our directory listings; pricing is as each maker states it and can change. Check official sites before you choose.