Judgeval
Open-source SDK for tracing agents and scoring them with agent judges, so failures in production can be found and fixes checked.
✨ Try AgentScout's free AI image generator →Good for: building AI agents, evaluating agents, monitoring agents in production
Use cases
About Judgeval
Judgeval is listed in our evals & observability category and is aimed at builders. Open-source SDK for tracing agents and scoring them with agent judges, so failures in production can be found and fixes checked. Pricing, as the maker states it: Open source + cloud. Maturity: Active. We last checked the official link on 2026-10-18.
Judgeval at a glance
- Category: Evals & observability
- Built for: Developers and teams building with AI agents
- Pricing: Open source + cloud (as the maker states it)
- Licence: Apache License 2.0 (OSI-approved open source)
- GitHub: JudgmentLabs/judgeval · 1,068 stars (as of Oct 18, 2026)
- Official site: judgmentlabs.ai
- Listed on AgentScout: Oct 18, 2026
Judgeval FAQ
What is Judgeval?
Open-source SDK for tracing agents and scoring them with agent judges, so failures in production can be found and fixes checked.
How much does Judgeval cost?
Judgeval's pricing, as the maker states it: Open source + cloud. Open-source tools are free to use, though you may still pay for model API usage or a hosted plan. Check the official site for current plans.
Is Judgeval open source?
Yes. Judgeval's GitHub repository is published under the Apache License 2.0, according to the licence file in the repo.
Where can I find Judgeval's source code?
On GitHub at JudgmentLabs/judgeval, which had 1,068 stars as of Oct 18, 2026.
What are alternatives to Judgeval?
Other evals & observability tools on AgentScout include MLflow, Opik, Arize Phoenix, SWE-bench, Kiln.
Facts from our link-checked listing, the GitHub repository and the official site. See something wrong? Suggest an edit.
Judgeval ratings and reviews
Judgeval alternatives
Other evals & observability tools in our directory. See all evals & observability tools →
Latitude
Observability platform for AI agents that captures traces, groups failures into signals and verifies fixes against real traces.
Opik
Comet's open-source platform for tracing, evaluating and monitoring LLM apps, RAG systems and agent workflows, self-hosted or in the cloud.
Kore.ai Autoloop
Optimization engine in Kore.ai Agent Platform Artemis that evaluates agents against business goals, diagnoses failures from traces, applies targeted fixes, and re-verifies; GA Oct 2026.
MLflow
Open-source platform for tracing, evaluating and managing agents, LLM apps and ML models.
Agenta
Open-source LLMOps platform for prompt management, evaluation and observability.
W&B Weave
Weights & Biases toolkit for tracing and evaluating agents.
Popular searches with this tool
🏆 Ranked in: Best AI Agent Observability & Eval Tools
Badge for makers
Built Judgeval? Add this badge to your README or site. It links back to this page. About badges
