Open-source AI testing agents (2026)
AI testing agents with public source code you can read, self-host and extend. Here are all 11 from the AgentScout directory.
Updated
How we picked and ranked: tools are matched by category, the pricing each maker states and what each tool says it does. They are ranked by GitHub stars as recorded on each listing (link checks 2026-10-18); tools without a tracked repo follow, by listing maturity. No paid placements.
1. Goose
Extensible open-source AI agent that installs, runs, edits and tests code on your machine with any LLM and MCP extensions.
Pricing (as stated): Open source · Mature
2. Midscene.js
Vision-driven UI automation and E2E testing framework where a GUI agent carries out natural-language steps on web, mobile and desktop apps.
Pricing (as stated): Open source · Mature
3. MCP Inspector
Official developer tool for testing and debugging MCP servers through a web UI, CLI or terminal UI.
Pricing (as stated): Open source · Mature
4. MobileBuildMCP
Sentry's MCP server and CLI that gives coding agents build, test and run tools for iOS and macOS projects.
Pricing (as stated): Open source · Mature
5. Unity MCP
MCP tools and CLI that let AI agents develop and test inside the Unity engine.
Pricing (as stated): Open source · Active
6. NitroStack
TypeScript framework with decorators and dependency injection for building, testing and deploying MCP servers and MCP apps, with a companion desktop studio.
Pricing (as stated): Open source + cloud · Active
7. Shippie
Extendable code review and QA agent that reads your diff, explores the codebase and posts focused review comments.
Pricing (as stated): Open source · Active
8. ripwire
Zero-dependency CLI and MCP server that gives coding agents a ranked call graph, blast radius and tests-to-run before and after edits.
Pricing (as stated): Open source · Active
9. Testkube MCP
Testkube MCP server for managing test workflows and executions.
Pricing (as stated): Open source · Active
10. BrowserGym
ServiceNow's Gym environment for building, testing and evaluating web agents across browser benchmarks.
Pricing (as stated): Open source · Active
11. Hercules
Open-source testing agent that turns plain Gherkin steps into end-to-end UI, API, accessibility and security tests.
Pricing (as stated): Open source · Active
FAQ
What are the best open-source AI testing agents?
Ranked by tracked GitHub stars, the top open-source AI testing agents in our directory are Goose, Midscene.js, MCP Inspector, MobileBuildMCP, Unity MCP. Stars measure popularity, not fit, so compare a few against your own task.
How many open-source AI testing agents does AgentScout list?
11 tools match. This page shows the top 11. Pricing mix on this page: Open source (10), Open source + cloud (1).
How much do these tools cost?
Pricing is what each maker states on their site. Open-source tools are free to self-host but may need a paid model API key or offer paid cloud plans; freemium tools have a free plan with limits. Check each official site.
Where can I see every tool for testing apps?
Our task page lists all 22 tools we match to testing apps.
Related: All tools for testing apps · coding (5) · MCP & infra (4) · browser/computer use (1) · evals & observability (1)
By role: For developers · For QA engineers
All popular searches · AI agents by role · Free tools
Best-of lists: Cursor alternatives · All best lists
Related categories: Coding · MCP & infra
Facts come from our directory listings; pricing is as each maker states it and can change. Check official sites before you choose.