InferenceX
InferenceX is SemiAnalysis' open-source inference research platform that continuously benchmarks how fast and how efficiently AI models are served on different accelerators and software stacks. It runs end-to-end serving benchmarks for new models from day zero on NVIDIA B200, B300 and GB200 and AMD MI355X systems with vLLM and SGLang, adds networking, kernel and hardware microbenchmarks, measures power draw, and publishes results on inferencex.com. Apache 2.0.
Updated
✨ Try AgentScout's free AI image generator →Good for: research, evaluating agents
Use cases
About InferenceX
InferenceX is listed in our evals & observability category and is aimed at builders. InferenceX is SemiAnalysis' open-source inference research platform that continuously benchmarks how fast and how efficiently AI models are served on different accelerators and software stacks. It runs end-to-end serving benchmarks for new models from day zero on NVIDIA B200, B300 and GB200 and AMD MI355X systems with vLLM and SGLang, adds networking, kernel and hardware microbenchmarks, measures power draw, and publishes results on inferencex.com. Apache 2.0. Pricing, as the maker states it: Open source. Maturity: Active. We last checked the official link on 2026-11-15.
InferenceX at a glance
- Category: Evals & observability
- Built for: Developers and teams building with AI agents
- Pricing: Open source (as the maker states it)
- GitHub: SemiAnalysisAI/InferenceX · 2,040 stars (as of Nov 15, 2026)
- Official site: inferencex.semianalysis.com
- Listed on AgentScout: Nov 15, 2026
InferenceX FAQ
What is InferenceX?
InferenceX is SemiAnalysis' open-source inference research platform that continuously benchmarks how fast and how efficiently AI models are served on different accelerators and software stacks. It runs end-to-end serving benchmarks for new models from day zero on NVIDIA B200, B300 and GB200 and AMD MI355X systems with vLLM and SGLang, adds networking, kernel and hardware microbenchmarks, measures power draw, and publishes results on inferencex.com. Apache 2.0.
How much does InferenceX cost?
InferenceX's pricing, as the maker states it: Open source. Open-source tools are free to use, though you may still pay for model API usage or a hosted plan. Check the official site for current plans.
Where can I find InferenceX's source code?
On GitHub at SemiAnalysisAI/InferenceX, which had 2,040 stars as of Nov 15, 2026.
What are alternatives to InferenceX?
Similar tools include Opik, ReLE Chinese LLM Benchmark, SWE-bench, Galileo Agent Leaderboard, Monocle and QAmcp.
Facts from our link-checked listing, the GitHub repository and the official site. See something wrong? Suggest an edit.
InferenceX ratings and reviews
Be the first to rate InferenceX: tap 1 to 5 stars below. A written review is optional.
InferenceX alternatives
Other evals & observability tools in our directory. See all evals & observability tools →
Opik
Comet's open-source platform for tracing, evaluating and monitoring LLM apps, RAG systems and agent workflows, self-hosted or in the cloud.
ReLE Chinese LLM Benchmark
ReLE (formerly CLiB) is NoneLinear's continuously updated benchmark of how well large language models handle Chinese, covering more than 400 commercial and open models such as GPT, Gemini, Claude, Grok, Qwen, DeepSeek, Kimi, GLM and ERNIE. Models are scored across about 300 dimensions in seven domains including education, medicine, finance, law, reasoning and math, instruction following, and agents and tool calling, and the project also publishes a defect library of over two million failure cases.
SWE-bench
Benchmark that tests whether language models and coding agents can resolve real GitHub issues, with multimodal and verified variants.
Galileo Agent Leaderboard
Galileo's leaderboard that ranks how well LLMs perform as agents on tool use and multi-step tasks.
Monocle
Monocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python.
QAmcp
Free open source MCP server tester that connects like an AI app and flags hidden instructions, leaked secrets, bad inputs, slow tools, and login problems.
Get your badge
Built InferenceX? Optional. Showing this badge does not change your rank or earn a Featured spot. It links to this page. About badges