AI tools for building AI agents: voice and phone agents
Platforms for agents that listen and talk: phone lines, real-time speech and voice assistants. Of the 212 tools on our building AI agents page, 12 fit this narrower job. By pricing: 11 open source, 1 source-available. The most-starred open repos here are LiteLLM (61,465 stars), FunASR (20,738 stars), Pipecat (16,446 stars). Most come from the voice (9), MCP & infra (1), security & governance (1) categories.
LiteLLM
Open-source AI gateway and Python SDK that calls 100+ LLM providers in the OpenAI format, with cost tracking, guardrails, load balancing and logging.
Added
FunASR
ModelScope's speech recognition toolkit for offline, streaming and edge ASR, with VAD, punctuation, speaker diarization and OpenAI-compatible/MCP serving.
Added
Pipecat
Open-source framework for voice and multimodal conversational agents.
Added
TEN Framework
Open-source framework for real-time multimodal voice agents, including the TEN Agent demo.
Added
Vision Agents
Stream's open framework for building real-time voice and video agents with any model.
Added
Superagent
SDK for AI agent safety that blocks prompt injections and unsafe tool calls, redacts PII and secrets, and runs red-team scenarios.
Added
vocode
Open-source library for building voice-based LLM agents for phone calls and apps.
Added
ElevenLabs UI
Component library on shadcn/ui with orbs, waveforms and voice-agent components for building audio and agentic apps.
Added
OpenInference
OpenTelemetry-based instrumentation that traces LLM and agent framework calls so any observability backend can show them.
Added
Patter
Open-source SDK that gives your AI agent a phone number, handling speech, real-time voice and telephony carriers in Python or TypeScript.
Added
Voiceloop
Zero-dependency JS library for an interruptible in-browser voice loop (VAD, STT, LLM, TTS).
Added
LiveKit Agents
Real-time voice and video infrastructure with an agents framework.
Added
GitHub repos (11)
| Repo | GitHub stars | Licence | Last push |
|---|---|---|---|
| LiteLLM ↗ | 61,465 | MIT | 2026-10-18 |
| FunASR ↗ | 20,738 | MIT | 2026-10-18 |
| Pipecat ↗ | 16,446 | BSD 2-Clause | 2026-10-17 |
| TEN Framework ↗ | 11,172 | Apache License 2.0 with additional conditions | 2026-10-18 |
| Vision Agents ↗ | 8,171 | Apache License 2.0 | 2026-09-23 |
| Superagent ↗ | 6,766 | — | 2026-08-25 |
| vocode ↗ | 3,804 | MIT | 2024-11-15 |
| ElevenLabs UI ↗ | 2,425 | — | 2026-10-16 |
| OpenInference ↗ | 1,268 | — | — |
| Patter ↗ | 1,076 | — | — |
| Voiceloop ↗ | 6 | MIT | 2026-10-13 |
11 repos. Stars and licence come from each repo, checked when the listing was last verified.
FAQ
What are the best AI tools for building AI agents focused on voice and phone agents?
We list 12. The first three by GitHub stars and featured status are LiteLLM: Open-source AI gateway and Python SDK that calls 100+ LLM providers in the OpenAI format, with cost tracking, guardrails, load balancing and logging.; FunASR: ModelScope's speech recognition toolkit for offline, streaming and edge ASR, with VAD, punctuation, speaker diarization and OpenAI-compatible/MCP serving.; Pipecat: Open-source framework for voice and multimodal conversational agents.
Are there free or open-source options?
Yes: 11 open source, 1 source-available. Pricing is as each maker states it.
More from building AI agents
Tools are grouped by what each says it does. Missing one? Submit it free.