Open-source AI web scraping tools (2026)
AI web scraping tools with public source code you can read, self-host and extend. Here are all 15 from the AgentScout directory.
Quick answer: the top picks on this list, ranked by GitHub stars and listing maturity, are Firecrawl, Scrapling and Crawl4AI, out of 15 tools listed below. At a glance: 15 open-source, 15 free or freemium and 13 local or self-hosted across 15 link-checked tools (last verified ).
Verified on : every tool link on this page was checked by our daily link check. No paid placements.
Ranked comparison (top 10)
| # | Tool | Pricing | Best for | Open source | Runs locally | GitHub stars |
|---|---|---|---|---|---|---|
| 1 |  Firecrawl | Open source + cloud | research the web | Yes | Self-hosted | 195,878 |
| 2 |  Scrapling | Open source | automate the browser, build an agent | Yes | Self-hosted | 89,984 |
| 3 |  Crawl4AI | Open source | automate the browser, research the web | Yes | Self-hosted | 86,378 |
| 4 |  ScrapeGraphAI | Open source + cloud | automate the browser, build an agent | Yes | Self-hosted | 32,269 |
| 5 |  Moli | Open source | automate the browser, research the web | Yes | Self-hosted | 21,082 |
| 6 |  Douyin TikTok Download API | Open source + cloud | voice & phone calls, connect tools via MCP | Yes | Offline | 20,962 |
| 7 |  Apify MCP Server | Open source + cloud | research the web, connect tools via MCP | Yes | Self-hosted | 13,411 |
| 8 |  Monid | Open source + cloud | voice & phone calls | Yes | Self-hosted | 6,732 |
| 9 |  Wigolo | Open source | write code, research the web | Yes | Offline | 5,507 |
| 10 |  Spider | Open source + cloud | automate the browser | Yes | Self-hosted | 2,805 |
Updated
How we picked and ranked: tools are matched by category, the pricing each maker states and what each tool says it does. They are ranked by GitHub stars as recorded on each listing (link checks 2026-10-28); tools without a tracked repo follow, by listing maturity. No paid placements.
1. Firecrawl
Turns websites into clean, LLM-ready data for agents.
Pricing (as stated): Open source + paid cloud · Mature
2. Scrapling
Python web-scraping framework with fast HTTP requests, stealth browsers, adaptive parsing and concurrent crawling, plus an MCP server for agents.
Pricing (as stated): Open source · Mature
3. Crawl4AI
Open-source crawler that turns websites into clean, LLM-ready Markdown for agents and RAG pipelines.
Pricing (as stated): Open source · Mature
4. ScrapeGraphAI
Python library that uses LLMs and graph logic to build scraping pipelines: describe the data you want and it extracts it from websites or documents.
Pricing (as stated): Open source + cloud · Mature
5. Moli
Lightweight Rust headless browser for AI agents that fetches pages, searches and extracts content, usable via CLI, CDP or WebDriver.
Pricing (as stated): Open source · Mature
6. Douyin TikTok Download API
、TikTok API， AI agent MCP ，Docker 。 Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async REST API, MCP server for AI agents, CLI and web console, plus offline speech-to-text.
Pricing (as stated): Open source + cloud · Mature
7. Apify MCP Server
Apify's MCP server giving agents access to thousands of scrapers for social media, search and maps.
Pricing (as stated): Open source + cloud · Mature
8. Monid
Gateway that gives an agent one base URL and one key to call thousands of paid tools, from web scraping and enrichment to image and voice generation, billed per call.
Pricing (as stated): Open source + cloud · Mature
9. Wigolo
Local-first web search, fetch and crawl for coding agents over MCP, with no API keys.
Pricing (as stated): Open source · Mature
10. Spider
Fast Rust web crawler for collecting data for agents and LLMs.
Pricing (as stated): Open source + cloud · Active
11. Bright Data MCP
Bright Data's MCP server for web search, scraping and browsing.
Pricing (as stated): Open source + cloud · Active
12. Crw
Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents.
Pricing (as stated): Open source + cloud · Active
13. Scout (Outcrawl)
Open-source browser agent from Outcrawl that opens a real browser and clicks, types and reads pages to finish a task you describe.
Pricing (as stated): Open source + cloud · Active
14. Scrawl
Mac app that lets you show a coding agent a problem: press a hotkey, talk, click and draw over your screen or a mirrored iPhone, and Scrawl turns it into transcribed, cropped frames you paste into your agent.
Pricing (as stated): Free, MIT licence · Active
15. ScrapeGraph MCP
ScrapeGraphAI's MCP server for AI-powered web scraping.
Pricing (as stated): Open source · Active
FAQ
What are the best open-source AI web scraping tools?
Ranked by tracked GitHub stars, the top open-source AI web scraping tools in our directory are Firecrawl, Scrapling, Crawl4AI, ScrapeGraphAI, Moli. Stars measure popularity, not fit, so compare a few against your own task.
How many open-source AI web scraping tools does AgentScout list?
15 tools match. This page shows the top 15. Pricing mix on this page: Open source + cloud (9), Open source (6).
How much do these tools cost?
Pricing is what each maker states on their site. Open-source tools are free to self-host but may need a paid model API key or offer paid cloud plans; freemium tools have a free plan with limits. Check each official site.
Where can I see every tool for scraping websites?
Our task page lists all 24 tools we match to scraping websites.
Related: All tools for scraping websites · browser/computer use (6) · MCP servers (6) · research (1) · agent infra (1)
By role: For researchers · For data analysts
All popular searches · AI agents by role · Free tools
Best-of lists: Crawl4AI alternatives · All best lists
Related categories: Browser/computer use · MCP servers
Facts come from our directory listings; pricing is as each maker states it and can change. Check official sites before you choose.