dalao.ai

Firecrawl

Turn any website into LLM-ready dataOpen source

About Firecrawl

Firecrawl is a web data API built for AI agents. Give it a URL or a search query and it returns clean Markdown or structured JSON, handling rendering, anti-bot measures and queues. The company started at Y Combinator in 2022. The core code is open source under the AGPL, there's a hosted service billed by usage, and the Fire-engine anti-bot engine is cloud-only. In September 2026 it announced a $75M Series B and launched Alexandria, which plugs official data providers and its own indexes into the same API.

EditorialUpdated 7 min read

What problem it solves

Before a language model can use a web page, someone has to turn it into readable text, and the hard parts are JavaScript rendering, bot protection, rate limits, queues and retries. Firecrawl wraps all of that in a set of endpoints. Search returns results with full page content. Scrape handles a single page and Crawl a whole site. Map lists a site's URLs and Batch Scrape does many pages at once. Parse handles files. Interact clicks and types on a page from a prompt or code. Agent browses on its own to gather data, and Monitor checks pages for changes on a schedule.

The company's About page says Firecrawl started in 2022 as part of Y Combinator, and that the team previously built Mendable and SideGuide. Y Combinator's company page lists the batch as Summer 2022 (S22) but gives the founding year as 2024; the GitHub repo dates from April 2024. The three co-founders are CEO Caleb Peffer, CTO Nicolas Silberstein Camara and Chief Growth Officer Eric Ciarla.

Mendable was an AI chat product for documentation. In the Series B announcement, Peffer writes that Mendable worked, but building it taught the team that getting clean, reliable information out of the web was the hardest part of the stack, and that other AI teams were solving the same problem from scratch. So they built Firecrawl. They figured a few hundred people had that problem; Peffer says more than 1.5 million users now build with it.

On September 22, 2026, the company announced a $75M Series B led by Smash Capital, with Altos, Nexus, Y Combinator, Freestyle and Offline Ventures joining. It also launched Alexandria, which puts official data providers, custom connectors and Firecrawl's own indexes alongside the live web, so an agent uses one interface to find a source, learn what it contains and pull from it. The indexes named include a Research Index with tens of millions of paper abstracts, a Developer Index covering docs, READMEs, issues and merged pull requests, and a Government Index of laws and regulations. Firecrawl already pays data providers such as Wikimedia Enterprise under individual agreements, and plans a self-service system so people and organizations get paid when agents use what they know.

The API to search, scrape, and interact with the web at scale. The context API for AI agents. Open source and available as a hosted service.

Firecrawl README

Highlights

Output a model can use directly. One request can return Markdown, HTML, JSON, screenshots and more, and JSON extraction can follow a supplied schema. The README says it respects robots.txt by default.

Built to plug into agents. The README lists three routes: install Skills into a coding agent (npx skills add firecrawl/skills), use the CLI, or connect the official MCP server, firecrawl-mcp. SDKs cover nine languages: Python, Node, Go, Java, Elixir, Rust, Ruby, .NET and PHP. There are also integrations for Lovable, Zapier and n8n.

The company's own benchmark. The homepage reports 96% coverage and a P95 latency of 3,387 ms on its public 1,000-URL dataset (firecrawl/scrape-content-dataset-v1), from a run on January 13, 2026. That's an internal test, and the dataset is public.

Alexandria. The data layer launched with the Series B lets agents query official data providers and Firecrawl's own indexes, not just the live web. Firecrawl says that across 845 tasks, using the same model and prompts with blind AI judging, agents using Alexandria scored 21% higher on answer quality than agents using built-in web tools. Again, that's the company's own test.

How it compares

Open-source libraries like Browser Use let a model drive a browser on the local machine. Firecrawl is mainly a hosted API billed per request: it runs the browsers, proxies and bot handling, and returns cleaned results. Its Interact and Agent features can also act on pages and browse on their own, but only in the cloud.

One user quote on the homepage says they moved their internal agent's scraping tool from Apify to Firecrawl because it benchmarked 50x faster (user's own claim).

On open source versus cloud, Firecrawl's docs say both expose the core APIs; the difference is who configures, secures and runs the supporting services. When self-hosting, authentication, TLS, persistence, monitoring, capacity, upgrades and recovery are the operator's responsibility. LLM-backed formats and advanced scraping services need extra setup. Agent, Browser, Interact, the managed dashboard and enterprise controls are cloud-only.

Who uses it

The homepage says Firecrawl is trusted by more than 150,000 companies, and the Series B post claims over 1.5 million users. Both are the company's own figures. The Series B post says Mendable's customers included Snapchat, Coinbase and MongoDB. Public numbers as of 2026-10-25: about 195,000 GitHub stars and 10,000 forks, and the npm downloads API shows 709,842 downloads of the Node SDK @mendable/firecrawl-js from 2026-10-16 to 2026-10-22 (as of 2026-10-26).

Who it's for

Teams building RAG, research assistants, or competitor and lead research. People who want coding agents like Claude Code, or any MCP client, to fetch web pages. Teams that need to watch pages for changes without running their own browser fleet and proxies. Products that can pay by usage and would rather hand bot protection to someone else.

Not for

Anyone scraping a handful of fixed pages, where a short script will do. Teams that must self-host everything and also need strong anti-bot capability, since Fire-engine is cloud-only. And companies that want to offer a modified version as a network service without publishing their source under the AGPL.

Getting started

Cloud: create an account at firecrawl.dev to get an API key, which starts with fc-. The web Playground runs a test URL; the Python SDK installs with pip install firecrawl-py and exposes scrape and search on Firecrawl(api_key=...). For a coding agent, run npx -y firecrawl-cli@latest init --all --browser to install the CLI and Skills in one go.

Self-hosted: the docs start it with Docker Compose; the API listens on localhost:3002. The quickstart turns API authentication off (USE_DB_AUTHENTICATION=false), and the queue admin UI is off by default. The docs say to get one scrape working before adding services. LLM features can use any OpenAI-compatible provider or Ollama.

Versions: as of 2026-10-26 the latest release is v2.11.0, published 2026-06-19.

Pricing & license

Billing runs on credits (as of 2026-10-25). Free includes 1,000 credits a month with no card and 2 concurrent requests. Hobby is $19 a month ($16 billed annually) for 5,000 credits and 5 concurrent. Standard is $99 ($83 annually) for 100,000 credits and 25 concurrent. Growth is $399 ($333 annually) for 500,000 credits and 50 concurrent. Scale is $749 ($599 annually) for 1,000,000 credits and 100 concurrent, with unused credits rolling over for a month. Enterprise is custom. Top-ups come in $5 increments, each adding 1,000 to 5,000 credits depending on the plan, and purchased credits expire on cancellation.

Unit costs: Scrape, Crawl and Map are 1 credit per page; Monitor is 1 per page per check; Search is 2 per 10 results; Agent gets 5 free runs a day, then dynamic pricing. Add-ons: JSON output adds 4 credits per page. The pricing page lists Interact at 2 credits per browser minute, but the billing docs say 2 to 7: 7 with a prompt, 2 for Playwright code only. PDF parsing adds 1 per page, prompt-injection checks add 4, and zero data retention (ZDR) adds 1. Failed scrapes are free, but 403 and 404 responses still cost 1 credit.

License: the core is AGPL-3.0; the SDKs and some UI components are MIT.

Risks & limits

Fire-engine, the anti-bot engine, isn't open source. A maintainer wrote in GitHub issue #468 that it's "not available for the self hosted version" and "not (yet) open-source," and issue #2815 in 2026 confirmed again that self-hosted setups don't get it; only the Playwright service supports waiting for page loads, and everything else falls back to plain requests. The self-hosting docs describe Fire-engine as something to "run and configure separately; it is not included," and the maintainers say in those issues that it isn't offered to self-hosters.

The quickstart runs without authentication. Firecrawl respects robots.txt by default, but compliance rests with the user. The 96% coverage figure and Alexandria's 21% gain are both the company's own tests.

Our verdict

Firecrawl turns web pages into Markdown or structured JSON through one set of endpoints, handling rendering, bot protection and queues; it ships SDKs in nine languages, an official MCP server, a CLI and Skills; the core is open source under AGPL-3.0 and can be self-hosted with Docker Compose.

Its limits: the Fire-engine anti-bot engine, along with Agent, Browser and Interact, is cloud-only; the self-hosting quickstart turns API authentication off; add-ons such as JSON output and Interact are billed extra; the 96% coverage figure and Alexandria's 21% gain are both the company's own tests.

Why it stands out

Firecrawl is the scraping layer that most retrieval pipelines end up depending on without announcing it. Give it a URL and it returns clean Markdown, structured data, or a full crawl with the boilerplate stripped away.

The hard parts are unglamorous: JavaScript rendering, anti-bot handling, rate control, and a job queue that survives a bad afternoon. The core is open source under AGPL-3.0, with the SDKs and some UI components under MIT; the Fire-engine anti-bot service is not part of the open-source release, and Agent and Browser are cloud-only. The team previously built Mendable.

It turns a URL into clean Markdown or structured data, handling JavaScript rendering and rate control, with the core open source under AGPL-3.0.

Sources

Official

  1. Firecrawl READMEGitHub · Accessed Oct 25, 2026
  2. Firecrawl LICENSEGitHub · Accessed Oct 25, 2026
  3. Firecrawl | Web Data API for AI Agentsfirecrawl.dev · Accessed Oct 25, 2026
  4. Firecrawl: Aboutfirecrawl.dev · Accessed Oct 25, 2026
  5. Introducing Alexandria and our $75M Series Bfirecrawl.dev
  6. Pricing | Firecrawlfirecrawl.dev · Accessed Oct 25, 2026
  7. Billing | Firecrawldocs.firecrawl.dev · Accessed Oct 25, 2026
  8. Self-hosting Firecrawl | Firecrawldocs.firecrawl.dev · Accessed Oct 25, 2026

Updates & reading

NewsX

Firecrawl Alexandria adds fantasy basketball data

Firecrawl says fantasy basketball data is now live in Alexandria, giving ChatGPT or Claude projections, expert rankings, injury updates and news for every NBA player via the Firecrawl plugin.

1 source · 1 post on X