Web data extraction
How to use
Firecrawl.
Turning websites into markdown, structured records, URL maps, search results, screenshots, and asynchronous crawl datasets for AI applications.
Firecrawl is a web scraping, crawling, mapping, search, and structured extraction platform. Turning websites into markdown, structured records, URL maps, search results, screenshots, and asynchronous crawl datasets for AI applications. This guide covers the whole path in one place: official access, a first session that produces something reviewable, the checks that make output trustworthy, and the permissions worth limiting before you connect real work.
AI should make the work easier to inspect. If the workflow removes the source, the owner, or the review step, redesign the workflow.
Access & setup
Find, install, and sign in to Firecrawl
Get into the official Firecrawl experience with the right account and a setup you understand.
- 01
Start at https://www.firecrawl.dev/ and confirm the domain before entering account or payment information.
- 02
Availability: Firecrawl is available through a hosted API, SDKs, CLI, MCP integration, and an open-source self-hosting path.
- 03
A hosted API key or a maintained self-hosted deployment, network access to authorized targets, and a secure application environment.
- 04
Sign in with the account you intend to keep using, then review plan, data, notification, and permission settings.
- 05
Run one low-risk test task before connecting sensitive files, repositories, or workspace data.
- Use official download pages.
- Review permissions during setup.
- Keep installers and applications updated.
First session
Your first useful Firecrawl session
Learn the interaction loop using a small task with a clear outcome.
- 01
Create a development key and choose a public page you are authorized to retrieve.
- 02
Request only main-content markdown with explicit freshness and timeout settings.
- 03
Inspect the final URL, metadata, completeness, cache state, and extracted text for unsafe or sensitive content.
- 04
Store the source, retrieval time, and content hash before using the result in a model or index.
- State the outcome before the background.
- Provide the real source material.
- Review the result before expanding the task.
Quality control
Check the quality of Firecrawl output
Establish that a retrieved answer is actually supported by the sources it cites.
- 01
Restate the question precisely; retrieval quality degrades fastest on vague or compound queries.
- 02
Open the cited sources and confirm each one genuinely contains the claim attributed to it.
- 03
Check the publication and capture dates — stale sources are the most common cause of confidently wrong answers.
- 04
Look for what is missing: retrieval returns what matches the query, not what would contradict it.
- 05
Confirm quotations are verbatim and that summaries have not reversed a qualification or a negation.
- Open the citation; do not trust that a link supports the claim beside it.
- Check dates on every retrieved source.
- Treat an absent contradiction as unverified, not as agreement.
Privacy & permissions
Use Firecrawl safely
Control what you index and what leaves your systems when you query it.
- 01
Classify the corpus before indexing: confidential, personal, and regulated material needs an explicit decision.
- 02
Confirm where the index is stored, who can query it, and how long content is retained.
- 03
Scope crawling and scraping to sources you are permitted to collect, and respect robots and terms of use.
- 04
Strip secrets and personal data before indexing rather than filtering them at query time.
- 05
Review query logs — the questions asked can be as sensitive as the documents indexed.
- Scraping can expose copyrighted, private, malicious, or personal data and browser actions can carry powerful credentials. Confirm authorization, protect keys and cookies, constrain scope, verify webhooks, manage caching and retention, and treat every retrieved page as untrusted input.
- Follow your organisation's approved-use policy.
- Never treat fluent output as authorization to act.
Core workflows
Step-by-step ways to use Firecrawl for the work it does best.
Each workflow is a separate guide with its own steps and review checkpoints.
Extract useful page content while controlling freshness, formats, browser behavior, caching, and retention.
↗ 02 WorkflowBuild a controlled asynchronous Firecrawl site crawlMap and crawl an authorized site with strict scope, verified webhooks, deduplication, and ingestion checks.
↗Official references
Check the current product documentation.
Features, plan limits, availability, and data controls change. These official pages are the starting points used for this guide.