Skip to content

Add fastCRW (open-source alternative to Firecrawl and Tavily) #560

@us

Description

@us

Add open source tool: fastCRW

  • Repository: https://github.com/us/crw
  • Website: https://fastcrw.com
  • Description: Open-source web scraping, crawling, AI search, and data extraction API in a single 8 MB Rust binary. Firecrawl-compatible /v1 endpoint and Tavily-compatible search responses. 710 ms p50 on a 1,000-URL benchmark, 6.6 MB idle RAM.
  • Category: API Development
  • Features: Self-hosted option, API integration, Markdown support, Plugin system, Data export

Open source alternative to

  1. Firecrawl - https://firecrawl.dev
  2. Tavily - https://tavily.com

Additional info

  • License: AGPL-3.0
  • Primary language: Rust
  • Endpoints: scrape, crawl, search, extract, map (Firecrawl /v1 compatible)
  • Search response format: Tavily-compatible
  • Free tier: 500 credits/month on managed cloud
  • Self-host: AGPL-3.0, single 8 MB Docker image
  • SDKs: Python, Node.js, CLI
  • Integrations: LangChain, LlamaIndex, CrewAI, n8n, Dify
  • Distribution: Homebrew, apt, Docker

Benchmark

On a public 1,000-URL benchmark fastCRW posts 710 ms p50 and 833 ms average per URL, runs in 6.6 MB idle RAM, and ships as an 8 MB Docker image.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions