Welcome to CrawlForge

We're excited to introduce CrawlForge MCP - a comprehensive web scraping platform designed specifically for AI engineers and developers building intelligent applications.

Why We Built CrawlForge

Modern AI applications need reliable, structured data from the web. Whether you're building RAG systems, training models, or creating intelligent agents, web data is essential. But traditional scraping tools are complex, unreliable, and not designed for AI workflows.

CrawlForge changes that.

What Makes CrawlForge Different

26 Specialized Tools

We've built 26 purpose-built tools for different scraping scenarios:

Basic Tools: fetch_url, extract_text, extract_links, extract_metadata
Extraction Tools: scrape_structured, extract_content, map_site, process_document
Advanced Tools: search_web, crawl_deep, batch_scrape, deep_research
And more: stealth_mode, localization, analyze_content, summarize_content

Credit-Based Pricing

Pay only for what you use. Each tool costs 1-10 credits based on complexity:

Simple fetches: 1 credit
Structured scraping: 2 credits
Browser automation: 5 credits
Deep research: 10 credits

Start with 1,000 free credits - no credit card required.

Built for Developers

RESTful API: Simple HTTP requests, no SDKs required
Comprehensive docs: Clear examples in cURL, TypeScript, Python
Generous rate limits: Scale from prototype to production
Real-time analytics: Track usage and optimize costs

Getting Started

Sign up for a free account at crawlforge.dev
Create an API key in your dashboard
Make your first request:

Bash

curl -X POST https://crawlforge.dev/api/v1/tools/fetch_url \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

That's it! You're scraping.

What's Next

We're just getting started. Here's what's coming:

Webhooks: Get notified when scraping jobs complete
Custom tools: Request specialized scrapers for your use case
Team collaboration: Share API keys and manage usage
Advanced analytics: Deep insights into your scraping patterns

Join Us

We're building CrawlForge for developers like you. We'd love to hear your feedback, feature requests, and success stories.

Happy scraping!

- The CrawlForge Team

Frequently Asked Questions

What is CrawlForge?

CrawlForge is an enterprise web scraping platform purpose-built for AI engineers building RAG systems, training models, and intelligent agents. It exposes 26 specialized scraping tools natively over the Model Context Protocol so AI assistants can use them as easily as built-in capabilities.

Do I need an SDK to use CrawlForge?

No. CrawlForge is a RESTful API that works with simple HTTP requests -- no SDKs required. Documentation includes clear examples in cURL, TypeScript, and Python so you can integrate from any language or framework.

How does CrawlForge's credit-based pricing work?

You pay only for what you use. Simple fetches cost 1 credit, structured scraping is 2 credits, browser automation is 5 credits, and deep research is 10 credits. New accounts start with 1,000 free credits and no credit card is required.

What is on the CrawlForge roadmap?

Upcoming features include webhooks for job completion notifications, custom tool requests for specialized scraping needs, team collaboration with shared API keys, and advanced analytics into your scraping patterns.

What Makes CrawlForge Different

26 Specialized Tools

We've built 26 purpose-built tools for different scraping scenarios:

Basic Tools: fetch_url, extract_text, extract_links, extract_metadata

Extraction Tools: scrape_structured, extract_content, map_site, process_document

Advanced Tools: search_web, crawl_deep, batch_scrape, deep_research

And more: stealth_mode, localization, analyze_content, summarize_content

Credit-Based Pricing

Pay only for what you use. Each tool costs 1-10 credits based on complexity:

Simple fetches: 1 credit

Structured scraping: 2 credits

Browser automation: 5 credits

Deep research: 10 credits

Start with 1,000 free credits - no credit card required.

Built for Developers

RESTful API: Simple HTTP requests, no SDKs required

Comprehensive docs: Clear examples in cURL, TypeScript, Python

Generous rate limits: Scale from prototype to production

Real-time analytics: Track usage and optimize costs