API Reference
CrawlForge MCP API Reference
Complete reference for all 23 CrawlForge MCP tools. Build powerful web scraping applications with our developer-first API.
API Overview
Base URL
Bash
Authentication
All API requests require an API key passed in the X-API-Key header:
Bash
Get your API key from the dashboard.
Request Format
All tool endpoints accept JSON in the request body:
Json
Response Format
All responses follow a standard format:
Json
HTTP Status Codes
| Code | Description |
|---|---|
| 200 | OK - Request successful |
| 400 | Bad Request - Invalid parameters |
| 401 | Unauthorized - Invalid or missing API key |
| 402 | Payment Required - Insufficient credits |
| 429 | Too Many Requests - Rate limit exceeded |
| 500 | Internal Server Error |
Rate Limits
Rate limits apply per API key and vary by plan. Exceeding the limit returns a
429 Too Many Requests response.Free Plan
1 requests per second
Hobby Plan
2 requests per second
Professional Plan
4 requests per second
Business Plan
10 requests per second
All Tools
Basic Tools 1-2 credits
list_ollama_models
List locally installed Ollama models (free discovery helper)
0 credits
fetch_url
Fetch and parse web pages with automatic handling
1 credit
extract_text
Extract clean text from HTML with intelligent parsing
1 credit
extract_links
Discover and extract all links from a webpage
1 credit
extract_metadata
Extract page metadata, OpenGraph, and Twitter Card tags
1 credit
scrape_template
Pre-built scrapers for 10 popular sites (Amazon, GitHub, YouTube, and more)
1 credit
scrape_structured
Extract structured data using CSS selectors
2 credits
extract_content
Extract main content from web pages with readability detection
2 credits
map_site
Discover and map website structure and URLs
2 credits
process_document
Process documents from multiple formats including PDFs
2 credits
localization
Multi-language and geo-location management for scraping
2 credits
Advanced Tools 3-5 credits
track_changes
Track page changes with baselines, scheduled monitoring, and alerts
3 credits
analyze_content
Analyze content for sentiment, language, and topics
3 credits
extract_structured
LLM-powered schema-driven extraction with CSS selector fallback
3 credits
extract_with_llm
LLM-powered extraction, local Ollama by default (OpenAI/Anthropic optional)
3 credits
summarize_content
Generate intelligent summaries of text content
4 credits
crawl_deep
Deep website crawling using breadth-first search
4 credits
stealth_mode
Anti-detection browser management with fingerprint randomization
5 credits
scrape_with_actions
Execute browser actions before scraping (clicks, forms, etc.)
5 credits
batch_scrape
Process multiple URLs simultaneously with async job management
5 credits
search_web
Search the web using Google Custom Search API
5 credits
generate_llms_txt
Generate standard-compliant llms.txt and llms-full.txt files
5 credits
Premium Tools 10 credits
Error Handling
All errors follow a standard format with
error.code and error.message fields.Json
See the FAQ for a complete list of error codes and solutions.
Ready to start building?
Pick a tool and explore its documentation