Get Started

Getting Started with CrawlForge MCP

Get your first API response in under 5 minutes. CrawlForge MCP provides 30 specialized web scraping tools designed for AI applications.

What is CrawlForge MCP?

CrawlForge MCP is a powerful web scraping server built on the Model Context Protocol (MCP). It provides 30 specialized tools for extracting, analyzing, and processing web data—all accessible through a simple REST API or native MCP integration.

Whether you're building an AI research assistant, content aggregator, or data pipeline, CrawlForge MCP handles the complexity of web scraping so you can focus on building your application.

Why Choose CrawlForge?

Built for AI

Native MCP integration with Claude Desktop and other AI tools

30 Specialized Tools

From basic scraping to AI-powered research, all in one API

Credit-Based Pricing

Pay only for what you use, starting at $0.003/request

Developer-First

Simple API, comprehensive docs, 1,000 free credits to start

Quick Start

Create a free account and get 1,000 credits to start building immediately.

Create Free Account

Install via NPM (MCP Integration)

Use CrawlForge MCP natively with Claude Desktop, Cursor IDE, or any MCP-compatible tool. Check out the GitHub repository for more details.

aPackage Installation

Install globally:

Bash

npm install -g crawlforge-mcp-server

Dependencies, including crawlforge-extractors, install automatically.

Quick setup wizard:

Bash

npx crawlforge-setup

bClaude Desktop Configuration

Add to claude_desktop_config.json:

Json

{
  "mcpServers": {
    "crawlforge": {
      "command": "npx",
      "args": ["-y", "crawlforge-mcp-server"]
    }
  }
}

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%/Claude/claude_desktop_config.json
Linux: ~/.config/Claude/claude_desktop_config.json

cCursor IDE Configuration

Add to ~/.cursor/mcp.json:

Json

{
  "mcpServers": {
    "crawlforge": {
      "type": "stdio",
      "command": "crawlforge-mcp"
    }
  }
}

dEnvironment Variables

Bash

export CRAWLFORGE_API_KEY="cf_live_your_api_key_here"
export CRAWLFORGE_API_URL="https://api.crawlforge.dev"

eUsage Examples

"Search for the latest AI news"

"Extract all links from example.com"

"Crawl the documentation site and summarize it"

Make Your First Request

Fetch your first page with cURL, TypeScript, or Python. The TypeScript and Python samples use the official SDKs, crawlforge-sdk on npm and crawlforge on PyPI, and cURL calls the REST API directly.

cURL

Bash

curl -X POST https://www.crawlforge.dev/api/v1/tools/fetch_url \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

TypeScript

Bash

npm install crawlforge-sdk

Typescript

// npm install crawlforge-sdk
import { CrawlForge } from 'crawlforge-sdk';

const client = new CrawlForge({ apiKey: process.env.CRAWLFORGE_API_KEY });

const result = await client.fetchUrl({ url: 'https://example.com' });

// result.data is untyped in crawlforge-sdk 0.1 — its shape is the response shown in the next step.
const { status, content_length } = result.data as { status: number; content_length: number };
console.log('Status:', status);
console.log('Content length:', content_length);
console.log('Credits used:', result.creditsUsed);
console.log('Credits remaining:', result.creditsRemaining);

Python

Bash

pip install crawlforge

Python

# pip install crawlforge
from crawlforge import CrawlForge

client = CrawlForge()  # reads CRAWLFORGE_API_KEY

result = client.fetch_url(url='https://example.com')

# result.data is a plain dict — its shape is the response shown in the next step.
print(f"Status: {result.data['status']}")
print(f"Content length: {result.data['content_length']}")
print(f"Credits used: {result.credits_used}")
print(f"Credits remaining: {result.credits_remaining}")

Understand the Response

All API responses follow a standard format with success status, data, and credit information.

Json

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "status": 200,
    "status_text": "OK",
    "content": "<!DOCTYPE html>...",
    "content_length": 1256,
    "content_type": "text/html; charset=UTF-8"
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 245
}

success: Whether the request was successful
data: The extracted content and metadata
credits_used: Credits deducted for this request
credits_remaining: Your remaining credit balance

The SDKs return the same envelope as result.data, creditsUsed / credits_used and creditsRemaining / credits_remaining, and raise a typed error instead of returning success: false.

Pro Tip: Store your API key in environment variables and never commit it to version control. You can rotate your keys anytime from the dashboard.

Next Steps

API Reference

Explore all 30 tools and their parameters

Tool Documentation

Learn about specific tools and use cases

Examples & Tutorials

Step-by-step guides for common use cases

Pricing & Plans

Choose the right plan for your needs

Ready to build?

Get Started Free View API Docs

// npm install crawlforge-sdk import { CrawlForge } from 'crawlforge-sdk'; const client = new CrawlForge({ apiKey: process.env.CRAWLFORGE_API_KEY }); const result = await client.fetchUrl({ url: 'https://example.com' }); // result.data is untyped in crawlforge-sdk 0.1 — its shape is the response shown in the next step. const { status, content_length } = result.data as { status: number; content_length: number }; console.log('Status:', status); console.log('Content length:', content_length); console.log('Credits used:', result.creditsUsed); console.log('Credits remaining:', result.creditsRemaining);

# pip install crawlforge from crawlforge import CrawlForge client = CrawlForge() # reads CRAWLFORGE_API_KEY result = client.fetch_url(url='https://example.com') # result.data is a plain dict — its shape is the response shown in the next step. print(f"Status: {result.data['status']}") print(f"Content length: {result.data['content_length']}") print(f"Credits used: {result.credits_used}") print(f"Credits remaining: {result.credits_remaining}")

{ "success": true, "data": { "url": "https://example.com", "status": 200, "status_text": "OK", "content": "<!DOCTYPE html>...", "content_length": 1256, "content_type": "text/html; charset=UTF-8" }, "credits_used": 1, "credits_remaining": 999, "processing_time": 245 }