Get Started

Getting Started with CrawlForge MCP

Get your first API response in under 5 minutes. CrawlForge MCP provides 23 specialized web scraping tools designed for AI applications.

What is CrawlForge MCP?

CrawlForge MCP is a powerful web scraping server built on the Model Context Protocol (MCP). It provides 23 specialized tools for extracting, analyzing, and processing web data—all accessible through a simple REST API or native MCP integration.

Whether you're building an AI research assistant, content aggregator, or data pipeline, CrawlForge MCP handles the complexity of web scraping so you can focus on building your application.

Why Choose CrawlForge?

Built for AI

Native MCP integration with Claude Desktop and other AI tools

23 Specialized Tools

From basic scraping to AI-powered research, all in one API

Credit-Based Pricing

Pay only for what you use, starting at $0.003/request

Developer-First

Simple API, comprehensive docs, 1,000 free credits to start

Quick Start

Create a free account and get 1,000 credits to start building immediately.

Create Free Account

Install via NPM (MCP Integration)

Use CrawlForge MCP natively with Claude Desktop, Cursor IDE, or any MCP-compatible tool. Check out the GitHub repository for more details.

aPackage Installation

Install globally:

Bash

npm install -g crawlforge-mcp-server

Quick setup wizard:

Bash

npx crawlforge-setup

bClaude Desktop Configuration

Add to claude_desktop_config.json:

Json

{
  "mcpServers": {
    "crawlforge": {
      "command": "npx",
      "args": ["-y", "crawlforge-mcp-server"]
    }
  }
}

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%/Claude/claude_desktop_config.json
Linux: ~/.config/Claude/claude_desktop_config.json

cCursor IDE Configuration

Add to ~/.cursor/mcp.json:

Json

{
  "mcpServers": {
    "crawlforge": {
      "type": "stdio",
      "command": "crawlforge-mcp"
    }
  }
}

dEnvironment Variables

Bash

export CRAWLFORGE_API_KEY="cf_live_your_api_key_here"
export CRAWLFORGE_API_URL="https://api.crawlforge.dev"

eUsage Examples

"Search for the latest AI news"

"Extract all links from example.com"

"Crawl the documentation site and summarize it"

Make Your First Request

Use cURL, TypeScript, Python, or any HTTP client to fetch your first page.

cURL

Bash

curl -X POST https://www.crawlforge.dev/api/v1/tools/fetch_url \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

TypeScript

Typescript

const response = await fetch('https://www.crawlforge.dev/api/v1/tools/fetch_url', {
  method: 'POST',
  headers: {
    'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ url: 'https://example.com' }),
});

const data = await response.json();
console.log(data);

Python

import requests
import os

response = requests.post(
    'https://www.crawlforge.dev/api/v1/tools/fetch_url',
    headers={
        'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
        'Content-Type': 'application/json',
    },
    json={'url': 'https://example.com'}
)

data = response.json()
print(data)

Understand the Response

All API responses follow a standard format with success status, data, and credit information.

Json

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "status": 200,
    "status_text": "OK",
    "content": "<!DOCTYPE html>...",
    "content_length": 1256,
    "content_type": "text/html; charset=UTF-8"
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 245
}

success: Whether the request was successful
data: The extracted content and metadata
credits_used: Credits deducted for this request
credits_remaining: Your remaining credit balance

Pro Tip: Store your API key in environment variables and never commit it to version control. You can rotate your keys anytime from the dashboard.

Next Steps

API Reference

Explore all 23 tools and their parameters

Tool Documentation

Learn about specific tools and use cases

Examples & Tutorials

Step-by-step guides for common use cases

Pricing & Plans

Choose the right plan for your needs

Ready to build?

Get Started Free View API Docs

const response = await fetch('https://www.crawlforge.dev/api/v1/tools/fetch_url', { method: 'POST', headers: { 'X-API-Key': process.env.CRAWLFORGE_API_KEY!, 'Content-Type': 'application/json', }, body: JSON.stringify({ url: 'https://example.com' }), }); const data = await response.json(); console.log(data);

import requests import os response = requests.post( 'https://www.crawlforge.dev/api/v1/tools/fetch_url', headers={ 'X-API-Key': os.environ['CRAWLFORGE_API_KEY'], 'Content-Type': 'application/json', }, json={'url': 'https://example.com'} ) data = response.json() print(data)

{ "success": true, "data": { "url": "https://example.com", "status": 200, "status_text": "OK", "content": "<!DOCTYPE html>...", "content_length": 1256, "content_type": "text/html; charset=UTF-8" }, "credits_used": 1, "credits_remaining": 999, "processing_time": 245 }