Skip to content
CrawlForge MCP
Integration

Integration Guide

Connect CrawlForge MCP to Claude Desktop, LangChain, LlamaIndex, and other AI frameworks. Built on the Model Context Protocol for universal compatibility.

What is the Model Context Protocol?

The Model Context Protocol (MCP) is an open standard created by Anthropic that enables AI assistants to securely connect to external data sources and tools. CrawlForge MCP implements this protocol to provide web scraping capabilities to any MCP-compatible AI application.

Why Use MCP Integration?

Universal Compatibility
Works with any MCP-compatible AI tool or framework
Zero Dependencies
No additional packages required beyond MCP client libraries
Real-Time Data
Give your AI access to live web data without training
Simple Setup
Configure once, use everywhere with standard MCP protocol

Integration Options

NPM Installation
Install CrawlForge MCP server globally via npm for use with any MCP client

Quick Start

MCP Protocol
Model Context Protocol specification and server configuration

Core

Claude Desktop
Install and configure CrawlForge MCP with Claude Desktop app

Popular

Cursor IDE
Enable AI-powered web scraping in Cursor IDE with .cursorrules config

IDE

LangChain
Integrate with LangChain for document loaders and retrieval chains

Framework

LlamaIndex
Use CrawlForge MCP as a data connector in LlamaIndex

Framework

n8n
Connect CrawlForge to n8n workflows via the MCP Client node or REST API

Automation

MCP Configurator
Generate the exact config for any client with your API key filled in

Tool

Claude Code
Register CrawlForge with one claude mcp add command or a project .mcp.json

IDE

VS Code
Give Copilot agent mode web access through .vscode/mcp.json

IDE

Zed
Add CrawlForge as a context server in Zed's settings.json

IDE

Gemini CLI
Register CrawlForge with gemini mcp add or ~/.gemini/settings.json

CLI

Windsurf
Give Cascade web access through mcp_config.json

IDE

Cline
Add CrawlForge to Cline's MCP settings with auto-approve for trusted tools

IDE

Quick Start

Option 1: Claude Desktop Integration
The fastest way to get started—configure CrawlForge MCP in Claude Desktop with a JSON config file.

Claude Desktop Setup Guide →

Option 2: Cursor IDE Integration
Enable AI-powered web scraping in your development workflow with Cursor IDE.

Cursor IDE Setup Guide →

Option 3: LangChain/LlamaIndex Integration
Integrate CrawlForge MCP into your AI application using popular frameworks.

LangChain Guide → · LlamaIndex Guide →

Option 4: Custom MCP Implementation
Build your own MCP client using the protocol specification.

MCP Protocol Docs →

Need Help with Integration?

Check our troubleshooting guides or contact support for personalized assistance.

View Getting StartedAPI Reference