CrawlForge
Cursor IDE

Cursor IDE Integration

Enable AI-powered web scraping in Cursor IDE with CrawlForge MCP. The setup wizard auto-configures Cursor, or add the server to ~/.cursor/mcp.json yourself.

Prerequisites

Cursor IDE
Download from cursor.com

Required

Node.js 18+
Install from nodejs.org

Required

API Key
Get from dashboard

Free

1
Install CrawlForge MCP Server
First, install the CrawlForge MCP server package globally.
Bash
npm install -g crawlforge-mcp-server
Quick setup: Run npx crawlforge-setup to configure your API key automatically.
2
Configure Cursor IDE
Choose your preferred configuration method:

AOption A: Auto-configure with the wizard

The setup wizard detects Cursor and writes ~/.cursor/mcp.json for you

Run the wizard—if Cursor is installed it adds the CrawlForge server automatically:

Bash
npx crawlforge-setup

Restart Cursor afterwards to load the server.

BOption B: Edit ~/.cursor/mcp.json

Add the server to Cursor's MCP config file yourself

Create or edit ~/.cursor/mcp.json (or .cursor/mcp.json in a project):

Json
{
  "mcpServers": {
    "crawlforge": {
      "type": "stdio",
      "command": "crawlforge-mcp"
    }
  }
}

crawlforge-mcp requires the global install from Step 1. With no global install, use "command": "npx" and "args": ["-y", "crawlforge-mcp-server"] instead.

Tip: Editing ~/.cursor/mcp.json applies to every project, while a project-level .cursor/mcp.json scopes the server to that repo.
3
Set Your API Key
Configure your API key using one of these methods:

Setup Wizard

Run the setup wizard to configure your API key:

Bash
npx crawlforge-setup

Manual Setup

Or set the environment variable manually:

Bash
export CRAWLFORGE_API_KEY="cf_live_your_api_key_here"

Add this to your shell profile (~/.bashrc, ~/.zshrc, etc.)

Security: Never commit your API key to version control. Use environment variables or the config file.
4
Test the Integration
Restart Cursor IDE and test the CrawlForge MCP integration.
  • Restart Cursor IDE — Close and reopen Cursor to load the MCP configuration
  • Open AI Chat — Press Cmd/Ctrl + L to open the AI chat
  • Test a Command — Try one of the example prompts below

Example Prompts

"Search for the latest AI news"

"Extract all links from example.com"

"Crawl the documentation site and summarize it"

"Extract product prices from this e-commerce site"

Available Tools

After setup, Cursor will have access to all 23 CrawlForge MCP tools:

ToolDescriptionCredits
fetch_urlFetch HTML from URLs1 credits
extract_textExtract clean text1 credits
extract_linksExtract all links1 credits
extract_metadataGet page metadata1 credits
scrape_structuredCSS selector extraction2 credits
batch_scrapeParallel batch scraping1/URL
deep_researchAI-powered research10 credits
search_webWeb search integration5 credits

See the full API reference for all 23 tools and their parameters.

Troubleshooting

MCP Server Not Loading?

  1. Check ~/.cursor/mcp.json is valid JSON
  2. Ensure Node.js 18+ is installed: node --version
  3. Verify the CLI is installed: crawlforge --version
  4. Restart Cursor IDE completely

Authentication Errors?

  1. Run npx crawlforge-setup to reconfigure
  2. Check API key starts with cf_
  3. Verify environment variable is set in your shell
  4. Generate a new API key from the dashboard

Tools Not Appearing in Chat?

  1. Check Cursor's MCP plugin is enabled in settings
  2. Try using the global settings instead of .cursorrules
  3. Clear Cursor's cache and restart
  4. Check Cursor's console for error messages
Still having issues? Contact support at support@crawlforge.dev with your Cursor version, Node.js version, and error messages.
Successfully integrated?
Explore all available tools and start building with AI-powered web scraping.
View All ToolsCredit Optimization Guide