CrawlForge
Integration
Cursor Ide
Cursor IDE3 Minutes

Cursor IDE Integration

Enable AI-powered web scraping in Cursor IDE with CrawlForge MCP. The setup wizard auto-configures Cursor, or add the server to ~/.cursor/mcp.json yourself.

Prerequisites

Cursor IDE

Download from cursor.com

Required
Node.js 18+

Install from nodejs.org

Required
API Key

Get from dashboard

Free

1Install CrawlForge MCP Server

First, install the CrawlForge MCP server package globally.

Bash
Quick setup: Run npx crawlforge-setup to configure your API key automatically.

2Configure Cursor IDE

Choose your preferred configuration method:

Option A: Auto-configure with the wizard
The setup wizard detects Cursor and writes ~/.cursor/mcp.json for you

Run the wizard—if Cursor is installed it adds the CrawlForge server automatically:

Bash

Restart Cursor afterwards to load the server.

Tip: Editing ~/.cursor/mcp.json applies to every project, while a project-level .cursor/mcp.json scopes the server to that repo.

3Set Your API Key

Configure your API key using one of these methods:

Run the setup wizard to configure your API key:

Bash
Security: Never commit your API key to version control. Use environment variables or the config file.

4Test the Integration

Restart Cursor IDE and test the CrawlForge MCP integration.

Restart Cursor IDE

Close and reopen Cursor to load the MCP configuration

Open AI Chat

Press Cmd/Ctrl + L to open the AI chat

Test a Command

Try one of the example prompts below

Example Prompts

"Search for the latest AI news"

"Extract all links from example.com"

"Crawl the documentation site and summarize it"

"Extract product prices from this e-commerce site"

Available Tools

After setup, Cursor will have access to all 23 CrawlForge MCP tools:

fetch_url

Fetch HTML from URLs

1 credits

extract_text

Extract clean text

1 credits

extract_links

Extract all links

1 credits

extract_metadata

Get page metadata

1 credits

scrape_structured

CSS selector extraction

2 credits

batch_scrape

Parallel batch scraping

1/URL

deep_research

AI-powered research

10 credits

search_web

Web search integration

5 credits

See the full API reference for all 23 tools and their parameters.

Troubleshooting

MCP Server Not Loading?

1. Check ~/.cursor/mcp.json is valid JSON

2. Ensure Node.js 18+ is installed: node --version

3. Verify the CLI is installed: crawlforge --version

4. Restart Cursor IDE completely

Authentication Errors?

1. Run npx crawlforge-setup to reconfigure

2. Check API key starts with cf_

3. Verify environment variable is set in your shell

4. Generate a new API key from the dashboard

Tools Not Appearing in Chat?

1. Check Cursor's MCP plugin is enabled in settings

2. Try using the global settings instead of .cursorrules

3. Clear Cursor's cache and restart

4. Check Cursor's console for error messages

Still having issues? Contact support at support@crawlforge.dev with your Cursor version, Node.js version, and error messages.
Successfully integrated?
Explore all available tools and start building with AI-powered web scraping.