How to Use CrawlForge with Windsurf IDE

Windsurf is a next-generation AI code editor built by Codeium, designed around the concept of "Flows" -- multi-step AI actions that understand your entire codebase (see the official Windsurf getting started docs). CrawlForge MCP extends Windsurf's capabilities beyond your local files and into the live web by plugging into Cascade's MCP support, letting the AI fetch documentation, scrape reference implementations, research APIs, and extract data -- all from within your editor.

This guide shows you how to configure CrawlForge as an MCP server in Windsurf and use it for real development tasks.

Table of Contents

• Prerequisites
• Why CrawlForge in Windsurf
• Step 1: Install CrawlForge MCP Server
• Step 2: Configure Windsurf MCP Settings
• Step 3: Verify the Connection
• Practical Use Cases
• Credit Cost Breakdown
• Troubleshooting
• Next Steps

Prerequisites

• Windsurf IDE installed (windsurf.com)
• Node.js 18+ installed
• A CrawlForge API key -- sign up free for 1,000 credits

Why CrawlForge in Windsurf

Windsurf's Cascade AI agent can already read your codebase and make multi-file edits. But it cannot access external resources:

• It cannot read the latest API documentation for a library you are using
• It cannot check how a competitor implements a specific feature
• It cannot pull real-world HTML to test your parser against
• It cannot research best practices across multiple sources

CrawlForge solves this. Once configured, Windsurf's AI agent gains access to all 31 CrawlForge tools -- and selects the right one based on what you ask.

Step 1: Install CrawlForge MCP Server

Open your terminal and install the CrawlForge MCP server globally:

npm install -g crawlforge-mcp-server

Verify the installation:

crawlforge-mcp-server --version

Step 2: Configure Windsurf MCP Settings

Windsurf reads MCP server configurations from a JSON file. Open your Windsurf MCP config:

macOS / Linux:

# Open or create the MCP config
code ~/.windsurf/mcp_config.json

Windows:

code %APPDATA%\Windsurf\mcp_config.json

Add the CrawlForge server to the mcpServers object:

// ~/.windsurf/mcp_config.json
{
  "mcpServers": {
    "crawlforge": {
      "command": "crawlforge-mcp-server",
      "args": [],
      "env": {
        "CRAWLFORGE_API_KEY": "cf_live_your_api_key_here"
      }
    }
  }
}

Replace cf_live_your_api_key_here with your actual API key from the CrawlForge dashboard.

Alternative: Project-Level Configuration

For team projects, create a .windsurf/mcp_config.json in your project root. This keeps the configuration version-controlled (but remember to add your API key via environment variables rather than hardcoding):

// .windsurf/mcp_config.json
{
  "mcpServers": {
    "crawlforge": {
      "command": "crawlforge-mcp-server",
      "args": [],
      "env": {
        "CRAWLFORGE_API_KEY": "${CRAWLFORGE_API_KEY}"
      }
    }
  }
}

Step 3: Verify the Connection

Restart Windsurf to pick up the new MCP config. Then open the Cascade panel (Cmd+L / Ctrl+L) and type:

Windsurf should automatically invoke the CrawlForge fetch_url or extract_content tool and return a summary. If you see the results, the connection is working.

You can verify which tools are available by asking:

Cascade will list all 31 tools with their descriptions.

Practical Use Cases

Fetch Documentation While Coding

Ask Cascade to read the latest docs for any library:

CrawlForge's extract_content tool fetches the documentation page, and Cascade applies it directly to your code. Cost: 2 credits.

Research API Design Patterns

Cascade uses search_web to find articles, then extract_content to read them. Cost: ~11 credits (5 for search + 2 per page for 3 pages).

Generate Test Data from Real Websites

CrawlForge's scrape_structured extracts the data, and Cascade generates typed interfaces from the response. Cost: 2 credits.

Monitor a Competitor's Changelog

Uses extract_content to read the page and filters by date. Cost: 2 credits.

Credit Cost Breakdown

A typical development session uses 10-30 credits. The Free tier (1,000 one-time credits) covers most initial developer needs.

Troubleshooting

"MCP server not found": Verify the global install path is in your system PATH. Run which crawlforge-mcp-server to check. If it returns nothing, reinstall with npm install -g crawlforge-mcp-server.

"Authentication failed": Double-check your API key in mcp_config.json. Keys start with cf_live_ for production or cf_test_ for development.

Tools not appearing: Restart Windsurf after editing the MCP config. The server loads on IDE startup.

Slow responses: CrawlForge tools that render JavaScript (scrape_with_actions, stealth_mode) take 3-8 seconds. Static tools like fetch_url and extract_text respond in under 1 second.

Next Steps

With CrawlForge configured in Windsurf, explore these workflows:

• Use stealth mode to access sites with anti-bot protection
• Build a research agent workflow for market analysis
• Explore the full CrawlForge Quick Start for MCP configuration options
• Check all 31 tools and their capabilities

Give your AI editor access to the live web. Start free with 1,000 credits -- no credit card required.