How to Use CrawlForge with Cline (VS Code)

Cline is an open-source AI coding assistant for VS Code that supports the Model Context Protocol. It turns VS Code into an agentic development environment where the AI can read files, run commands, and -- with CrawlForge MCP -- scrape the web. Unlike closed-source alternatives, Cline lets you choose your own LLM provider (Anthropic, OpenAI, Google, or local models via Ollama) and gives you full control over what the AI can access.

This tutorial shows you how to connect CrawlForge to Cline and use 18 web scraping tools directly in your VS Code workflow.

Table of Contents

• Prerequisites
• How Cline Uses MCP Servers
• Step 1: Install CrawlForge MCP Server
• Step 2: Add CrawlForge to Cline's MCP Config
• Step 3: Test the Integration
• Real-World Development Workflows
• Credit Cost Breakdown
• Tips for Efficient Credit Usage
• Troubleshooting
• Next Steps

Prerequisites

• VS Code with the Cline extension installed
• Node.js 18+
• An LLM API key (Anthropic, OpenAI, or other supported providers)
• A CrawlForge API key -- sign up free for 1,000 credits

How Cline Uses MCP Servers

Cline discovers MCP servers through a JSON configuration file. Once configured, the AI can see all available tools and invoke them during conversations. The key advantage of Cline's MCP implementation:

• Approval workflow: Cline shows you exactly which tool it wants to call and with what parameters before executing. You approve or deny each action.
• Multi-provider support: Use CrawlForge tools with any LLM, not just Claude.
• Open source: You can inspect exactly how tool calls are made.

Step 1: Install CrawlForge MCP Server

Verify it installed correctly:

Step 2: Add CrawlForge to Cline's MCP Config

Open VS Code and access Cline's MCP settings:

1. Open the Command Palette (Cmd+Shift+P / Ctrl+Shift+P)
2. Type "Cline: Open MCP Settings"
3. This opens the cline_mcp_settings.json file

Add CrawlForge to the server list:

Replace the API key with your actual key from the CrawlForge dashboard.

Security Note

Cline stores MCP configs locally in your VS Code settings directory. Your API key is not committed to version control. For shared team setups, use environment variable references:

Then set the environment variable in your shell profile (~/.zshrc or ~/.bashrc):

Step 3: Test the Integration

Open the Cline panel in VS Code (click the Cline icon in the sidebar) and send this message:

Use CrawlForge to fetch the homepage of https://crawlforge.dev and summarize what the product does.

Cline will:

1. Show you the proposed tool call (extract_content with the URL)
2. Wait for your approval
3. Execute the tool and display the results
4. Generate a summary based on the extracted content

If this works, you are connected. All 18 CrawlForge tools are now available.

Real-World Development Workflows

Fetch Library Documentation

When you encounter an unfamiliar API, ask Cline to look it up:

I'm trying to use the Stripe API to create a subscription with a trial period. Fetch the relevant Stripe docs and show me a TypeScript example.

Cline uses CrawlForge's search_web (5 credits) to find the right docs page, then extract_content (2 credits) to read it, and generates a code example tailored to your project. Total: 7 credits.

Scrape and Type External Data

Scrape the API response structure from https://jsonplaceholder.typicode.com/posts/1 and generate a TypeScript interface for it.

CrawlForge's fetch_url (1 credit) retrieves the JSON, and Cline generates the interface:

Investigate a Bug from Stack Overflow

I'm getting "TypeError: Cannot read properties of undefined (reading 'map')" in my React component. Search for solutions and apply the best fix to my code.

Cline searches with search_web (5 credits), reads the top Stack Overflow answer with extract_content (2 credits), and applies the fix directly to your file. Total: 7 credits.

Research Before Implementing

Before I implement WebSocket support, research the current best practices for WebSocket authentication in Node.js. Check at least 3 sources.

Cline chains search_web + multiple extract_content calls, then summarizes the consensus. Total: ~11 credits.

Credit Cost Breakdown

The Free tier gives you 1,000 credits/month -- enough for ~140 single-page fetches or ~90 search-and-read workflows.

Tips for Efficient Credit Usage

1. Be specific in your prompts. "Fetch the Stripe subscription docs" triggers one extract_content call (2 credits). "Research Stripe subscriptions" might trigger search_web + multiple extracts (11+ credits).

2. Provide URLs when you have them. "Read https://docs.stripe.com/billing/subscriptions" costs 2 credits. "Find the Stripe subscription docs" costs 7 credits (search + extract).

3. Use fetch_url for JSON APIs. At 1 credit, it is the cheapest option for structured data.

4. Batch your research. Ask Cline to research multiple topics in one conversation rather than starting new ones.

Troubleshooting

Cline does not show CrawlForge tools: Restart VS Code after editing the MCP settings. If tools still do not appear, check for JSON syntax errors in cline_mcp_settings.json.

"Server disconnected" error: Ensure crawlforge-mcp-server is in your PATH. Run which crawlforge-mcp-server in your terminal. If it returns nothing, run npm install -g crawlforge-mcp-server again.

Tool calls fail with 401: Your API key is invalid or expired. Generate a new key at crawlforge.dev/dashboard.

Slow tool responses: Browser-based tools (scrape_with_actions, stealth_mode) take 3-8 seconds. Static tools (fetch_url, extract_text) respond in under 1 second.

Next Steps

With CrawlForge connected to Cline, you have a web-powered coding assistant. Explore further:

• CrawlForge Quick Start for detailed setup options
• All 18 tools explained with usage examples
• Stealth mode for sites behind Cloudflare or CAPTCHAs
• Deep research automation for multi-source analysis

---

Turn VS Code into a web-scraping powerhouse. Start free with 1,000 credits -- no credit card required.