MCP Protocol
Complete Model Context Protocol specification for CrawlForge MCP. Learn how to implement custom MCP clients and integrate with any AI framework.
What is MCP?
The Model Context Protocol (MCP) is an open standard created by Anthropic that enables AI assistants to securely access external data sources and tools. It defines a standardized way for AI models to discover, invoke, and receive results from external services.
Built on standard JSON-RPC for universal compatibility
API key authentication and rate limiting built-in
Add custom tools without changing the protocol
Server Configuration
Configure your MCP client to connect to the CrawlForge MCP server.
crawlforge-mcp ships with the global crawlforge-mcp-server install. With no global install, use "command": "npx" and "args": ["-y", "crawlforge-mcp-server"]. If you ran crawlforge-setup, the env block is optional—the key is read from ~/.crawlforge/config.json.Tool Discovery
MCP clients discover available tools by calling the tools/list method.
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
name | string | Required | - | Tool identifier (e.g., "fetch_url") Example: fetch_url |
description | string | Required | - | Human-readable tool description Example: Fetch and parse HTML content from a URL |
inputSchema | object | Required | - | JSON Schema defining tool parameters Example: { "type": "object", "properties": {...} } |
Tool Invocation
Invoke tools using the tools/call method with tool name and arguments.
await in JavaScript or async/await in Python to wait for results.Response Format
Tool responses follow the standard MCP format with content, metadata, and error handling.
MCP Message Structure
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
method | string | Required | - | MCP method name (e.g., "tools/call") Example: tools/call |
params | object | Required | - | Method-specific parameters Example: { "name": "fetch_url", "arguments": {...} } |
id | string | Required | - | Unique request identifier for correlation Example: req_12345 |
Error Codes
| Code | Name | Description |
|---|---|---|
-32700 | Parse Error | Invalid JSON |
-32600 | Invalid Request | Malformed request object |
-32601 | Method Not Found | Unknown MCP method |
-32602 | Invalid Params | Missing or invalid parameters |
-32603 | Internal Error | Server error during execution |
-32001 | Insufficient Credits | Not enough credits to execute tool |
Best Practices
Use Unique Request IDs
Generate unique IDs for each request to correlate responses
Handle Errors Gracefully
Check for both JSON-RPC errors and tool-specific errors
Implement Timeouts
Set reasonable timeouts for tool calls (10-30 seconds)
Monitor Credit Usage
Track _meta.credits_remaining in responses
Troubleshooting
For detailed troubleshooting, see the Claude Desktop guide or contact support.
MCP Resources & Prompts
v3.6+Beyond tools, CrawlForge exposes MCP Resources (URI-addressable data) and Prompts (pre-built workflows). Both are available when using the server with any MCP-compatible client.
Resources
Resources live under the crawlforge:// URI scheme and address long-lived artifacts produced by earlier tool calls—research sessions, batch jobs, crawl sitemaps, screenshots, and page snapshots. The five registered URI templates are:
crawlforge://research/{sessionId}crawlforge://job/{jobId}crawlforge://crawl/{sessionId}/sitemapcrawlforge://screenshot/{actionId}crawlforge://snapshot/{urlHash}/{timestamp}
Resources are read-only, TTL-bound, and do not consume credits.
Prompts
Prompts are pre-built workflows the model can discover and invoke. CrawlForge ships five out of the box:
| Prompt | Purpose |
|---|---|
competitive-analysis | Side-by-side scrape and compare of competitor sites |
monitor-changes | Set up a change-tracking workflow for a target URL |
rag-ingest | Crawl a domain and emit clean, chunked content for RAG pipelines |
site-audit | Full SEO + accessibility + content audit of a site |
research-deep-dive | Multi-stage deep research with citations |
Example: how Claude Desktop discovers and invokes a prompt.