MCP Protocol
Complete Model Context Protocol specification for CrawlForge MCP. Learn how to implement custom MCP clients and integrate with any AI framework.
What is MCP?
The Model Context Protocol (MCP) is an open standard created by Anthropic that enables AI assistants to securely access external data sources and tools. It defines a standardized way for AI models to discover, invoke, and receive results from external services.
Server Configuration
Configure your MCP client to connect to the CrawlForge MCP server.
crawlforge-mcp ships with the global crawlforge-mcp-server install. With no global install, use "command": "npx" and "args": ["-y", "crawlforge-mcp-server"]. If you ran crawlforge-setup, the env block is optional—the key is read from ~/.crawlforge/config.json.Tool Discovery
MCP clients discover available tools by calling the tools/list method.
Tool Manifest Fields
| Name | Type | Required | Description |
|---|---|---|---|
name | string | Required | Tool identifier (e.g., "fetch_url") · Example: fetch_url |
description | string | Required | Human-readable tool description · Example: Fetch and parse HTML content from a URL |
inputSchema | object | Required | JSON Schema defining tool parameters · Example: { "type": "object", "properties": {...} } |
Tool Invocation
Invoke tools using the tools/call method with tool name and arguments.
await in JavaScript or async/await in Python to wait for results.Response Format
Tool responses follow the standard MCP format with content, metadata, and error handling.
MCP Message Structure
Standard Message Fields
| Name | Type | Required | Description |
|---|---|---|---|
method | string | Required | MCP method name (e.g., "tools/call") · Example: tools/call |
params | object | Required | Method-specific parameters · Example: { "name": "fetch_url", "arguments": {...} } |
id | string | Required | Unique request identifier for correlation · Example: req_12345 |
Error Codes
| Code | Name | Description |
|---|---|---|
-32700 | Parse Error | Invalid JSON |
-32600 | Invalid Request | Malformed request object |
-32601 | Method Not Found | Unknown MCP method |
-32602 | Invalid Params | Missing or invalid parameters |
-32603 | Internal Error | Server error during execution |
-32001 | Insufficient Credits | Not enough credits to execute tool |
Best Practices
- Use Unique Request IDs — Generate unique IDs for each request to correlate responses
- Handle Errors Gracefully — Check for both JSON-RPC errors and tool-specific errors
- Implement Timeouts — Set reasonable timeouts for tool calls (10-30 seconds)
- Monitor Credit Usage — Track
_meta.credits_remainingin responses
Troubleshooting
For detailed troubleshooting, see the Claude Desktop guide or contact support.
MCP Resources & Prompts
Beyond tools, CrawlForge exposes MCP Resources (URI-addressable data) and Prompts (pre-built workflows). Both are available when using the server with any MCP-compatible client.
Resources
Resources live under the crawlforge:// URI scheme and address long-lived artifacts produced by earlier tool calls—research sessions, batch jobs, crawl sitemaps, screenshots, and page snapshots. The five registered URI templates are:
crawlforge://research/{sessionId}crawlforge://job/{jobId}crawlforge://crawl/{sessionId}/sitemapcrawlforge://screenshot/{actionId}crawlforge://snapshot/{urlHash}/{timestamp}
Resources are read-only, TTL-bound, and do not consume credits.
Prompts
Prompts are pre-built workflows the model can discover and invoke. CrawlForge ships five out of the box:
| Prompt | Purpose |
|---|---|
competitive-analysis | Side-by-side scrape and compare of competitor sites |
monitor-changes | Set up a change-tracking workflow for a target URL |
rag-ingest | Crawl a domain and emit clean, chunked content for RAG pipelines |
site-audit | Full SEO + accessibility + content audit of a site |
research-deep-dive | Multi-stage deep research with citations |
Example: how Claude Desktop discovers and invokes a prompt.