On this page
Smithery is the largest MCP server registry, hosting over 100,000 tools and integrations for AI agents. CrawlForge is listed on Smithery, which means you can discover, install, and manage your web scraping MCP server without touching a config file manually.
This guide walks you through the complete Smithery workflow -- from searching the registry to calling CrawlForge tools in production.
Table of Contents
- What Is Smithery?
- Prerequisites
- Step 1: Install the Smithery CLI
- Step 2: Find CrawlForge on Smithery
- Step 3: Connect to CrawlForge
- Step 4: Browse Available Tools
- Step 5: Call Tools from the CLI
- Step 6: Use Namespaces for Environments
- Credit Cost Reference
- When to Use Smithery vs Direct Installation
- Next Steps
What Is Smithery?
Smithery is a marketplace and registry for MCP servers. Think of it as npm for AI agent tools. Instead of manually configuring each MCP server in your client's settings file, Smithery provides a CLI and web interface to search, install, and manage MCP connections.
Smithery handles OAuth flows, credential storage, token refresh, and session lifecycle automatically. For teams running multiple MCP servers across dev and production environments, this eliminates significant configuration overhead.
Prerequisites
- Node.js 18+ installed
- A CrawlForge account with an API key (free tier includes 1,000 credits)
- Terminal access (macOS, Linux, or Windows with WSL)
Step 1: Install the Smithery CLI
Authenticate with your Smithery account:
This opens your browser for confirmation. Once authenticated, the CLI stores your credentials locally.
Step 2: Find CrawlForge on Smithery
Search the Smithery registry for CrawlForge:
You will see CrawlForge listed with its description, tool count (18 tools), and connection URL. You can also browse the Smithery web catalog to read reviews and documentation before installing.
Step 3: Connect to CrawlForge
Add CrawlForge as a managed MCP connection:
Verify the connection status:
You should see crawlforge with status connected. If it shows auth_required, follow the authorization URL provided in the output.
Step 4: Browse Available Tools
List all 18 CrawlForge tools through Smithery:
The tool get command shows the full JSON schema for each tool, including required parameters, optional fields, and response format. This is useful for building automation scripts.
Step 5: Call Tools from the CLI
Execute CrawlForge tools directly from your terminal:
When piped, Smithery outputs JSONL, which integrates cleanly with jq and other CLI tools:
Step 6: Use Namespaces for Environments
Smithery namespaces let you isolate CrawlForge connections across dev, staging, and production:
This prevents accidental credit consumption on your production account during development.
Credit Cost Reference
| Credits | Tools |
|---|---|
| 1 | fetch_url, extract_text, extract_links, extract_metadata |
| 2 | scrape_structured, extract_content, summarize_content, generate_llms_txt |
| 3 | map_site, process_document, analyze_content, localization |
| 5 | search_web, crawl_deep, batch_scrape, scrape_with_actions, stealth_mode |
| 10 | deep_research |
The free tier includes 1,000 credits. A typical research workflow -- search (5) + extract 3 pages (6) + summarize (2) -- costs 13 credits total.
When to Use Smithery vs Direct Installation
| Approach | Best For | Setup Time |
|---|---|---|
| Smithery | Teams managing multiple MCP servers, CI/CD pipelines, multi-environment deployments | 2 minutes |
| Direct npm install | Solo developers, Claude Code / Cursor users who want zero dependencies | 1 minute |
| API-only | Non-MCP clients, custom integrations, server-to-server workflows | 5 minutes |
If you are already using Smithery for other MCP servers (GitHub, Slack, Notion), adding CrawlForge through Smithery keeps everything in one management layer. If CrawlForge is your only MCP server, direct installation is simpler.
Next Steps
- CrawlForge Quick Start -- direct installation guide for Claude Code
- 18 Web Scraping Tools Overview -- what each tool does and when to use it
- View Pricing -- credit packs and subscription plans
- Smithery Documentation -- full Smithery CLI reference
Ready to start scraping? Sign up free and get 1,000 credits -- no credit card required. Then install via Smithery or npm and start extracting web data in under 60 seconds.
