CLI

CrawlForge CLI

The crawlforge command ships with the crawlforge-mcp-server package, so a single global install gives you both the MCP server and a standalone CLI for one-off commands and scripting—no MCP client required. Set an API key and you're scraping in seconds.

Install & Authenticate

Install crawlforge-mcp-server globally—the crawlforge command is included. The fastest way to authenticate is the setup wizard, which stores your key in ~/.crawlforge/config.json for the CLI to read automatically. Get a key from your API Keys dashboard.

Bash

npm install -g crawlforge-mcp-server
npx crawlforge-setup        # guided API-key setup (recommended)
crawlforge --help

Other ways to authenticate: export CRAWLFORGE_API_KEY=cf_live_... in your ~/.zshrc / ~/.bashrc, or pass --api-key cf_live_... on any command. Priority: --api-key > environment variable > stored config.

Commands

The 15 tool commands below cover all 23 CrawlForge tools (some commands expose multiple tools via flags). Run crawlforge <command> --help to see flags and options for any command.

Command	Purpose
`scrape`	Fetch a URL (fetch_url); add --extract for cleaned text/markdown
`search`	Web search (--provider crawlforge or searxng)
`crawl`	Deep multi-page crawl with depth/page/concurrency controls
`map`	Generate a sitemap (--format json or xml)
`extract`	Structured extraction via --schema (CSS) or --prompt (LLM)
`track`	Track content changes (--selector, --threshold)
`analyze`	Sentiment, entities, and readability analysis
`research`	Multi-source deep research with source verification
`stealth`	Anti-bot browser mode (--engine playwright or camoufox)
`batch`	Scrape many URLs from a newline-delimited file
`actions`	Browser automation from a JSON action script (--script)
`localize`	Locale/geo-aware fetch (--locale, --country, --currency)
`llmstxt`	Generate llms.txt and llms-full.txt (--include-full)
`template`	Run a pre-built site template (--list shows all 10)
`monitor`	Scheduled change monitoring (--interval, --webhook)

Skills installers

Two helper commands copy CrawlForge skill files into your AI coding tool so it knows how to use every CrawlForge command.

Command	Purpose
`install-skills`	Install CrawlForge skills into Claude Code, Cursor, or VS Code (--target, --dry-run)
`uninstall-skills`	Remove installed CrawlForge skill files (--target)

Running the MCP server by hand: the same binary also starts the server over stdio with crawlforge mcp (alias crawlforge serve). MCP clients normally launch it for you—see the Claude Desktop and Cursor guides.

Examples

1. Scrape a page as clean markdown

Bash

crawlforge scrape https://example.com --extract --format markdown

Without --extract the command runs fetch_url (raw headers + body); with it, extract_content returns cleaned text or markdown. Add --pretty to print indented JSON.

2. Run deep research with multiple sources

Bash

crawlforge research "AI agent frameworks 2026" --depth deep --max-urls 10

Analyzes up to 10 sources, verifies citations, and emits a structured research report. Depth is basic, standard, or deep. Costs 10 credits.

3. Use a pre-built site template

Bash

crawlforge template github-repo https://github.com/anthropics/claude-code --pretty
crawlforge template --list   # show all 10 templates

Templates extract a known schema for popular sites (GitHub repos, Amazon products, YouTube videos, and more) without writing selectors yourself. The target URL is a positional argument.

4. Pipe JSON into other tools

Bash

crawlforge search "nodejs MCP server" --json | jq '.results[].url'

--json emits single-line JSON for shell pipelines and CI; --quiet returns only an exit code.

Global Flags

These flags work with any command and shape how output is delivered.

Flag	Description
`--json`	Output compact JSON (pipe-friendly)
`--pretty`	Output pretty-printed JSON
`--quiet`	Suppress all stdout output (exit code only)
`--api-key <key>`	Override the CRAWLFORGE_API_KEY env var
`--timeout <ms>`	Global request timeout in ms (default: 30000)

Next Steps

API Reference

All 23 tools, parameters, and credit costs.

MCP Protocol

Use the same backend from any MCP-compatible client.

Getting Started

5-minute quickstart for new accounts.

Ready to scrape from the terminal?

Get Free Credits Manage API Keys

Command

Purpose

scrape

Fetch a URL (fetch_url); add --extract for cleaned text/markdown

search

Web search (--provider crawlforge or searxng)

crawl

Deep multi-page crawl with depth/page/concurrency controls

map

Generate a sitemap (--format json or xml)

extract

Structured extraction via --schema (CSS) or --prompt (LLM)

track

Track content changes (--selector, --threshold)

analyze

Sentiment, entities, and readability analysis

research

Multi-source deep research with source verification

stealth

Anti-bot browser mode (--engine playwright or camoufox)

batch

Scrape many URLs from a newline-delimited file

actions

Browser automation from a JSON action script (--script)

localize

Locale/geo-aware fetch (--locale, --country, --currency)

llmstxt

Generate llms.txt and llms-full.txt (--include-full)

template

Run a pre-built site template (--list shows all 10)

monitor

Scheduled change monitoring (--interval, --webhook)

Command

Purpose

install-skills

Install CrawlForge skills into Claude Code, Cursor, or VS Code (--target, --dry-run)

uninstall-skills

Remove installed CrawlForge skill files (--target)

Flag

Description

--json

Output compact JSON (pipe-friendly)

--pretty

Output pretty-printed JSON

--quiet

Suppress all stdout output (exit code only)

--api-key <key>

Override the CRAWLFORGE_API_KEY env var

--timeout <ms>

Global request timeout in ms (default: 30000)