Every page
Every public page on CrawlForge in one place: documentation, all 31 MCP tools, the blog, use cases, competitor comparisons, templates, integrations, and the web scraping glossary.
Main
6 pages
Documentation
13 pages
MCP tools
29 pages
- fetch url
- scrape
- extract text
- extract links
- extract metadata
- scrape template
- scrape structured
- extract content
- map site
- process document
- localization
- track changes
- analyze content
- extract structured
- extract with llm
- summarize content
- crawl deep
- stealth mode
- scrape with actions
- browser session
- batch scrape
- get batch results
- read result
- search web
- serp rank
- generate llms txt
- agent
- deep research
- list ollama models
Blog
97 pages
- Blog home
- Product Updates
- Tutorials
- Use Cases
- Web Scraping
- AI Engineering
- Lead Enrichment API Pay As You Go: Spend Only on Good Leads
- Market Research API for Agencies: One Setup, Every Client
- Lead Enrichment API Without a Sales Call: First Lead Today
- CrawlForge MCP v6.7–v6.12: Scrape Cloudflare-Protected Sites
- Market Research API for Consultants: Zero to Sourced
- Market Research API for RevOps: One Question, Every Vendor
- CrawlForge MCP v6.6.0: Browser Sessions That Stay Open
- Competitor Monitoring API for Consultants: One Per Client
- Competitor Monitoring API Pay As You Go: What a Check Costs
- Competitor Monitoring API for RevOps: No Scraper to Run
- Shipping an MCP Server Inside Your Product: Five Decisions
- MCP Server for Due Diligence: What an Agent Can Verify
- Reddit Scraper for Claude Code: What Claude Actually Calls
- Web Search MCP Server With Local LLM: Which Model Actually Runs
- Web Search MCP Server: Self-Hosted With SearXNG
- CrawlForge MCP v5.2: Every Change Across 28 Tools, Tested Against Live Sites
- CrawlForge MCP v5.2.0: Shopify Product Data Without Parsing HTML
- Web Scraping API Credits That Don't Expire
- AI Crawler List 2026: Every Bot + Should You Block Them?
- CrawlForge MCP v5.1.0: Search Reddit Without the API
- Reddit API Alternatives That Still Work in 2026
- How to Scrape Reddit Without the API (2026)
- Reddit Data for AI Agents: The MCP Route
- Answer 3 Questions, Get 1,000 Free CrawlForge Credits
- The Web Scraping Keyword Gap: A 2026 SERP Study
- How to Run a Keyword Gap Analysis with an MCP Server
- Agent Scraper: What It Is and How to Build One
- CrawlForge v5.0.4: 34 Fixes From Live-Testing 27 MCP Tools
- Is Web Scraping Legal in 2026? The EDPB's New AI Rules
- CrawlForge v5.0.0: Security, Correctness, MCP Spec
- SSRF in MCP Servers: Why Scrapers Leak Cloud Secrets
- How to Actually Use CrawlForge MCP in Claude Code
- CrawlForge v4.8.0: Claude Skills That Auto-Activate
- CrawlForge vs Firecrawl vs Tavily vs Exa: Best Web Data API for AI Agents (2026)
- How to Build a Web-Scraping MCP Server in TypeScript (2026)
- How to Give ChatGPT Web Scraping with MCP Connectors (2026)
- Web Scraping with Claude: The Complete Guide (2026)
- Best MCP Servers for Web Scraping in 2026 (Top 8 Ranked)
- Best Web Scraping Tools for AI Agents in 2026
- How to Use Web Scraping Templates the Right Way
- Scrape Amazon, LinkedIn & 8 More Sites With One Tool
- Extract Web Data With Local LLMs (Ollama + CrawlForge)
- Web Scraping From the CLI: The CrawlForge CLI Guide
- CrawlForge v4.2.2: New CLI + 3 Tools for Local AI Scraping
- Web Scraping: Python vs MCP in 2026
- MCP Protocol Explained: A Developer Guide for 2026
- Best Web Scraping Tools in 2026: The Definitive Guide
- How to Use CrawlForge with LangGraph Agents
- How to Use CrawlForge with Make and Zapier
- How to Use CrawlForge with Dify Workflows
- How to Use CrawlForge with Mastra AI Agents
- How to Use CrawlForge with Cursor Rules
- How to Use CrawlForge with Smithery: MCP Registry Guide
- E-commerce Product Data Extraction at Scale
- Build a Research Agent with CrawlForge Deep Research
- How to Use CrawlForge with Anthropic Claude API
- Build a Lead Enrichment Engine with CrawlForge
- How to Build a RAG Pipeline with Web Data
- Web Scraping by Industry: 2026 Playbook
- How to Scrape Websites with Claude Code (2026 Guide)
- How to Scrape Websites in Cursor IDE with CrawlForge MCP
- How to Scrape Websites in Zed AI with CrawlForge MCP
- How to Scrape Websites with GitHub Copilot Agents in VS Code
- LlamaIndex Web Scraping Guide with CrawlForge MCP
- How to Use CrawlForge with OpenAI Agents SDK
- Content Migration Automation with CrawlForge
- How to Use CrawlForge with Cline (VS Code)
- Web Scraping for AI Training Data Pipelines
- How to Use CrawlForge with Windsurf IDE
- Real-Time Competitive Intelligence with AI Agents
- How to Use CrawlForge with Vercel AI SDK
- Automate SEO Audits with CrawlForge MCP
- How to Use CrawlForge with n8n: Workflow Automation Guide
- Build an AI-Powered Price Monitoring System
- CrawlForge MCP Is Now Live: Free Web Scraping for AI Agents
- CrawlForge MCP Quick Start: Zero to Web Scraping in 60 Seconds
- The Complete Guide to MCP Web Scraping: Everything Developers Need to Know
- From 10 Hours to 10 Minutes: Automating Research with CrawlForge Deep Research
- Stealth Mode Scraping: How CrawlForge Bypasses Anti-Bot Detection
- How to Build a Competitive Intelligence Agent with Claude + CrawlForge
- CrawlForge vs Firecrawl: Which MCP Web Scraper Is Right for You?
- How to Install CrawlForge MCP and Use It in Claude Code: A Beginner's Guide
- CrawlForge vs Apify vs ScrapingBee: 2026 Web Scraping Comparison
- 5 Ways to Use CrawlForge with LangChain: AI Web Scraping Tutorial
- 31 Web Scraping Tools in One MCP Server: The Complete CrawlForge Guide
- How to Add Web Scraping to Claude Desktop in 5 Minutes
- Introducing deep_research: AI-Powered Multi-Source Analysis
- Building an AI Research Assistant with Claude and MCP
- Web Scraping for AI Training Data: A Complete 2026 Guide
- MCP vs REST: Why We Built a Native MCP Scraping Server
- Welcome to CrawlForge: Enterprise Web Scraping for AI
Use cases
16 pages
- Use cases home
- AI Agent Data Pipelines
- E-commerce Price Monitoring
- SEO Analysis & Auditing
- Competitive Intelligence
- Content Migration
- Lead Enrichment
- AI Training Data Collection
- Real-Time Research Agents
- RAG Knowledge Base Ingestion
- PDF & Document Data Extraction
- Job Listing Aggregation
- Keyword Rank Tracking
- llms.txt & AI Search Visibility
- Review & Sentiment Monitoring
- Geo Pricing & Localization Testing
Alternatives & comparisons
21 pages
- Alternatives home
- Best Firecrawl Alternatives 2026
- CrawlForge vs ScrapingBee
- CrawlForge vs Scrapy
- CrawlForge vs Playwright
- CrawlForge vs Puppeteer
- Best Firecrawl Alternative — CrawlForge
- Best Apify Alternative — CrawlForge
- Web Scraping API Comparison 2026
- CrawlForge vs Scrapfly
- Best Scrapfly Alternative — CrawlForge
- Best Zyte Alternative — CrawlForge
- CrawlForge vs ScrapeGraphAI
- Best Browse AI Alternative for Developers — CrawlForge
- Best Crawl4AI Alternative (Hosted) — CrawlForge
- Best Bright Data Alternative for Developers — CrawlForge
- CrawlForge vs Octoparse
- CrawlForge vs Jina Reader
- CrawlForge vs Browserbase
- CrawlForge vs Tavily
- CrawlForge vs Exa
Templates
25 pages
- Templates home
- Competitor Pricing Monitor
- News Aggregation Pipeline
- E-commerce Product Extraction
- Multi-Source Research Agent
- Job Listings Scraper
- Compliance Monitoring
- Documentation Knowledge Base
- Contact Enrichment Pipeline
- SEO Site Audit
- Social Media Monitoring
- Real Estate Listings Tracker
- Content Migration Tool
- Academic Paper Research
- Website Change Detector
- AI Training Data Collector
- Link Building Prospector
- Dynamic SPA Scraper
- Localization Content Audit
- PDF Document Processor
- Stealth Data Extraction
- Market Intelligence Dashboard
- Review Sentiment Analyzer
- Tech Stack Detector
- Government Data Extractor
Integrations
12 pages
Reference
71 pages
- Web scraping glossary
- Web Scraping Terms
- AI and MCP Terms
- Data and API Terms
- Web Data Industry Terms
- Web Scraping
- Web Crawler
- DOM Parsing
- CSS Selector
- XPath
- Headless Browser
- Proxy Rotation
- Rate Limiting
- CAPTCHA Solving
- Robots.txt
- Sitemap
- User Agent
- HTTP Headers
- Pagination
- Dynamic Content
- Model Context Protocol (MCP)
- MCP Server
- MCP Client
- AI Agent
- Large Language Model (LLM)
- Retrieval-Augmented Generation (RAG)
- Embeddings
- Vector Database
- Structured Output
- Prompt Engineering
- Fine-Tuning
- Token
- Context Window
- Function Calling
- Tool Use
- JSON-LD
- Structured Data
- Schema Markup
- API Endpoint
- Webhook
- REST API
- GraphQL
- JSON
- Markdown
- HTML Parsing
- Competitive Intelligence
- Price Monitoring
- Lead Enrichment
- Content Migration
- SEO Audit
- Data Pipeline
- ETL (Extract, Transform, Load)
- Data Quality
- Data Governance
- Web Data
- Web Scraping API
- Bot Detection
- Residential Proxy
- Geo-Targeted Scraping
- MCP Tool
- AI Search API
- Browser Agent
- LLM-Ready Data
- SERP API
- Embedded Page State
- Schema-Based Extraction
- Batch Scraping
- Managed Scraping Service
- Credit-Based Pricing
- No-Code Web Scraper
- Real-Time Web Data
Legal
5 pages