On this page
Most web scraping projects are not unique snowflakes. Track competitor prices. Enrich a list of leads. Audit a site for SEO. Pull training data for a model. It is the same handful of recipes, over and over. A web scraping template is one of those recipes, pre-wired: a ready-to-use JSON config that chains the right CrawlForge tools in the right order, so you can copy it, point it at your targets, and run. CrawlForge ships 24 of them in the templates gallery. This guide is about using them well -- not just copy-paste, but read, adapt, and cost them out before you scale.
Table of Contents
- What Is a Web Scraping Template?
- Templates Gallery vs the scrape_template Tool
- How to Use a Template the Right Way
- 8 Templates Worth Copying First
- The Other 16 Templates
- Customizing or Building Your Own
What Is a Web Scraping Template?
A template is a saved configuration that orchestrates two or three CrawlForge tools into one workflow with a business outcome attached. Instead of wiring search_web then scrape_structured then analyze_content yourself -- and guessing every parameter -- you copy a config that already does it.
Each template in the gallery carries:
- A category -- E-commerce, Research, Data Collection, Monitoring, AI & LLM, Sales, SEO, Content, or Advanced Scraping (nine in total).
- A difficulty -- beginner, intermediate, or advanced.
- The tool chain it runs and a fixed credit cost per run (3 to 19 credits).
- A copy-paste JSON config with sensible default parameters.
You run that config from any MCP client (Claude, Cursor, Windsurf), the crawlforge CLI, or the REST API. Same config, same shape of result.
Templates Gallery vs the scrape_template Tool
This trips people up, so let's be precise. CrawlForge has two different things with "template" in the name:
| Templates gallery | scrape_template tool | |
|---|---|---|
| What it is | A library of multi-tool config chains | A single tool with 10 site schemas |
| Scope | Any workflow (pricing, SEO, research...) | 10 specific sites (Amazon, LinkedIn, GitHub...) |
| Output | Whatever the chained tools return | Structured JSON for that one site |
| Cost | 3-19 credits/run (sum of its tools) | 1 credit/call |
| Use when | You want a whole workflow, ready-made | You want data from one popular site |
If your target is one of the ten supported sites, reach for the tool -- it is covered in depth in Scrape Amazon, LinkedIn & 8 More Sites With One Tool. For everything else -- a full pricing-monitoring or lead-enrichment pipeline -- you want a gallery template. This guide is about the gallery.
How to Use a Template the Right Way
Copy-paste is step one. Using a template well is six.
1. Pick by outcome, not by tool. Start from the job ("monitor competitor prices") and filter the gallery by category and difficulty. New to this? Start with the cheap, two-tool, beginner templates before reaching for a 19-credit research pipeline.
2. Read the config before you run it. Look at the tool order, the parameters, and whether it carries a schedule (hourly, daily, or weekly). Order matters: search_web finds the URLs, then scrape_structured extracts from them.
3. Swap the placeholders. Every config ships with example values -- https://competitor-a.com/pricing, {company_name}, "product name", a default schema. Replace those with your real targets and the exact fields you want back. The schema is your output contract; trim it to what you'll actually use.
4. Do the credit math before you scale. A template's cost is just the sum of its tools' costs. Multiply by frequency: a 7-credit template run hourly is 168 credits/day. Here is the per-tool table the configs draw from:
| Credits | Tools |
|---|---|
| 1 | fetch_url, extract_text, extract_links, extract_metadata, scrape_template |
| 2 | scrape_structured, extract_content, map_site, process_document, localization |
| 3 | analyze_content, track_changes, extract_structured, extract_with_llm |
| 4 | summarize_content, crawl_deep |
| 5 | stealth_mode, scrape_with_actions, batch_scrape, search_web, generate_llms_txt |
| 10 | deep_research |
5. Run it from your stack of choice. In an MCP client, paste the goal and let the agent call the tools. From a terminal or cron job, use the crawlforge CLI. In an app, hit the REST API. All three share one API key and one credit balance.
6. Schedule and monitor. Templates built for monitoring carry a schedule. Pair them with track_changes so you act on diffs, not on every identical run.
8 Templates Worth Copying First
The gallery has 24. These eight cover the highest-demand jobs and span beginner to advanced.
1. Competitor Pricing Monitor
E-commerce · intermediate · 7 credits/run · batch_scrape + scrape_structured
Scrape a set of competitor pricing pages on a schedule and normalize them into a clean plan / price / features structure.
Adapt it: replace urls with your competitors' pricing pages, then tune selectors and schema to the fields you track. Keep schedule at daily for most pricing work. Full walkthrough: build an AI price-monitoring system.
2. Contact Enrichment Pipeline
Sales · intermediate · 7 credits/run · search_web + extract_metadata + extract_links
Turn a bare company name into an enriched record -- official site, social handles, and key links.
Adapt it: drive {company_name} from your CRM export, and widen the extract_links filter to the domains you care about. Run it per row to enrich a whole list. Full walkthrough: build a lead-enrichment engine.
3. SEO Site Audit
SEO · beginner · 6 credits/run · map_site + extract_metadata + analyze_content
Crawl a site, pull every page's metadata, and score content quality -- a fast, repeatable audit.
Adapt it: point url at your domain and raise or lower max_depth to control crawl breadth (and cost). One of the cheapest templates to run regularly. Full walkthrough: automating SEO audits with CrawlForge.
4. AI Training Data Collector
AI & LLM · intermediate · 7 credits/run · batch_scrape + extract_content
Collect and clean web pages at scale into model-ready text -- no navigation, no boilerplate.
Adapt it: feed urls from a sitemap or CSV, and keep remove_navigation on so menus and footers don't pollute your dataset. Full walkthrough: web scraping for AI training data pipelines.
5. Market Intelligence Dashboard
Research · advanced · 19 credits/run · deep_research + batch_scrape + summarize_content
The flagship. Run multi-source research, scrape the key industry sources, and summarize it all into a daily briefing.
Adapt it: change the query to your market and swap urls for your trusted sources. At 19 credits/run it is the most expensive template here -- run it daily, not hourly. Related reading: competitive intelligence with AI agents.
6. Review Sentiment Analyzer
E-commerce · intermediate · 10 credits/run · search_web + scrape_structured + analyze_content
Find reviews across platforms, structure them, and score sentiment and topics.
Adapt it: put your product in the query, raise max_results for more coverage, and keep the schema tight so sentiment scoring stays clean. Related reading: e-commerce product data extraction at scale.
7. Job Listings Scraper
Data Collection · intermediate · 7 credits/run · search_web + scrape_structured
Search job boards and pull listings into a structured feed -- title, company, location, salary, date.
Adapt it: change the query to your role and region, and add schema fields (remote flag, seniority) as needed. See it live on the Job Listings Scraper template page.
8. Website Change Detector
Monitoring · beginner · 6 credits/run · fetch_url + extract_content + analyze_content
Watch a single page and surface when its content shifts -- pricing, terms, or announcements.
Adapt it: set url to the page you care about and dial schedule to your tolerance for staleness -- hourly for fast-moving pages, daily for the rest. Related reading: build a competitive-intelligence agent.
The Other 16 Templates
The remaining gallery entries, grouped by category -- each is a copy-paste config on the templates page:
- Research: News Aggregation Pipeline (11cr), Multi-Source Research Agent (12cr), Academic Paper Research (14cr).
- Data Collection: Real Estate Listings Tracker (7cr), PDF Document Processor (6cr), Government Data Extractor (5cr).
- Monitoring: Compliance Monitoring (9cr), Social Media Monitoring (12cr).
- E-commerce: E-commerce Product Extraction (3cr).
- AI & LLM: Documentation Knowledge Base (10cr).
- Sales: Tech Stack Detector (3cr).
- SEO: Link Building Prospector (7cr).
- Content: Content Migration Tool (7cr), Localization Content Audit (7cr).
- Advanced Scraping: Dynamic SPA Scraper (7cr), Stealth Data Extraction (7cr).
Customizing or Building Your Own
No template is a perfect fit out of the box -- that is the point of step three. When a config gets you 80% there, swap the parameters and schema and you are done. When nothing fits:
- Start from the closest template and rewrite its schema and parameters.
- Compose tools yourself. Use
scrape_structuredwhen you know stable CSS selectors, orextract_with_llmwhen the layout shifts and you want schema-driven, layout-resilient extraction. - Request a template. If you want a recipe we don't ship yet, ask on Discord -- popular requests get added to the gallery.
Ready to run your first template? Start free with 1,000 credits -- no credit card required. Browse the full templates gallery, grab the scrape_template tool for single-site jobs, or watch a template run in production in the e-commerce extraction guide.
