The Problem
Competitor websites actively block scrapers with CAPTCHAs, rate limiting, and bot detection. Getting reliable, structured data from protected sites requires expensive proxy infrastructure and constant maintenance.
Use Cases
Gather competitor data discreetly using stealth browsing and structured extraction. Watch pricing, messaging, and launches without tripping bot defenses.
Estimated cost: ~7 credits per protected page
01Quick Answer
Use CrawlForge stealth_mode (5 credits) to render in a real browser with randomized fingerprints past rate limits and bot detection, then scrape_structured (2 credits) to map selectors to typed JSON -- product names, prices, specs. You monitor protected competitor pages on a schedule without running your own browser infrastructure.
02The brief
The Solution
CrawlForge stealth_mode renders in a real browser with fingerprint rotation to get past common anti-bot systems. Pair it with scrape_structured for clean, schema-based data extraction.
03In code
Code Example
competitive-intelligence.js
// Gather competitor data through anti-bot protectionsconst page = await mcp.stealth_mode({ url: "https://competitor.com/pricing", stealthConfig: { level: "advanced", randomizeFingerprint: true },}); // Extract structured pricing dataconst pricing = await mcp.scrape_structured({ url: "https://competitor.com/pricing", schema: { plans: [{ name: "string", price: "string", features: ["string"], }], },}); console.log(pricing.plans);04The pipeline
Tools Used
7◆ credits
Estimated cost: ~7 credits per protected page
- https://…
- 01stealth_
mode 5 credits - 02scrape_
structured 2 credits - JSON · markdown
05Questions
Frequently Asked Questions
01How do I scrape competitor sites that block bots?
Use CrawlForge stealth_mode, which renders in a real browser and randomizes fingerprints to get past rate limits and bot detection. Combine it with scrape_structured to return clean, schema-based data from the protected pages.
02Is stealth scraping reliable enough for ongoing monitoring?
Yes. stealth_mode handles the browser and the fingerprinting for you and rotates through any proxies you supply, so you can monitor protected competitor pages on a schedule without running your own browser infrastructure. Pair it with track_changes to watch for launches or pricing moves.
03What does stealth_mode cost per request?
stealth_mode is 5 credits per request because it runs a full browser with proxy and fingerprint rotation. Reserve it for sites that actually block standard fetches, and use cheaper tools like fetch_url at 1 credit for open pages.
04Can I extract structured data from competitor pages?
Yes. scrape_structured at 2 credits maps CSS selectors to fields so you get typed JSON — product names, prices, specs — rather than raw HTML. For AI-driven extraction without selectors, use extract_structured at 3 credits.
06Keep exploring
Related Use Cases
E-commerce Price Monitoring
Track competitor prices and product availability across hundreds of e-commerce sites. Detect undercutting the day it happens and reprice with confidence.
batch_scrape 5 creditstrack_changes 3 credits
Lead Enrichment
Enrich sales leads with company data, tech stacks, and contact information from the web. Turn a bare domain list into a qualified, sales-ready pipeline.
search_web 5 creditsextract_metadata 1 credit
Start forging
Ready to Get Started?
Every new account gets 1,000 free credits. No credit card required.