CrawlForge v5.0.4: 34 Fixes From Live-Testing 27 MCP Tools

On August 20, 2026 we shipped four patch releases in a single day — v5.0.1, v5.0.2, v5.0.3, and v5.0.4 — closing 34 defects. Not one of them came from a bug report. All of them came from a simpler idea: run all 27 MCP tools and every CLI subcommand against real, live websites, and judge the content that comes back — not the exit code.

v5.0.0 hardened the codebase from the inside: a seven-phase remediation driven by a full internal audit. But an audit reads code. It cannot tell you that GitHub's logged-out layout stopped shipping the markup your template targets, that your stealth engine's upstream package quietly broke, or that your research agent fetches exactly the right page and then claims it cannot access the internet. Only live traffic tells you that.

So that is what we ran — three full rounds of it, fixing everything each round found before running the next.

Table of contents

• What shipped in v5.0.1 through v5.0.4
• How we live-test an MCP server
• v5.0.1: 13 fixes, one of them critical
• v5.0.2: content quality on real markup
• v5.0.3: 15 fixes and an un-broken stealth engine
• v5.0.4: the agent tool can finally answer
• Which local model should the agent use?
• Do I need to change anything?
• How to upgrade
• What is next

What shipped in v5.0.1 through v5.0.4

The unit suite grew from 914 to 980 tests across the four releases — every defect got a regression test before its fix shipped — and MCP protocol compliance held at 100% throughout. No tool schemas, output shapes, or credit costs changed. All four are drop-in upgrades.

How we live-test an MCP server

Each round is a full-surface pass: all 27 tools invoked over MCP stdio, and every CLI subcommand run in a real shell, against live production websites — news front pages, e-commerce catalogs, GitHub repositories, documentation sites. A workflow of parallel agents then judges what came back. Is the markdown clean, or is it welded-together text? Are the extracted entities real? Does the answer name what is actually on the page right now?

That last question is the point. HTTP 200 with plausible-looking output is the easiest way for a web scraping tool to be wrong, and it is invisible to both unit tests and code audits. The failures below all passed CI. Several had passed CI for multiple releases.

v5.0.1: 13 fixes, one of them critical

The critical one: crawl_deep could starve the whole server. For every link on a page, the BFS crawler re-parsed the entire page HTML — O(links × page size) of synchronous CPU. On link-dense pages the process pegged a core for 13+ minutes at multi-gigabyte memory while the Node.js event loop starved: every concurrent MCP call hung, and even graceful shutdown could not run. The page is now parsed once and reused for every link.

Also in crawl_deep: concurrent queue tasks could all pass the max_pages check before any of them registered, letting crawls overshoot the cap. The cap is now exact.

The rest of round one, quickly:

• serp_rank no longer throws away paid lookups. DataForSEO emits snippet: null for organic results without a description; the output schema required a string, discarding an already-billed response. Now nullable.
• generate_llms_txt honors analysisOptions.maxPages and maxDepth. The caps never reached the analyzer before.
• CLI stealth failed on every default invocation — --engine defaulted to playwright, a value the enum stopped accepting in v4.0.0. The default is now chromium, with playwright kept as an alias.
• CLI track can finally compare across invocations. Baselines lived only in process memory, so a fresh CLI process (or a restarted MCP server) had nothing to compare against. Baselines now rehydrate from the newest persisted snapshot — fixing a latent bug from v4.8.0 where stored snapshots came back as Buffers and were silently rejected by a string-type guard.
• monitor:stop told the truth about nothing. It reported stopped: false while succeeding, and unknown monitor ids were a silent no-op. Both paths now load the store first and report real results.
• crawlforge init had two setup bugs. Creator machines no longer need an API key to generate a config. And the generated MCP stanza invoked npx -y crawlforge@latest — a package that does not exist. The published package is crawlforge-mcp-server, so every init-generated config was broken until edited by hand. Fixed, and the config now points at crawlforge-mcp-server@latest.
• Change snapshots stored junk. Every detected change wrote an empty snapshot, because change records carry diff analysis rather than content. Snapshots now store only real content, and rehydration skips empty ones so previously polluted stores still recover.

One environment note for self-hosters: stealth_mode, scrape_with_actions, and scrape's screenshot format need the Playwright browsers installed — npx playwright install chromium.

v5.0.2: content quality on real markup

Round one also flagged four tools that succeeded while returning degraded content. All four were markup problems on real websites:

• analyze_content always returned zero entities. The analyzer called an NLP method that requires a plugin we never installed; the throw aborted extraction and a silent catch returned empties — even though people, places, and organizations extraction worked fine underneath. Dates now use the core library's tag matching, and "Apple and Microsoft announced record earnings in California" yields the entities you would expect.
• extract_structured silently omitted prices. The literal .price selector cannot match class names like price_color. The semantic table now includes [itemprop="price"] and [class*="price"].
• generate_llms_txt produced no page inventory. Substring matching classified "S-api-ens" as an API link, and one false positive suppressed the entire sitemap fallback. API detection now uses word boundaries, and the ## Pages listing always renders for uncategorized sites.
• scrape_template returned null watchers and empty topics on GitHub. GitHub's logged-out React layout dropped the markup the selectors targeted. New selectors read the current layout, with the classic-layout ones kept as fallbacks.

v5.0.3: 15 fixes and an un-broken stealth engine

Round two went deeper and found fifteen more, three of them structural:

The agent tool stopped fabricating. Preamble-line filtering, a per-source synthesis context budget with relevance ordering, explicit anti-fabrication rules, and a guarantee that sites named in your prompt are fetched first.

The Camoufox engine had been silently broken since v5.0.0. Camoufox's ESM entry point is a broken esbuild bundle, so loading it simply failed — which also disabled deep_research's anti-bot fallback without any visible error. The fix loads it via CJS, uses the real Camoufox() API, and creates viewport: null contexts to bypass a playwright-core/Firefox protocol mismatch.

Scheduled monitors now persist in ~/.crawlforge/monitors. They previously wrote to a ./monitors directory relative to wherever the process happened to start, so monitor:list, monitor:stop, and the cron workflow only worked from that exact directory. Legacy stores migrate automatically.

The remaining twelve were content-quality fixes across the surface: track_changes similarity clamped to [0,1] with real baseline stats, batch_scrape markdown keeping span and small content, extract_text stripping noscript blocks, French dates formatted DD/MM/YYYY in localization, extractive summaries including the lead sentence, analyze_content returning real topics, scrape_with_actions honoring scroll coordinates and reporting scrolledTo, llms.txt links getting real page titles, and more — about 50 new regression tests in this release alone.

v5.0.4: the agent tool can finally answer

The third round had one hard defect left, and it was the most interesting bug of the day.

The agent tool would fetch exactly the right page — HTTP 200, evidence recorded — and then answer "I'm unable to access the internet." Three coupled causes, each verified insufficient to fix alone:

1. Text flattening destroyed structure. Page text was collapsed with a whitespace regex that welded adjacent block elements into one string: a ranked list became "1.Story title329 points". A new flattener marks block-element boundaries before collapsing, so every table row and list item survives as its own line.
2. Source ordering ignored your intent. Synthesis sources were ranked purely by prompt-term overlap, so a generic article containing the task's words could outrank the site the prompt explicitly named. Seed URLs and prompt-named sites now carry a fixed priority boost above any term-overlap score.
3. Small local models refused perfectly good input. The synthesis prompt now states that the sources were already fetched and forbids refusing for lack of browsing ability.

With all three in place, the agent — running live over MCP stdio — names the actual current Hacker News #1 story. That is the end-to-end bar every future release has to clear.

v5.0.4 also fixes CLI search ignoring the API key stored by crawlforge setup: the key was read from the environment at import time, before the CLI's key-resolution hook ran. It now resolves at call time.

Which local model should the agent use?

One finding from the synthesis work is worth acting on if you run the agent tool against local Ollama models. On the real ~13KB multi-source synthesis prompt, llama3.2 — the code default — still fails where qwen2.5:3b (the same 3B size), mistral:7b, and gemma3:12b all succeed.

The default stays llama3.2 for ecosystem compatibility, but if agent answers degrade locally, switch:

export OLLAMA_DEFAULT_MODEL=qwen2.5:3b

Do I need to change anything?

No. All four releases are patches: no tool schemas, output shapes, or credit costs changed, the tool count stays at 27, and pricing is untouched. The Node.js floor of >= 20.16.0 from v5.0.0 still applies.

One exception: if you ever ran crawlforge init before v5.0.1 and your MCP client config invokes crawlforge@latest, that stanza never worked — re-run crawlforge init or point it at crawlforge-mcp-server@latest.

How to upgrade

npm install -g crawlforge-mcp-server@latest

New users:

npm install -g crawlforge-mcp-server
npx crawlforge init

Existing MCP client users can simply trigger an /mcp reconnect. The full defect-by-defect record for all four releases is in the changelog and on the releases page.

What is next

Full-surface live testing is now part of the release gate, not a one-off: every future release gets a live run of all 27 tools against real websites before it ships. The deferred v5.0.0 tracks — a hosted remote endpoint with OAuth, scheduled monitoring as a service, persistent sessions — remain queued behind it.

And the standing invitation holds: if a tool returns content that does not match what is actually on the page, that is exactly the bug we want to hear about.

Ready to try it? Start free with 1,000 credits — then run npx crawlforge init to register the MCP server. See the full docs or the v5.0.0 release post for the overhaul these releases build on.