On this page
Adding an MCP server to your own editor is a five-line config change and the blast radius is your machine. Shipping one inside a product other people run is a different decision. You inherit the server's permissions, its unit cost and its failure mode on behalf of every user, and when it invents a source, the fabricated link has your product's name above it, not the model's.
We just did this. LightAudit Score is a free, open-source console that runs Lighthouse against localhost, staging and VPN hosts that PageSpeed Insights cannot reach. Click a low score and an AI reads the actual audit data and returns prioritised fixes. Those fixes carry citations only when a research MCP server is connected — and CrawlForge is the named option.
This post is the architecture, not the pitch. Five decisions, each of which someone in your security review is going to ask about.
LightAudit Score is launching soon. The repository goes public shortly, and it is free and MIT-licensed when it does — no account, no licence, no usage cap. To be notified the day it ships, leave your email on lightauditscore.com: one email with the repository and the quick start, and nothing else.
Table of Contents
- Why an AI feature needs a research server at all
- Decision 1: Off by default, and honest when absent
- Decision 2: Ten tools out of twenty-nine
- Decision 3: The user's key never touches your app
- Decision 4: Pin the version, don't resolve it at launch
- Decision 5: Name one server, drive any server
- What it costs per analysis
- The general shape
Why an AI feature needs a research server at all
A Lighthouse score of 71 tells a developer that something is wrong. It does not tell them which of forty audits to act on first, and it certainly does not tell them why. That is the gap an AI panel fills: read the audit tree, name the cause, rank the fixes.
The moment those fixes cite web.dev or developer.chrome.com, you have a problem that is not about model quality. A model asked for citations will produce plausible URLs whether or not it opened them. Ask it not to, and it mostly complies — until the one time it doesn't, which is the time a customer clicks through to a 404 and stops trusting the whole panel.
A research server closes that loop mechanically rather than by instruction. The model can only cite a source it fetched through a tool call, because the citation is assembled from the tool result, not from the model's recollection of the web. This is the same traceability property that makes a general web-data server useful in due diligence work: the value is not the data, it is the auditable path from the claim back to the page.
Decision 1: Off by default, and honest when absent
The research server is a switch in Settings, and it starts off.
With it off, the analysis is diagnosed from audit data alone and the result carries a plain No web research badge. Every analysis is also stamped with the provider and model that produced it. Nothing silently downgrades, and nothing invents a source to fill the space where a citation would have been.
This is the decision most teams get backwards. The instinct is to make the integration mandatory so the feature always looks its best; the consequence is that your feature's trust story now depends on a third party being reachable. Degrade visibly instead. A user who sees No web research knows exactly what they are reading. A user who sees a citation that turns out to be invented has learned something much worse about your product.
Decision 2: Ten tools out of twenty-nine
CrawlForge exposes twenty-nine tools. LightAudit's analysis path is handed ten of them: search, fetch, extract, summarise. Site crawlers, batch jobs, browser automation and deep research are denied outright.
The reason is unit cost, not paranoia. A citation pass needs to find a page and read it. It does not need to crawl a domain, and the difference shows up on the user's bill:
| Tool | Credits | Granted? |
|---|---|---|
fetch_url | 1 | Yes — raw HTTP read |
extract_text | 1 | Yes |
extract_content | 2 | Yes — main content, boilerplate stripped |
summarize_content | 4 | Yes |
search_web | 5 | Yes — the entry point |
crawl_deep | 4 | No — walks a whole site |
batch_scrape | 5 | No — fans out across a URL list |
stealth_mode | 5 | No — launches a browser |
deep_research | 10 | No — multi-source, expands the query itself |
deep_research is the instructive one. It is the best tool in the catalogue for a research question, and it is denied. A model that reaches for it on every low score turns an 11-credit analysis into a 10-credit-plus-fan-out one, and the user who clicked a score ring did not ask for a research project. The tool your agent would enjoy most is not the tool your product should grant it.
If you take one thing from this post: enumerate the tools you grant. An MCP server's catalogue is its author's idea of useful, and the intersection with your feature's job is usually a third of it.
Decision 3: The user's key never touches your app
LightAudit forwards nothing to the research server. CrawlForge authenticates from its own configuration, written by its own setup wizard, so the user's API key never passes through the app that benefits from it.
That sentence is worth more in a security review than any amount of encryption-at-rest language, because it removes the question rather than answering it. You are not a custodian of a third-party credential. There is no key in your database to leak, no key in your logs to redact, no key in a support bundle, and no incident-response paragraph to write about a vendor you do not control.
The general form: when an integration can authenticate itself out-of-band, let it. Every credential your product holds on a user's behalf is a liability you took on to save them one configuration step.
Decision 4: Pin the version, don't resolve it at launch
Almost every MCP config you will copy from a README looks like this:
{
"mcpServers": {
"crawlforge": {
"command": "npx",
"args": ["-y", "crawlforge-mcp-server"],
"env": {
"CRAWLFORGE_API_KEY": "cf_live_YOUR_API_KEY_HERE"
}
}
}
}That is the right config for your own machine and the wrong one to ship. -y suppresses the install prompt and a bare package name resolves to whatever is latest at launch time, which means the code your customers run tomorrow is code nobody on your team has read. It is a supply-chain decision disguised as a convenience flag.
Pin it:
{
"mcpServers": {
"crawlforge": {
"command": "npx",
"args": ["-y", "crawlforge-mcp-server@5.6.6"]
}
}
}Then move the version bump into your normal dependency review, where a human reads a changelog before your users get new behaviour. This costs you a pull request every few weeks. The alternative costs you an incident you cannot reproduce, because the version that caused it is no longer what latest resolves to.
Decision 5: Name one server, drive any server
LightAudit names CrawlForge in its documentation and ships nothing. Any research server declared in a standard .mcp.json will be driven instead.
This is not altruism, it is how you survive a procurement conversation. An MCP dependency your buyer cannot swap is a lock-in question in every enterprise review you will ever sit in. Because MCP is a protocol rather than an SDK, honouring the standard config file costs you approximately nothing and converts "which vendor are you tied to" into "whichever one you prefer".
It also keeps you honest. A named default that can be replaced has to stay the best option on merit.
What it costs per analysis
One analysis is one search and a few reads:
| Step | Tool | Credits |
|---|---|---|
| Find the authoritative pages for the diagnosis | search_web | 5 |
| Read three of them | extract_content × 3 | 6 |
| Total | 11 |
Analyses are stored with the run, so reopening a report costs nothing — the second developer to read the same report spends zero credits. A new CrawlForge account starts with 1,000 free credits and no card, which is roughly ninety cited analyses before anyone makes a purchasing decision. Per-tool pricing is on the pricing page.
That number is the one to work out before you wire the server in, not after. The tools your feature grants and the number of times it calls them are the two variables in your customers' bill, and the first one is fixed at design time.
The general shape
Strip out the Lighthouse specifics and the same four questions apply to any MCP server you put inside a shipped product:
- What is the blast radius? Enumerate the granted tools. Deny the rest by name.
- Who holds the credential? If the server can authenticate itself, do not hold it for them.
- What is the unit cost? Calculate it per user action, not per month.
- What happens when it is not there? Degrade visibly. A missing feature is recoverable; a fabricated one is not.
None of this is exotic. It is the same discipline you would apply to any third-party call in a hot path — it just tends to get skipped for MCP servers, because the config is five lines and it feels like a settings change rather than a dependency.
LightAudit Score is launching soon. It is free, MIT-licensed and runs entirely on your own machine, and the repository goes public shortly. If you want to see this pattern working rather than described, the site has the full feature breakdown and an early-access list that will send you the link the day it ships — and the research server it names is the one you have just been reading about.
Try this yourself — no signup needed
Run any of CrawlForge's 29 scraping and extraction tools in the playground, then start free with 1,000 credits.
1,000 free credits • One-time • No credit card required
Tags
About the Author
Stay updated with the latest insights
Get tutorials, product updates, and web scraping tips delivered to your inbox.
No spam. Unsubscribe anytime.