CrawlForge MCP
HomePlaygroundUse CasesIntegrationsPricingDocumentationBlog
Shipping an MCP Server Inside Your Product: Five Decisions
AI Engineering
Back to Blog
AI Engineering

Shipping an MCP Server Inside Your Product: Five Decisions

C
CrawlForge Team
Engineering Team
September 4, 2026
7 min read

On this page

Quick Answer

Putting an MCP server inside a product you ship is a different problem from adding one to your own editor: you inherit its blast radius, its unit cost and its failure mode on behalf of every user. LightAudit Score, a free Lighthouse console, connects CrawlForge as a research server so its AI fixes carry citations to pages it actually opened. Five decisions made that safe to ship — the server is off by default and the feature degrades honestly without it, ten of twenty-nine tools are granted while the crawlers and batch jobs are denied, the user API key never passes through the app, the version is pinned instead of resolved by npx at launch, and any server declared in a standard .mcp.json can replace it. Budget roughly 11 credits per analysis: one search and three page reads.

Adding an MCP server to your own editor is a five-line config change and the blast radius is your machine. Shipping one inside a product other people run is a different decision. You inherit the server's permissions, its unit cost and its failure mode on behalf of every user, and when it invents a source, the fabricated link has your product's name above it, not the model's.

We just did this. LightAudit Score is a free, open-source console that runs Lighthouse against localhost, staging and VPN hosts that PageSpeed Insights cannot reach. Click a low score and an AI reads the actual audit data and returns prioritised fixes. Those fixes carry citations only when a research MCP server is connected — and CrawlForge is the named option.

This post is the architecture, not the pitch. Five decisions, each of which someone in your security review is going to ask about.

LightAudit Score is launching soon. The repository goes public shortly, and it is free and MIT-licensed when it does — no account, no licence, no usage cap. To be notified the day it ships, leave your email on lightauditscore.com: one email with the repository and the quick start, and nothing else.

Table of Contents

  • Why an AI feature needs a research server at all
  • Decision 1: Off by default, and honest when absent
  • Decision 2: Ten tools out of twenty-nine
  • Decision 3: The user's key never touches your app
  • Decision 4: Pin the version, don't resolve it at launch
  • Decision 5: Name one server, drive any server
  • What it costs per analysis
  • The general shape

Why an AI feature needs a research server at all

A Lighthouse score of 71 tells a developer that something is wrong. It does not tell them which of forty audits to act on first, and it certainly does not tell them why. That is the gap an AI panel fills: read the audit tree, name the cause, rank the fixes.

The moment those fixes cite web.dev or developer.chrome.com, you have a problem that is not about model quality. A model asked for citations will produce plausible URLs whether or not it opened them. Ask it not to, and it mostly complies — until the one time it doesn't, which is the time a customer clicks through to a 404 and stops trusting the whole panel.

A research server closes that loop mechanically rather than by instruction. The model can only cite a source it fetched through a tool call, because the citation is assembled from the tool result, not from the model's recollection of the web. This is the same traceability property that makes a general web-data server useful in due diligence work: the value is not the data, it is the auditable path from the claim back to the page.

Decision 1: Off by default, and honest when absent

The research server is a switch in Settings, and it starts off.

With it off, the analysis is diagnosed from audit data alone and the result carries a plain No web research badge. Every analysis is also stamped with the provider and model that produced it. Nothing silently downgrades, and nothing invents a source to fill the space where a citation would have been.

This is the decision most teams get backwards. The instinct is to make the integration mandatory so the feature always looks its best; the consequence is that your feature's trust story now depends on a third party being reachable. Degrade visibly instead. A user who sees No web research knows exactly what they are reading. A user who sees a citation that turns out to be invented has learned something much worse about your product.

Decision 2: Ten tools out of twenty-nine

CrawlForge exposes twenty-nine tools. LightAudit's analysis path is handed ten of them: search, fetch, extract, summarise. Site crawlers, batch jobs, browser automation and deep research are denied outright.

The reason is unit cost, not paranoia. A citation pass needs to find a page and read it. It does not need to crawl a domain, and the difference shows up on the user's bill:

ToolCreditsGranted?
fetch_url1Yes — raw HTTP read
extract_text1Yes
extract_content2Yes — main content, boilerplate stripped
summarize_content4Yes
search_web5Yes — the entry point
crawl_deep4No — walks a whole site
batch_scrape5No — fans out across a URL list
stealth_mode5No — launches a browser
deep_research10No — multi-source, expands the query itself

deep_research is the instructive one. It is the best tool in the catalogue for a research question, and it is denied. A model that reaches for it on every low score turns an 11-credit analysis into a 10-credit-plus-fan-out one, and the user who clicked a score ring did not ask for a research project. The tool your agent would enjoy most is not the tool your product should grant it.

If you take one thing from this post: enumerate the tools you grant. An MCP server's catalogue is its author's idea of useful, and the intersection with your feature's job is usually a third of it.

Decision 3: The user's key never touches your app

LightAudit forwards nothing to the research server. CrawlForge authenticates from its own configuration, written by its own setup wizard, so the user's API key never passes through the app that benefits from it.

That sentence is worth more in a security review than any amount of encryption-at-rest language, because it removes the question rather than answering it. You are not a custodian of a third-party credential. There is no key in your database to leak, no key in your logs to redact, no key in a support bundle, and no incident-response paragraph to write about a vendor you do not control.

The general form: when an integration can authenticate itself out-of-band, let it. Every credential your product holds on a user's behalf is a liability you took on to save them one configuration step.

Decision 4: Pin the version, don't resolve it at launch

Almost every MCP config you will copy from a README looks like this:

Json
{
  "mcpServers": {
    "crawlforge": {
      "command": "npx",
      "args": ["-y", "crawlforge-mcp-server"],
      "env": {
        "CRAWLFORGE_API_KEY": "cf_live_YOUR_API_KEY_HERE"
      }
    }
  }
}

That is the right config for your own machine and the wrong one to ship. -y suppresses the install prompt and a bare package name resolves to whatever is latest at launch time, which means the code your customers run tomorrow is code nobody on your team has read. It is a supply-chain decision disguised as a convenience flag.

Pin it:

Json
{
  "mcpServers": {
    "crawlforge": {
      "command": "npx",
      "args": ["-y", "crawlforge-mcp-server@5.6.6"]
    }
  }
}

Then move the version bump into your normal dependency review, where a human reads a changelog before your users get new behaviour. This costs you a pull request every few weeks. The alternative costs you an incident you cannot reproduce, because the version that caused it is no longer what latest resolves to.

Decision 5: Name one server, drive any server

LightAudit names CrawlForge in its documentation and ships nothing. Any research server declared in a standard .mcp.json will be driven instead.

This is not altruism, it is how you survive a procurement conversation. An MCP dependency your buyer cannot swap is a lock-in question in every enterprise review you will ever sit in. Because MCP is a protocol rather than an SDK, honouring the standard config file costs you approximately nothing and converts "which vendor are you tied to" into "whichever one you prefer".

It also keeps you honest. A named default that can be replaced has to stay the best option on merit.

What it costs per analysis

One analysis is one search and a few reads:

StepToolCredits
Find the authoritative pages for the diagnosissearch_web5
Read three of themextract_content × 36
Total11

Analyses are stored with the run, so reopening a report costs nothing — the second developer to read the same report spends zero credits. A new CrawlForge account starts with 1,000 free credits and no card, which is roughly ninety cited analyses before anyone makes a purchasing decision. Per-tool pricing is on the pricing page.

That number is the one to work out before you wire the server in, not after. The tools your feature grants and the number of times it calls them are the two variables in your customers' bill, and the first one is fixed at design time.

The general shape

Strip out the Lighthouse specifics and the same four questions apply to any MCP server you put inside a shipped product:

  1. What is the blast radius? Enumerate the granted tools. Deny the rest by name.
  2. Who holds the credential? If the server can authenticate itself, do not hold it for them.
  3. What is the unit cost? Calculate it per user action, not per month.
  4. What happens when it is not there? Degrade visibly. A missing feature is recoverable; a fabricated one is not.

None of this is exotic. It is the same discipline you would apply to any third-party call in a hot path — it just tends to get skipped for MCP servers, because the config is five lines and it feels like a settings change rather than a dependency.

LightAudit Score is launching soon. It is free, MIT-licensed and runs entirely on your own machine, and the repository goes public shortly. If you want to see this pattern working rather than described, the site has the full feature breakdown and an early-access list that will send you the link the day it ships — and the research server it names is the one you have just been reading about.

Try this yourself — no signup needed

Run any of CrawlForge's 29 scraping and extraction tools in the playground, then start free with 1,000 credits.

1,000 free credits • One-time • No credit card required

Tags

mcparchitectureb2bcitationslighthouse

About the Author

C

CrawlForge Team

Engineering Team

Building the most comprehensive web scraping MCP server. We create tools that help developers extract, analyze, and transform web data for AI applications.

Stay updated with the latest insights

Get tutorials, product updates, and web scraping tips delivered to your inbox.

No spam. Unsubscribe anytime.

Put this into practice

Test CrawlForge's tools on any URL — free, no signup.

On this page

Frequently Asked Questions

Is LightAudit Score available yet?+

Not quite — it is launching soon. The repository goes public shortly, and until it does the site carries an early-access list that sends you the link on the day it ships. Everything described in this post is how the shipped product works: free, MIT-licensed, running on your own machine, with the research server off by default behind a switch in Settings.

Should I ship an MCP server inside my product, or just call the API?+

Ship the MCP server when the calling side is a model that decides which tool to use, and call the REST API when your own code decides. LightAudit ships the server because the analysis is a model reading audit data and choosing what to look up; a fixed pipeline that always searches then always reads three pages would be simpler as three HTTP calls. The protocol earns its place by letting the model choose, which is also exactly why the granted tool list matters so much.

How do I stop the model calling expensive tools?+

Do not rely on the prompt. Grant the tools you want and deny the rest at the connection, so the expensive ones are not in the catalogue the model can see. LightAudit grants ten of the twenty-nine CrawlForge tools and denies site crawlers, batch jobs, browser automation and deep_research outright, which caps a single analysis at roughly 11 credits regardless of what the model would have preferred to do.

What happens if the research server is unavailable?+

The feature has to keep working without it, and it has to say so. In LightAudit the analysis is diagnosed from the audit data alone and the result carries a plain "No web research" badge, alongside the provider and model that produced it. It never falls back to citing from memory. Designing this path first is what stops an outage in a dependency turning into fabricated output in your product.

Do my users need to give my app their CrawlForge API key?+

They should not have to, and in LightAudit they do not. CrawlForge authenticates from its own configuration, written by its own setup wizard, so the key never passes through the app. That removes the whole custody question from your security review: no key in your database, your logs, or a support bundle. Prefer this arrangement for any integration that can authenticate itself out-of-band.

Why pin the MCP server version instead of using npx -y package@latest?+

Because on a shipped product, @latest means your customers run code nobody on your team has read. Pinning to a reviewed version — crawlforge-mcp-server@5.6.6 rather than the bare package name — moves the upgrade into your normal dependency review, where a human reads a changelog first. It also makes incidents reproducible, which a floating version does not, since latest will have moved by the time you investigate.

How much does a cited analysis cost in credits?+

About 11 credits: one search_web call at 5 credits plus three extract_content reads at 2 each. Stored analyses are re-read for free, so a report opened by a second developer costs nothing. A new account gets 1,000 free credits with no card, which is roughly ninety cited analyses — enough to decide whether the feature is worth paying for before anyone is asked to.

Related Articles

Web Search MCP Server With Local LLM: Which Model Actually Runs
AI Engineering

Web Search MCP Server With Local LLM: Which Model Actually Runs

search_web finds the pages; a local Ollama model reads them. CrawlForge picks that model by measured accuracy rather than parameter count — and a 4B model beats a 20B one.

C
CrawlForge Team
|
Aug 31
|
4m
Web Search MCP Server: Self-Hosted With SearXNG
AI Engineering

Web Search MCP Server: Self-Hosted With SearXNG

Most "self-hosted" search MCP servers only run the process locally. CrawlForge can point search_web at your own SearXNG instance — and here is the part that still isn't self-hosted.

C
CrawlForge Team
|
Aug 28
|
4m
MCP Protocol Explained: A Developer Guide for 2026
AI Engineering

MCP Protocol Explained: A Developer Guide for 2026

Learn how the Model Context Protocol works, why it matters for AI agents, and how to build MCP servers and clients with architecture diagrams and code.

C
CrawlForge Team
|
Apr 27
|
10m

Footer

CrawlForge MCP

Enterprise web scraping for AI Agents. 29 specialized MCP tools designed for modern developers building intelligent systems.

Product

  • Features
  • Playground
  • Pricing
  • Use Cases
  • Integrations
  • Alternatives
  • Changelog

Resources

  • Getting Started
  • API Reference
  • Templates
  • Guides
  • Blog
  • Glossary
  • FAQ
  • Sitemap

Developers

  • MCP Protocol
  • Claude Desktop
  • Cursor IDE
  • LangChain
  • LlamaIndex

Company

  • About
  • Contact
  • Privacy
  • Terms
  • Acceptable Use
  • Cookies

Stay updated

Get the latest updates on new tools and features.

Built with Next.js and MCP protocol

© 2025-2026 CrawlForge. All rights reserved.