Web Search MCP Server With Local LLM: Which Model Actually Runs

Most guides to running a web search MCP server "with a local LLM" stop at the install command. The more useful question is which model your machine actually ends up using, because CrawlForge does not hardcode one — it ranks the models you already have installed and picks the best of them.

Two jobs are worth separating first. search_web finds pages; it is a search API call and no LLM is involved. The local model matters for what happens next — extract_with_llm, summarize_content, analyze_content, deep_research. So "web search MCP server with local LLM" really means: the search is remote, the reading is local.

Does a web search MCP server use a local LLM by default?

CrawlForge does, with no configuration. Providers are registered in a fixed order of preference — openai, anthropic, ollama — but the two cloud providers only register when their API key is present. Ollama needs no key, so on a machine with no keys set it is the only provider registered, and it becomes the default.

Set OPENAI_API_KEY and OpenAI outranks Ollama. That is deliberate: a cloud key is only there because someone deliberately configured it, so it wins over the zero-config local option. Set DISABLE_OLLAMA=true to keep Ollama out entirely.

Which local model does CrawlForge actually pick?

In this order:

1. OLLAMA_DEFAULT_MODEL if set — an explicit pin always wins.
2. Otherwise the highest-ranked model you actually have installed.
3. Otherwise the first installed model, whatever it is.
4. llama3.2 only when Ollama cannot be reached at all — so the error message names a real model rather than a placeholder.

The installed list comes from a single GET /api/tags against OLLAMA_BASE_URL (default http://localhost:11434) with a three-second timeout, cached for the life of the process.

Step 1 is the one that bites. A stray OLLAMA_DEFAULT_MODEL left in a local .env silently defeats the entire ranking, and nothing warns you.

Why does a 4B model outrank a 20B one?

Because the ranking is measured, not assumed. Scored against three live product pages with independently verified ground truth, counting correct fields out of 18:

Parameter count did not predict accuracy. gemma3:4b matched the 20B model's score at roughly a third of the latency, and held 45/45 across five runs.

The failures are the interesting part. llama3.2 — the old hardcoded default — invented a compare-at price on a product that has none, on every single run. That is why choosing the best installed model replaced hardcoding one.

There is a second ranking for a different job. Judging claims — relevance, same-meaning grouping, contradiction — is not extraction, and only gemma3:12b is listed as fit for it. gemma3:4b, the extraction winner, scored a Playwright-versus-Selenium marketing page 0.9 relevant to an anti-bot research topic. A model absent from that list is never asked to judge, because a model that invents disagreement between sources that agree is worse than one that reports none.

What happens when Ollama is not running?

It is probed once. If the host is unreachable the Ollama provider is de-registered and selection re-runs, so a machine holding a cloud key falls back to that, and a machine with neither reports no provider available instead of hanging. Tools that can degrade to CSS or keyword extraction do so rather than failing.

Does running the model locally make the search free?

No. Local inference costs nothing to run, but the tools still cost credits: search_web 5, summarize_content 4, extract_with_llm 3, analyze_content 3, list_ollama_models 1. Metering runs before a provider is chosen, so it never learns that a local model answered.

What local inference changes is where your data goes. With Ollama, scraped page content is never sent to a third-party model. Pair it with provider: "searxng" on search_web and the query stays on your network too — though that is a privacy change, not a cost one.