search_web
Google's index, as JSON. Each result carries the title, URL, snippet and Google's own highlighted variants, alongside a metadata block with the total match count and how long the search took. Filters for site, file type, language, recency and safe search are applied as real Google operators.
Use Cases
Research Pipelines
Find sources for a topic, then feed the URLs straight into batch_scrape or extract_content.
Site-Scoped Search
Set site to search inside one domain — a substitute for a site search box that does not exist or works badly.
Document Hunting
Combine file_type with a query to find PDFs, spreadsheets or slide decks on a subject.
Recency Monitoring
Restrict to the last day or week with time_range to catch new coverage of a topic or a brand.
Competitive Discovery
See which pages Google surfaces for the terms you care about, and how many results exist in total.
Agent Grounding
Give a model current search results to work from, instead of relying on what it memorised.
Endpoint
/api/v1/tools/search_webParameters
limit accepts up to 100, but Google returns at most 10 results per request — a larger limit does not fail, it just gives you 10. Page through the rest with offset, one search (and 5 credits) per page.| Name | Type | Required | Default | Description |
|---|---|---|---|---|
query | string | Required | - | The search terms. Google operators you type here work as they do in the search box, and `site` and `file_type` are appended as operators on top. Example: mcp server for web scraping |
limit | number | Optional | 10 | Results to return, 1-100 — but Google caps a single request at 10, so anything above 10 behaves as 10. Example: 10 |
offset | number | Optional | 0 | Zero-based index of the first result. Use 10, 20, 30 to page. Each page is a separate search and costs 5 credits. Example: 10 |
lang | string | Optional | - | Restrict to documents in one language, as an ISO 639-1 code. Echoed back in `search_metadata.lang`, which defaults to `en` when you omit it. Example: es |
site | string | Optional | - | Limit to one domain. Appended to your query as `site:<domain>`, so it behaves exactly like typing it. Example: docs.anthropic.com |
safe_search | boolean | Optional | - | Turn on Google's SafeSearch filtering. Off unless you set it. Example: true |
time_range | string | Optional | - | Restrict by recency: `day`, `week`, `month`, `year` or `all`. `all` applies no restriction, which is also the default. Example: week |
file_type | string | Optional | - | Limit to one file extension. Appended as `filetype:<ext>`, so `pdf`, `xlsx` and `pptx` all work. Example: pdf |
Request Examples
curl -X POST https://crawlforge.dev/api/v1/tools/search_web \
-H "X-API-Key: cf_test_YOUR_KEY" \
-H "Content-Type: application/json" \
-d '{
"query": "mcp server for web scraping",
"limit": 10,
"time_range": "month",
"safe_search": true
}'Response Example
{ "success": true, "data": { "query": "mcp server for web scraping", "search_metadata": { "total_results": 48200, "results_returned": 1, "offset": 0, "search_time": 0.312, "lang": "en", "safe_search": false, "time_range": "month", "site": null, "file_type": null }, "results": [ { "title": "CrawlForge MCP Server — 29 web tools for Claude", "url": "https://www.crawlforge.dev/", "snippet": "Search, scrape, crawl and extract from any site through one MCP server.", "displayLink": "www.crawlforge.dev", "formattedUrl": "https://www.crawlforge.dev/", "htmlSnippet": "Search, scrape, crawl and extract from any site through one MCP server.", "htmlTitle": "CrawlForge MCP Server — 29 web tools for Claude", "cacheId": "x8Kd0PqR2mUJ", "pagemap": { "metatags": [ { "og:type": "website" } ] } } ], "spelling_correction": "mcp server for web scraping", "related_searches": [ "mcp web scraping tools", "model context protocol scraper" ] }, "credits_used": 5, "credits_remaining": 995, "processing_time": 460}data.queryThe terms Google actually searched, which can differ from what you sentdata.search_metadata.total_resultsGoogle's estimate of matching documents. An estimate, not a count you can page throughdata.search_metadata.results_returnedLength of `results` — at most 10data.search_metadata.search_timeSeconds Google spent, as reported by Google. Not your request's latencydata.results[].urlThe destination link. `formattedUrl` is Google's display version of the same addressdata.results[].htmlSnippetThe snippet with Google's `<b>` term highlighting left in. Escape it before renderingdata.results[].cacheIdGoogle's cache identifier, when it has one. Absent for results Google has not cacheddata.results[].pagemapStructured data Google lifted from the page — metatags, images, schema.org types. Shape varies per result and can be absentdata.spelling_correctionGoogle's suggested spelling, or `null` when it has nonedata.related_searchesRelated query suggestions. Frequently an empty arraycredits_usedFixed 5 credits per search, whatever `limit` you asked forError Handling
Invalid Query (400 Bad Request)
VALIDATION_ERROR. query is required and must not be empty. The same status covers a limit outside 1-100, a negative offset, or a time_range that is not one of the five accepted values.
Search Failed (500 Internal Server Error)
TOOL_ERROR. Covers an upstream Google error, an exhausted daily quota, and a timeout after 10 seconds. The message names the cause. Nothing is charged for a failed search.
results array and total_results: 0 — not an error. Check results_returned rather than relying on the status code.Credit Cost
offset means another search, so ten pages cost 50 credits. Failed searches are not charged.What's Included:
Google's live index, not a cached copy
Title, URL, snippet and Google's highlighted variants
Total match estimate and search time
site, file_type, lang, time_range and safe-search filters
Spelling correction and related searches when Google supplies them
Plan Recommendations:
Free Plan: 1,000 one-time trial credits = 200 searches
Hobby Plan: 5,000 credits = 1,000 searches ($19/mo)
Professional Plan: 50,000 credits = 10,000 searches ($99/mo)