Skip to content

CrawlForge TeamEngineering Team

4 min read

Market Research API for RevOps: One Question, Every Vendor

A RevOps question about the market usually sounds simple. How many vendors in our category publish a free tier? Who still sells per seat? Which of them gate SSO behind the enterprise plan?

The answers sit on forty pricing pages. Someone opens them one at a time and pastes into a spreadsheet. A quarter later the sheet is stale, and nobody can say which cell came from where.

A market research API for RevOps should turn that into a script: one question, asked of every vendor's page, returning a row for each vendor with the sentence that answers it.

What kind of market research API does RevOps need?

There are two kinds, and most of the pages ranking for this phrase sell the first. Data vendors license datasets: firmographics, intent signals, market-size estimates. You query their database, and they decide what goes in it.

CrawlForge is the second kind. It owns no market data, no company records and no TAM figures. It reads public pages you point it at. That makes it the right tool when the answer is published on the vendors' own sites and no dataset has a column for it: packaging, plan limits, and which features sit behind which tier.

How do you ask one question across a whole category?

Use scrape with a question format. The hosted API runs no model for it. It ranks the page's sentences against your question and returns the five best verbatim, each with an offset into the page's markdown.

Bash
curl -X POST https://crawlforge.dev/api/v1/tools/scrape \
  -H "X-API-Key: cf_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://vendor-a.com/pricing",
    "formats": [
      "markdown",
      { "type": "question", "question": "Is SSO included on every plan or only on enterprise?" }
    ]
  }'

The answer is in formats.answer.evidence: up to five sentences, each with a score, an offset and a length. Asking for markdown in the same call costs nothing extra and lets you read the lines around each offset.

Run that over your vendor list and write one row per vendor: URL, question, top evidence, date checked. In n8n it is one HTTP Request node inside a loop; the n8n setup is in the docs.

Why not let a model write the answer?

Because nobody can audit that cell afterwards. A model can put a price on a row that the page never printed, and the fake looks exactly like a real one. The due diligence post covers that failure in detail. It matters just as much when the row feeds a deal desk's discount policy.

The extractive answer is less convenient and easier to audit. You get ranked passages, not a verdict. When we asked our own pricing page whether there is a free tier, the top sentence was "That's the free tier." It was correct, but it means nothing without the context its offset points to. Treat the evidence as a locator. A person, or a model that reads only those five sentences, fills in the cell, and the source sentence goes in the row beside it.

Two limits to plan around. The hosted API flattens tables to text, so a price that only appears in a pricing grid comes back as a run of cells rather than a clean row; the MCP server keeps table rows intact. And ask one question per call, so each row maps to one question.

What does a category survey cost?

A question format adds 1 credit to the scrape's 2, so each page costs 3.

SurveyPagesQuestionsCredits
One question, one category401120
Pricing and packaging pass403360
That pass, quarterly for a year403 × 41,440

The free tier's one-time 1,000 credits cover two full pricing-and-packaging passes, with credits left over.

What won't it do?

It won't size a market, and it won't tell you who is buying. If the question is how many companies in EMEA have more than 200 employees, buy the dataset. It also won't watch the pages between surveys. For "tell me when a competitor changes its pricing", use hosted monitors, and per-account profiles belong in an enrichment pipeline.

Start free with 1,000 credits and run one question across your category before you build anything around it.

Try this yourself — no signup needed

Explore all 31 CrawlForge scraping and extraction tools in the playground, then start free with 1,000 credits.

1,000 free credits • One-time • No credit card required

Tags

  • revops
  • market research
  • scrape
  • b2b
  • pricing research

About the Author

CrawlForge Team

Engineering Team

Building the most comprehensive web scraping MCP server. We create tools that help developers extract, analyze, and transform web data for AI applications.

Newsletter

Stay updated with the latest insights

Get tutorials, product updates, and web scraping tips delivered to your inbox.

No spam. Unsubscribe anytime.

FAQ

Frequently asked questions

01Is CrawlForge a market research data provider?

No. It owns no market data: no firmographics, no intent signals, no market-size estimates and no contact records. It reads public pages you point it at and returns what they say. For RevOps that makes it the research layer for questions answered on vendors' own sites, such as packaging, plan limits and feature tiers, alongside whatever dataset you already license.

02Does the question format use an LLM?

Not on the hosted REST API. There it runs in extractive mode only: it ranks the page's sentences against your question and returns the best five verbatim, each with an offset and length into the page's markdown. A request for mode "model" is rejected with a 400 and charged nothing. Model mode is available on the MCP server, which can call a local Ollama model or a configured LLM provider.

03How do I find the pricing page for every vendor in a category?

If you already have the vendors' domains, map_site lists a site's URLs for 2 credits, so you can pick out the pricing page without guessing the path. If you are still working out who is in the category, search_web costs 5 credits per query, and deep_research costs 10 per run and returns a source URL for every finding.

04Can I run the survey from n8n or another workflow tool?

Yes. Every tool is a plain POST endpoint on the REST API, authenticated with an X-API-Key header, so an HTTP Request node in n8n, or the equivalent step in Make or Zapier, can call it inside a loop over your vendor list. Nothing needs to be self-hosted for the REST path.

Keep reading

Related Articles