Competitor Monitoring API for RevOps: No Scraper to Run

Competitor pricing changes on a Tuesday. The revenue team hears about it on Thursday, from a rep who lost the deal on Wednesday.

The usual fix is a scraper: a cron job, a headless browser, a diff, a Slack post — owned by whoever had spare time last quarter. It works for about a month. Then a layout shifts or the box it runs on rotates, and nobody notices it stopped, because a monitor finding nothing and a monitor that is dead produce exactly the same silence.

A competitor monitoring API for RevOps moves the running part off your side. You describe what to watch; the checks, the diffing and the retries become somebody else's uptime problem.

What does a competitor monitoring API do for RevOps?

You POST one monitor — up to 20 URLs, a cron schedule, and where the results go. CrawlForge's scheduler runs the checks, diffs each page against the previous one, and delivers what changed.

curl -X POST https://crawlforge.dev/api/v1/monitors \
  -H "X-API-Key: cf_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "competitor pricing",
    "targets": [
      { "url": "https://competitor-a.com/pricing", "selector": ".pricing-table" },
      { "url": "https://competitor-b.com/pricing", "selector": "main" }
    ],
    "schedule_cron": "0 7 * * 1-5",
    "timezone": "America/New_York",
    "notify_emails": ["revops@example.com"],
    "webhook_url": "https://example.com/hooks/crawlforge"
  }'

The selector decides whether this survives contact with a real website. Scope the diff to the pricing table and a navigation redesign is not a pricing event; leave it off and every marketing tweak fires.

The first check of a target captures a baseline and reports new. Every check after that diffs against the previous check and rolls the baseline forward, so what reaches you is "changed since last time" rather than "changed since some Tuesday in March". Nothing runs on your infrastructure: the scheduler wakes every five minutes and starts whatever is due. The full endpoint reference is in the docs.

How does the signal reach the rest of the stack?

Over a webhook, which is the reason this belongs in a RevOps stack rather than in somebody's browser tab. Each check sends a monitor.page event for every page that is not unchanged, then one monitor.check.completed summary.

The page payload carries change_percent, added_count and removed_count, plus up to 20 sample lines from each side of the diff — enough for a handler to decide without fetching anything itself:

• change_percent past your threshold on a pricing URL — open the task, post to the deal channel, page the PMM.
• Anything else — log it and move on.

Two headers matter. X-Webhook-Signature is sha256= followed by an HMAC-SHA256 of the raw body, keyed with the monitor's webhook_secret (returned on create and on a single-monitor GET, never in a list) — verify it before you parse. X-Webhook-ID is stable across retries, and there are up to four attempts, so deduplicate on it or one competitor price change becomes four CRM tasks.

What does watching a competitor cost?

There is no per-monitor fee, and creating, listing or pausing a monitor is free. Each check holds 3 credits per target — the track_changes price — and keeps them for targets that come back new, changed or unchanged. Targets that came back blocked or error are not charged, so a Cloudflare wall costs nothing.

Every monitor reports its own estimated_credits_per_month, which is the ceiling — the figure assumes every target is fetched and compared on every run. Five competitor pages checked each morning is 450 credits a month, so the free tier's one-time 1,000 credits cover the first two months of it.

What won't it do?

It reads pages you point it at. It holds no contact records, no firmographics and no org charts, so it does not replace the database your enrichment runs on — it is the monitoring layer above your own sources. Two limits are worth knowing before you wire it in: robots.txt is always respected with no override, so a disallowed path reports error with ROBOTS_DISALLOWED on every check and is never charged, and consecutive runs must be at least five minutes apart.

If you would rather have Claude read and summarise each diff than route it in a webhook handler, that loop is a different post, and track_changes is the same primitive to call by hand.

Start free with 1,000 credits, or create the first monitor from the dashboard and read a real payload before you write any code.