MonitoringAdvanced3 credits

track_changes

Store a snapshot of a page, then compare the live page against it whenever you want. You get back whether anything changed, how much, which lines were added or removed, and a structural similarity score telling you whether the markup itself was rebuilt. Or pass operation: "monitor" and CrawlForge checks the page on a schedule for you.

Use Cases

Detect a scraper about to break

A low structural_similarity means the page's markup was rebuilt, which is what actually breaks selectors — catch it before your extraction pipeline starts returning empty results.

Watch competitor pricing pages

Scope to the pricing table with selector, compare on your own schedule, and read the added and removed lines to see what moved.

Track legal and policy documents

Compare Terms of Service, privacy policies, or regulatory pages and keep an auditable record of exactly which lines changed.

Catch breaking changes in API docs

Baseline a vendor's reference or changelog page and compare before each of your releases.

Verify a deploy changed only what you expected

Baseline a page before shipping, compare after, and confirm the diff matches the intended change.

Endpoint

POST/api/v1/tools/track_changes

Auth Required

1 req/s on Free plan

3 credits

Parameters

This endpoint takes eight parameters; schedule, notify_emails and webhook_url apply only to operation: "monitor". Older examples passed trackingOptions, monitoringOptions and storageOptions objects — the hosted REST API accepts those keys and ignores them, so sending them changes nothing. The equivalent controls exist on the CrawlForge MCP server.

Name	Type	Required	Default	Description
url	string	Required	-	The page to capture or compare. Example: https://competitor.com/pricing
operation	string	Optional	compare	One of `"create_baseline"`, `"compare"` or `"monitor"`. `monitor` creates a [hosted monitor](/docs/api-reference/monitors) for the URL and charges nothing for the call. Any other value is rejected with 400. Example: compare
selector	string	Optional	-	CSS selector scoping tracking to part of the page, e.g. `.pricing-table`. Baselines are stored per (url, selector) pair, so the same URL can be tracked at several scopes independently. Returns 422 if the selector matches nothing. Example: .pricing-table
update_baseline	boolean	Optional	false	`compare` only. After diffing, overwrite the stored baseline with the content just fetched. Use this for rolling comparison, where each call reports the change since the previous call rather than since the original capture. Example: false
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Left at `true`, a path disallowed for `CrawlForge` is refused with 403 before anything is fetched and no credits are charged. Set it to `false` only for a target you have your own agreement with — the response then carries a `warnings` entry and the override is recorded against your API key. A hosted monitor always respects robots.txt. Example: true
schedule	string	Optional	0 * * * *	`monitor` only. Five-field cron expression the hosted monitor runs on, evaluated in UTC. Consecutive runs must be at least 5 minutes apart, or the call is rejected with 400. Example: /15 * * *
notify_emails	string[]	Optional	-	`monitor` only. Up to 5 addresses emailed when a check finds new, changed, blocked or errored pages. Example: ["alerts@example.com"]
webhook_url	string	Optional	-	`monitor` only. Public `https` URL that receives the signed `monitor.page` and `monitor.check.completed` events. A `webhook_secret` is generated and returned with the monitor. Example: https://example.com/hooks/crawlforge

Operations

Create a baseline once, then compare against it as often as you like — or hand the page to a hosted monitor.

create_baseline

Fetches the page, reduces it to normalized visible text, and stores it with a content hash and a structural signature. Baselines are kept 90 days per API account. Run this first — comparing without one returns 404.

compare

Fetches the page again and diffs it line by line against the stored baseline. Returns the change percentage, structural similarity, counts, and samples of the added and removed lines. This is the default operation.

monitor

Creates a hosted monitor named after the URL's host, with this one target, the schedule (hourly by default), timezone UTC and 30 days of retention, and returns it under data.monitor — webhook_secret included. The call itself costs nothing; each scheduled check bills 3 credits per compared target, and blocked or errored targets are free. Manage it through the monitors API or the dashboard.

Local or hosted

A scheduled monitor runs in one of two places, and where it runs decides who has to keep a process alive and how you get told.

On your machine — the MCP server

operation: "create_scheduled_monitor" on the CrawlForge MCP server stores the monitor under ~/.crawlforge/monitors and fires it in-process, so checks run only while the server process is alive; missed runs are caught up on restart, and crawlforge monitor:run-due from system cron is the guaranteed path. It notifies by webhook or Slack from your machine and never sends email. Each local check you run costs 3 credits.

Hosted — CrawlForge's scheduler

Created with operation: "monitor" here, through /api/v1/monitors, or from the MCP server with hosted: true. CrawlForge's own scheduler fetches, diffs and records every check without any process of yours, and notifies by email and signed webhook. 3 credits per compared target per check, with blocked and errored targets free. Manage it from the dashboard.

Reading a comparison

The two scores answer different questions, and the combination is more useful than either alone.

change_percent — how much text moved

Added plus removed lines as a share of the larger document, 0-100. High on any real content update; also high on pages carrying timestamps, view counters or rotating testimonials, so expect a noisy floor on dynamic pages.

structural_similarity — is the markup still the same

0-1 score comparing tag vocabulary and nesting depth. High change_percent with high structural_similarity is the same layout with new copy. A low score means the page was rebuilt, and that is what breaks selectors.

Detection is text-based: the page is fetched over HTTP and JavaScript is never executed, so content rendered client-side is invisible to this endpoint. For checks that run on a schedule without your own process, with webhooks and a check history, create a hosted monitor. For browser-rendered tracking, use the CrawlForge MCP server, which can also hand a monitor to the hosted scheduler with hosted: true.

structural_similarity is null, not 0, when the stored baseline predates the field. Zero is a real score meaning nothing structural survived, so an unmeasured comparison reports null instead. Re-run create_baseline, or pass update_baseline once, to start scoring it.

Request Examples

terminalBash

# Step 1: capture the baseline (once per url + selector, kept 90 days)
curl -X POST https://crawlforge.dev/api/v1/tools/track_changes \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.com/pricing",
    "operation": "create_baseline",
    "selector": ".pricing-table"
  }'

# Step 2: compare against it, as often as you like
curl -X POST https://crawlforge.dev/api/v1/tools/track_changes \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.com/pricing",
    "operation": "compare",
    "selector": ".pricing-table"
  }'

# Rolling comparison: diff against the previous call, not the original capture
curl -X POST https://crawlforge.dev/api/v1/tools/track_changes \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.com/pricing",
    "operation": "compare",
    "selector": ".pricing-table",
    "update_baseline": true
  }'

# Hosted monitor: CrawlForge checks the page every 15 minutes on its own
# scheduler and posts signed webhooks. Creating it costs nothing; each check
# bills 3 credits per compared target.
curl -X POST https://crawlforge.dev/api/v1/tools/track_changes \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://competitor.com/pricing",
    "operation": "monitor",
    "selector": ".pricing-table",
    "schedule": "*/15 * * * *",
    "webhook_url": "https://example.com/hooks/crawlforge"
  }'

Response Example

200 OK1,180ms

{
  "success": true,
  "data": {
    "operation": "compare",
    "url": "https://competitor.com/pricing",
    "selector": ".pricing-table",
    "changed": true,
    "change_percent": 12.5,
    "structural_similarity": 0.9713,
    "added_count": 4,
    "removed_count": 2,
    "added_samples": [
      "Pro $79 / month",
      "Includes 50,000 credits",
      "Enterprise",
      "Contact sales"
    ],
    "removed_samples": [
      "Pro $99 / month",
      "Includes 25,000 credits"
    ],
    "baseline_captured_at": "2026-08-01T12:00:00.000Z",
    "compared_at": "2026-08-26T14:30:00.000Z",
    "baseline_updated": false
  },
  "credits_used": 3,
  "credits_remaining": 997,
  "processing_time": 1180
}

Field Descriptions

data.changedTrue when the content hash differs from the baseline — the fastest check if you only need a yes or no.

data.change_percentAdded plus removed lines as a percentage of the larger document, 0-100.

data.structural_similarity0-1 markup similarity. Null when the baseline predates this field.

data.added_samplesUp to 20 added lines, each truncated to 500 characters. Not the full diff — use added_count for the true total.

data.removed_samplesUp to 20 removed lines, same caps as added_samples.

data.baseline_updatedWhether this call overwrote the baseline, echoing the update_baseline parameter.

credits_used3 credits, charged per successful call. Failed calls are not charged.

Error Handling

No baseline found (404 BASELINE_NOT_FOUND)

You called compare before storing a baseline, or the 90-day baseline expired. Run create_baseline for this exact (url, selector) pair first.

Selector matched nothing (422 SELECTOR_NOT_FOUND)

The CSS selector returned no elements. Check it against the live markup — a selector that works in devtools after JavaScript runs may not exist in the raw HTML this endpoint fetches.

Target unreachable (502 FETCH_FAILED)

The page returned a non-2xx status. Sites behind bot protection commonly land here — fetch them with stealth_mode instead.

Invalid parameters (400 VALIDATION_ERROR)

Malformed URL, or an operation outside create_baseline, compare and monitor. The response details array names the failing field. A schedule that is not a five-field cron expression, or that fires more often than every 5 minutes, is refused with the same code.

Storage unavailable (503 STORAGE_UNAVAILABLE)

Baseline storage could not be reached. Retry — no credits are charged for failed calls.

Blocked by robots.txt (403 Forbidden)

The target site's robots.txt disallows this path for CrawlForge. Set respect_robots: false to override if you have your own agreement with the target — the override is recorded against your API key. The override does not reach a host on CrawlForge's permanent opt-out list, which is refused whatever respect_robots is set to.

Reducing false positives:

Scope with selector rather than tracking whole pages. A full-page baseline picks up nav, footer, cookie banners and rotating content, which is usually what produces a change on every single comparison.

Credit Cost

3 credits

3 credits per call

Both create_baseline and compare cost 3 credits. Failed calls are not charged. monitor costs nothing to call; the hosted monitor it creates bills 3 credits per compared target on every scheduled check, with blocked and errored targets free. With compare on your own schedule, your comparison frequency — and therefore your cost — is entirely under your control.

Cost Breakdown:

create_baseline: 3 credits, once per (url, selector) pair, valid 90 days

compare: 3 credits per call

monitor: 0 credits to create; 3 credits per compared target on each scheduled check

Polling Cost Example, per URL:

Hourly: 24 calls/day = 72 credits/day

Every 6 hours: 4 calls/day = 12 credits/day

Daily: 1 call/day = 3 credits/day

Plan Recommendations:

Free Plan: 1,000 one-time trial credits = about 5 URLs compared every 6 hours for a month

Hobby Plan: 5,000 credits/mo = about 13 URLs compared every 6 hours ($19/mo)

Professional Plan: 100,000 credits/mo = about 276 URLs compared every 6 hours ($99/mo)

Related Tools

/api/v1/monitors

Create, pause, run and inspect hosted monitors with several targets, and verify their webhooks

fetch_url

Raw HTTP fetch when you want to diff content yourself (1 credit)

extract_content

Pull main content out before comparing, for noisy pages (2 credits)

stealth_mode

Reach pages behind bot protection that return 502 here (5 credits)

batch_scrape

Fetch many URLs in one call (5 credits per URL)

Ready to try track_changes? Sign up for free and get 1,000 credits to baseline your first pages.