scrape_with_actions

Execute browser action chains including click, scroll, type, and form auto-fill with screenshot capture. Perfect for login flows, infinite scroll, modal dialogs, and complex JavaScript-heavy sites.

Use Cases

Automate login forms and access authenticated content behind login walls

Infinite Scroll

Scrape content from infinite scroll pages like social media feeds and product listings

Modal Dialogs

Interact with popups, modals, and dynamic overlays

JavaScript-Heavy Sites

Handle SPAs and sites with dynamic content loading via AJAX

Multi-Step Forms

Navigate through multi-step wizards and complex form submissions

Visual Testing

Capture screenshots at each step for debugging and visual regression testing

Endpoint

POST/api/v1/tools/scrape_with_actions

Auth Required

1 req/s on Free plan

5 credits

Parameters

Name	Type	Required	Default	Description
url	string	Required	-	The URL to load before running the action chain Example: https://app.example.com/dashboard
actions	array	Required	-	Ordered list of 1-20 browser actions to run before scraping. Each item is an object with a `type` of `wait`, `click`, `type`, `press`, `scroll`, `screenshot`, `executeJavaScript`, `select`, `hover`, or `navigate`, plus type-specific fields: `selector` (CSS target), `text` (for `type`), `key` (for `press`), `script` (for `executeJavaScript`), `duration`/`condition` (for `wait`), `button`/`clickCount`/`delay` (for `click`), `direction`/`distance`/`toElement` (for `scroll`), `fullPage`/`quality`/`format` (for `screenshot`), `value` or `values` (for `select`), and `url`/`waitUntil` (for `navigate`). `select` takes either a single `value` or a `values` array, and a plain string matches an option by its value or its visible label; `hover` needs `selector`, optionally with `force` and `position`. Optional on any action: `timeout` (per action, default 10000ms — distinct from `browserOptions.timeout`, which budgets the whole chain), `description`, `continueOnError` (default false), `retries` (0-5, default 1), and `captureAfter` (default false). Example: [{"type": "click", "selector": "#login"}, {"type": "type", "selector": "#email", "text": "user@example.com"}, {"type": "wait", "duration": 1000}, {"type": "screenshot"}]
formats	array	Optional	["json"]	Output formats to return: `markdown`, `html`, `json`, `text`, or `screenshots` Example: ["markdown", "screenshots"]
captureScreenshots	boolean	Optional	true	Take screenshots during action execution Example: true
formAutoFill	object	Optional	-	Fill and submit a form in one step. Shape: `{ fields: [{ selector, value, type: text\|select\|checkbox\|radio\|file, waitAfter }], submitSelector, waitAfterSubmit }` — `waitAfterSubmit` defaults to 2000ms. Example: {"fields": [{"selector": "#email", "value": "user@example.com", "type": "text"}], "submitSelector": "#login"}
browserOptions	object	Optional	-	Browser configuration: `headless` (default true), `userAgent`, `viewportWidth` (default 1280, range 800-1920), `viewportHeight` (default 720, range 600-1080), `timeout` (default 30000ms, range 10000-120000), and `stealth` (default false). Keep `timeout` within the ~25s REST window. Set `stealth` to true to run the action chain in the stealth Chromium engine at its `medium` profile instead of the standard browser pool — the boolean is the only knob, so the level, fingerprint randomization and engine choice are not settable from here. It is slower to start, and it renders JavaScript rather than solving challenges. Example: {"viewportWidth": 1440, "viewportHeight": 900, "timeout": 20000}
extractionOptions	object	Optional	-	Content extraction options: `selectors` (a CSS key→value map of data to pull out), `includeMetadata` (default true), `includeLinks` (default true), and `includeImages` (default true). Example: {"selectors": {"title": "h1", "price": ".price"}}
continueOnActionError	boolean	Optional	false	Keep running the remaining actions when one fails instead of aborting the chain Example: false
maxRetries	number	Optional	1	Maximum retry attempts for the overall run on failure (0-3) Example: 1
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Omit it and the compliant default (`true`) applies: a URL disallowed for `CrawlForge` is refused before the browser opens and this endpoint charges nothing for it, and every `navigate` action is checked the same way. Set it to `false` only for a target you have your own agreement with — the override is recorded against your API key. Example: true

Available Action Types

wait

Wait for specified milliseconds

action.jsonJson

{"type": "wait", "duration": 2000}

click

Click an element by CSS selector

action.jsonJson

{"type": "click", "selector": "#button"}

type

Type text into an input field

action.jsonJson

{"type": "type", "selector": "#search", "text": "query"}

press

Press a keyboard key

action.jsonJson

{"type": "press", "key": "Enter"}

scroll

Scroll to element or position

action.jsonJson

{"type": "scroll", "toElement": "#content"}

screenshot

Capture screenshot

action.jsonJson

{"type": "screenshot"}

executeJavaScript

Execute custom JavaScript

action.jsonJson

{"type": "executeJavaScript", "script": "window.scrollTo(0, 0)"}

select

Choose options in a <select>: selector names the dropdown, then either value (one option) or values (several). A plain string matches an option by its value or its visible label.

hover

Move the pointer over the element named by selector — for menus and tooltips that only appear on hover. force and position behave as they do for click.

navigate

Load another url in the same browser session, so cookies, localStorage and login state from earlier actions carry over. Optional waitUntil is load, domcontentloaded (the default), networkidle, or commit. The new URL passes the same robots.txt and SSRF checks as the initial one, so it is not a way around the gate.

Request Examples

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/scrape_with_actions \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com/dashboard",
    "actions": [
      {"type": "click", "selector": "#login"},
      {"type": "type", "selector": "#email", "text": "user@example.com"},
      {"type": "type", "selector": "#password", "text": "secret123"},
      {"type": "wait", "duration": 1000},
      {"type": "screenshot"}
    ],
    "formats": ["markdown", "screenshots"]
  }'

Response Example

200 OK9000ms

{
  "success": true,
  "data": {
    "url": "https://app.example.com/dashboard",
    "actionsExecuted": 4,
    "results": {
      "markdown": "# Dashboard\n\nWelcome back — you are now signed in...",
      "screenshots": [
        "data:image/png;base64,iVBORw0KGgoAAAANS...(truncated)"
      ]
    },
    "actionLog": [
      {
        "type": "click",
        "selector": "#login",
        "success": true
      },
      {
        "type": "type",
        "selector": "#email",
        "success": true
      },
      {
        "type": "wait",
        "duration": 1000,
        "success": true
      },
      {
        "type": "screenshot",
        "success": true
      }
    ]
  },
  "credits_used": 5,
  "credits_remaining": 995,
  "processing_time": 9000
}

Field Descriptions

data.urlThe URL that was loaded before the action chain ran

data.actionsExecutedNumber of actions that ran successfully

data.results.markdownPage content in each requested format after all actions completed

data.results.screenshotsBase64-encoded screenshots captured during the run (data:image/png;base64,…)

data.actionLogPer-action log with type, selector, and success status

credits_usedCredits deducted for this request (5 per scrape)

processing_timeTotal time in ms including all actions and waits

Error Handling

Browser runtime not configured (503 TOOL_NOT_AVAILABLE)

This tool needs a browser automation runtime. When the hosted execution backend is not configured the call returns 503 immediately and no credits are charged.

Execution backend timed out (504 MCP_UPSTREAM_TIMEOUT)

Navigation plus the action chain exceeded the execution backend's time budget. Shorten waits, reduce the number of actions, or retry. Failures are not charged.

Blocked by robots.txt (502 TOOL_ERROR)

The target's robots.txt disallows this URL for CrawlForge, so no browser was launched and nothing was charged. Every navigate action is checked the same way, not just the initial URL. Set respect_robots: false to override if you have your own agreement with the target — the override is recorded against your API key.

Invalid Action (400 Bad Request)

One or more actions have invalid parameters. Check action type and required fields.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits (need 5). Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Pro Tip: Use wait actions between interactions to ensure JavaScript has time to execute. For debugging, enable captureScreenshots to see what the browser sees at each step.

Credit Cost

5 credits

5 credits per request

Each successful scrape_with_actions request costs 5 credits, regardless of the number of actions.

Free Plan: 1,000 one-time trial credits = 200 action chains

Hobby Plan: 5,000 credits/month = 1,000 action chains ($19/mo)

Professional Plan: 50,000 credits/month = 10,000 action chains ($99/mo)

Business Plan: 250,000 credits/month = 50,000 action chains ($399/mo)

Related Tools

form_submit

Simplified form submission without action chains (3 credits)

screenshot

Capture screenshots without actions (2 credits)

Ready to try scrape_with_actions? Sign up for free and get 1,000 credits to start building.

Use Cases

Automate login forms and access authenticated content behind login walls

Infinite Scroll

Scrape content from infinite scroll pages like social media feeds and product listings

Modal Dialogs

Interact with popups, modals, and dynamic overlays

JavaScript-Heavy Sites

Handle SPAs and sites with dynamic content loading via AJAX

Multi-Step Forms

Navigate through multi-step wizards and complex form submissions

Visual Testing

Capture screenshots at each step for debugging and visual regression testing

Parameters

Name	Type	Required	Default	Description
url	string	Required	-	The URL to load before running the action chain Example: https://app.example.com/dashboard
actions	array	Required	-	Ordered list of 1-20 browser actions to run before scraping. Each item is an object with a `type` of `wait`, `click`, `type`, `press`, `scroll`, `screenshot`, `executeJavaScript`, `select`, `hover`, or `navigate`, plus type-specific fields: `selector` (CSS target), `text` (for `type`), `key` (for `press`), `script` (for `executeJavaScript`), `duration`/`condition` (for `wait`), `button`/`clickCount`/`delay` (for `click`), `direction`/`distance`/`toElement` (for `scroll`), `fullPage`/`quality`/`format` (for `screenshot`), `value` or `values` (for `select`), and `url`/`waitUntil` (for `navigate`). `select` takes either a single `value` or a `values` array, and a plain string matches an option by its value or its visible label; `hover` needs `selector`, optionally with `force` and `position`. Optional on any action: `timeout` (per action, default 10000ms — distinct from `browserOptions.timeout`, which budgets the whole chain), `description`, `continueOnError` (default false), `retries` (0-5, default 1), and `captureAfter` (default false). Example: [{"type": "click", "selector": "#login"}, {"type": "type", "selector": "#email", "text": "user@example.com"}, {"type": "wait", "duration": 1000}, {"type": "screenshot"}]
formats	array	Optional	["json"]	Output formats to return: `markdown`, `html`, `json`, `text`, or `screenshots` Example: ["markdown", "screenshots"]
captureScreenshots	boolean	Optional	true	Take screenshots during action execution Example: true
formAutoFill	object	Optional	-	Fill and submit a form in one step. Shape: `{ fields: [{ selector, value, type: text\|select\|checkbox\|radio\|file, waitAfter }], submitSelector, waitAfterSubmit }` — `waitAfterSubmit` defaults to 2000ms. Example: {"fields": [{"selector": "#email", "value": "user@example.com", "type": "text"}], "submitSelector": "#login"}
browserOptions	object	Optional	-	Browser configuration: `headless` (default true), `userAgent`, `viewportWidth` (default 1280, range 800-1920), `viewportHeight` (default 720, range 600-1080), `timeout` (default 30000ms, range 10000-120000), and `stealth` (default false). Keep `timeout` within the ~25s REST window. Set `stealth` to true to run the action chain in the stealth Chromium engine at its `medium` profile instead of the standard browser pool — the boolean is the only knob, so the level, fingerprint randomization and engine choice are not settable from here. It is slower to start, and it renders JavaScript rather than solving challenges. Example: {"viewportWidth": 1440, "viewportHeight": 900, "timeout": 20000}
extractionOptions	object	Optional	-	Content extraction options: `selectors` (a CSS key→value map of data to pull out), `includeMetadata` (default true), `includeLinks` (default true), and `includeImages` (default true). Example: {"selectors": {"title": "h1", "price": ".price"}}
continueOnActionError	boolean	Optional	false	Keep running the remaining actions when one fails instead of aborting the chain Example: false
maxRetries	number	Optional	1	Maximum retry attempts for the overall run on failure (0-3) Example: 1
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Omit it and the compliant default (`true`) applies: a URL disallowed for `CrawlForge` is refused before the browser opens and this endpoint charges nothing for it, and every `navigate` action is checked the same way. Set it to `false` only for a target you have your own agreement with — the override is recorded against your API key. Example: true

Available Action Types

wait

Wait for specified milliseconds

action.jsonJson

{"type": "wait", "duration": 2000}

click

Click an element by CSS selector

action.jsonJson

{"type": "click", "selector": "#button"}

type

Type text into an input field

action.jsonJson

{"type": "type", "selector": "#search", "text": "query"}

press

Press a keyboard key

action.jsonJson

{"type": "press", "key": "Enter"}

scroll

Scroll to element or position

action.jsonJson

{"type": "scroll", "toElement": "#content"}

screenshot

Capture screenshot

action.jsonJson

{"type": "screenshot"}

executeJavaScript

Execute custom JavaScript

action.jsonJson

{"type": "executeJavaScript", "script": "window.scrollTo(0, 0)"}

select

Choose options in a <select>: selector names the dropdown, then either value (one option) or values (several). A plain string matches an option by its value or its visible label.

hover

Move the pointer over the element named by selector — for menus and tooltips that only appear on hover. force and position behave as they do for click.

navigate

Request Examples

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/scrape_with_actions \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com/dashboard",
    "actions": [
      {"type": "click", "selector": "#login"},
      {"type": "type", "selector": "#email", "text": "user@example.com"},
      {"type": "type", "selector": "#password", "text": "secret123"},
      {"type": "wait", "duration": 1000},
      {"type": "screenshot"}
    ],
    "formats": ["markdown", "screenshots"]
  }'

Response Example

200 OK9000ms

{
  "success": true,
  "data": {
    "url": "https://app.example.com/dashboard",
    "actionsExecuted": 4,
    "results": {
      "markdown": "# Dashboard\n\nWelcome back — you are now signed in...",
      "screenshots": [
        "data:image/png;base64,iVBORw0KGgoAAAANS...(truncated)"
      ]
    },
    "actionLog": [
      {
        "type": "click",
        "selector": "#login",
        "success": true
      },
      {
        "type": "type",
        "selector": "#email",
        "success": true
      },
      {
        "type": "wait",
        "duration": 1000,
        "success": true
      },
      {
        "type": "screenshot",
        "success": true
      }
    ]
  },
  "credits_used": 5,
  "credits_remaining": 995,
  "processing_time": 9000
}

Field Descriptions

data.urlThe URL that was loaded before the action chain ran

data.actionsExecutedNumber of actions that ran successfully

data.results.markdownPage content in each requested format after all actions completed

data.results.screenshotsBase64-encoded screenshots captured during the run (data:image/png;base64,…)

data.actionLogPer-action log with type, selector, and success status

credits_usedCredits deducted for this request (5 per scrape)

processing_timeTotal time in ms including all actions and waits

Error Handling

Browser runtime not configured (503 TOOL_NOT_AVAILABLE)

This tool needs a browser automation runtime. When the hosted execution backend is not configured the call returns 503 immediately and no credits are charged.

Execution backend timed out (504 MCP_UPSTREAM_TIMEOUT)

Navigation plus the action chain exceeded the execution backend's time budget. Shorten waits, reduce the number of actions, or retry. Failures are not charged.

Blocked by robots.txt (502 TOOL_ERROR)

Invalid Action (400 Bad Request)

One or more actions have invalid parameters. Check action type and required fields.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits (need 5). Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Pro Tip: Use wait actions between interactions to ensure JavaScript has time to execute. For debugging, enable captureScreenshots to see what the browser sees at each step.

Credit Cost

5 credits

5 credits per request

Each successful scrape_with_actions request costs 5 credits, regardless of the number of actions.

Free Plan: 1,000 one-time trial credits = 200 action chains

Hobby Plan: 5,000 credits/month = 1,000 action chains ($19/mo)

Professional Plan: 50,000 credits/month = 10,000 action chains ($99/mo)

Business Plan: 250,000 credits/month = 50,000 action chains ($399/mo)