CrawlForge
Advanced Tool5 credits

scrape_with_actions

Execute browser action chains including click, scroll, type, and form auto-fill with screenshot capture. Perfect for login flows, infinite scroll, modal dialogs, and complex JavaScript-heavy sites.

Use Cases

Login Flows

Automate login forms and access authenticated content behind login walls

Infinite Scroll

Scrape content from infinite scroll pages like social media feeds and product listings

Modal Dialogs

Interact with popups, modals, and dynamic overlays

JavaScript-Heavy Sites

Handle SPAs and sites with dynamic content loading via AJAX

Multi-Step Forms

Navigate through multi-step wizards and complex form submissions

Visual Testing

Capture screenshots at each step for debugging and visual regression testing

Endpoint

POST/api/v1/tools/scrape_with_actions
Auth Required
2 req/s on Free plan
5 credits

Parameters

NameTypeRequiredDefaultDescription
url
stringRequired-
The starting URL for the scraping session
Example: https://example.com/login
actions
arrayRequired-
Array of browser actions to perform sequentially
Example: [{"type": "wait", "value": 2000}, {"type": "click", "selector": "#submit"}]
formAutoFill
objectOptional-
Auto-fill form fields with specified values
Example: {"#username": "user@example.com", "#password": "secret"}
captureScreenshots
booleanOptionalfalse
Capture screenshots after each action
Example: true
waitForNavigation
booleanOptionaltrue
Wait for navigation to complete after click actions
Example: true

Available Action Types

wait
Wait for specified milliseconds
action.jsonJson
click
Click an element by CSS selector
action.jsonJson
type
Type text into an input field
action.jsonJson
press
Press a keyboard key
action.jsonJson
scroll
Scroll to element or position
action.jsonJson
screenshot
Capture screenshot
action.jsonJson
executeJavaScript
Execute custom JavaScript
action.jsonJson

Request Examples

terminalBash

Response Example

200 OK8450ms
{
  "success": true,
  "data": {
    "content": "...final page content after all actions...",
    "screenshots": [
      "data:image/png;base64,iVBORw0KGgoAAAANS...",
      "data:image/png;base64,iVBORw0KGgoAAAANS..."
    ],
    "intermediateStates": [
      {
        "action": "type",
        "selector": "#search-input",
        "success": true,
        "timestamp": 1705334523000
      },
      {
        "action": "click",
        "selector": "#submit",
        "success": true,
        "timestamp": 1705334524500
      }
    ],
    "finalState": {
      "url": "https://example.com/search?q=AI+web+scraping",
      "title": "Search Results - AI web scraping",
      "cookies": 12,
      "localStorage": 5
    }
  },
  "credits_used": 5,
  "credits_remaining": 995,
  "processing_time": 8450
}
Field Descriptions
data.contentFinal HTML content after executing all actions
data.screenshotsArray of base64-encoded screenshots (if captureScreenshots: true)
data.intermediateStatesLog of each action with success status and timestamp
data.finalStateFinal browser state including URL, cookies, and localStorage
credits_usedCredits deducted for this request (5 per scrape)
processing_timeTotal time including all actions and waits

Error Handling

Action Failed (422 Unprocessable Entity)

One or more actions failed to execute. Check that selectors are correct and elements are visible.

Timeout (504 Gateway Timeout)

The action chain took too long to complete (max 30s). Reduce wait times or number of actions.

Invalid Action (400 Bad Request)

One or more actions have invalid parameters. Check action type and required fields.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits (need 5). Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Credit Cost

5 credits
5 credits per request
Each successful scrape_with_actions request costs 5 credits, regardless of the number of actions.

Free Plan: 1,000 credits/month = 200 action chains

Hobby Plan: 5,000 credits/month = 1,000 action chains ($19/mo)

Professional Plan: 50,000 credits/month = 10,000 action chains ($99/mo)

Business Plan: 250,000 credits/month = 50,000 action chains ($399/mo)

Related Tools

form_submit
Simplified form submission without action chains (3 credits)
screenshot
Capture screenshots without actions (2 credits)