AI-PoweredAutonomous8 credits

agent

Autonomous research and extraction from a natural-language prompt — no URLs required. The agent plans its own steps, finds and reads its own sources, and shapes an answer under hard safety stops that the orchestrator enforces, never the model.

Use Cases

Open-Ended Research

Answer questions that span sites you haven't identified yet — the agent discovers its own sources instead of taking a URL list.

Competitive Snapshots

Ask for a competitor's current pricing tiers or feature set and get a synthesized answer rather than a pile of raw HTML.

Bounded Autonomy

max_steps, max_urls, and max_seconds are enforced outside the model, so a run can never exceed the budget you set.

Machine-Readable Answers

Set output_format to json when the result feeds a downstream system rather than a human reader.

Endpoint

POST/api/v1/tools/agent

Auth Required

1 req/s on Free plan

8 credits

Parameters

The three max_* limits are hard stops enforced by the orchestrator, not suggestions passed to the model. A run that hits one returns whatever it has gathered so far, with stop_reason explaining which limit fired.

Name	Type	Required	Default	Description
prompt	string	Required	-	Natural-language description of what you need. Must be at least 10 characters. Example: Find the current pricing tiers for the top 3 MCP web-scraping providers
max_steps	number	Optional	10	Hard cap on planning and execution steps (1-50). Example: 10
max_urls	number	Optional	20	Hard cap on how many URLs the agent may visit (1-100). Example: 20
max_seconds	number	Optional	120	Hard wall-clock limit for the whole run, in seconds (10-600). Example: 120
output_format	string	Optional	"markdown"	Shape of the answer: `text`, `json`, or `markdown`. Example: markdown

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/agent \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Find the current pricing tiers for the top 3 MCP web-scraping providers",
    "max_steps": 10,
    "max_urls": 20,
    "max_seconds": 120,
    "output_format": "markdown"
  }'

TypeScript

agent.tsTypescript

const response = await fetch('https://crawlforge.dev/api/v1/tools/agent', {
  method: 'POST',
  headers: {
    'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    prompt: 'Find the current pricing tiers for the top 3 MCP web-scraping providers',
    max_steps: 10,
    max_urls: 20,
    max_seconds: 120,
    output_format: 'markdown',
  }),
});

const data = await response.json();

if (data.success) {
  console.log('Answer:', data.data.answer);
  console.log('Steps taken:', data.data.steps_taken);
  console.log('Sources read:', data.data.urls_visited);

  // Anything other than 'completed' means a hard limit stopped the run early.
  if (data.data.stop_reason !== 'completed') {
    console.warn('Stopped early:', data.data.stop_reason);
  }

  console.log('Credits used:', data.credits_used);
  console.log('Credits remaining:', data.credits_remaining);
} else {
  console.error('Error:', data.error);
}

Python

agent.pyPython

import requests
import os

response = requests.post(
    'https://crawlforge.dev/api/v1/tools/agent',
    headers={
        'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
        'Content-Type': 'application/json',
    },
    json={
        'prompt': 'Find the current pricing tiers for the top 3 MCP web-scraping providers',
        'max_steps': 10,
        'max_urls': 20,
        'max_seconds': 120,
        'output_format': 'markdown'
    }
)

data = response.json()

if data['success']:
    print(f"Answer: {data['data']['answer']}")
    print(f"Steps taken: {data['data']['steps_taken']}")
    print(f"Sources read: {data['data']['urls_visited']}")

    # Anything other than 'completed' means a hard limit stopped the run early.
    if data['data']['stop_reason'] != 'completed':
        print(f"Stopped early: {data['data']['stop_reason']}")

    print(f"Credits used: {data['credits_used']}")
    print(f"Credits remaining: {data['credits_remaining']}")
else:
    print(f"Error: {data['error']}")

Response Example

200 OK8420ms

{
  "success": true,
  "data": {
    "prompt": "Find the current pricing tiers for the top 3 MCP web-scraping providers",
    "answer": "## Pricing comparison\n\n- **CrawlForge** — Free (1,000 credits), Hobby $19/mo, Professional $99/mo...",
    "output_format": "markdown",
    "steps_taken": 6,
    "urls_visited": [
      "https://example.com/pricing",
      "https://example.org/plans"
    ],
    "limits": {
      "max_steps": 10,
      "max_urls": 20,
      "max_seconds": 120
    },
    "stop_reason": "completed"
  },
  "credits_used": 8,
  "credits_remaining": 992,
  "processing_time": 8420
}

Field Descriptions

data.answerThe synthesized answer, rendered in the requested `output_format`

data.steps_takenHow many planning/execution steps the run actually used

data.urls_visitedEvery URL the agent read while answering — use it to audit sources

data.limitsEchoes the hard stops that were in force for this run

data.stop_reason`completed` when the agent finished on its own; otherwise the limit that stopped it

credits_usedCredits deducted for this run (8 per run, regardless of steps taken)

credits_remainingYour remaining credit balance

Error Handling

Invalid Input (400 Bad Request)

The prompt is shorter than 10 characters, or a max_* value is outside its allowed range (max_steps 1-50, max_urls 1-100, max_seconds 10-600).

Agent Run Failed (500 Internal Server Error)

The run could not be completed. Credits are not deducted for a failed run — retry with a narrower prompt or a smaller max_steps.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits. Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Pro Tip: A run that returns stop_reason other than completed still costs 8 credits. Start with a tight max_seconds while you tune the prompt, then raise it once the agent reliably finishes on its own.

Credit Cost

8 credits

8 credits per run

Each agent run costs a flat 8 credits regardless of how many steps it takes or how many URLs it visits — the limits you set cap wall-clock time, not price.

Free Plan: 1,000 one-time credits = 125 runs

Hobby Plan: 5,000 credits/month = 625 runs ($19/mo)

Professional Plan: 50,000 credits/month = 6,250 runs ($99/mo)

Business Plan: 250,000 credits/month = 31,250 runs ($399/mo)

Related Tools

deep_research

Multi-stage research with source verification and synthesis (10 credits)

search_web

Structured web search when you want to pick the sources yourself (5 credits)

extract_with_llm

LLM extraction from a page you already have (3 credits)

scrape

Multi-format extraction from a single known URL (2 credits)

Ready to try agent? Sign up for free and get 1,000 credits to start building.

Use Cases

Open-Ended Research

Answer questions that span sites you haven't identified yet — the agent discovers its own sources instead of taking a URL list.

Competitive Snapshots

Ask for a competitor's current pricing tiers or feature set and get a synthesized answer rather than a pile of raw HTML.

Bounded Autonomy

max_steps, max_urls, and max_seconds are enforced outside the model, so a run can never exceed the budget you set.

Machine-Readable Answers

Set output_format to json when the result feeds a downstream system rather than a human reader.

Parameters

Name	Type	Required	Default	Description
prompt	string	Required	-	Natural-language description of what you need. Must be at least 10 characters. Example: Find the current pricing tiers for the top 3 MCP web-scraping providers
max_steps	number	Optional	10	Hard cap on planning and execution steps (1-50). Example: 10
max_urls	number	Optional	20	Hard cap on how many URLs the agent may visit (1-100). Example: 20
max_seconds	number	Optional	120	Hard wall-clock limit for the whole run, in seconds (10-600). Example: 120
output_format	string	Optional	"markdown"	Shape of the answer: `text`, `json`, or `markdown`. Example: markdown

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/agent \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Find the current pricing tiers for the top 3 MCP web-scraping providers",
    "max_steps": 10,
    "max_urls": 20,
    "max_seconds": 120,
    "output_format": "markdown"
  }'

TypeScript

agent.tsTypescript

const response = await fetch('https://crawlforge.dev/api/v1/tools/agent', {
  method: 'POST',
  headers: {
    'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    prompt: 'Find the current pricing tiers for the top 3 MCP web-scraping providers',
    max_steps: 10,
    max_urls: 20,
    max_seconds: 120,
    output_format: 'markdown',
  }),
});

const data = await response.json();

if (data.success) {
  console.log('Answer:', data.data.answer);
  console.log('Steps taken:', data.data.steps_taken);
  console.log('Sources read:', data.data.urls_visited);

  // Anything other than 'completed' means a hard limit stopped the run early.
  if (data.data.stop_reason !== 'completed') {
    console.warn('Stopped early:', data.data.stop_reason);
  }

  console.log('Credits used:', data.credits_used);
  console.log('Credits remaining:', data.credits_remaining);
} else {
  console.error('Error:', data.error);
}

Python

agent.pyPython

import requests
import os

response = requests.post(
    'https://crawlforge.dev/api/v1/tools/agent',
    headers={
        'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
        'Content-Type': 'application/json',
    },
    json={
        'prompt': 'Find the current pricing tiers for the top 3 MCP web-scraping providers',
        'max_steps': 10,
        'max_urls': 20,
        'max_seconds': 120,
        'output_format': 'markdown'
    }
)

data = response.json()

if data['success']:
    print(f"Answer: {data['data']['answer']}")
    print(f"Steps taken: {data['data']['steps_taken']}")
    print(f"Sources read: {data['data']['urls_visited']}")

    # Anything other than 'completed' means a hard limit stopped the run early.
    if data['data']['stop_reason'] != 'completed':
        print(f"Stopped early: {data['data']['stop_reason']}")

    print(f"Credits used: {data['credits_used']}")
    print(f"Credits remaining: {data['credits_remaining']}")
else:
    print(f"Error: {data['error']}")

Response Example

200 OK8420ms

{
  "success": true,
  "data": {
    "prompt": "Find the current pricing tiers for the top 3 MCP web-scraping providers",
    "answer": "## Pricing comparison\n\n- **CrawlForge** — Free (1,000 credits), Hobby $19/mo, Professional $99/mo...",
    "output_format": "markdown",
    "steps_taken": 6,
    "urls_visited": [
      "https://example.com/pricing",
      "https://example.org/plans"
    ],
    "limits": {
      "max_steps": 10,
      "max_urls": 20,
      "max_seconds": 120
    },
    "stop_reason": "completed"
  },
  "credits_used": 8,
  "credits_remaining": 992,
  "processing_time": 8420
}

Field Descriptions

data.answerThe synthesized answer, rendered in the requested `output_format`

data.steps_takenHow many planning/execution steps the run actually used

data.urls_visitedEvery URL the agent read while answering — use it to audit sources

data.limitsEchoes the hard stops that were in force for this run

data.stop_reason`completed` when the agent finished on its own; otherwise the limit that stopped it

credits_usedCredits deducted for this run (8 per run, regardless of steps taken)

credits_remainingYour remaining credit balance

Error Handling

Invalid Input (400 Bad Request)

The prompt is shorter than 10 characters, or a max_* value is outside its allowed range (max_steps 1-50, max_urls 1-100, max_seconds 10-600).

Agent Run Failed (500 Internal Server Error)

The run could not be completed. Credits are not deducted for a failed run — retry with a narrower prompt or a smaller max_steps.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits. Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Credit Cost

8 credits

8 credits per run

Each agent run costs a flat 8 credits regardless of how many steps it takes or how many URLs it visits — the limits you set cap wall-clock time, not price.

Free Plan: 1,000 one-time credits = 125 runs

Hobby Plan: 5,000 credits/month = 625 runs ($19/mo)

Professional Plan: 50,000 credits/month = 6,250 runs ($99/mo)

Business Plan: 250,000 credits/month = 31,250 runs ($399/mo)