Basic ToolBatch1 credit

get_batch_results

Retrieve paginated results for a batch_scrape job. Submit the batch once, then poll this endpoint with the returned batchId to check status and page through results as they complete.

Use Cases

Poll a Running Batch

Check job status and pull the pages that have finished while the rest of the batch is still running.

Page Through Large Jobs

Walk results 25 at a time (up to 100 per page) instead of holding an entire batch in memory.

Resume After a Restart

The batchId outlives your process — a worker that crashes can pick up exactly where it left off.

Cheap Status Checks

At 1 credit per call, polling costs a fraction of re-running the batch you already paid for.

Endpoint

POST/api/v1/tools/get_batch_results

Auth Required

1 req/s on Free plan

1 credit

Parameters

batchId comes from the batch_scrape response — this tool never starts a job, it only reads one back.

Name	Type	Required	Default	Description
batchId	string	Required	-	The batch job identifier returned by `batch_scrape`. Example: batch_1700000000000_abc123def
page	number	Optional	1	Page number to retrieve (1-based). Example: 1
limit	number	Optional	25	Results per page (1-100). Example: 50

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/get_batch_results \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "batchId": "batch_1700000000000_abc123def",
    "page": 1,
    "limit": 25
  }'

TypeScript

getBatchResults.tsTypescript

async function getPage(batchId: string, page: number) {
  const response = await fetch('https://crawlforge.dev/api/v1/tools/get_batch_results', {
    method: 'POST',
    headers: {
      'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ batchId, page, limit: 25 }),
  });

  return response.json();
}

// Read total_pages from the first response, then loop — one fewer call than
// incrementing until you get an empty array.
const first = await getPage('batch_1700000000000_abc123def', 1);

if (first.success) {
  const all = [...first.data.results];

  for (let page = 2; page <= first.data.total_pages; page++) {
    const next = await getPage(first.data.batch_id, page);
    all.push(...next.data.results);
  }

  console.log(`Batch ${first.data.status}: ${all.length} of ${first.data.total} results`);
  console.log('Credits remaining:', first.credits_remaining);
} else {
  console.error('Error:', first.error);
}

Python

get_batch_results.pyPython

import requests
import os

def get_page(batch_id, page):
    response = requests.post(
        'https://crawlforge.dev/api/v1/tools/get_batch_results',
        headers={
            'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
            'Content-Type': 'application/json',
        },
        json={'batchId': batch_id, 'page': page, 'limit': 25}
    )
    return response.json()

# Read total_pages from the first response, then loop — one fewer call than
# incrementing until you get an empty list.
first = get_page('batch_1700000000000_abc123def', 1)

if first['success']:
    all_results = list(first['data']['results'])

    for page in range(2, first['data']['total_pages'] + 1):
        nxt = get_page(first['data']['batch_id'], page)
        all_results.extend(nxt['data']['results'])

    print(f"Batch {first['data']['status']}: {len(all_results)} of {first['data']['total']} results")
    print(f"Credits remaining: {first['credits_remaining']}")
else:
    print(f"Error: {first['error']}")

Response Example

200 OK96ms

{
  "success": true,
  "data": {
    "batch_id": "batch_1700000000000_abc123def",
    "status": "completed",
    "page": 1,
    "limit": 25,
    "total": 3,
    "total_pages": 1,
    "results": [
      {
        "url": "https://example.com/page-1",
        "status": "completed",
        "data": {
          "title": "Example page 1"
        }
      },
      {
        "url": "https://example.com/page-2",
        "status": "completed",
        "data": {
          "title": "Example page 2"
        }
      }
    ]
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 96
}

Field Descriptions

data.batch_idThe batch job you queried, echoed back

data.statusStatus of the batch as a whole

data.totalTotal number of results across every page

data.total_pagesHow many pages exist at the current `limit`

data.resultsOne entry per URL in this page, each with its own per-URL `status`

credits_usedCredits deducted for this request (1 per retrieval)

credits_remainingYour remaining credit balance

Error Handling

Invalid Input (400 Bad Request)

batchId is empty, page is below 1, or limit falls outside the 1-100 range.

Retrieval Failed (500 Internal Server Error)

The batch could not be read back — most often an unknown or expired batchId. Credits are not deducted for a failed retrieval.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits. Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Pro Tip: Read total_pages from the first response and loop until you've fetched them all, rather than incrementing page until you get an empty array — it costs one fewer call per batch.

Credit Cost

1 credit

1 credit per request

Each get_batch_results request costs 1 credit, whether you pull 1 result or 100. The scraping itself was already billed by batch_scrape.

Free Plan: 1,000 one-time credits = 1,000 requests

Hobby Plan: 5,000 credits/month = 5,000 requests ($19/mo)

Professional Plan: 50,000 credits/month = 50,000 requests ($99/mo)

Business Plan: 250,000 credits/month = 250,000 requests ($399/mo)

Related Tools

batch_scrape

Submit the batch job this tool reads back (5 credits)

scrape

Multi-format extraction from a single URL, synchronously (2 credits)

crawl_deep

Breadth-first crawl when you don't have the URL list up front (4 credits)

map_site

Discover a site's URLs before batching them (2 credits)

Ready to try get_batch_results? Sign up for free and get 1,000 credits to start building.

Use Cases

Poll a Running Batch

Check job status and pull the pages that have finished while the rest of the batch is still running.

Page Through Large Jobs

Walk results 25 at a time (up to 100 per page) instead of holding an entire batch in memory.

Resume After a Restart

The batchId outlives your process — a worker that crashes can pick up exactly where it left off.

Cheap Status Checks

At 1 credit per call, polling costs a fraction of re-running the batch you already paid for.

Parameters

batchId comes from the batch_scrape response — this tool never starts a job, it only reads one back.

Name	Type	Required	Default	Description
batchId	string	Required	-	The batch job identifier returned by `batch_scrape`. Example: batch_1700000000000_abc123def
page	number	Optional	1	Page number to retrieve (1-based). Example: 1
limit	number	Optional	25	Results per page (1-100). Example: 50

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/get_batch_results \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "batchId": "batch_1700000000000_abc123def",
    "page": 1,
    "limit": 25
  }'

TypeScript

getBatchResults.tsTypescript

async function getPage(batchId: string, page: number) {
  const response = await fetch('https://crawlforge.dev/api/v1/tools/get_batch_results', {
    method: 'POST',
    headers: {
      'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ batchId, page, limit: 25 }),
  });

  return response.json();
}

// Read total_pages from the first response, then loop — one fewer call than
// incrementing until you get an empty array.
const first = await getPage('batch_1700000000000_abc123def', 1);

if (first.success) {
  const all = [...first.data.results];

  for (let page = 2; page <= first.data.total_pages; page++) {
    const next = await getPage(first.data.batch_id, page);
    all.push(...next.data.results);
  }

  console.log(`Batch ${first.data.status}: ${all.length} of ${first.data.total} results`);
  console.log('Credits remaining:', first.credits_remaining);
} else {
  console.error('Error:', first.error);
}

Python

get_batch_results.pyPython

import requests
import os

def get_page(batch_id, page):
    response = requests.post(
        'https://crawlforge.dev/api/v1/tools/get_batch_results',
        headers={
            'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
            'Content-Type': 'application/json',
        },
        json={'batchId': batch_id, 'page': page, 'limit': 25}
    )
    return response.json()

# Read total_pages from the first response, then loop — one fewer call than
# incrementing until you get an empty list.
first = get_page('batch_1700000000000_abc123def', 1)

if first['success']:
    all_results = list(first['data']['results'])

    for page in range(2, first['data']['total_pages'] + 1):
        nxt = get_page(first['data']['batch_id'], page)
        all_results.extend(nxt['data']['results'])

    print(f"Batch {first['data']['status']}: {len(all_results)} of {first['data']['total']} results")
    print(f"Credits remaining: {first['credits_remaining']}")
else:
    print(f"Error: {first['error']}")

Response Example

200 OK96ms

{
  "success": true,
  "data": {
    "batch_id": "batch_1700000000000_abc123def",
    "status": "completed",
    "page": 1,
    "limit": 25,
    "total": 3,
    "total_pages": 1,
    "results": [
      {
        "url": "https://example.com/page-1",
        "status": "completed",
        "data": {
          "title": "Example page 1"
        }
      },
      {
        "url": "https://example.com/page-2",
        "status": "completed",
        "data": {
          "title": "Example page 2"
        }
      }
    ]
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 96
}

Field Descriptions

data.batch_idThe batch job you queried, echoed back

data.statusStatus of the batch as a whole

data.totalTotal number of results across every page

data.total_pagesHow many pages exist at the current `limit`

data.resultsOne entry per URL in this page, each with its own per-URL `status`

credits_usedCredits deducted for this request (1 per retrieval)

credits_remainingYour remaining credit balance

Error Handling

Invalid Input (400 Bad Request)

batchId is empty, page is below 1, or limit falls outside the 1-100 range.

Retrieval Failed (500 Internal Server Error)

The batch could not be read back — most often an unknown or expired batchId. Credits are not deducted for a failed retrieval.

Insufficient Credits (402 Payment Required)

Your account doesn't have enough credits. Purchase more credits or upgrade your plan.

Rate Limit Exceeded (429 Too Many Requests)

You've exceeded your plan's rate limit. Wait a moment or upgrade your plan for higher limits.

Pro Tip: Read total_pages from the first response and loop until you've fetched them all, rather than incrementing page until you get an empty array — it costs one fewer call per batch.

Credit Cost

1 credit

1 credit per request

Each get_batch_results request costs 1 credit, whether you pull 1 result or 100. The scraping itself was already billed by batch_scrape.

Free Plan: 1,000 one-time credits = 1,000 requests

Hobby Plan: 5,000 credits/month = 5,000 requests ($19/mo)

Professional Plan: 50,000 credits/month = 50,000 requests ($99/mo)

Business Plan: 250,000 credits/month = 250,000 requests ($399/mo)