extract_structured

Give the extractor a JSON Schema and a natural-language prompt. The LLM reads the page and returns data matching your schema. When no LLM provider is configured it falls back to CSS selector extraction using your hints.

Use Cases

Schema-First Product Extraction

Define the fields you want once; the LLM maps any e-commerce site to your schema.

Resume & Document Parsing

Extract candidate names, skills, and work history directly into a typed object.

Knowledge Graph Seeding

Extract entities and relationships from articles into structured JSON for graph loaders.

Endpoint

POST/api/v1/tools/extract_structured

Auth Required

1 req/s on Free plan

3 credits

Parameters

LLM vs. selector fallback: Provide llmConfig to use LLM-powered extraction. Without it, the tool uses selectorHints for deterministic CSS extraction — cheaper and no LLM key required.

Name	Type	Required	Default	Description
url	string	Required	-	URL to extract data from Example: https://example.com/product/123
schema	object	Required	-	JSON Schema describing the data to extract Example: {"type":"object","properties":{"title":{"type":"string"},"price":{"type":"number"}},"required":["title"]}
prompt	string	Optional	-	Natural-language instructions guiding the LLM extraction Example: Extract the product name, current price, and whether it is in stock
llmConfig	object	Optional	-	Optional LLM provider configuration (provider, apiKey). Omit to use CSS selector fallback. Example: {"provider": "openai", "apiKey": "sk-..."}
selectorHints	object	Optional	-	CSS selector hints to guide extraction (also used by selector fallback) Example: {"title": "h1.product-title", "price": ".price"}
fallbackToSelectors	boolean	Optional	true	Fall back to CSS selector extraction when LLM is unavailable Example: true
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Left at `true`, a path disallowed for `CrawlForge` is refused with 403 before anything is fetched and no credits are charged. Set it to `false` only for a target you have your own agreement with — the response then carries a `warnings` entry and the override is recorded against your API key. Example: true

Request Examples

cURL — LLM extraction

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/extract_structured \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/product/123",
    "schema": {
      "type": "object",
      "properties": {
        "title": { "type": "string" },
        "price": { "type": "number" },
        "in_stock": { "type": "boolean" }
      },
      "required": ["title", "price"]
    },
    "prompt": "Extract the product name, price in USD, and availability",
    "llmConfig": { "provider": "openai", "apiKey": "sk-..." }
  }'

TypeScript — selector fallback

extractStructured.tsTypescript

const response = await fetch('https://crawlforge.dev/api/v1/tools/extract_structured', {
  method: 'POST',
  headers: {
    'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://example.com/product/123',
    schema: {
      type: 'object',
      properties: {
        title: { type: 'string' },
        price: { type: 'number' },
      },
      required: ['title'],
    },
    selectorHints: {
      title: 'h1.product-title',
      price: '.price-value',
    },
    fallbackToSelectors: true,
  }),
});

const data = await response.json();
if (data.success) {
  console.log(data.data.extracted.title, data.data.extracted.price);
}

Python

extract_structured.pyPython

import requests, os

response = requests.post(
    'https://crawlforge.dev/api/v1/tools/extract_structured',
    headers={
        'X-API-Key': os.environ['CRAWLFORGE_API_KEY'],
        'Content-Type': 'application/json',
    },
    json={
        'url': 'https://example.com/article/42',
        'schema': {
            'type': 'object',
            'properties': {
                'headline': {'type': 'string'},
                'author': {'type': 'string'},
                'published_at': {'type': 'string'},
                'tags': {'type': 'array'},
            },
            'required': ['headline'],
        },
        'prompt': 'Extract headline, author, publish date (ISO 8601), and tags',
    },
)

data = response.json()
if data['success']:
    print(data['data']['extracted'])

Response Example

200 OK1.2s

{
  "success": true,
  "data": {
    "url": "https://example.com/product/123",
    "data": {
      "title": "Premium Wireless Headphones",
      "price": 299.99,
      "in_stock": true
    },
    "extraction": {
      "method_by_field": {
        "title": "selector",
        "price": "json-ld",
        "in_stock": "meta"
      },
      "llm_used": false,
      "note": "Selector/structured-data extraction; LLM-guided extraction is available via the CrawlForge MCP server"
    },
    "extracted_at": "2026-08-26T14:30:00.000Z"
  },
  "credits_used": 3,
  "credits_remaining": 997,
  "processing_time": 1240
}

Field Descriptions

data.dataOne key per property in your schema, coerced to the declared type. A field that could not be found is null.

data.extraction.method_by_fieldHow each field was obtained: `selector` from a selectorHints entry, `json-ld` from structured data, `meta` from a meta tag, or `none` when nothing matched.

data.extraction.llm_usedAlways false on the hosted REST API — extraction is selector and structured-data based, never generative.

data.extraction.noteRestates where LLM-guided extraction is available.

data.extracted_atISO 8601 timestamp of the extraction.

Error Handling

Blocked by robots.txt (403 Forbidden)

The target site's robots.txt disallows this path for CrawlForge. Set respect_robots: false to override if you have your own agreement with the target — the override is recorded against your API key. The override does not reach a host on CrawlForge's permanent opt-out list, which is refused whatever respect_robots is set to.

Credit Cost