Skip to content
AI Tool3 credits

extract_with_llm

AI-powered extraction that runs on a local Ollama model by default — no LLM API key required. Optionally route to OpenAI or Anthropic when you need a hosted model. Give it a prompt (and optionally a JSON Schema) and get back structured data.

Use Cases

Local-First Extraction

Run extractions against Ollama on your own machine — zero LLM API costs and private by default.

Schema-Driven Data Lakes

Combine a prompt with a JSON Schema to populate typed rows for your warehouse or graph store.

Multi-Provider Failover

Start on local Ollama, fall back to OpenAI or Anthropic for higher-stakes pages by toggling one parameter.

Endpoint

POST/api/v1/tools/extract_with_llm
Auth Required
1 req/s on Free plan
3 credits

Parameters

NameTypeRequiredDefaultDescription
url
stringOptional-
URL to fetch and extract from. Either url or content is required.
Example: https://example.com/article/42
content
stringOptional-
Raw text or HTML content to extract from. Either url or content is required.
Example: "<html>...</html>"
prompt
stringRequired-
Natural-language instructions guiding the LLM extraction
Example: Extract the headline, author, and three key takeaways
schema
objectOptional-
Optional JSON Schema describing the data structure to extract
Example: {"type":"object","properties":{"title":{"type":"string"}},"required":["title"]}
provider
stringOptionalauto
LLM provider: "ollama" (local, default), "openai", "anthropic", or "auto"
Example: ollama
model
stringOptional-
Model identifier. Defaults per provider: llama3.2, gpt-4o-mini, claude-haiku-4-5-20251001
Example: llama3.2
maxTokens
numberOptional4096
Maximum tokens for the LLM response
Example: 4096

Request Examples

cURL — local Ollama (default, no API key)

terminalBash
curl -X POST https://crawlforge.dev/api/v1/tools/extract_with_llm \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/article/42",
    "prompt": "Extract the headline, author, and three key takeaways",
    "provider": "ollama"
  }'

TypeScript — OpenAI with schema

extractWithLlm.tsTypescript
// npm install crawlforge-sdk
import { CrawlForge } from 'crawlforge-sdk';

const client = new CrawlForge({ apiKey: process.env.CRAWLFORGE_API_KEY });

const result = await client.extractWithLlm({
  url: 'https://example.com/product/123',
  prompt: 'Extract product name, price in USD, and stock status',
  provider: 'openai',
  model: 'gpt-4o-mini',
  schema: {
    type: 'object',
    properties: {
      title: { type: 'string' },
      price: { type: 'number' },
      in_stock: { type: 'boolean' },
    },
    required: ['title', 'price'],
  },
});

// result.data is untyped in crawlforge-sdk 0.1 — its shape is the Response Example below.
const { extracted } = result.data as {
  extracted: { title: string; price: number; in_stock: boolean };
};
console.log(extracted);

Python — Anthropic

extract_with_llm.pyPython
# pip install crawlforge
from crawlforge import CrawlForge

client = CrawlForge()  # reads CRAWLFORGE_API_KEY

result = client.extract_with_llm(
    url='https://example.com/article/42',
    prompt='Extract headline, author, publish date (ISO 8601), and tags',
    provider='anthropic',
    model='claude-haiku-4-5-20251001',
    schema={
        'type': 'object',
        'properties': {
            'headline': {'type': 'string'},
            'author': {'type': 'string'},
            'published_at': {'type': 'string'},
            'tags': {'type': 'array'},
        },
        'required': ['headline'],
    },
)

# result.data is a plain dict — its shape is the Response Example below.
print(result.data['extracted'])

Response Example

200 OK3.5s
{
"success": true,
"data": {
"extracted": {
"headline": "How Local LLMs Are Changing Data Pipelines",
"author": "Jane Doe",
"takeaways": [
"Lower cost",
"Better privacy",
"Faster iteration"
]
},
"model_used": "llama3.2",
"provider": "ollama"
},
"credits_used": 3,
"credits_remaining": 997,
"processing_time": 3500
}
Field Descriptions
data.providerResolved provider — "ollama" when provider is "auto"
data.model_usedDefault model per provider unless you specify one
credits_usedFlat 3 credits regardless of provider

Credit Cost

3 credits
3 credits per request
Flat 3 credits whether you use local Ollama, OpenAI, or Anthropic.

Tip: Use list_ollama_models first to discover which local models are available before sending an extraction.

Related Tools