extract_metadata

Extract comprehensive page metadata including title, description, OpenGraph tags, Twitter Card data, and SEO information.

Use Cases

SEO Analysis

Analyze page titles, descriptions, and keywords for optimization

Social Media Preview Data

Get OpenGraph and Twitter Card data for rich social previews

Content Categorization

Use metadata to classify and organize web content

Link Preview Generation

Build rich link previews with titles, images, and descriptions

Endpoint

POST/api/v1/tools/extract_metadata

Auth Required

1 req/s on Free plan

1 credit

Parameters

Name	Type	Required	Default	Description
url	string	Optional	-	Page to fetch metadata from. Either `url` or `html` is required. Example: https://example.com/article
html	string	Optional	-	Raw HTML to parse instead of fetching. Example: <html>...</html>
include_social	boolean	Optional	true	Include Open Graph, Twitter card, Facebook app id and site-verification tags. Example: true
include_seo	boolean	Optional	true	Include description, keywords, author, robots, canonical, headings and viewport. Example: true
include_technical	boolean	Optional	true	Include generator, favicons, stylesheets and scripts. Example: true
include_structured_data	boolean	Optional	true	Include JSON-LD structured data blocks found on the page. Example: true
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Left at `true`, a path disallowed for `CrawlForge` is refused with 403 before anything is fetched and no credits are charged. Set it to `false` only for a target you have your own agreement with — the response then carries a `warnings` entry and the override is recorded against your API key. Example: true
max_inline_chars	number	Optional	40000	Largest result to return inline, in characters of its JSON (1,000-10,000,000). Over it the response carries `preview` (the first `max_inline_chars` characters of the pretty-printed JSON), `result_handle`, `total_chars`, `truncated: true` and `expires_at`, and [read_result](/docs/api-reference/tools/read-result) reads the rest for 1 credit per call. Stored results are kept for 1 hour. Example: 40000

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/extract_metadata \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article"}'

TypeScript

extractMetadata.tsTypescript

// npm install crawlforge-sdk
import { CrawlForge } from 'crawlforge-sdk';

const client = new CrawlForge({ apiKey: process.env.CRAWLFORGE_API_KEY });

const result = await client.extractMetadata({
  url: 'https://example.com/article',
});

// result.data is untyped in crawlforge-sdk 0.1 — its shape is the Response Example below.
const { metadata } = result.data as {
  metadata: {
    basic: { title: string };
    seo: { description: string };
    social: { open_graph: { image?: string } | null; twitter: { card?: string } | null };
  };
};
console.log('Title:', metadata.basic.title);
console.log('Description:', metadata.seo.description);
// open_graph and twitter are null when the page declares no such tags.
console.log('OG Image:', metadata.social.open_graph?.image);
console.log('Twitter Card:', metadata.social.twitter?.card);

Python

extract_metadata.pyPython

# pip install crawlforge
from crawlforge import CrawlForge

client = CrawlForge()  # reads CRAWLFORGE_API_KEY

result = client.extract_metadata(
    url='https://example.com/article',
)

# result.data is a plain dict — its shape is the Response Example below.
metadata = result.data['metadata']
print(f"Title: {metadata['basic']['title']}")
print(f"Description: {metadata['seo']['description']}")
# open_graph and twitter are None when the page declares no such tags.
open_graph = metadata['social']['open_graph'] or {}
twitter = metadata['social']['twitter'] or {}
print(f"OG Image: {open_graph.get('image')}")
print(f"Twitter Card: {twitter.get('card')}")

Response Example

200 OK195ms

{
  "success": true,
  "data": {
    "metadata": {
      "basic": {
        "title": "Example Article - Best Practices Guide",
        "url": "https://example.com/article",
        "lang": "en",
        "charset": "utf-8"
      },
      "seo": {
        "description": "Learn the best practices for web development",
        "keywords": "web development, best practices, tutorial",
        "author": "Jane Doe",
        "robots": "index, follow",
        "canonical": "https://example.com/article",
        "headings": {
          "h1": [
            "Best Practices Guide"
          ],
          "h2": [
            "Getting Started",
            "Common Pitfalls"
          ],
          "h3": []
        },
        "meta_refresh": null,
        "viewport": "width=device-width, initial-scale=1"
      },
      "social": {
        "open_graph": {
          "title": "Example Article - Best Practices Guide",
          "image": "https://example.com/og-image.jpg",
          "type": "article",
          "site_name": "Example Site"
        },
        "twitter": {
          "card": "summary_large_image",
          "site": "@examplesite"
        },
        "facebook_app_id": null,
        "google_site_verification": null
      },
      "technical": {
        "generator": "Next.js",
        "favicons": [
          {
            "rel": "icon",
            "href": "/favicon.ico",
            "type": "image/x-icon"
          }
        ]
      },
      "structured_data": []
    },
    "summary": {
      "has_title": true,
      "has_description": true,
      "has_open_graph": true,
      "has_twitter_cards": true,
      "has_structured_data": false,
      "total_headings": 3
    },
    "extracted_from": "https://example.com/article",
    "redirected": false,
    "extraction_time": "2026-08-26T14:30:00.000Z"
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 195
}

Field Descriptions

data.metadata.basicAlways present: title, resolved url, html lang and charset.

data.metadata.seoPresent when `include_seo` is true. Includes the full h1/h2/h3 heading structure.

data.metadata.social.open_graphEvery `og:` property the page declares, verbatim — null when the page has none.

data.metadata.social.twitterEvery `twitter:` property the page declares. Null when there are none.

data.metadata.technicalPresent when `include_technical` is true: generator, favicons, stylesheets and scripts.

data.metadata.structured_dataJSON-LD blocks found on the page. Empty array when there are none.

data.summaryBooleans for a quick completeness check without walking the whole metadata object.

data.extracted_fromThe source URL — the final URL after any redirect — or the literal string 'provided HTML' when you passed `html`.

data.redirectedAlways present. `true` when the fetch ended on a different URL from the one you sent, compared after parsing, so `https://example.com` and `https://example.com/` are the same page. The metadata then describes the page redirected to, and `data.requested_url` carries the URL you sent. Always `false` for `html`.

data.extraction_timeISO 8601 timestamp of the extraction.

Error Handling

Blocked by robots.txt (403 Forbidden)

The target site's robots.txt disallows this path for CrawlForge. Set respect_robots: false to override if you have your own agreement with the target — the override is recorded against your API key. The override does not reach a host on CrawlForge's permanent opt-out list, which is refused whatever respect_robots is set to.

Related Tools

fetch_url

Fetch pages before extracting metadata (1 credit)

content_analysis

Analyze content quality and SEO (4 credits)

Ready to extract metadata? Sign up for free and get 1,000 credits.