extract_metadata

Extract comprehensive page metadata including title, description, OpenGraph tags, Twitter Card data, and SEO information.

Use Cases

SEO Analysis

Analyze page titles, descriptions, and keywords for optimization

Social Media Preview Data

Get OpenGraph and Twitter Card data for rich social previews

Content Categorization

Use metadata to classify and organize web content

Link Preview Generation

Build rich link previews with titles, images, and descriptions

Endpoint

POST/api/v1/tools/extract_metadata

Auth Required

1 req/s on Free plan

1 credit

Parameters

Name	Type	Required	Default	Description
url	string	Optional	-	Page to fetch metadata from. Either `url` or `html` is required. Example: https://example.com/article
html	string	Optional	-	Raw HTML to parse instead of fetching. Example: <html>...</html>
include_social	boolean	Optional	true	Include Open Graph, Twitter card, Facebook app id and site-verification tags. Example: true
include_seo	boolean	Optional	true	Include description, keywords, author, robots, canonical, headings and viewport. Example: true
include_technical	boolean	Optional	true	Include generator, favicons, stylesheets and scripts. Example: true
include_structured_data	boolean	Optional	true	Include JSON-LD structured data blocks found on the page. Example: true
respect_robots	boolean	Optional	true	Respect the target site's robots.txt. Left at `true`, a path disallowed for `CrawlForge` is refused with 403 before anything is fetched and no credits are charged. Set it to `false` only for a target you have your own agreement with — the response then carries a `warnings` entry and the override is recorded against your API key. Example: true

Request Examples

cURL

terminalBash

curl -X POST https://crawlforge.dev/api/v1/tools/extract_metadata \
  -H "X-API-Key: cf_test_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article"}'

TypeScript

extractMetadata.tsTypescript

const response = await fetch('https://crawlforge.dev/api/v1/tools/extract_metadata', {
  method: 'POST',
  headers: {
    'X-API-Key': process.env.CRAWLFORGE_API_KEY!,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    url: 'https://example.com/article'
  }),
});

const data = await response.json();

if (data.success) {
  const { title, description, og, twitter } = data.data;
  console.log('Title:', title);
  console.log('Description:', description);
  console.log('OG Image:', og.image);
  console.log('Twitter Card:', twitter.card);
}

Response Example

200 OK195ms

{
  "success": true,
  "data": {
    "metadata": {
      "basic": {
        "title": "Example Article - Best Practices Guide",
        "url": "https://example.com/article",
        "lang": "en",
        "charset": "utf-8"
      },
      "seo": {
        "description": "Learn the best practices for web development",
        "keywords": "web development, best practices, tutorial",
        "author": "Jane Doe",
        "robots": "index, follow",
        "canonical": "https://example.com/article",
        "headings": {
          "h1": [
            "Best Practices Guide"
          ],
          "h2": [
            "Getting Started",
            "Common Pitfalls"
          ],
          "h3": []
        },
        "meta_refresh": null,
        "viewport": "width=device-width, initial-scale=1"
      },
      "social": {
        "open_graph": {
          "title": "Example Article - Best Practices Guide",
          "image": "https://example.com/og-image.jpg",
          "type": "article",
          "site_name": "Example Site"
        },
        "twitter": {
          "card": "summary_large_image",
          "site": "@examplesite"
        },
        "facebook_app_id": null,
        "google_site_verification": null
      },
      "technical": {
        "generator": "Next.js",
        "favicons": [
          {
            "rel": "icon",
            "href": "/favicon.ico",
            "type": "image/x-icon"
          }
        ]
      },
      "structured_data": []
    },
    "summary": {
      "has_title": true,
      "has_description": true,
      "has_open_graph": true,
      "has_twitter_cards": true,
      "has_structured_data": false,
      "total_headings": 3
    },
    "extracted_from": "https://example.com/article",
    "extraction_time": "2026-08-26T14:30:00.000Z"
  },
  "credits_used": 1,
  "credits_remaining": 999,
  "processing_time": 195
}

Field Descriptions

data.metadata.basicAlways present: title, resolved url, html lang and charset.

data.metadata.seoPresent when `include_seo` is true. Includes the full h1/h2/h3 heading structure.

data.metadata.social.open_graphEvery `og:` property the page declares, verbatim — null when the page has none.

data.metadata.social.twitterEvery `twitter:` property the page declares. Null when there are none.

data.metadata.technicalPresent when `include_technical` is true: generator, favicons, stylesheets and scripts.

data.metadata.structured_dataJSON-LD blocks found on the page. Empty array when there are none.

data.summaryBooleans for a quick completeness check without walking the whole metadata object.

data.extracted_fromThe source URL, or the literal string 'provided HTML' when you passed `html`.

data.extraction_timeISO 8601 timestamp of the extraction.

Error Handling

Blocked by robots.txt (403 Forbidden)

The target site's robots.txt disallows this path for CrawlForge. Set respect_robots: false to override if you have your own agreement with the target — the override is recorded against your API key. The override does not reach a host on CrawlForge's permanent opt-out list, which is refused whatever respect_robots is set to.

Related Tools

fetch_url

Fetch pages before extracting metadata (1 credit)

content_analysis

Analyze content quality and SEO (4 credits)

Ready to extract metadata? Sign up for free and get 1,000 credits.