Introducing deep_research: AI-Powered Multi-Source Analysis

Today we're launching deep_research - the most powerful tool in the CrawlForge suite. It transforms how AI applications gather and synthesize information from the web.

The Research Problem

Manual research is slow and fragmented:

1. Search for sources (5-10 minutes)
2. Open and read each result (20-30 minutes)
3. Take notes and extract key facts (15-20 minutes)
4. Cross-reference and verify (10-15 minutes)
5. Synthesize into a coherent summary (15-20 minutes)

Total: 65-95 minutes for a single research topic.

Existing tools help with pieces:

• Search APIs find sources
• Scraping tools extract content
• LLMs can summarize text

But nothing combines them into a unified research workflow. Until now.

Announcing deep_research

deep_research does what a human researcher does, but in seconds:

curl -X POST https://crawlforge.dev/api/v1/tools/deep_research \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "topic": "Next.js 15 App Router performance improvements",
    "depth": "moderate",
    "maxSources": 10
  }'

Response (15-30 seconds later):

{
  "success": true,
  "data": {
    "summary": "Next.js 15 introduces significant performance improvements including Partial Prerendering (PPR), faster cold starts, and optimized client-side navigation...",
    "keyFindings": [
      {
        "claim": "PPR reduces initial page load by 40-60%",
        "confidence": "high",
        "sources": ["vercel.com/blog", "nextjs.org/docs"]
      },
      {
        "claim": "Cold start times improved by 30%",
        "confidence": "high",
        "sources": ["vercel.com/blog", "github.com/vercel/next.js"]
      }
    ],
    "sources": [
      {
        "url": "https://nextjs.org/blog/next-15",
        "title": "Next.js 15 Release Notes",
        "relevance": 0.95,
        "extractedAt": "2026-01-20T10:30:00Z"
      }
    ],
    "conflicts": [],
    "metadata": {
      "searchQueries": 3,
      "sourcesAnalyzed": 10,
      "processingTime": 18.5
    }
  },
  "credits": {
    "used": 10,
    "remaining": 990
  }
}

How It Works Under the Hood

deep_research runs a multi-stage pipeline:

Stage 1: Query Expansion

Your topic is expanded into multiple search queries:

Stage 2: Source Discovery

Multiple web searches find relevant sources:

• Google Custom Search API integration
• Filters for recency and relevance
• Automatic deduplication
• Domain reputation scoring

Stage 3: Content Extraction

Each source is scraped and processed:

• Main content extraction (removes ads, navigation)
• Metadata capture (author, date, domain)
• Key quote identification
• Readability scoring

Stage 4: Verification

Facts are cross-referenced across sources:

• Claim extraction using NLP
• Source agreement scoring
• Conflict detection
• Confidence assignment (high/medium/low)

Stage 5: Synthesis

AI synthesizes findings into a coherent summary:

• Key findings with citations
• Conflicting viewpoints highlighted
• Source ranking by relevance
• Actionable recommendations

Key Features

Source Verification

Every claim includes confidence scoring:

{
  "claim": "React Server Components reduce JavaScript bundle size",
  "confidence": "high",
  "sources": ["react.dev", "vercel.com", "github.com/facebook/react"],
  "agreementScore": 0.92
}

Conflict Detection

When sources disagree, we tell you:

{
  "conflicts": [
    {
      "topic": "Default cache behavior",
      "positions": [
        { "claim": "fetch is cached by default", "sources": ["docs.nextjs.org"] },
        { "claim": "fetch is no longer cached by default", "sources": ["vercel.com/blog"] }
      ],
      "resolution": "Changed in Next.js 15 - fetch is now uncached by default"
    }
  ]
}

Configurable Depth

Choose how deep to research:

Real-World Use Cases

Competitor Analysis

{
  "topic": "Firecrawl vs Apify pricing and features comparison",
  "depth": "deep",
  "focus": "comparative"
}

Returns feature comparison tables, pricing breakdowns, and user sentiment analysis.

Market Research

{
  "topic": "Web scraping market size and growth 2025-2026",
  "depth": "moderate",
  "sources": ["news", "reports"]
}

Aggregates market data from multiple analyst reports with citations.

Technical Documentation

{
  "topic": "How to implement rate limiting in Node.js",
  "depth": "moderate",
  "focus": "tutorial"
}

Synthesizes best practices from official docs, Stack Overflow, and tutorials.

News Aggregation

{
  "topic": "AI regulation developments Europe March 2026",
  "depth": "shallow",
  "timeRange": "week"
}

Latest news with source diversity and credibility scoring.

Pricing and Credits

deep_research costs 10 credits per query.

Compared to doing it manually:

• search_web (5 credits) × 4 queries = 20 credits
• extract_content (2 credits) × 12 sources = 24 credits
• Manual total: 44 credits

deep_research saves 77% while providing better results.

Plan Capacity

Getting Started

1. Basic Research

const response = await fetch('https://crawlforge.dev/api/v1/tools/deep_research', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${process.env.CRAWLFORGE_API_KEY}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    topic: 'Your research topic here',
    depth: 'moderate'
  })
});

const { data } = await response.json();
console.log(data.summary);
console.log(data.keyFindings);

2. With Source Filtering

{
  "topic": "Machine learning best practices 2026",
  "depth": "deep",
  "options": {
    "excludeDomains": ["medium.com", "quora.com"],
    "preferDomains": ["arxiv.org", "github.com"],
    "timeRange": "year"
  }
}

3. In Claude Desktop

Just ask naturally:

Claude will automatically use deep_research and present synthesized results.

What's Next

We're actively improving deep_research:

• Real-time sources - Include live news and social media
• Custom source lists - Research only from your approved domains
• Export formats - PDF reports, Markdown, structured JSON
• Scheduled research - Run recurring research jobs

Start Researching

Sign up at crawlforge.dev and get 1,000 free credits - enough for 100 research queries. See the pricing page for plan details once you scale beyond the free tier.

Have feedback? We'd love to hear it. Reach out on GitHub or Twitter.

API Reference: /docs/api-reference/tools/deep-research