How to Build a Competitive Intelligence Agent with Claude + CrawlForge

// competitors.json
{
  "competitors": [
    {
      "name": "Competitor A",
      "website": "https://competitor-a.com",
      "pricing_page": "https://competitor-a.com/pricing",
      "blog": "https://competitor-a.com/blog"
    },
    {
      "name": "Competitor B",
      "website": "https://competitor-b.com",
      "pricing_page": "https://competitor-b.com/pricing",
      "blog": "https://competitor-b.com/blog"
    }
  ],
  "focus_areas": [
    "pricing",
    "features",
    "content_strategy"
  ]
}

You are a competitive intelligence analyst. Your job is to:

1. Research competitors using CrawlForge tools
2. Track pricing and feature changes
3. Analyze content strategies
4. Identify market trends
5. Generate actionable insights

Always use the most credit-efficient tool:
- fetch_url (1 credit) for basic page content
- extract_content (2 credits) for clean article text
- search_web (5 credits) only when you need to find new sources
- analyze_content (3 credits) for NLP analysis

Never use deep_research (10 credits) for known competitor URLs.

// In Claude Code, ask:
"Fetch the pricing page for [competitor] and extract:
1. Plan names and prices
2. Feature lists for each tier
3. Any promotional offers
4. Compare to last known pricing"

// CrawlForge will use extract_content (2 credits)
// to get clean, structured pricing data

// Ask Claude:
"Check [competitor]'s changelog or product updates page.
Summarize any new features released in the past month.
Flag anything that competes directly with our offerings."

// Uses: fetch_url (1 credit) + Claude's analysis

// Ask Claude:
"Analyze [competitor]'s blog for the past month:
1. What topics are they covering?
2. What keywords are they targeting?
3. How frequently are they publishing?
4. What's their content format (tutorials, news, case studies)?"

// Uses: extract_links (1 credit) to find blog posts
// Then: batch_scrape (5 credits) to get content
// Then: analyze_content (3 credits) for NLP insights

// Set up monitoring for a competitor's pricing page:
"Create a change monitor for https://competitor.com/pricing
Alert me when:
- Prices change by more than 10%
- New plans are added
- Features are added or removed"

// Uses: track_changes (3 credits for setup)
// Ongoing monitoring included

{
  "url": "https://competitor.com/pricing",
  "trackingOptions": {
    "trackText": true,
    "trackStructure": true,
    "granularity": "section",
    "significanceThresholds": {
      "minor": 0.1,
      "moderate": 0.3,
      "major": 0.7
    }
  },
  "monitoringOptions": {
    "interval": 86400000, // Daily check
    "notificationThreshold": "moderate"
  }
}

Generate a competitive intelligence report for the week of [date]:

## Executive Summary
[2-3 sentence overview of key findings]

## Pricing Intelligence
[Any pricing changes detected across competitors]

## Product Updates
[New features or changes from competitors]

## Content Analysis
[What competitors are publishing and why it matters]

## Market Trends
[Patterns observed across multiple competitors]

## Recommended Actions
[3-5 specific actions we should consider]

---
Sources: [List all URLs analyzed]
Credits Used: [Total credits consumed]

# Create a weekly cron job (macOS/Linux)
crontab -e

# Add this line (runs every Monday at 9 AM):
0 9 * * 1 cd /path/to/project && claude "Run weekly competitive intelligence report" > /path/to/reports/report-$(date +%Y%m%d).md

// Instead of 10 separate fetch_url calls (10 credits):
// Use batch_scrape for multiple URLs (5 credits total)
"Batch scrape these 10 competitor pages: [urls]"

// Fetch once, analyze multiple times:
"1. Fetch competitor.com/pricing
 2. Extract pricing tiers
 3. Compare to our pricing
 4. Identify positioning gaps
 5. Suggest response strategies"

// All analysis after step 1 uses Claude (no additional credits)

# Competitive Intelligence Report
## Week of January 20, 2026

### Executive Summary
Competitor A launched a new enterprise tier at $199/month, positioning
directly against our Pro plan. Competitor B increased their content
velocity by 40%, focusing heavily on AI automation tutorials.

### Pricing Intelligence
| Competitor | Change | Details |
|------------|--------|---------|
| Competitor A | New Tier | Enterprise plan launched at $199/mo |
| Competitor B | No Change | Prices stable for 3 months |

### Product Updates
**Competitor A:**
- Released API v2 with GraphQL support
- Added team collaboration features
- New dashboard analytics

**Competitor B:**
- Mobile app beta launched
- Webhook improvements

### Content Analysis
**Competitor A (12 posts):**
- Primary topics: AI automation, enterprise use cases
- Target keywords: "enterprise AI", "team collaboration"
- Format: Long-form tutorials (avg 2,500 words)

**Competitor B (8 posts):**
- Primary topics: Developer tutorials, integrations
- Target keywords: "API integration", "developer tools"
- Format: Code-heavy how-tos (avg 1,800 words)

### Recommended Actions
1. Consider matching Competitor A's enterprise pricing
2. Accelerate our content velocity to maintain SEO position
3. Prioritize GraphQL support in our roadmap
4. Create competitive comparison landing page

---
**Credits Used:** 47/1,000 (free tier)
**Sources Analyzed:** 24 pages across 2 competitors

const competitors = [
  { name: "A", url: "https://a.com" },
  { name: "B", url: "https://b.com" },
  { name: "C", url: "https://c.com" },
  // ... up to 50 URLs
];

// Use batch_scrape for all pricing pages at once
"Batch scrape all competitor pricing pages and compare:
${competitors.map(c => c.url + '/pricing').join('\n')}"

// 5 credits for up to 50 URLs
// vs 50 credits for individual fetch_url calls