map_site
Quickly discover and analyze website structure using sitemaps, robots.txt, and intelligent URL discovery. Perfect for SEO audits and understanding site architecture.
Use Cases
SEO Audits
Analyze site structure, URL patterns, and sitemap quality for SEO optimization
Site Architecture Planning
Understand how competitors organize content before building your own site
Content Inventory
Quickly discover all pages on a site without crawling each one
Sitemap Validation
Verify sitemap.xml is complete and properly formatted
Migration Planning
Map old site structure before migrating to a new platform
Competitive Analysis
Understand competitor site structure and content organization
Endpoint
/api/v1/tools/map_siteParameters
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
url | string | Required | - | Base URL to map (sitemap.xml will be auto-discovered) Example: https://example.com |
includeSitemap | boolean | Optional | true | Parse sitemap.xml if found Example: true |
maxUrls | number | Optional | 1000 | Maximum URLs to discover (100-10000) Example: 5000 |
groupByPath | boolean | Optional | true | Group URLs by path structure Example: true |
includeMetadata | boolean | Optional | false | Include lastmod, priority, and changefreq from sitemap Example: true |
excludePatterns | string[] | Optional | - | Skip URLs matching these regex patterns Example: ["/admin/.*", ".*\\.(pdf|zip)$"] |
Request Examples
Response Example
{ "success": true, "data": { "url": "https://example.com", "totalUrls": 1247, "sitemapFound": true, "sitemapUrls": [ "https://example.com/sitemap.xml", "https://example.com/sitemap-posts.xml" ], "groups": { "/": [ { "url": "https://example.com", "priority": 1, "lastmod": "2025-09-15" } ], "/blog": [ { "url": "https://example.com/blog/post-1", "priority": 0.8, "lastmod": "2025-09-28" }, { "url": "https://example.com/blog/post-2", "priority": 0.8, "lastmod": "2025-09-27" } ], "/products": [ { "url": "https://example.com/products/item-1", "priority": 0.9, "lastmod": "2025-09-20" }, { "url": "https://example.com/products/item-2", "priority": 0.9, "lastmod": "2025-09-19" } ] }, "statistics": { "totalGroups": 15, "avgUrlsPerGroup": 83, "largestGroup": "/blog", "largestGroupSize": 420, "sitemapCoverage": 0.95 }, "metadata": { "robotsTxtFound": true, "sitemapIndexFound": true, "totalSitemaps": 3, "oldestUpdate": "2024-01-15", "newestUpdate": "2025-09-28" } }, "credits_used": 2, "credits_remaining": 998, "processing_time": 3450}data.totalUrlsTotal unique URLs discovereddata.sitemapFoundWhether sitemap.xml was discovereddata.groupsURLs organized by path structuredata.statistics.sitemapCoveragePercentage of URLs from sitemap vs discovereddata.metadataAdditional site informationcredits_usedFixed 2 credits per site mappingprocessing_timeUsually 2-5 seconds for most sitesError Handling
No Sitemap Found (200 OK with warning)
Site doesn't have sitemap.xml. Tool will attempt alternative discovery methods.
Invalid Sitemap Format (400 Bad Request)
Sitemap.xml exists but is malformed or invalid XML. Check sitemap syntax.
Max URLs Exceeded (200 OK with truncation)
Site has more URLs than maxUrls limit. Results truncated. Increase limit if needed.
Site Unreachable (500 Internal Server Error)
Cannot connect to the site. Check if URL is correct and site is accessible.
Credit Cost
Cost Comparison:
map_site (1000 URLs): 2 credits
crawl_deep (1000 pages): 1,000 credits
Savings: 99.8% fewer credits with map_site
Plan Recommendations:
Free Plan: 1,000 credits = 500 sites mapped
Hobby Plan: 5,000 credits = 2,500 sites mapped ($19/mo)
Professional Plan: 50,000 credits = 25,000 sites mapped ($99/mo)