Acceptable Use Policy
Guidelines for ethical, legal, and responsible use of CrawlForge web scraping tools. Our policy ensures a fair and sustainable ecosystem for all users and website owners.
Last Updated: December 2025
Quick Navigation
1. Purpose of This Policy
About This Policy
This Acceptable Use Policy (AUP) governs your use of CrawlForge's web scraping tools and services. It is designed to:
- •Protect the rights and privacy of website owners and their users
- •Ensure CrawlForge is used for legitimate, legal purposes
- •Maintain the stability and performance of our infrastructure
- •Promote ethical web scraping practices across the industry
- •Comply with applicable laws and regulations (GDPR, CCPA, CFAA, etc.)
Agreement
By using CrawlForge, you agree to comply with this policy. Violations may result in immediate account suspension or termination without refund.
2. Prohibited Activities
The following activities are strictly prohibited when using CrawlForge:
Illegal Content Scraping
Do not use CrawlForge to scrape, collect, or distribute illegal content, including copyrighted materials without authorization, personal data in violation of privacy laws, or content that violates intellectual property rights.
Examples of prohibited use:
- ×Scraping paywalled content without permission
- ×Collecting personal data in violation of GDPR/CCPA
- ×Harvesting copyrighted materials for redistribution
Security Bypass Attempts
Do not attempt to bypass, circumvent, or disable security measures, authentication systems, or access controls on target websites. This includes attempting to exploit vulnerabilities or use our tools for unauthorized access.
Examples of prohibited use:
- ×Bypassing login requirements to access private content
- ×Exploiting website vulnerabilities or security flaws
- ×Attempting to access restricted areas or admin panels
Denial of Service (DoS) Attacks
Do not use CrawlForge to conduct denial-of-service attacks, overload servers, or intentionally harm target websites. Excessive request volumes that degrade website performance are prohibited.
Examples of prohibited use:
- ×Sending thousands of simultaneous requests to overwhelm servers
- ×Repeatedly scraping the same pages to consume bandwidth
- ×Intentionally targeting small websites with high-volume requests
Data Harvesting for Spam
Do not collect email addresses, phone numbers, or other contact information for the purpose of sending unsolicited communications (spam), phishing attempts, or fraudulent activities.
Examples of prohibited use:
- ×Scraping email addresses for spam campaigns
- ×Harvesting phone numbers for robocalls or SMS spam
- ×Collecting personal data for phishing or identity theft
3. Rate Limits and Fair Usage
To ensure fair access for all users and maintain system stability, CrawlForge enforces the following rate limits:
Free Plan
60 requests/min
3,600 requests per hour
Hobby Plan
120 requests/min
7,200 requests per hour
Professional Plan
240 requests/min
14,400 requests per hour
Business Plan
600 requests/min
36,000 requests per hour
Note: Rate limits are enforced per API key. Exceeding these limits will result in 429 (Too Many Requests) responses. If you require higher limits, please contact our sales team for enterprise options.
Fair Usage Guidelines
Respect robots.txt
Always check and respect the robots.txt file of target websites. If a site explicitly disallows scraping in their robots.txt, do not use CrawlForge to access those restricted areas.
Implement Reasonable Delays
Space out your requests appropriately. For most websites, adding a 1-2 second delay between requests is a good practice to avoid overwhelming servers.
Use Appropriate Tools
Choose the most efficient CrawlForge tool for your use case. Don't use deep_research (10 credits) when fetch_url (1 credit) would suffice. This reduces load on both our systems and target websites.
Monitor Your Impact
Be aware of your scraping volume and its potential impact on target websites. If you notice errors or slowdowns, reduce your request rate or contact the website owner for permission.
4. Respecting Target Website Terms
You are responsible for ensuring your use of CrawlForge complies with the terms of service and policies of the websites you scrape. This includes:
- Reading Terms of Service: Review the target website's terms of service and robots.txt file before scraping.
- Seeking Permission: When in doubt, contact the website owner to request permission for scraping activities.
- Honoring robots.txt: Respect the directives in robots.txt files, including crawl-delay and disallowed paths.
- Limiting Request Volume: Implement appropriate delays and avoid overwhelming target servers with excessive requests.
- Respecting API Alternatives: If a website offers an official API, use it instead of scraping HTML content.
Important Legal Notice
CrawlForge provides tools for web scraping, but you are solely responsible for how you use these tools. Ensure your scraping activities comply with applicable laws in your jurisdiction, including but not limited to the Computer Fraud and Abuse Act (CFAA), GDPR, CCPA, and copyright laws.
5. Enforcement and Account Suspension
Violations of this Acceptable Use Policy may result in the following actions:
1. Warning Notice
For minor or first-time violations, we will send a warning notice explaining the issue and requesting corrective action.
2. Temporary Suspension
For repeated or moderate violations, your account may be temporarily suspended (typically 7-30 days) while we investigate the issue.
3. Permanent Termination
For severe violations (illegal activity, security breaches, DoS attacks), we reserve the right to immediately and permanently terminate your account without prior notice or refund.
4. Legal Action
We reserve the right to pursue legal action against users who engage in illegal activities or cause harm to CrawlForge, our infrastructure, or third parties.
Appeals Process
If you believe your account was suspended in error, you may submit an appeal to support@crawlforge.dev within 14 days of suspension. Include your account details and an explanation of why you believe the suspension was incorrect.
6. Reporting Policy Violations
If you believe someone is using CrawlForge in violation of this Acceptable Use Policy, or if you are a website owner experiencing abuse from a CrawlForge user, please report it immediately.
How to Report
- Email: support@crawlforge.dev
- Subject Line: "AUP Violation Report"
- Include: Date/time of incidents, IP addresses or API keys (if known), URLs being scraped, and description of the violation
We take all abuse reports seriously and will investigate within 48 hours. All reports are kept confidential.
7. Contact Information
Questions About This Policy?
We're here to help you understand our guidelines and ensure responsible use of CrawlForge.
Contact Details
- •Email: support@crawlforge.dev
- •Website: crawlforge.dev
- •Response Time: Within 48 business hours
Related Legal Documents
Questions About This Policy?
If you have any questions about this Acceptable Use Policy or need clarification on any guidelines, please don't hesitate to contact us.
support@crawlforge.devReady to get started?
Start with 1,000 free credits. No credit card required.