Large Language Model (LLM)
AI / MCPDefinition
A large language model is a neural network trained on vast amounts of text data that can understand and generate human language. LLMs power AI assistants, code generators, and autonomous agents.
How It Relates to CrawlForge
LLMs like Claude, GPT, and Llama are powerful reasoners but have a fundamental limitation: they cannot access the live web. Their knowledge is frozen at training time, making them unreliable for tasks requiring current information.
CrawlForge bridges this gap by giving LLMs real-time web access through MCP tools. When an LLM needs current data -- live prices, recent news, updated documentation -- it calls CrawlForge tools to fetch and extract that information, then reasons about the fresh data.
Related CrawlForge Tools
Related Terms
AI Agent
An AI agent is an autonomous system powered by a large language model that can reason about tasks, make decisions, and take actions by using tools. Agents go beyond simple chatbots by planning and executing multi-step workflows.
Token
A token is the basic unit of text that language models process. Text is split into tokens (roughly 4 characters or 0.75 words each) before being processed by the model. Token counts determine costs and context limits.
Context Window
The context window is the maximum amount of text (measured in tokens) that a language model can process in a single request. It includes both the input prompt and the generated output.
Retrieval-Augmented Generation (RAG)
RAG is an AI architecture that combines information retrieval with text generation. It first retrieves relevant documents from external sources, then uses them as context for the language model to generate accurate, grounded responses.
Start Scraping with 1,000 Free Credits
Get started with CrawlForge today. No credit card required.
Start scraping with 1,000 free credits