llms-full.txt

llms-full.txt is an expanded version of llms.txt that includes a site's full page content, rather than short summaries, in one consolidated markdown file intended for AI systems to ingest directly. It trades the compact, curated nature of llms.txt for completeness, at the cost of a much larger file.

Why it matters

Because llms-full.txt bundles complete content rather than links, it removes any dependency on an AI crawler successfully rendering or retrieving individual pages, which matters most for sites with heavy client-side rendering or crawl-access friction. It's a supplementary format, not a replacement for llms.txt or normal crawlability.

Related terms

• llms.txt

  llms.txt is a markdown file placed at a website's root that summarizes its content and structure specifically for AI systems and their crawlers, functioning as the closest thing this space has to an emerging technical standard. It gives an AI system a curated map of a site instead of forcing it to infer structure from a full crawl.

• Chunking

  Chunking is how AI systems split content into smaller passages, typically 200 to 500 tokens, before indexing and retrieving it, meaning a page is rarely evaluated as one continuous whole but as a set of independent segments competing separately for retrieval.

• Schema Markup / Structured Data

  Schema markup, or structured data, is machine-readable metadata embedded in a page, most commonly as JSON-LD, that explicitly tells crawlers what the content actually is: an article, an organization, a product, a set of questions and answers, rather than leaving that classification to inference.