GEO is real —
and Cloudflare
just proved it.

There is a conversation happening right now between your website and AI systems — and you are almost certainly not part of it. Every time someone asks ChatGPT, Perplexity, Gemini, or Claude a question that touches your industry, those systems consult what they know about your business. In most cases, that is very little — or nothing at all. GEO is the discipline of changing that. And it is more straightforward than its name suggests.

Where GEO came from

The term "Generative Engine Optimisation" entered the technical literature in 2023 through a research paper out of Princeton and Georgia Tech. The paper — authored by Aggarwal, Manzoor, and colleagues — asked a question that now seems obvious in hindsight: if search engines can be optimised for, and if AI systems are becoming a primary interface for information retrieval, can those AI systems also be optimised for?

The answer the researchers found was yes. Structured content, authoritative citations, clear entity definitions, and specific writing patterns all measurably increased how often a brand appeared in AI-generated responses. The paper gave the field a name and a framework. Practitioners ran with it.

In the SEO community, figures like Rand Fishkin (SparkToro), Neil Patel, and the editorial teams at Search Engine Journal and Search Engine Land began writing about GEO throughout 2024. Jeremiah Owyang was among the early voices connecting GEO to the emerging llms.txt standard. Jeremy Howard of Answer.AI formally proposed the llms.txt specification — a structured file hosted at a site's root that gives AI systems a clean, authoritative summary of what the business is and does.

Why people dismissed it

The scepticism was reasonable. In 2023 and early 2024, the counterarguments had real weight.

The first objection: large language models don't crawl the web in real time. They are trained on a static snapshot of data. Optimising for them is like optimising for a library that was printed two years ago and never updated. This was true, and remains partly true for training-time data.

The second objection: LLMs hallucinate. They fabricate citations, invent facts, and confidently state things that are wrong. If the models are unreliable, why build an optimisation strategy around them? Also true, and still a genuine limitation.

The third objection: there is no standard. Unlike Google's well-documented guidelines, GEO had no official spec, no authoritative source, and no way to verify that any specific action produced any specific result. Reasonable.

But each of these objections has been overtaken by events.

Retrieval-Augmented Generation — RAG — changed the equation entirely. Modern AI systems increasingly pull live web content at query time to supplement their training data. When someone asks Perplexity which accommodation platform has the lowest commission in South Africa, Perplexity crawls the web, retrieves content, and synthesises an answer. The quality and structure of that content directly determines whether your business appears in the answer.

The theory vs the reality

The original GEO research identified several specific techniques that increased AI citation rates. Structured data and schema markup increased visibility. Citing authoritative sources within content increased it further. Fluency and clarity of writing mattered. Quotation and statistical content performed well. These were not hunches — they were measured across hundreds of queries and multiple AI systems.

In practice, the field has consolidated around a few core actions that produce measurable results:

Entity definition. AI models build knowledge graphs — mental maps of what entities exist in the world and how they relate to each other. If your business is not a well-defined entity in those graphs, you are invisible. Entity definition means creating structured, consistent, machine-readable descriptions of your business across multiple independent sources.

The llms.txt file. A plain text file at your domain root that gives AI systems a structured brief about your business. Think of it as a cover letter written directly to the AI. It covers what you do, who you serve, what makes you different, and how to cite you accurately. Growing numbers of AI systems check for this file explicitly.

JSON-LD Schema markup. Structured data embedded in your HTML that tells search engines and AI systems exactly what type of entity your page describes — an Organisation, a Product, a Service, a Person, a FAQ. Schema is the most battle-tested GEO signal because it predates the term by years.

Robots.txt AI crawler permissions. Explicitly allowing or denying specific AI crawlers. This is where Cloudflare's recent announcement becomes directly relevant.

The Cloudflare proof point

In July 2026, Matt Silverlock, VP of Cloudflare, sent the following to the millions of website owners using Cloudflare's network:

Read that carefully. Cloudflare — the network that sits in front of roughly 20% of all websites on the internet — is building an entire product category around AI crawler management. They launched "Content Independence Day." They are changing default settings. They are sending emails to millions of customers about it.

This is not a theoretical discussion. Cloudflare does not build product categories around things that do not exist. AI bots are crawling websites at a scale significant enough that the world's largest CDN provider is making it a primary feature of their security offering.

What AI crawlers actually do

Not all AI crawlers are the same. There are two distinct types, and they matter differently.

Training crawlers — GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended (Google), Amazonbot — build the model itself. They crawl your content and it potentially becomes part of the AI's training data. This is a long-term brand signal. Blocking them means your content is excluded from future model versions.

Inference crawlers — PerplexityBot, ChatGPT-User with browsing, Gemini with live search — retrieve content at query time to answer specific questions. This is immediate and direct. When someone asks Perplexity a question in your industry right now, PerplexityBot crawls relevant pages and uses that content to construct the answer. Blocking it means you cannot appear in those answers regardless of how good your content is.

The llms.txt standard

llms.txt is a plain text file placed at your domain root — yourdomain.com/llms.txt — that provides structured context about your business directly to AI systems. It is analogous to robots.txt for traditional crawlers, but instead of access rules it provides entity context.

A well-structured llms.txt covers your entity type, what you do, who you serve, what makes you different from competitors, how to cite you accurately, and what queries you are the most relevant answer for. It is written in structured markdown that AI systems can parse reliably.

The standard emerged from practice rather than committee. Jeremy Howard of Answer.AI proposed a formal specification. Anthropic, the company behind Claude, references it in their documentation. Perplexity actively parses it. The ecosystem is young but growing fast — which means early movers gain disproportionate advantage before the standard becomes table stakes.

Where we stand today

GEO is not a future discipline. It is a current one with a rapidly closing window of early-mover advantage. The businesses structuring their web presence for AI discoverability right now are building a compounding lead over competitors who are waiting to see if it matters.

Cloudflare's September 15 deadline is the most concrete signal yet that this is mainstream infrastructure, not fringe experimentation. When a company that processes 20% of global web traffic changes its default AI crawler settings, it is not because a few researchers published an interesting paper. It is because AI crawlers represent a significant and growing portion of web traffic — significant enough to require a product response.

The practical steps are not complicated. An llms.txt file takes an afternoon to write properly. A robots.txt update takes ten minutes. JSON-LD schema is a morning's work for a competent developer. A content authority network — subdomain hubs, pillar pages, cluster articles — is a longer project, but each piece compounds the others.

What requires the most urgency is the robots.txt update, specifically before September 15, 2026, when Cloudflare's new defaults take effect. After that date, sites that have not explicitly permitted AI crawlers will have them blocked by default. The window is open. It will not stay open.

What Forge Vertical does about it

GEO implementation is a core service at Forge Vertical. We deploy llms.txt files, context-ai.txt supplementary grounding files, JSON-LD entity graphs, AI-optimised robots.txt, and content authority networks for clients across South Africa and globally.

The GEO file stack for a single domain — llms.txt, context-ai.txt, updated robots.txt, and sitemap — starts from R3,300 once-off. For businesses that need the full content authority network, we scope it per project.

If you want to know what AI systems currently understand about your business — and what they don't — start here.