The Math Is Clear: LLMs Have Fundamentally Changed Search
There’s a very heated debate going on in the Search Engine Optimization (SEO) community right now. The debate centralizes around what to call techniques used for optimizing visibility in AI answers produced by Large Language Models (LLMs).
One side of the debate feels there should be a new name for these techniques; “AI Search”, “Generative Engine Optimization” (GEO), “Answer Engine Optimization” (AEO) and AI SEO have all been proposed.
The other side feels that there are very few new methodologies at play, if any, with the introduction of LLMs to the space and therefore no new nomenclature is actually warranted. Simply put, it’s still “just SEO”
The following meme has become ubiquitous due to its ability to capture this debate:
The naming convention itself is not of interest to me. As I’m not an SEO expert myself, I honestly don’t feel best equipped to weigh in on this aspect of the debate.
To that end, by no means is this article making the rage-bait case that “SEO is dead”. I am of the strong opinion that SEO experts are actually best positioned to optimize visibility in LLM’s.
However, what I illustrate in this article, is that from a mathematical/algorithmic perspective, the introduction of LLM answers to the field of search is the single largest mathematical delta the field has experienced since its nascency.
And, while I can agree there are no net new methodologies at play, I will also show - mathematically - how the relative importance of certain strategies has changed significantly in response to the new search infrastructure at play.
Introduction
For most of the last 25 years, web search has been defined by a single paradigm: ranking documents.
Google, Bing, DuckDuckGo — all of them take a query q, compute a relevance score R(q,d) for every document d, and return the top-k.
Entire industries (SEO, content marketing, programmatic media) evolved around that function.
But over the last 18-24 months, something very different from “better ranking” has emerged.
Search switched categories altogether.
The underlying type of the function changed.
LLM-based search doesn’t return a list of ranked documents. It returns an answer generated by a conditional language model, built on a retrieval pipeline whose scoring behavior differs from any prior search system.
As a machine learning engineer, this is my key assertion:
Mathematically, classical SEO and LLM search optimize different objective functions that are not monotonic transforms of each other.
Even perfect classical SEO cannot ensure the same level of LLM visibility or influence.Hence, the introduction of LLM’s to search is the largest mathematical/algorithmic shift in the space since it started.
Below, I’ll show why, and demonstrate a concrete example that proves it.
But first, I’ll list some definitions, assumptions and disclaimers.
0. Definitions, Assumptions and Disclaimers
Definitions
“Classical SEO” —> The field of search engine optimization before LLM-generated answers were introduced to the user experience
“LLM-based search” —> The field of search after LLMs were introduced.
“Monotonic” —> A function that is entirely non-decreasing or non-increasing over its entire domain. Functions that are non-monotonic relative to one another are, mathematically, fundamentally different.
“SERP” —> Search Engine Results Page i.e. the web page shown by an engine in response to a user’s query
Assumptions
It is widely established that there is more complexity to any search algorithm than math itself. Specifically, there are developers in the loop hired by for-profit companies that have a certain level of deterministic control over the output of the models. For this proof, I will assume that these developers have the same level of control over all iterations of search products - pre- and post-introduction of LLM’s. Hence, their influence can effectively be “canceled out” for the purposes of this document.
This proof does not take any paid motions (e.g. paid ads) into account. Nor should it need to. Direct injection of capital into the search problem supersedes the math at play.
Disclaimer
I am a co-founder and the CTO of a company called Noble, which uses automation to get companies mentioned by the sources LLM’s cite. However, it is widely agreed by experts in the field that this form of digital PR has value for every embodiment of SEO (pre- and post-LLMs); this proof, therefore, is not intended to convince anyone of the value of our product. I worked on this solely as a quasi-academic exercise and motivated by recent, “passionate” online debates in which I’ve engaged.
1. Classical Search: A Ranking Function
For decades, search followed the same structure:
Ranking formulation
This formula describes a ranker R that ranks a set of results d in response to a query q.
Simply, the entire system returns a ranked list of documents. In other words, they looked something like this:
Classical SEO’s optimization objective
Earlier embodiments of SEO, when searches resulted in blue links only, aimed to maximize expected click-through rate:
But Rand Fishkin showed over 7 years ago that clicks started to decline as the primary metric to evaluate SEO performance. This drop in clicks aligned with Google’s release of features like instant answers, knowledge panels and people also ask (PAA). Searches started to look like this:
Or like this:
It is essential to note, however, that in both cases the problem was still formulated as a ranking problem. These developments were merely UI changes and not evoked by changes to the underlying math.
It is now widely established that the most reliable metric of SEO performance is organic traffic to a page. This makes our life easier: we can leverage an apples-to-apples performance metric knowing that LLM-generated answers rarely explicitly result in clicks.
In math terms, modern SEO techniques - the modern, credible kind based on SERP-shape and intent alignment — aim to maximize organic traffic to a site/page. We can describe the objective like this:
Where, instead of trying to maximize for CTR, modern SEO is trying to optimize for organic traffic OT.
However, even as underlying ranking models (BM25 → RankBrain → BERT) and performance metrics (CTR —> OT) evolved, the underlying type signature never changed:
A set of documents, ranked by a scoring function.
2. LLM Search: A Generative Modeling Function
LLM-based search does something categorically different. Specifically, it generates an aggregated answer in response to a query.
Generative formulation
Where:
Cq = retrieved passages
pϕ = decoder (transformer)
a = answer, notably not a ranked list
The above formula essentially describes the process of retrieval augmented generation (RAG), in which semantic chunks that are highly relevant to the original query are pulled from a vector database of internet data and used to tune the LLM’s learned answer in near real-time.
LLM visibility objective
To influence an answer, content must both be retrieved and influence the generated string:
In layman’s terms, the above formula is simply stating that the new formulation of LLM-based search optimization requires creating a document d that influences the answer in the LLM’s response. We can take this one step further and frame this using our agreed upon optimization metric, organic traffic, from the previous section:
We can clearly see that this objective is different from rank-based optimization, which defined the field of search before the introduction of LLM’s.
Unequivocally, a different mathematical objective changes the nature of the problem. Hard stop.
But what does this actually mean in practical terms? Does SEO really need to change in any way? Do we actually need to do anything new?
3. The Example Using Strong, Modern SEO
Applying the mathematical assessment to the real world of SEO, my hypothesis is that optimizing for LLM’s doesn’t require any “net new” techniques. But, I do think the hierarchy of importance of various SEO strategies are shifting significantly as a byproduct of the magnitude of the mathematical changes.
I’ll show this by crafting two theoretical “sample pages” both using modern, top-tier but divergent SEO techniques recommended by experts. I’ll show how mathematically one of the pages would be expected to rank higher in the pre-LLM SEO paradigm, while the other would rank higher after the introduction of LLM’s.
Strategy A (θA): SERP-Consolidated Pillar Page
One massive “ultimate guide”
Consolidates multiple intents
Matches SERP shape
Typically ranks higher for head terms
Very modern, very strong SEO
It’ll look something like this:
Strategy B (θB): Intent-Split Scoped Pages
One page per question
Short, specific, tightly scoped
Exact-phrase alignment
Early answer in first 100–200 words
Zero topic drift
Also very modern and very strong SEO
This one looks a bit more like this:
A user will now search with a query q. To keep it consistent with our examples above, we can say our query is “what does zapier say about remote work”
Empirically:
In response to the query q, the pillar page, A, will most likely rank higher in a traditional SEO paradigm, pre-LLM’s. This is corroborated by empirical evidence as recent as 2023.
Now let’s analyze how LLM’s would answer query q.
4. Why LLM Retrieval Strongly Prefers θB (the Opposite Strategy)
Here, I’ll show why an LLM-generated answer to the same query will most likely pick the page leveraging strategy B.
4.1 BM25 Length Normalization Penalizes Long Documents
The retrieval step in RAG leverages the BM25 algorithm to rank semantic chunks.
BM25 scoring:
Longer pillar pages → high |p| → dilution → lower sparse match →less likely to be retrieved
4.2 Dense Embeddings Penalize Mixed Topics
Pillar pages will be rich in content, containing various topics and will therefore mix multiple “semantic regions”. For a pillar page mixing two semantic regions:
Convex mixtures (as above) lower cosine similarity:
Scoped pages as in strategy B preserve embedding purity.
4.3 Retrieval Uses Max-Over-Passages
Most systems retrieve by chunking a document into passages pj:
Scoped pages putting relevant content in the first chunk will lead to a higher retrieval probability based on the above equation.
4.4 Therefore, Retrieval Selects θB Over θA
As shown above, it is highly likely that documents following strategy B will be chosen over documents following strategy A with LLM-based answer generation
Thus:
and therefore:
Combining the inequalities, and to re-iterate the above:
but
Simply put, strategy A is expected to perform better in a pre-LLM SEO paradigm, and strategy B is expected to perform better with LLM-based answers.
No strictly monotonic function f can satisfy
This is the exact condition required to prove the paradigms are mathematically inequivalent.
QED.
And you don’t need to just trust my math, here’s the traditional search/ranked result to the query “what does zapier say about remote work” (pillar page is #1):
But the pillar page is nowhere to be found in the AI Overview, yet the intent-scoped page is*:
*Kudos to Zapier, they’re optimizing for all search paradigms
5. Why This Is the Biggest Shift in Search History
Based on the math at play and a concrete application of that math, I assert that the introduction of LLM-based answer generation to search is the largest shift in the industry since its begining.
There have been a long list of algorithmic changes to core search since its nascency including:
RankBrain
BERT
But with all these changes, the framing of the problem remained inside the structure:
i.e. a ranked list of documents.
The introduction of LLM’s alter:
the type of the output,
the objective function and
the probabilistic nature of search.
In the following ways:
(1) Type Signature/Type of Output
As stated above, the type of output for LLM-based search moved from a ranked list of documents to the following framing:
Essentially, query to a single answer.
(2) Optimization Objective
Pre-LLM search was a learn-to-rank problem; from the perspective of the algorithm: how can we ensure the most relevant and authoritative source shows up first. It could be measured by metrics like Precision@K
Post-LLM search seeks to ensure the single answer to a query is as accurate and reflective of the sources (RAG chunks) as possible. Here, a metric like Attributed QA or ColBERT might be used to measure the quality of a search result.
Different objectives represent a major mathematical shift.
(3) Probabilistic Nature
Lastly, traditional search is deterministic: for a fixed index and fixed query, the ranking output is identical every time.
In contrast, many have remarked on the fact that different users asking the same query to an LLM will generate slightly differing answers from one another. In fact, the same user writing the same query with a time gap between the queries may even see slightly different answers. This is because LLM’s use probabilistic sampling to generate their answers, a stark contrast with the deterministic argmax framing of traditional search.
Objectively, no previous shift in search history changed the math this fundamentally.
6. Why I’m Writing About This
I’ve spent my entire career (and much of college) studying machine learning and building production machine learning systems. I did this first at Tissue Analytics, where I worked on digital imaging models leveraging computer vision before and after deep machine learning became ubiquitous.
I am now attempting to study the changes in search with the same level of rigor. In other words: how modern search actually works under the hood.
There’s a lot of debate in the SEO world right now with phrases like “SEO is dead,” “LLMs kill SERPs,” or “AEO is just SEO with a new name.” But beneath all the discourse is something surprisingly missing: the mathematics.
Search has always been a mathematical field yet most of the current debates are happening without reference to the underlying mechanics.
This post exists to change that.
I’m not here to declare SEO dead. Far from it: SEOs are clearly the best-positioned operators to win in the new ecosystem because they already understand search intent, content structure, and the economics of visibility. But pretending that generative search isn’t a massive shift is bordering on irresponsible. The math clearly proves it.
My goal is to bridge the gap between ML research and practical search strategy:
clear math, real retrieval behavior, reproducible evidence, and zero ragebait. Changes in any field bring with them a massive opportunity, and this should excite us all.
Cool read. This will go to our next release
Love this 😍. I am building a solution to address exactly this. Getting cited by these LLM’s is such a flex.