Approaching AI Search Optimization And Classic Google Search Differently

AI search and classic Google search overlap, but they don’t behave the same way. Here’s how we separate the two in our own SEO work, and why the difference changes what we optimize, measure, and protect.

A note on how we hold these claims. A lot of what follows is built on close study of observable data: our own client logs and tests, plus published research. But the systems underneath, such as retrieval pipelines, AI indexers, answer engines, are largely opaque, and a pattern we can see is still a theory, not a law.

So treat the mechanism-level claims here as tested working models we’re confident enough to act on, not settled facts. When we say a signal “matters,” read it as “behaves as if it matters in everything we can measure.” We’d rather be precise about our certainty than oversell it.

There’s a bad habit floating around the SEO industry right now. A lot of people keep treating AI search optimization like a slightly newer label for the same old work, then act surprised when the results don’t line up.

In practice, the overlap is real, but the systems are not behaving the same way, and the difference shows up in how content gets found, how it gets pulled into answers, and how brands get represented across search surfaces.

From the agency side, this matters because clients do not care about labels. They care about visibility, qualified traffic, leads, and whether their brand is being described accurately.

When we look at AI Overviews, ChatGPT, Gemini, Perplexity, Copilot, and classic Google search side by side, the pattern is clear enough to shape a serious strategy around it. The work still lives inside SEO, but the inputs, the priorities, and the measurement layer need a wider lens.

This is how we think about it in our own SEO and AI/LLM optimization work.

The Main Mistake Most Teams Are Making

A lot of content teams still build around a single target keyword, publish a page, and expect search systems to reward that one focus. That used to be a workable mental model when search was narrower and more linear. Today, it is too small.

AI search works across a different retrieval pattern. The system may not simply look for the page that ranks for the exact query typed by the user. It may expand that query into several related searches, scan multiple sources, compare passages, and build an answer from a wider set of inputs. In other words, the original query is only part of the story.

That is why the old “rank for the keyword and you are done” mindset falls apart so quickly in AI surfaces. The page might rank well in Google, yet fail to appear in AI results. Or it might appear in AI results because it answers a related subtopic exceptionally well, even if it was never built around the primary keyword in the first place.

For anyone managing organic growth, this changes the job. We are no longer optimizing for one search string alone. We are optimizing for a topic ecosystem, a set of related questions, a content structure that can be parsed cleanly, and a brand presence that can survive synthesis from multiple sources.

Why The Research Matters Here

This topic gets debated a lot, so the research is the part worth paying attention to. Several independent studies point the same way, even when the exact numbers differ.

ZipTie found that traditional Google rankings line up with AI visibility only about 45% of the time, and that the link is weakening fast.

By their measure, only 38% of pages cited in AI Overviews still rank in the organic top 10, down from 76% a year earlier. Because AI answers typically draw from just three to five sources, a strong blue-link position is now a weak guarantee of being one of them.

That alone is enough to push back against the claim that AI optimization is nothing more than classic SEO with a different name.

Profound makes the same point from another angle: Answer Engines rarely use the query as typed, they fan it out into multiple high-intent sub-queries, and those are what actually decide which sources get cited.

As they put it, creating content without considering query fan-outs is “like doing SEO without knowing your keywords.” The system often isn’t working from the exact query the user typed; it expands the question into related sub-questions, then pulls from the pages that answer those side questions well.

Semrush study of more than 10 million keywords added another layer: AI Overviews appear most often on low-volume queries, nearly 60% of triggering keywords have 100 or fewer monthly searches. Classic keyword research, built around volume thresholds, simply doesn’t see most of that surface.

Now the honest part, because we said up top we wouldn’t cherry-pick. Two findings in that same Semrush data cut against the easy “AI is eating your traffic” narrative, and we’d rather state them plainly than bury them.

First, AI Overviews did not automatically increase zero-click behavior, on the keywords studied, the zero-click rate actually dipped slightly, from about 33.8% to 31.5%, after AI Overviews appeared. Second, AI Overviews still skew toward low-commercial-value queries: roughly 95% show no ads or minimal commercial intent, and high-CPC commercial terms remain largely untouched for now.

The implication is not “ignore AI search.”

It is that, today, AI search is mostly a top- and mid-funnel visibility and brand layer, not yet where most transactional clicks are won. That’s a real nuance, and it’s also exactly why the shape of the funnel is changing. We’d rather a client hear this framing from us than discover it after over-indexing on the wrong layer.

From an agency perspective, these studies change how we diagnose opportunity.

We do not just ask, “What keyword should this page rank for?” We also ask, “What related questions would an AI system generate, which passages on this page can answer them, and where are the holes in the surrounding content ecosystem?”

What Query Fan-Out Means In Practical Terms

Query fan-out is one of the most useful concepts in this entire conversation, and it deserves a clean explanation.

A user asks one question. The system then breaks that question into several smaller or related questions behind the scenes.

Those related queries may include definitions, comparisons, use cases, supporting facts, local intent, commercial intent, or even adjacent problems the user did not type directly.

The AI then retrieves content across those related searches and uses the pieces it trusts most.

So the page that wins in AI search is not always the page that most aggressively repeats the exact target keyword. It may be the page that covers the broader topic set cleanly, has clear passage-level meaning, and supports the adjacent questions the system expects to answer.

This is where a lot of “just SEO” arguments break down.

Traditional keyword targeting still matters, but it is now only one layer inside a bigger retrieval pattern. A strong page needs to be relevant to the primary topic and cleanly connected to the related topics surrounding it.

Why Traditional Keyword Tools Miss So Much

Most SEO tools were built for a world where keyword volume was the main proxy for opportunity. That worked well enough when search behavior was easier to measure. It works much less well now.

AI systems can generate or use queries that never show up in standard keyword databases, because those databases rely on observed searches above a certain threshold. A query that matters inside AI retrieval may be too small, too new, too implicit, or too synthetic to register in classic research tools.

That does not make it unimportant. It just makes it harder to see and, as the Semrush data above shows, that “invisible” slice is where a lot of AI Overviews actually fire.

This creates a serious blind spot.

Teams can spend weeks optimizing around visible keywords while the AI system expands into related question sets that were never in the brief. The result is a content strategy that looks tidy in a spreadsheet and incomplete in the real environment.

As an SEO provider, this is one reason we never treat keyword research as the final answer. It is the starting map, not the destination. We use it to understand the topic, then widen the frame to include query fan-out, semantic relationships, user intent layers, and the content structure needed for machine retrieval.

Why AI Search Feels Closer To Reputation Management

One of the most useful shifts in thinking is this: AI search often behaves more like reputation management than standard ranking work.

That sounds dramatic, but it isn’t. A brand’s answer in AI results is often built from a mix of the brand’s own pages, reviews, articles, third-party listings, community posts, and other sources that contribute to a broader consensus. In many cases the system is not just asking, “What does this site say?” It is asking, “What is the collective picture around this brand, product, or topic?”

This changes the strategy in a few ways.

First, your own website matters more when it helps define the canonical version of the truth. A clear pricing page, a well-written about page, a strong FAQ section, or a direct services page can help anchor the narrative. Second, what other sites say about you can shape the answer just as much as your own copy does. Third, your content needs to be consistent across the ecosystem, not fragmented into contradictory signals.

This also reframes the hardest question in AI search: what do you do when the system is simply wrong about you? You can’t edit the model.

What you can do is strengthen the entity, the machine-readable, cross-source record of who you are. In practice that means consistent Organization and about schema, an identical set of brand facts (services, locations, founding details, specializations) everywhere they appear, and, where it’s warranted, a presence in the sources that feed knowledge graphs, such as Wikipedia and Wikidata. When the underlying entity is clear and consistent, stale or hallucinated descriptions have less room to survive. When it’s fragmented, the system fills the gaps with whatever it happens to find.

That is one reason many brands need to rethink whether hiding useful information is still a good commercial decision. In classic lead-gen SEO, a company might avoid publishing pricing because it prefers to funnel people into a sales conversation. In AI search, that can backfire, because the system will go elsewhere to fill the gap, and it may rely on older, weaker, or less accurate third-party sources.

We see this as a brand strategy issue, not just a page optimization issue.

What To Do About Pricing, Positioning, And Brand Narrative

One of the clearest examples of this change is pricing.

If a brand does not publish pricing, AI systems will often look for that information somewhere else, directories, review platforms, competitor comparisons, forum posts, or old content that no longer reflects the current offer. The brand loses control of the story. In the AI layer, silence rarely stays silent. The system fills it.

For service businesses, this can affect more than pricing. It can change how the brand is described in terms of service scope, specializations, years of experience, audience, turnaround time, and even the type of client it serves. A stale third-party summary can linger far longer than it should, especially when the source ecosystem keeps repeating it.

The practical answer is not to publish every detail everywhere without thinking. Some businesses genuinely can’t post fixed prices because the work is scoped per client, and that’s fine.

But “we can’t publish an exact number” is not the same as silence. AI systems reward a concrete answer over a blank, so give them one: a starting rate, a typical range, a “from $X” anchor. The goal is to decide what the brand needs to control, then make sure the core pages say it clearly:

• a clear services page
• a current pricing page or pricing guidance (even a range)
• an updated about page
• a strong FAQ page
• comparison pages where relevant
• review and testimonial signals that support the current positioning

The goal is visibility with the right message attached.

The Technical Step Almost Everyone Skips: Let The AI Crawlers In

Before any of the on-page work matters, the AI systems have to be allowed to fetch your pages at all, and a surprising number of sites quietly block them without realizing it.

The major AI crawlers each have their own user agent: GPTBot (OpenAI/ChatGPT), ClaudeBot (Anthropic), PerplexityBot (Perplexity), and Google-Extended (Google’s AI training and grounding).

If your robots.txt disallows them or, just as common, your CDN or WAF blocks them by default, with Cloudflare’s bot controls being a frequent culprit; you can do everything else in this article perfectly and still be invisible in the answer layer.

So step zero of an AI-search audit is boring and decisive: confirm that the crawlers you want are actually allowed, at both the robots.txt and the edge/WAF level.

This is also a genuine decision, not just a checkbox. Some publishers deliberately block AI training crawlers to protect their content, and that’s a legitimate choice, but it should be one you made on purpose, not a default you inherited from a boilerplate config.

Why Crawl Timeouts And Server Behavior Matter More Than People Think

Once the crawlers are allowed in, they still have to successfully fetch the page, and some AI systems fetch in real time, under tight latency budgets, while a user waits for an answer. A page can be excellent and still get skipped because the fetch timed out before the content came back.

One signal we’ve dug into is the HTTP 499 status, a non-standard, NGINX-specific code that records the client giving up before the server finished responding. (On Apache, IIS, Cloudflare, Vercel, or Fastly you’ll see different signals — 408s, 524s, dropped connections — for the same underlying behavior.)

We treat it as one log signal among several, not a universal law, and we ran a small paired test on our own client portfolio to pressure-test the idea. The full write-up, caveats and all, is here: Investigating 499 Response Codes as an AI-Search Ranking Factor.

The general point holds regardless of stack: slow responses, rendering issues, and blocked or JavaScript-dependent content all reduce the odds of being used in an AI answer. In our process, technical fetch quality is now part of the AI-search foundation, not a separate cleanup task.

Where We Land On llms.txt

llms.txt is an emerging convention: a short Markdown file at your site’s root that hands AI tools a curated map of your most important pages. Think of it as robots.txt’s counterpart for AI. And, like robots.txt, it changes nothing for human visitors.

It’s also genuinely contested, so here’s our honest read rather than a hard sell: Google’s own guidance famously depends on which team you ask.

Google Search says you don’t need it, John Mueller has compared it to the long-ignored keywords meta tag, while Google’s Chrome team added an llms.txt audit to Lighthouse under its new “agentic browsing” checks.

The reconciliation is that the two teams are optimizing for different things: Search is talking about rankings and AI Overviews, where llms.txt does nothing today; Chrome is talking about readiness for browser-based AI agents, where a clean machine-readable map of your site genuinely helps.

So we won’t claim llms.txt lifts rankings; the evidence doesn’t support that, and we said at the top we wouldn’t oversell. The argument for it is simpler, and it’s about asymmetry.

The file takes an hour to write, it’s harmless, agent and developer tooling already reads it, and Google’s own Chrome tooling now audits for it.

Set that against Google’s long track record of being opaque about what actually moves its systems, and stubbornly refusing a 30-minute file because Search says it isn’t a ranking factor is the worse bet. We treat llms.txt as cheap insurance and basic hygiene, a clean, curated map of your best pages, not a growth hack.

On that framing, it’s a clear should-have today and, as agentic browsing matures, plausibly a must-have. The risk of skipping something this small outweighs the risk of being stubborn about it.

Why Metadata Matters More Than Many SEOs Expected

In classic Google SEO, title tags still matter and meta descriptions still matter indirectly, but they are often treated as click-through assets rather than core visibility drivers. In AI environments, our testing suggests they can play a larger role, because they appear to help the system decide whether to fetch the page in the first place.

A useful way to think about it: before committing to a deeper read, a retrieval system has little more to go on than a page’s title, description, and URL.

When those signals are clear, relevant, and aligned with the page content, the page seems to have a better chance of being fetched and used.

When they’re vague or mismatched, it seems to move on quickly. We can’t see inside the pipeline to prove the weighting, but the observable behavior lines up with treating metadata as machine-facing context, not just a SERP snippet.

For that reason, we now write metadata to tell the machine what the page is about, who it is for, and why it exists. This is especially useful on commercial pages, service pages, and content that needs to be understood fast.

Why Semantic URLs Matter More In AI Search

URL structure has always been a topic people argue about too much. In classic SEO it is usually a small signal, and changing a live URL can cause more harm than good.

In AI search, the slug appears to carry more interpretive weight than many teams expect. A URL that clearly reflects the topic seems to help the system understand the page faster; a messy, generic, or overly shortened structure gives it almost nothing to work with.

This does not mean you should rewrite every existing URL on your site, in most cases that’s a bad trade. But moving forward, it’s smart to build cleaner slugs:

• Better: /seo-services/ai-search-optimization/
• Weaker: /blog/post-1847/

We recommend treating the slug as part of the page’s meaning, not just its file path.

Why Recency Has Become More Visible

Recency has always mattered in some search categories, but AI systems seem especially sensitive to freshness signals. That doesn’t mean older content can’t be cited, it means the system may place more weight on visible recency than many marketers expect, particularly for topics where information changes often.

The point is not to fake freshness. Bumping a date without changing the substance is exactly the kind of signal these systems are learning to discount, and it erodes trust when readers notice. The point is to make genuine updates visible: change the content, then let the date reflect that real change. For evergreen content, that usually means:

• refreshing the introduction
• updating examples
• replacing outdated stats
• adding newer internal links
• revisiting related FAQs
• noting the date of the last meaningful update

This works in classic SEO too, but it matters more in AI retrieval, because the system is often trying to resolve present-tense questions, not just archive material.

Content Chunking Is Not A Trendy Buzzword

A lot of people dismiss content chunking as a fancy word for breaking up paragraphs. That’s too simplistic.

Chunking is about creating smaller units of meaning that can stand on their own. It helps both audiences. For people, it makes content easier to scan, read, and trust.

For AI systems, it makes information easier to isolate, compare, and cite, because modern retrieval works at the passage level, it doesn’t need a wall of text, it needs passages with clear semantic boundaries.

A strong chunk usually does a few things well: it covers one idea, uses a clear heading, starts with the point rather than the preamble, includes a supporting detail or example, and avoids blending multiple topics into one paragraph.

And no, bigger context windows don’t make this obsolete, a common objection we don’t buy. Larger models still compress, rank, and filter what they read. When the source material is messy, that compression loses more meaning; when it’s structured, more of the useful signal survives.

The system can only extract so much from a page at once, so clear sections, clear boundaries, and clear topic progression make the extraction job easier. Good structure isn’t a compromise you make for machines. It’s a performance asset that happens to help humans at the same time.

Good Structure And Good Writing Aren’t In Conflict

When we restructure content for clients, we look for three recurring problems.

First, paragraphs that cover too much at once, definitions, benefits, limitations, and implementation crammed into one block, forcing both the reader and the system to do extra work.

Second, sections without clear boundaries, where a heading says one thing but the paragraphs underneath drift into three other topics.

Third, sentences that hide the core point halfway through, fine for a novel, not ideal for search content.

A stronger version opens with the direct answer, supports it with detail, keeps adjacent ideas in separate blocks, and uses headings that match real user intent.

There’s a misconception that AI-friendly content has to sound flat. It doesn’t. We write plenty of content with tone, personality, and rhythm.

What changes is the internal discipline: we’re more deliberate about where the point lives in a paragraph, how much one section tries to do, and how clearly the text signals meaning.

A useful test, read a paragraph and ask whether the main point can be extracted quickly without losing the rest of the message. If the answer is no, the paragraph probably needs to be split.

Strong writing and strong structure support each other; the trick is keeping the voice while making the meaning easy to lift.

The Role Of Structured Data And Accessibility

There is still debate about how much schema markup matters for AI visibility. From what we see in the field, structured data remains valuable because it helps systems understand entities, relationships, and page purpose more cleanly. We wouldn’t reduce it to “use schema because Google likes rich results”, that’s too narrow. In many cases the value comes from machine readability across systems, not only from visible SERP enhancements.

Accessibility matters for the same reason. Clean HTML, meaningful headings, proper labeling, and content that doesn’t depend too heavily on JavaScript all make a page easier to process.

If a page is difficult to render or interpret, the AI system has less to work with. Structured data, accessibility, and clean markup all support the same goal: making the page easier to understand.

Why Reddit, YouTube, And UGC Keep Showing Up

When people ask what sources AI systems trust, the answer is not always the sources they expect.

Community and platform content often play a major role, especially for comparison queries, product questions, service research, and troubleshooting.

Reddit is frequently cited because it holds a huge volume of conversational, problem-solving content. YouTube matters because it packages useful information in formats that are easy to mine, especially when transcripts and metadata are available.

This tells us something about strategy: we shouldn’t only think about the brand’s own domain. We should also think about where the surrounding discussion happens.

That doesn’t mean chasing every platform at once, it means knowing which ones influence your category, which might include Reddit threads, YouTube explainers, review platforms, comparison articles, industry directories, forums, and durable social posts.

The obvious next question is how you influence those sources, and this is where we’d caution against the easy answer. You don’t manufacture Reddit threads or seed fake reviews.

Beyond breaking platform rules, it backfires: communities are good at spotting and punishing it, and a burned brand reputation is far harder to repair than a thin one. The durable approach is to earn the mentions, show up as a real participant, let employees and founders engage transparently, support actual customers where they’re already talking, and make your own content good enough to be referenced on its merits.

The goal is to be part of the consensus honestly, and not to fake it.

Why PR Still Belongs In The Mix

Public relations remains highly relevant because AI systems need source material, and PR helps seed consistent coverage across the web.

This is not old-school vanity PR for the sake of mentions. It is strategic distribution. When the brand story appears in multiple credible places, the model has more material to work with and less room to rely on stale summaries or conflicting third-party descriptions.

For SEO teams, this means working more closely with PR than many did in the past. A link is useful. A mention is useful. A clear, consistent brand statement in the right publication can be even more useful when AI systems are assembling answers. PR is not only about awareness, it’s about feeding the broader knowledge environment.

Why Classic SEO Still Matters

None of this means classic SEO stopped mattering.

Google still relies on strong content, clear structure, relevant entities, internal linking, topical authority, technical health, and backlinks. Those fundamentals still support visibility in the classic search layer and often help in the AI layer too. The difference is not that classic SEO became obsolete. The difference is that it is no longer enough on its own.

We still care about technical crawlability, topical relevance, internal linking, content depth, page speed, semantic clarity, strong title tags, and search intent alignment. We now also need to understand how those elements behave inside AI retrieval, not just inside the blue links.

Why We Treat AI Search Optimization As A Separate Workstream

In our agency, we treat AI search optimization as a distinct workstream inside SEO because the goals and success conditions are different.

Classic SEO is mostly about winning ranked placement and earning clicks. AI search optimization is about being selected, summarized, cited, and represented accurately across answer environments. Sometimes that leads to traffic. Sometimes it leads to brand lift. Sometimes it leads to a qualified click from a user who has already educated themselves inside the AI interface.

The value flows differently. The work cannot be measured only by traffic sessions, because the influence often happens before the visit. This is also why AI search gets budget attention from decision makers who tuned out traditional SEO, the use case is easier to explain in business terms.

A brand wants to be in the answer. A brand wants to be described correctly. A brand wants to show up where buyers are asking questions. That’s a different conversation from “we need 12 more blog posts for this keyword cluster.”

How We Actually Start A Content Restructure Project

When we take on a page that needs to perform better in AI search, we usually begin with the page itself, then widen out to the surrounding ecosystem.

1. Read the page like a machine and like a human. Look for sections that are overloaded, vague, or off-topic. Good information is often buried inside too many mixed paragraphs.
2. Split mixed ideas into separate chunks. Each section should carry one job.
3. Tighten the headings. Headings should tell the reader what comes next and create a clear retrieval boundary.
4. Add specific supporting details. Data, examples, comparisons, and direct statements. Strong content is content with evidence.
5. Review metadata and URL structure. Check that the title, description, and slug support the topic clearly.
6. Check for technical friction. Confirm the AI crawlers are allowed, then look at rendering, server response, and crawl logs.
7. Review off-site signals. Look at Reddit, YouTube, PR coverage, review sites, and directories to see what consensus already exists.
8. Measure the impact over time. We don’t expect a single prompt to tell the whole story. We look for directional improvement across multiple checks.

Why Measurement Is Messy And Still Worth Doing

A lot of people get stuck on the fact that AI visibility is probabilistic. They’re not wrong, the same prompt can return different results depending on context, platform, and time. But classic rank tracking was never perfect either. Search Console already gives averages, not pure truth. AI systems just add more variability, which means measurement needs more sampling, not less seriousness.

In practice, that means sampling deliberately rather than spot-checking. We run a defined set of prompts across multiple surfaces, ChatGPT, Gemini, Perplexity, Copilot, and Google’s AI Overviews, and repeat them on a schedule, because one prompt on one day tells you almost nothing.

Purpose-built tools (Profound, Ziptie, Otterly, and Peec AI, among others) help automate that sampling at scale. And we separate three metrics that often get blurred together:

• Citation rate — are you linked as a source?
• Mention rate — are you named, even without a link?
• Share of voice — how often you appear versus competitors for a topic.

Read this way, AI visibility is a precision problem, not a certainty problem.

The right questions are directional: Is the brand showing up more often? Is it being represented more accurately? Are the right pages being cited? Are competitor answers getting weaker while ours get stronger?

Where Most Brands Should Start

For most businesses, the best starting point is not a giant AI overhaul. It’s a careful review of the pages that matter most: service pages, product pages, pricing pages, about pages, comparison pages, FAQ pages, and high-value blog posts.

Then ask a few practical questions:

• Are the AI crawlers even allowed to fetch the page?
• Is the topic clear in the title, slug, and description?
• Does each section cover one thing well?
• Are there enough explicit answers for related questions?
• Does the site tell the same story as third-party mentions?
• Are there technical barriers to crawl and fetch?
• Does the page look good for both people and machines?

That’s usually enough to reveal the first wave of improvements.

Final Thought

We’re past the point where it makes sense to pretend AI search is just a rebrand of classic SEO. The fundamentals still matter, but the retrieval behavior has changed enough to require a wider strategy. Query fan-out, structured content, metadata, semantic URLs, recency, crawler access, technical fetch quality, and brand consensus all play a role now.

The SEO teams doing the best work aren’t throwing out the old playbook, they’re extending it. They balance human readability with clear structure, while actively shaping how their brand is interpreted so the final output stays consistent with what they actually offer.

That’s the work now. If you’d like a second set of eyes on where your site already wins and where it’s invisible to AI, that’s exactly what we do at ZetaTheorem, from AI/LLM optimization to content strategy. Send us your site for a free review.