ZetaTheorem retro arcade banner for the article: 7 skills marketers need in 2026.
ZETATHEOREM
ONLINE
NOW PLAYING
7 SKILLS TOLEVEL UP IN 2026
A MARKETER’S GUIDE TO THE AI ERA
XP
P1 READY
Quick disclaimer before we dive in: this isn't a peer-reviewed study, it's a field report. Everything below comes from years spent in the marketing trenches: shipping campaigns, breaking dashboards, sheepishly rebuilding them, and collaborating with a lot of brilliant people along the way. Content folks who can make a single CTA sing. Analytics wizards who build dashboards so intricate they deserve their own README. GTM teams who can turn a boring feature release into an actual moment. These are my observations, not commandments. Take what's useful and argue with the rest.
The marketing job description has quietly mutated. Five years ago you could pick a lane, like "I'm a content person" or "I'm a paid person," and comfortably ride it for a decade. In 2026, the lanes are merging, and the merge lane is full of robots.
Here's the thing nobody says out loud: AI didn't replace marketers. It raised the floor. Anyone can now generate a passable blog post in 30 seconds, which means a passable blog post is worth roughly nothing. The edge didn't disappear. It just moved somewhere harder to copy. Here are the seven that matter most in 2026: five new edges the AI shift made urgent, and two old truths the robots still can't touch.
1. Knowing how AI search actually works (GEO/AEO)
This is the big one, so it goes first. People are increasingly asking ChatGPT, Perplexity, and Google's AI Overviews instead of scrolling ten blue links. ChatGPT alone reached roughly 800 million weekly users in 2025, and Gartner predicts traditional search volume will fall 25% by 2026 as that behavior shifts.
So the new skill is making your brand quotable by machines: clean structure, clear claims, real expertise the models can lift. We call it AI/LLM optimization, and honestly, it's the most fun part of SEO right now.
2. Marketing automation (the unglamorous superpower)
Nobody puts "built a Zapier flow" in their bio, but they should. By 2025, around three-quarters of marketers had adopted AI, yet most still use it to fire off one-off, generic campaigns. The gap between "I used AI once" and "AI quietly runs my lead routing, enrichment, and follow-ups while I sleep" is enormous.
The skill isn't knowing every tool. It's systems thinking: spotting the repetitive task, mapping the trigger, and wiring it together so it runs without you. The marketers who compound their output are the ones who automate the boring 80% and spend their human hours on the 20% that actually needs a brain.
3. Go-to-market (GTM) thinking
You can be the best campaign executor alive and still flop if the positioning is wrong. GTM is the connective tissue: who's the customer, what's the wedge, what's the message, and how do all the channels move together at launch.
It matters more than ever because buyers do their homework alone now. Gartner found B2B buyers spend just 17% of their total buying time meeting with potential suppliers. The rest is independent research, comparison, and quiet judgment. You have to win the room you're not even in. That's a strategy problem before it's a content problem, and it's exactly why GTM-minded marketers are gold in fast-moving spaces like SaaS.
4. Data and analytics fluency
You don't need to become a data scientist. You need to read a dashboard without panicking and ask the right question of it. "Traffic is up" is not insight. "Traffic is up but conversions are flat because the new landing page loads in four seconds on mobile" is a marketer earning their salary.
This skill is getting more important in the AI era, not less, because AI will hand you a confident answer whether or not it's correct. Someone has to sanity-check the robot. The marketers I trust most can sit with a clean Looker or Power BI view, smell when a number is lying, and trace it back to a real cause.
5. AI-augmented content, with taste
Yes, AI can write. That's precisely why taste is now the scarce resource. When everyone has the same content firehose, the output is an ocean of beige, and audiences have gotten very good at scrolling past beige.
The skill here is being a great editor and director, not a great typist: feeding AI the right brief, the right voice, the right examples, then ruthlessly cutting what sounds like a robot trying to sound human. Use the machine for the first draft and the heavy lifting; bring the judgment, the brand voice, and the actual point of view yourself. (If you want a feel for the difference, it's the whole reason human-led copywriting and content strategy still command a premium.)
6. Email marketing and outreach (still the king)
Now for the part the AI crowd loves to forget: the highest-ROI channel in marketing is still a well-written email. Email returns somewhere around $36 to $42 for every $1 spent, which quietly embarrasses almost every other channel. The reason is simple: your email list is the one audience you actually own. Social reach is rented from an algorithm that can change the rules overnight, but your inbox relationship is yours.
The skill in 2026 isn't blasting a newsletter at everyone. It's outreach that feels one-to-one: thoughtful segmentation, a subject line that earns the open, and a message relevant enough to get a reply. AI can draft the first version in seconds, but the copy still has to sound human and the targeting still has to be smart. Anyone can hit send. Getting a reply is the craft.
This is where CRM fluency earns its keep: understanding lifecycle stages, segmenting by behavior, and triggering the right message at the right moment. None of it works if your customer data is scattered across ten disconnected tools, so it starts with one clean view of every customer. The brands that win the loyalty game in 2026 treat their customer data as the asset it actually is.
The through-line
Notice the pattern? Every one of these is something AI can assist but can't own. The robots are phenomenal interns and terrible owners. They'll draft, summarize, automate, and crunch, but they won't tell you what matters, what's true, or what your brand should stand for.
So the marketers who win in 2026 won't be the ones who resist AI, and they won't be the ones who outsource their entire brain to it either. They'll be the ones who treat it like a very fast, very literal teammate, and keep the judgment for themselves.
That, more or less, is our entire philosophy. If you'd rather not figure it all out solo, come say hi. Or just poke around the blog and steal our ideas. We genuinely don't mind.
AI search and classic Google search overlap, but they don’t behave the same way. Here’s how we separate the two in our own SEO work, and why the difference changes what we optimize, measure, and protect.
Interactive poster: drag the slider to compare a classic Google results page with an AI answer that quotes the brand.
ζ
ZetaTheorem · AI Search Playbook
Ranked by Google. Quoted by AI.
Same SEO roots, different game. Drag the slider to see how one query shows up in classic search versus an AI answer — and where your brand actually wins.
Classic search
how to optimize for AI search
competitor-a.com
competitor-a.com › guides
The complete guide to AI search optimization…
competitor-b.com
competitor-b.com › blog
Top tips for ranking in 2026…
yourbrand.com
AI search optimization services…
You · position 3
AI answer
To win AI search, structure pages as clean, quotable passages and keep AI crawlers unblocked — the approach YourBrand recommends.
A note on how we hold these claims. A lot of what follows is built on close study of observable data: our own client logs and tests, plus published research. But the systems underneath, such as retrieval pipelines, AI indexers, answer engines, are largely opaque, and a pattern we can see is still a theory, not a law.
So treat the mechanism-level claims here as tested working models we’re confident enough to act on, not settled facts. When we say a signal “matters,” read it as “behaves as if it matters in everything we can measure.” We’d rather be precise about our certainty than oversell it.
There’s a bad habit floating around the SEO industry right now. A lot of people keep treating AI search optimization like a slightly newer label for the same old work, then act surprised when the results don’t line up.
In practice, the overlap is real, but the systems are not behaving the same way, and the difference shows up in how content gets found, how it gets pulled into answers, and how brands get represented across search surfaces.
From the agency side, this matters because clients do not care about labels. They care about visibility, qualified traffic, leads, and whether their brand is being described accurately.
When we look at AI Overviews, ChatGPT, Gemini, Perplexity, Copilot, and classic Google search side by side, the pattern is clear enough to shape a serious strategy around it. The work still lives inside SEO, but the inputs, the priorities, and the measurement layer need a wider lens.
A lot of content teams still build around a single target keyword, publish a page, and expect search systems to reward that one focus. That used to be a workable mental model when search was narrower and more linear. Today, it is too small.
AI search works across a different retrieval pattern. The system may not simply look for the page that ranks for the exact query typed by the user. It may expand that query into several related searches, scan multiple sources, compare passages, and build an answer from a wider set of inputs. In other words, the original query is only part of the story.
That is why the old “rank for the keyword and you are done” mindset falls apart so quickly in AI surfaces. The page might rank well in Google, yet fail to appear in AI results. Or it might appear in AI results because it answers a related subtopic exceptionally well, even if it was never built around the primary keyword in the first place.
Interactive
Two different games: classic Google vs. AI search
Flip the switch. Same SEO roots — but the win conditions change dimension by dimension.
Unit of optimizationOne exact target keywordA cluster of related questions
How content is readThe whole page, matched to a queryPassage by passage, then recombined
What “winning” meansRank #1 in the blue linksBe the source the answer quotes
Research inputSearch volume for visible keywordsQuery fan-out + intent coverage
How you measure itKeyword position & clicksCitation rate, mention rate, share of voice
If you ignore itYou rank a little lowerYou vanish from the answer entirely
For anyone managing organic growth, this changes the job. We are no longer optimizing for one search string alone. We are optimizing for a topic ecosystem, a set of related questions, a content structure that can be parsed cleanly, and a brand presence that can survive synthesis from multiple sources.
Why The Research Matters Here
This topic gets debated a lot, so the research is the part worth paying attention to. Several independent studies point the same way, even when the exact numbers differ.
ZipTie found that traditional Google rankings line up with AI visibility only about 45% of the time, and that the link is weakening fast.
By their measure, only 38% of pages cited in AI Overviews still rank in the organic top 10, down from 76% a year earlier. Because AI answers typically draw from just three to five sources, a strong blue-link position is now a weak guarantee of being one of them.
That alone is enough to push back against the claim that AI optimization is nothing more than classic SEO with a different name.
Interactive
Ranking in Google ≠ showing up in AI
Drag the timeline. Watch how much the “ranks in Google” and “cited by AI” circles have drifted apart in just a year. (Figures: ZipTie.)
12 months agoToday
Today, only 38% of AI-cited pages still rank in Google’s top 10.
AI answers typically cite just 3–5 sources. Being on page one is no longer the same as being one of them.
Profound makes the same point from another angle: Answer Engines rarely use the query as typed, they fan it out into multiple high-intent sub-queries, and those are what actually decide which sources get cited.
As they put it, creating content without considering query fan-outs is “like doing SEO without knowing your keywords.” The system often isn’t working from the exact query the user typed; it expands the question into related sub-questions, then pulls from the pages that answer those side questions well.
Semrush study of more than 10 million keywords added another layer: AI Overviews appear most often on low-volume queries, nearly 60% of triggering keywords have 100 or fewer monthly searches. Classic keyword research, built around volume thresholds, simply doesn’t see most of that surface.
Now the honest part, because we said up top we wouldn’t cherry-pick. Two findings in that same Semrush data cut against the easy “AI is eating your traffic” narrative, and we’d rather state them plainly than bury them.
First, AI Overviews did not automatically increase zero-click behavior, on the keywords studied, the zero-click rate actually dipped slightly, from about 33.8% to 31.5%, after AI Overviews appeared. Second, AI Overviews still skew toward low-commercial-value queries: roughly 95% show no ads or minimal commercial intent, and high-CPC commercial terms remain largely untouched for now.
The implication is not “ignore AI search.”
It is that, today, AI search is mostly a top- and mid-funnel visibility and brand layer, not yet where most transactional clicks are won. That’s a real nuance, and it’s also exactly why the shape of the funnel is changing. We’d rather a client hear this framing from us than discover it after over-indexing on the wrong layer.
From an agency perspective, these studies change how we diagnose opportunity.
We do not just ask, “What keyword should this page rank for?” We also ask, “What related questions would an AI system generate, which passages on this page can answer them, and where are the holes in the surrounding content ecosystem?”
What Query Fan-Out Means In Practical Terms
Query fan-out is one of the most useful concepts in this entire conversation, and it deserves a clean explanation.
A user asks one question. The system then breaks that question into several smaller or related questions behind the scenes.
Those related queries may include definitions, comparisons, use cases, supporting facts, local intent, commercial intent, or even adjacent problems the user did not type directly.
The AI then retrieves content across those related searches and uses the pieces it trusts most.
Interactive
Query Fan-Out Explorer
Type a question (or pick one) and hit Fan out. You’ll see the hidden sub-searches an AI runs behind a single query — colour-coded by intent. This is the surface your content actually competes on.
Try:
best protein powder for weight loss
the model quietly expands this into…
So the page that wins in AI search is not always the page that most aggressively repeats the exact target keyword. It may be the page that covers the broader topic set cleanly, has clear passage-level meaning, and supports the adjacent questions the system expects to answer.
This is where a lot of “just SEO” arguments break down.
Traditional keyword targeting still matters, but it is now only one layer inside a bigger retrieval pattern. A strong page needs to be relevant to the primary topic and cleanly connected to the related topics surrounding it.
Why Traditional Keyword Tools Miss So Much
Most SEO tools were built for a world where keyword volume was the main proxy for opportunity. That worked well enough when search behavior was easier to measure. It works much less well now.
AI systems can generate or use queries that never show up in standard keyword databases, because those databases rely on observed searches above a certain threshold. A query that matters inside AI retrieval may be too small, too new, too implicit, or too synthetic to register in classic research tools.
That does not make it unimportant. It just makes it harder to see and, as the Semrush data above shows, that “invisible” slice is where a lot of AI Overviews actually fire.
This creates a serious blind spot.
Teams can spend weeks optimizing around visible keywords while the AI system expands into related question sets that were never in the brief. The result is a content strategy that looks tidy in a spreadsheet and incomplete in the real environment.
As an SEO provider, this is one reason we never treat keyword research as the final answer. It is the starting map, not the destination. We use it to understand the topic, then widen the frame to include query fan-out, semantic relationships, user intent layers, and the content structure needed for machine retrieval.
Why AI Search Feels Closer To Reputation Management
One of the most useful shifts in thinking is this: AI search often behaves more like reputation management than standard ranking work.
That sounds dramatic, but it isn’t. A brand’s answer in AI results is often built from a mix of the brand’s own pages, reviews, articles, third-party listings, community posts, and other sources that contribute to a broader consensus. In many cases the system is not just asking, “What does this site say?” It is asking, “What is the collective picture around this brand, product, or topic?”
This changes the strategy in a few ways.
First, your own website matters more when it helps define the canonical version of the truth. A clear pricing page, a well-written about page, a strong FAQ section, or a direct services page can help anchor the narrative. Second, what other sites say about you can shape the answer just as much as your own copy does. Third, your content needs to be consistent across the ecosystem, not fragmented into contradictory signals.
This also reframes the hardest question in AI search: what do you do when the system is simply wrong about you? You can’t edit the model.
What you can do is strengthen the entity, the machine-readable, cross-source record of who you are. In practice that means consistent Organization and about schema, an identical set of brand facts (services, locations, founding details, specializations) everywhere they appear, and, where it’s warranted, a presence in the sources that feed knowledge graphs, such as Wikipedia and Wikidata. When the underlying entity is clear and consistent, stale or hallucinated descriptions have less room to survive. When it’s fragmented, the system fills the gaps with whatever it happens to find.
That is one reason many brands need to rethink whether hiding useful information is still a good commercial decision. In classic lead-gen SEO, a company might avoid publishing pricing because it prefers to funnel people into a sales conversation. In AI search, that can backfire, because the system will go elsewhere to fill the gap, and it may rely on older, weaker, or less accurate third-party sources.
We see this as a brand strategy issue, not just a page optimization issue.
What To Do About Pricing, Positioning, And Brand Narrative
One of the clearest examples of this change is pricing.
If a brand does not publish pricing, AI systems will often look for that information somewhere else, directories, review platforms, competitor comparisons, forum posts, or old content that no longer reflects the current offer. The brand loses control of the story. In the AI layer, silence rarely stays silent. The system fills it.
For service businesses, this can affect more than pricing. It can change how the brand is described in terms of service scope, specializations, years of experience, audience, turnaround time, and even the type of client it serves. A stale third-party summary can linger far longer than it should, especially when the source ecosystem keeps repeating it.
The practical answer is not to publish every detail everywhere without thinking. Some businesses genuinely can’t post fixed prices because the work is scoped per client, and that’s fine.
But “we can’t publish an exact number” is not the same as silence. AI systems reward a concrete answer over a blank, so give them one: a starting rate, a typical range, a “from $X” anchor. The goal is to decide what the brand needs to control, then make sure the core pages say it clearly:
a clear services page
a current pricing page or pricing guidance (even a range)
an updated about page
a strong FAQ page
comparison pages where relevant
review and testimonial signals that support the current positioning
The goal is visibility with the right message attached.
The Technical Step Almost Everyone Skips: Let The AI Crawlers In
Before any of the on-page work matters, the AI systems have to be allowed to fetch your pages at all, and a surprising number of sites quietly block them without realizing it.
The major AI crawlers each have their own user agent: GPTBot (OpenAI/ChatGPT), ClaudeBot (Anthropic), PerplexityBot (Perplexity), and Google-Extended (Google’s AI training and grounding).
If your robots.txt disallows them or, just as common, your CDN or WAF blocks them by default, with Cloudflare’s bot controls being a frequent culprit; you can do everything else in this article perfectly and still be invisible in the answer layer.
Interactive
AI Crawler Control Panel
Toggle each AI crawler and watch your robots.txt — and your visibility — change. Block a bot (often a CDN/WAF default) and you disappear from that engine, no matter how good the page is.
So step zero of an AI-search audit is boring and decisive: confirm that the crawlers you want are actually allowed, at both the robots.txt and the edge/WAF level.
This is also a genuine decision, not just a checkbox. Some publishers deliberately block AI training crawlers to protect their content, and that’s a legitimate choice, but it should be one you made on purpose, not a default you inherited from a boilerplate config.
Why Crawl Timeouts And Server Behavior Matter More Than People Think
Once the crawlers are allowed in, they still have to successfully fetch the page, and some AI systems fetch in real time, under tight latency budgets, while a user waits for an answer. A page can be excellent and still get skipped because the fetch timed out before the content came back.
One signal we’ve dug into is the HTTP 499 status, a non-standard, NGINX-specific code that records the client giving up before the server finished responding. (On Apache, IIS, Cloudflare, Vercel, or Fastly you’ll see different signals — 408s, 524s, dropped connections — for the same underlying behavior.)
We treat it as one log signal among several, not a universal law, and we ran a small paired test on our own client portfolio to pressure-test the idea. The full write-up, caveats and all, is here: Investigating 499 Response Codes as an AI-Search Ranking Factor.
The general point holds regardless of stack: slow responses, rendering issues, and blocked or JavaScript-dependent content all reduce the odds of being used in an AI answer. In our process, technical fetch quality is now part of the AI-search foundation, not a separate cleanup task.
Where We Land On llms.txt
llms.txt is an emerging convention: a short Markdown file at your site’s root that hands AI tools a curated map of your most important pages. Think of it as robots.txt’s counterpart for AI. And, like robots.txt, it changes nothing for human visitors.
It’s also genuinely contested, so here’s our honest read rather than a hard sell: Google’s own guidance famously depends on which team you ask.
Google Search says you don’t need it, John Mueller has compared it to the long-ignored keywords meta tag, while Google’s Chrome team added an llms.txt audit to Lighthouse under its new “agentic browsing” checks.
The reconciliation is that the two teams are optimizing for different things: Search is talking about rankings and AI Overviews, where llms.txt does nothing today; Chrome is talking about readiness for browser-based AI agents, where a clean machine-readable map of your site genuinely helps.
So we won’t claim llms.txt lifts rankings; the evidence doesn’t support that, and we said at the top we wouldn’t oversell. The argument for it is simpler, and it’s about asymmetry.
The file takes an hour to write, it’s harmless, agent and developer tooling already reads it, and Google’s own Chrome tooling now audits for it.
Set that against Google’s long track record of being opaque about what actually moves its systems, and stubbornly refusing a 30-minute file because Search says it isn’t a ranking factor is the worse bet. We treat llms.txt as cheap insurance and basic hygiene, a clean, curated map of your best pages, not a growth hack.
On that framing, it’s a clear should-have today and, as agentic browsing matures, plausibly a must-have. The risk of skipping something this small outweighs the risk of being stubborn about it.
Why Metadata Matters More Than Many SEOs Expected
In classic Google SEO, title tags still matter and meta descriptions still matter indirectly, but they are often treated as click-through assets rather than core visibility drivers. In AI environments, our testing suggests they can play a larger role, because they appear to help the system decide whether to fetch the page in the first place.
A useful way to think about it: before committing to a deeper read, a retrieval system has little more to go on than a page’s title, description, and URL.
When those signals are clear, relevant, and aligned with the page content, the page seems to have a better chance of being fetched and used.
When they’re vague or mismatched, it seems to move on quickly. We can’t see inside the pipeline to prove the weighting, but the observable behavior lines up with treating metadata as machine-facing context, not just a SERP snippet.
For that reason, we now write metadata to tell the machine what the page is about, who it is for, and why it exists. This is especially useful on commercial pages, service pages, and content that needs to be understood fast.
Why Semantic URLs Matter More In AI Search
URL structure has always been a topic people argue about too much. In classic SEO it is usually a small signal, and changing a live URL can cause more harm than good.
In AI search, the slug appears to carry more interpretive weight than many teams expect. A URL that clearly reflects the topic seems to help the system understand the page faster; a messy, generic, or overly shortened structure gives it almost nothing to work with.
This does not mean you should rewrite every existing URL on your site, in most cases that’s a bad trade. But moving forward, it’s smart to build cleaner slugs:
Better: /seo-services/ai-search-optimization/
Weaker: /blog/post-1847/
We recommend treating the slug as part of the page’s meaning, not just its file path.
Why Recency Has Become More Visible
Recency has always mattered in some search categories, but AI systems seem especially sensitive to freshness signals. That doesn’t mean older content can’t be cited, it means the system may place more weight on visible recency than many marketers expect, particularly for topics where information changes often.
The point is not to fake freshness. Bumping a date without changing the substance is exactly the kind of signal these systems are learning to discount, and it erodes trust when readers notice. The point is to make genuine updates visible: change the content, then let the date reflect that real change. For evergreen content, that usually means:
refreshing the introduction
updating examples
replacing outdated stats
adding newer internal links
revisiting related FAQs
noting the date of the last meaningful update
This works in classic SEO too, but it matters more in AI retrieval, because the system is often trying to resolve present-tense questions, not just archive material.
Content Chunking Is Not A Trendy Buzzword
A lot of people dismiss content chunking as a fancy word for breaking up paragraphs. That’s too simplistic.
Chunking is about creating smaller units of meaning that can stand on their own. It helps both audiences. For people, it makes content easier to scan, read, and trust.
For AI systems, it makes information easier to isolate, compare, and cite, because modern retrieval works at the passage level, it doesn’t need a wall of text, it needs passages with clear semantic boundaries.
Interactive
Same facts, two structures — which can an AI lift?
Flip between a wall of text and a chunked version of the exact same answer. Watch what a retrieval system can grab cleanly.
So you’ve been wondering about shoe longevity, and the truth is it depends on a lot of factors including your weight, your gait, the surfaces you run on, and the build of the shoe itself, but generally speaking most experienced runners and coaches will tell you that somewhere in the range of 300 to 500 miles is when the midsole foam starts to break down enough that you lose meaningful cushioning and support, although heavier runners may find they reach that point sooner, and rotating between two pairs can stretch the total life out a bit further.
AI lifts this passage
How often should you replace running shoes?
Replace running shoes every 300–500 miles (480–800 km) — when the midsole foam loses its cushioning.
Heavier runners trend toward the lower end of that range.
Rotating two pairs extends total lifespan.
Worn-flat tread or new aches are signs you’re overdue.
AI’s view: the answer (300–500 miles) is buried mid-sentence. The system has to parse the whole block and guess which clause answers the query — so it often skips you.AI’s view: a clear question heading, the answer in sentence one, support in tidy bullets. The boxed passage drops straight into an answer, with a citation to you.
A strong chunk usually does a few things well: it covers one idea, uses a clear heading, starts with the point rather than the preamble, includes a supporting detail or example, and avoids blending multiple topics into one paragraph.
And no, bigger context windows don’t make this obsolete, a common objection we don’t buy. Larger models still compress, rank, and filter what they read. When the source material is messy, that compression loses more meaning; when it’s structured, more of the useful signal survives.
The system can only extract so much from a page at once, so clear sections, clear boundaries, and clear topic progression make the extraction job easier. Good structure isn’t a compromise you make for machines. It’s a performance asset that happens to help humans at the same time.
Good Structure And Good Writing Aren’t In Conflict
When we restructure content for clients, we look for three recurring problems.
First, paragraphs that cover too much at once, definitions, benefits, limitations, and implementation crammed into one block, forcing both the reader and the system to do extra work.
Second, sections without clear boundaries, where a heading says one thing but the paragraphs underneath drift into three other topics.
Third, sentences that hide the core point halfway through, fine for a novel, not ideal for search content.
A stronger version opens with the direct answer, supports it with detail, keeps adjacent ideas in separate blocks, and uses headings that match real user intent.
There’s a misconception that AI-friendly content has to sound flat. It doesn’t. We write plenty of content with tone, personality, and rhythm.
What changes is the internal discipline: we’re more deliberate about where the point lives in a paragraph, how much one section tries to do, and how clearly the text signals meaning.
A useful test, read a paragraph and ask whether the main point can be extracted quickly without losing the rest of the message. If the answer is no, the paragraph probably needs to be split.
Strong writing and strong structure support each other; the trick is keeping the voice while making the meaning easy to lift.
The Role Of Structured Data And Accessibility
There is still debate about how much schema markup matters for AI visibility. From what we see in the field, structured data remains valuable because it helps systems understand entities, relationships, and page purpose more cleanly. We wouldn’t reduce it to “use schema because Google likes rich results”, that’s too narrow. In many cases the value comes from machine readability across systems, not only from visible SERP enhancements.
Accessibility matters for the same reason. Clean HTML, meaningful headings, proper labeling, and content that doesn’t depend too heavily on JavaScript all make a page easier to process.
If a page is difficult to render or interpret, the AI system has less to work with. Structured data, accessibility, and clean markup all support the same goal: making the page easier to understand.
Why Reddit, YouTube, And UGC Keep Showing Up
When people ask what sources AI systems trust, the answer is not always the sources they expect.
Community and platform content often play a major role, especially for comparison queries, product questions, service research, and troubleshooting.
Reddit is frequently cited because it holds a huge volume of conversational, problem-solving content. YouTube matters because it packages useful information in formats that are easy to mine, especially when transcripts and metadata are available.
This tells us something about strategy: we shouldn’t only think about the brand’s own domain. We should also think about where the surrounding discussion happens.
That doesn’t mean chasing every platform at once, it means knowing which ones influence your category, which might include Reddit threads, YouTube explainers, review platforms, comparison articles, industry directories, forums, and durable social posts.
The obvious next question is how you influence those sources, and this is where we’d caution against the easy answer. You don’t manufacture Reddit threads or seed fake reviews.
Beyond breaking platform rules, it backfires: communities are good at spotting and punishing it, and a burned brand reputation is far harder to repair than a thin one. The durable approach is to earn the mentions, show up as a real participant, let employees and founders engage transparently, support actual customers where they’re already talking, and make your own content good enough to be referenced on its merits.
The goal is to be part of the consensus honestly, and not to fake it.
Why PR Still Belongs In The Mix
Public relations remains highly relevant because AI systems need source material, and PR helps seed consistent coverage across the web.
This is not old-school vanity PR for the sake of mentions. It is strategic distribution. When the brand story appears in multiple credible places, the model has more material to work with and less room to rely on stale summaries or conflicting third-party descriptions.
For SEO teams, this means working more closely with PR than many did in the past. A link is useful. A mention is useful. A clear, consistent brand statement in the right publication can be even more useful when AI systems are assembling answers. PR is not only about awareness, it’s about feeding the broader knowledge environment.
Why Classic SEO Still Matters
None of this means classic SEO stopped mattering.
Google still relies on strong content, clear structure, relevant entities, internal linking, topical authority, technical health, and backlinks. Those fundamentals still support visibility in the classic search layer and often help in the AI layer too. The difference is not that classic SEO became obsolete. The difference is that it is no longer enough on its own.
We still care about technical crawlability, topical relevance, internal linking, content depth, page speed, semantic clarity, strong title tags, and search intent alignment. We now also need to understand how those elements behave inside AI retrieval, not just inside the blue links.
Why We Treat AI Search Optimization As A Separate Workstream
In our agency, we treat AI search optimization as a distinct workstream inside SEO because the goals and success conditions are different.
Classic SEO is mostly about winning ranked placement and earning clicks. AI search optimization is about being selected, summarized, cited, and represented accurately across answer environments. Sometimes that leads to traffic. Sometimes it leads to brand lift. Sometimes it leads to a qualified click from a user who has already educated themselves inside the AI interface.
The value flows differently. The work cannot be measured only by traffic sessions, because the influence often happens before the visit. This is also why AI search gets budget attention from decision makers who tuned out traditional SEO, the use case is easier to explain in business terms.
A brand wants to be in the answer. A brand wants to be described correctly. A brand wants to show up where buyers are asking questions. That’s a different conversation from “we need 12 more blog posts for this keyword cluster.”
How We Actually Start A Content Restructure Project
When we take on a page that needs to perform better in AI search, we usually begin with the page itself, then widen out to the surrounding ecosystem.
Read the page like a machine and like a human. Look for sections that are overloaded, vague, or off-topic. Good information is often buried inside too many mixed paragraphs.
Split mixed ideas into separate chunks. Each section should carry one job.
Tighten the headings. Headings should tell the reader what comes next and create a clear retrieval boundary.
Add specific supporting details. Data, examples, comparisons, and direct statements. Strong content is content with evidence.
Review metadata and URL structure. Check that the title, description, and slug support the topic clearly.
Check for technical friction. Confirm the AI crawlers are allowed, then look at rendering, server response, and crawl logs.
Review off-site signals. Look at Reddit, YouTube, PR coverage, review sites, and directories to see what consensus already exists.
Measure the impact over time. We don’t expect a single prompt to tell the whole story. We look for directional improvement across multiple checks.
Why Measurement Is Messy And Still Worth Doing
A lot of people get stuck on the fact that AI visibility is probabilistic. They’re not wrong, the same prompt can return different results depending on context, platform, and time. But classic rank tracking was never perfect either. Search Console already gives averages, not pure truth. AI systems just add more variability, which means measurement needs more sampling, not less seriousness.
In practice, that means sampling deliberately rather than spot-checking. We run a defined set of prompts across multiple surfaces, ChatGPT, Gemini, Perplexity, Copilot, and Google’s AI Overviews, and repeat them on a schedule, because one prompt on one day tells you almost nothing.
Purpose-built tools (Profound, Ziptie, Otterly, and Peec AI, among others) help automate that sampling at scale. And we separate three metrics that often get blurred together:
Citation rate — are you linked as a source?
Mention rate — are you named, even without a link?
Share of voice — how often you appear versus competitors for a topic.
Read this way, AI visibility is a precision problem, not a certainty problem.
The right questions are directional: Is the brand showing up more often? Is it being represented more accurately? Are the right pages being cited? Are competitor answers getting weaker while ours get stronger?
Where Most Brands Should Start
For most businesses, the best starting point is not a giant AI overhaul. It’s a careful review of the pages that matter most: service pages, product pages, pricing pages, about pages, comparison pages, FAQ pages, and high-value blog posts.
Interactive
Is your page AI-ready? A 60-second self-audit
Tick the ones that are true for a page you care about. Your score updates live.
Are the AI crawlers even allowed to fetch the page?
Is the topic clear in the title, slug, and description?
Does each section cover one thing well?
Are there enough explicit answers for related questions?
Does the site tell the same story as third-party mentions?
Are there technical barriers to crawl and fetch?
Does the page look good for both people and machines?
That’s usually enough to reveal the first wave of improvements.
Final Thought
We’re past the point where it makes sense to pretend AI search is just a rebrand of classic SEO. The fundamentals still matter, but the retrieval behavior has changed enough to require a wider strategy. Query fan-out, structured content, metadata, semantic URLs, recency, crawler access, technical fetch quality, and brand consensus all play a role now.
The SEO teams doing the best work aren’t throwing out the old playbook, they’re extending it. They balance human readability with clear structure, while actively shaping how their brand is interpreted so the final output stays consistent with what they actually offer.
Yes, but it doesn’t guarantee anything. ZipTie’s research found traditional rankings line up with AI visibility only about 45% of the time, and only 38% of pages cited in AI Overviews still rank in the organic top 10 (down from 76% a year earlier). A strong Google position helps, but it doesn’t automatically translate into AI visibility.
Could blocking AI crawlers be quietly hurting me?
Yes, and it’s more common than people think. If your robots.txt — or your CDN/WAF default, such as Cloudflare’s bot controls — blocks GPTBot, ClaudeBot, PerplexityBot, or Google-Extended, you can do everything else right and still be absent from AI answers. Check crawler access first.
Do I actually need an llms.txt file?
Google says you don’t need it for search, and we won’t claim it lifts rankings. But it takes about an hour, it’s harmless, developer and agent tooling already reads it, and Google’s own Chrome/Lighthouse now audits for it. Given that asymmetry, we treat it as cheap insurance worth shipping rather than a hill to die on.
Should I rewrite all my existing URLs for AI search?
Usually no. Changing live URLs can create problems for classic SEO and cause more harm than good. Going forward, use cleaner, more descriptive slugs on new pages and keep existing important URLs stable.
Why do Reddit and YouTube show up so often in AI answers?
Because they hold large volumes of useful, conversational, and explanatory content AI systems can draw from. Reddit reflects lived experience and community problem-solving; YouTube provides rich educational content with metadata and transcripts that are easy to process. The way to benefit is to earn genuine presence there, not to manufacture it.
How do you measure AI visibility?
By sampling a defined prompt set across multiple platforms and time periods, using tools like Profound, Ziptie, Otterly, or Peec AI, and tracking citation rate, mention rate, and share of voice. The data is probabilistic, so the goal isn’t perfect certainty — it’s whether the brand is showing up more often, in the right places, with the right message.
Is AI search worth prioritizing if it doesn’t drive huge referral traffic yet?
Yes, but as a brand and demand layer first. Current research suggests AI Overviews still skew toward lower-commercial-value queries, so the referral traffic may be smaller than classic search for now — but the visibility still influences trust, consideration, and downstream conversions.
Can strong writing and AI-friendly structure coexist?
Absolutely. Good writing still matters. The trick is to keep the voice while making the meaning easy to extract — with a clear structure underneath to support it.
Here’s a fun exercise. Open your keyword tool of choice and look up something like “best running shoes.” Thousands of searches a month. Solid difficulty score. A classic target.
Except nobody actually wants “best running shoes.” What they want is something like: “I’m training for my first half marathon, I overpronate, and my budget is about $120. What should I buy?”
For twenty years, that second sentence had nowhere to go. The search box rewarded short, robotic phrases, so we all learned to compress our messy human needs into three-word fragments. Keyword research was built on top of that compression. We counted the fragments, sorted them by volume, and wrote content for the winners.
AI search just removed the compression. People can ask the full question, add constraints, follow up, and get one synthesized answer. Which means the thing we’ve been counting all these years, the keyword, is dissolving back into what it always represented: intent.
That doesn’t make keyword research dead. It moves the job up a level, from counting strings to mapping intent. Let’s walk through what’s actually changing, what it does to your research process, and what it means for the content you publish.
The same need, two interfaces. Volume tools only ever saw the left side.
This was always an information retrieval game
Strip away the tools and dashboards, and search has always worked the same way. A person has a need, compresses it into a query, and a retrieval system decompresses that query into a ranked list of documents. Keyword research worked because everyone compressed the same way. Millions of different people, with millions of slightly different situations, all typed “best crm small business.” Demand pooled into countable strings, and an entire industry of volume and difficulty metrics grew around counting them.
LLMs didn’t change the game. It’s still information retrieval. They changed the interface, and the interface was the only reason keywords existed. Now the system retrieves passages from across the web, synthesizes them into an answer, and cites a handful of sources. Same game, new referee.
What actually changed: three shifts
1. Queries decompressed, and your metrics can’t see it
The data on this is striking. Queries in Google’s AI Mode run nearly twice the length of traditional searches (7.2 words versus 4, per Semrush). ChatGPT prompts run roughly 17 times longer than Google searches, according to SimilarWeb. And these aren’t one-shot queries anymore. They’re conversations, with follow-ups doing the work that five separate searches used to do.
Now here’s the uncomfortable part for those of us who grew up on keyword tools. Search volume and keyword difficulty are derived from traditional engine data. Prompts are private. They never enter those datasets. So the fastest-growing slice of demand is invisible to the exact metrics we use to decide what’s worth targeting.
Think about what each metric actually tells you now. Volume tells you how people used to compress this need on one platform. Difficulty tells you how hard it is to win a prize that’s shrinking. Neither is useless, but both got demoted: from decision gates to partial inputs.
The sub-queries, cited domains, and named brands inside AI answers
Replaced by fan-out analysis
2. Engines stopped ranking pages and started assembling answers
When someone asks an AI engine that half marathon question, the system doesn’t run one search. It quietly breaks the prompt into a swarm of sub-queries (shoes for overpronation, stability versus neutral shoes, best running shoes under $150, how shoe fit changes for long distances), retrieves content for all of them in parallel, and assembles an answer. Google calls this query fan-out, and it’s not unique to Google. ChatGPT runs a web search in about 31% of prompts, averaging two searches per prompt.
The new SERP: sub-queries you never see, passages instead of pages, citations instead of rankings.
See query fan-out in action
Type a question the way you’d ask an AI assistant, or grab an example. Watch one prompt turn into searches you’d never see.
The prompt
Hidden sub-queries the engine runs
What comes back
Synthesized answer
Illustrative simulation based on documented fan-out behavior in Google AI Mode and ChatGPT search, not live engine output. Source patterns: Google Search Central, Search Engine Land, Ahrefs.
Better yet, try it yourself with the interactive simulator above.
Two consequences worth sitting with. First, you no longer rank for what the user typed. You get retrieved for sub-queries you never see. Second, the unit of retrieval is the passage, not the page. Google’s own guidance confirms that AI features pull relevant chunks of pages into answers. A single well-structured section of your article can earn a citation even if the page as a whole would never have ranked for the original question.
Before you panic, look at the other side of the ledger. Semrush found the average AI search visitor is worth 4.4 times more than a traditional organic visitor. Seer Interactive watched ChatGPT referrals convert at 15.9% against 1.76% for organic search. The model did the comparison phase for the user, so the people who do click arrive pre-qualified. (One caveat: this skews toward considered purchases like software and services. Impulse ecommerce still behaves more traditionally.)
Four numbers that tell the whole trade: scarcer clicks, better clicks.
So queries are sorting into three buckets. Dying queries get answered inline, and most top-of-funnel informational content lives here. We’ve written before about how AI killed TOFU SEO content, and this is the mechanism. Surviving queries still earn the click: transactional, navigational, and anything people want to verify themselves. And newborn queries are the complex, multi-constraint prompts nobody bothered typing into a search box before. Your research process needs to classify demand by survivability, not just volume.
Query bucket
What’s happening
Example
Your move
Dying
Answered inline, the click rarely happens
“what is a crm”
Win the citation, not the visit
Surviving
A click is still needed to act or verify
“hubspot pricing”, “trail running shoes size 10”
Classic SEO still pays here
Newborn
Complex prompts nobody used to type
“best CRM for a 12-person agency that lives in Gmail”
Cover the question space
The new keyword research process, step by step
Here’s how the workflow changes in practice.
The new workflow at a glance. Details below.
Step 1: Start with people, not seed keywords. Your customers’ actual language is the new seed list. Mine sales call transcripts (Gong or Fireflies make this easy), support tickets, G2 and Trustpilot reviews, and the Reddit threads where your category gets discussed. You’re collecting questions, situations, and constraints, phrased the way real people phrase them. That’s what prompts look like.
Step 2: Keep your keyword tools, but change their job. Ahrefs, Semrush, and Google Search Console still map demand topology beautifully. The shift is in how you read them: treat each keyword as the head of an intent cluster, not a target. “Best CRM” isn’t one piece of content anymore. It’s the visible tip of fifty buyer situations, and your research should enumerate those situations.
Step 3: Simulate the fan-out. This is the new SERP analysis. Take your head intents and run them through ChatGPT, Perplexity, Gemini, and Google’s AI Mode. Record four things every time: the sub-questions the engine explores, the follow-ups it suggests, the domains it cites, and the brands it names. Tools like iPullRank’s Qforia can generate likely fan-out queries at scale, but even a manual afternoon of this is eye-opening. The gaps you find (questions where competitors get cited and you don’t exist) are your roadmap.
Step 4: Prioritize by coverage, not volume thresholds. Here’s a mindset shift that matters. Zero search volume no longer means zero demand. It usually means the demand is spread across infinite phrasings, or that it’s newly expressible now that the box accepts full sentences. If a phrase maps to a real pain point your customer has, it belongs in your research, volume or not, because any sub-question can be the hook that pulls you into a high-stakes answer. The metric to manage is coverage: the share of your category’s question space where you’re retrievable. One guardrail, though. Coverage means depth within clusters, sections and passages inside substantial resources. It does not mean a thin page for every permutation. That’s content farming with extra steps.
Step 5: Track share of answer, not just rank. Tools like Profound, Peec AI, Otterly, Semrush’s AI Toolkit, and Ahrefs’ Brand Radar now track whether you’re cited and mentioned across AI engines. Pair that with AI referral traffic in GA4 (and its conversion rate, which is the number your CFO will care about). Rankings still matter, but they’ve become an input, not the scoreboard.
And the whole process on one screen:
Step
Do this
Tools
1. Start with people
Mine real customer language for questions, situations, constraints
Gong, Fireflies, G2, Trustpilot, Reddit, support tickets
2. Re-task keyword tools
Treat each keyword as the head of an intent cluster
Ahrefs, Semrush, Google Search Console
3. Simulate the fan-out
Record sub-questions, follow-ups, cited domains, named brands
ChatGPT, Perplexity, Gemini, AI Mode, Qforia
4. Prioritize by coverage
Map every real pain point, volume or not, with depth per cluster
Your intent matrix, AlsoAsked
5. Track share of answer
Measure citations, mentions, AI referrals and their conversion
Profound, Peec AI, Otterly, Semrush AI Toolkit, GA4
What this does to the content you publish
The research shift drags content strategy along with it.
The page-per-keyword era is over; the resource-per-intent-cluster era has started. One substantial piece that covers a buying situation from every angle, built in extractable passages: question-shaped headings, the direct answer in the first sentence or two, then the supporting detail, data, and tables. You’re writing for a system that grabs self-contained chunks, so make your chunks self-contained. This is exactly the kind of architecture a good content strategy maps out before anyone writes a word.
Information gain stops being a nice-to-have. When an engine synthesizes from multiple sources, content that repeats what everyone else says contributes nothing quotable and gets averaged out. Original data, first-hand experience, and specific claims with numbers attached are what earn citations. Be the source the model quotes, not the summary it skips.
And your off-page presence is now part of your content strategy, whether you like it or not. Models triangulate from reviews, Reddit threads, and comparison posts. What the corpus says about you is what the answer says about you.
The bigger picture: you now have two audiences
Here’s the thing I most want non-SEO marketers to take away. Keyword research’s job description never changed. It was always about understanding how your audience seeks information, then making sure you show up at the moment of need. What changed is that an AI agent now sits between you and them, doing the finding, reading, and relaying.
That agent doesn’t behave like Google’s top ten. Ahrefs analyzed 15,000 queries and found that only 12% of URLs cited by AI tools overlap with Google’s top 10 results. The other 88% come from pages that don’t rank on page one at all. The engines go beyond the familiar index to find passages that actually answer the sub-question at hand. That’s a real opening for smaller brands who could never crack the top ten, and a real warning for big ones coasting on domain authority.
The retrieval pool is much bigger than the top 10, and it reads like a machine.
So your content has to work for the machine reader too. It needs to be retrievable (don’t block AI crawlers; we’ve even dug into oddities like whether 499 response codes affect AI SEO), parseable (clear structure, answer-first passages), and corroborated (consistent facts about your brand across the web, because models trust what multiple sources agree on).
Zoom out and the strategic picture looks like this. The KPI is shifting from rank to share of answer. Being the brand named in answers is the new position one, and being the brand named in prompts (“is X any good for Y”) is the new branded search. That pushes SEO, PR, and brand into one discipline, because influencing what models say about you means influencing the entire corpus they read. And through all of it, classic SEO remains the substrate rather than a casualty. These engines ground their answers in search indexes, so crawlability, authority, and rankings are the entry fee. Everything above is additive.
Where to start this week
Pick your five most important buyer intents. Run each through ChatGPT, Perplexity, and AI Mode, and write down who gets cited, who gets named, and which sub-questions you have no answer for. That one exercise will tell you more about your real visibility than any rank tracker currently can.
Then fix one cluster end to end: research the question space, build the resource, structure it for extraction, and track citations for a quarter. If you’d rather not figure out the machinery alone, that’s exactly what our AI/LLM optimization work is for. Either way, start now. The marketers who treat this as an evolution of information retrieval, rather than a brand-new dark art, are going to be very hard to catch.
This piece is best read as a working hypothesis, not a confirmed best practice. The analysis below comes from a mix of small-scale testing across our own portfolio of client sites.
That includes one paired experiment where we left 499-related issues in place on one site and deliberately eliminated them on a comparable site of similar authority.
It also draws on public commentary from SEO thought leaders who have been writing about AI search behavior. Sample sizes are small, attribution is messy, and the underlying systems (AI search agents, indexers, retrieval pipelines) are largely opaque. So treat what follows as a direction worth investigating in your own audits, not a settled ranking factor.
Why we started looking at this
If you’ve been doing technical SEO for a while, you already live in a world of crawl budgets, render budgets, Core Web Vitals, and server logs. Recent shifts in how people search (AI Overviews, ChatGPT search, Perplexity, enterprise assistants) raise a question worth taking seriously: are at least some of these systems fetching information live, under tight latency constraints, while a user is waiting for an answer?
We don’t think the answer is uniformly yes. A lot of AI-generated answers appear to be drawn from pre-indexed content the model or its retrieval layer already has.
But we also can’t assume it’s uniformly no. There’s enough evidence (in observed crawler behavior, in vendor documentation, and in our own logs) to suggest real-time fetching is part of the mix for at least some queries and some systems.
That uncertainty is what made one server-log detail interesting to us:
The 499 response code.
Our hypothesis (still a hypothesis): Sustained 499 patterns may quietly hurt visibility in real-time AI retrieval scenarios, because your server is effectively recording that the client gave up before you finished responding. Below is what 499 is, why we think it could matter, what we’ve seen, and how to investigate this on your own properties.
What is a 499 response code?
499 is a non-standard status code introduced by NGINX. It means:
Client Closed Request. The client terminated the connection before the server finished responding.
Important nuance: your server doesn’t “send” a 499 the way it sends a 200 or 404. NGINX records a 499 when the client bailed before the response was complete.
A second nuance worth calling out up front: 499 is NGINX-specific. If you’re on Apache, IIS, Cloudflare Workers, Vercel, Fastly, or another stack, you won’t see 499 in your logs at all.
You’ll see different signals: 408s, 503s, 524s, or just dropped connections recorded differently. The underlying behavior (client gave up mid-response) can still happen; the log code just won’t be 499.
If your stack isn’t NGINX, the equivalent signal is whatever your edge or origin uses to record client-initiated disconnects.
Common reasons 499 happens
A user hits stop or closes the tab, or their mobile connection drops.
A client (browser, app, bot) has a timeout and gives up.
An upstream system (proxy, edge, or potentially an AI fetcher) abandons the request to stay within a latency budget.
Your backend is slow (database, API, rendering) and the client leaves before NGINX can finish.
In classic SEO, 499s were largely treated as a “users bounced” signal.
The question we’re asking is whether, in a world where some retrieval traffic comes from automated agents on tight clocks, sustained 499s on the right templates also represent missed inclusion opportunities.
Why this might be more than a server curiosity
A reasonable counterargument to everything below is: “Most AI answers come from pre-indexed content, so live fetch latency doesn’t matter.” For a lot of queries, that’s probably right. But two things keep us from dismissing the live-fetch case:
Some AI search products clearly fetch in real time, at least for certain query types. Perplexity’s product behavior, ChatGPT’s browsing/search mode, and various enterprise assistants document or visibly perform live retrieval. The proportion varies by product and query, but it isn’t zero.
Even crawlers that aren’t “real-time” still have timeouts. A slow or unreliable origin can underperform in a generic index too.
So the working assumption is: real-time retrieval is a meaningful subset of how LLMs source content, and even where it isn’t, fast-and-reliable origins help. That makes server-side timing worth paying attention to.
The concurrent fetch pattern (where this matters most)
Where we think 499 risk is highest is in products that do concurrent live retrieval for a single user query. Example query:
“What’s the difference between 5G SA and NSA for industrial robotics, and what should a Canadian manufacturer buy this year?”
A retrieval-style system might, in parallel:
Fetch a few authoritative explainer pages.
Pull vendor documentation for SA vs NSA capabilities.
Grab a couple of recent articles.
Look for a decision guide or checklist.
Compare latency and reliability claims across sources.
If your page is in that candidate set but the fetch doesn’t complete inside the agent’s budget, you’re probably not in the final answer for that query.
Being late isn’t literally the same as being missing. The agent may have alternates and may try you again later. But for that specific synthesis, you’re out.
A simplified retrieval funnel
A rough mental model for live retrieval looks like this:
Step 1:
Discovery: the agent decides your URL might have the information.
Step 2:
Fetch: it requests your page.
Step 3:
Extract: it parses the content and pulls relevant passages.
Step 4:
Synthesize: it combines passages from multiple sources into an answer.
A 499 (or the equivalent on a non-NGINX stack) breaks step 2. No fetch, no extraction, no inclusion, at least for that turn. That’s not a “ranking factor” in the traditional sense, but if it happens often enough on your money pages, it’s plausibly a visibility tax.
What we tested (and the caveats)
Inside our own portfolio we ran an informal paired comparison:
Site A: a client property where we had identified a non-trivial pattern of 499s on key editorial and collection templates, and (with the client’s knowledge) left the underlying performance issues in place for a defined observation window.
Site B: a comparable property of similar authority, topic mix, and template structure, where we addressed TTFB, edge caching, and the specific endpoints producing 499s before the same window began.
Over the window, Site B showed more frequent appearance in AI Overview–style results and citations in chat-based search for the topics it should have been competitive on. Site A’s presence was patchier and noisier. That’s suggestive, not conclusive. Some honest caveats:
Two sites is a tiny sample, and “similar authority” is doing a lot of work in that sentence.
We can’t cleanly attribute 499s on Site A to AI agents specifically. Some are clearly humans on flaky mobile networks.
AI search surfaces are themselves moving targets, so changes during the window could be product-side, not site-side.
We didn’t isolate variables tightly enough to claim causation. We made a basket of performance fixes on Site B, not just 499 mitigation.
Independently, public commentary from practitioners like Michael King has pushed the broader idea that AI systems are sensitive to retrieval performance in ways traditional SEO didn’t emphasize. Our test isn’t a replication of anyone else’s work. It’s a small directional check on our own properties, consistent with that broader line of thinking.
Net: we think there’s enough signal to make 499 patterns worth investigating, and not enough to claim a confirmed mechanism.
Where 499 spikes tend to show up
In our logs, the templates most prone to 499s are also the ones most likely to matter for retrieval:
Heavy dynamic pages
Collection or hub pages assembled from multiple APIs.
Product comparison pages.
Filter- and facet-heavy URLs (tags, search endpoints).
These usually combine slow backend queries, cache misses, and long TTFB.
Overbuilt editorial templates
A dozen JavaScript bundles.
Third-party widgets and personalization calls.
Hero video, heavy fonts, client-side rendered content.
Humans will often wait three to five seconds. Automated clients are less patient.
“It’s fine on my laptop”
Edge misconfiguration.
Origin slow under concurrency.
Heavy TLS negotiation.
Backend queueing during peaks.
Bursty access patterns (several quick hits across many properties) expose fragility that a single browser session won’t.
Why we’re adding this lens to technical audits
Traditional technical audits focus on crawlability, indexation, performance for users, and structured data. We’re adding a fifth lens, tentatively, around retrievability under real-time constraints. The reasoning:
Users are asking longer, more complex questions.
They expect synthesized answers, not just blue links.
At least some systems pull from multiple sources concurrently and live.
On those queries, “slow to respond” looks a lot like “not included.”
Even if real-time retrieval turns out to be a smaller share of traffic than we think, the fixes for 499 patterns largely overlap with performance work you should be doing anyway. The downside of investigating is low.
How to audit 499s on your own properties
1) Confirm where 499 (or its equivalent) is logged
On NGINX you’ll have access logs with status codes. On other stacks, find the equivalent client-disconnect signal. You’re looking for patterns across URLs, user agents, time of day, and upstream response time. A single 499 isn’t a problem. A sustained pattern on important templates is.
2) Segment by URL type
Break the 499s down across blog posts, category/collection pages, search endpoints, API routes, auth/redirect chains, and CDN vs origin. Concentration on a few templates is good news, because you can fix it surgically.
3) Correlate with TTFB and upstream latency
Sustained 499s usually travel with slow backend responses, slow database calls, origin overload, and cache misses. If your logs include $request_time and $upstream_response_time, you’ll often see high values for both alongside the 499. Translation: the client wasn’t willing to wait.
4) Look at who is canceling
Not all 499s are the same. Look at known bots, proxy ranges, user agents associated with AI fetchers (these change frequently, so don’t overfit), and spikes that match known AI traffic patterns. Even when you can’t cleanly label an agent as “LLM,” a concentrated 499 pattern on retrieval-relevant pages is worth treating as a signal.
5) Compare with 408 / 504 / 524
499: client left early.
408: server timed out waiting for client.
504: gateway timeout (proxy didn’t get a response from upstream).
524 (Cloudflare): origin took too long.
499 clustered with 504/524 suggests a performance reliability problem, not just isolated disconnects.
What to do about it
Most of these are standard performance hygiene. We’re not claiming they’re uniquely “LLM-specific.” They’re things good technical SEOs already know to do. The argument is that real-time retrieval raises the cost of not doing them.
A) Make important pages fast to start
Retrieval agents need usable text quickly, not a fully hydrated app. Prioritize TTFB, server-side rendering or server-side content availability, and cached HTML for key pages. If your content only appears after client-side JS, you’re betting every agent will execute your app like a browser. That’s an uncomfortable bet.
B) Cache smarter, closer to the edge
Cache HTML at the CDN/edge for content pages where possible.
Many fetchers may hit a URL once. If every hit is an origin miss, you pay the latency every time.
C) Reduce backend complexity per request
Eliminate N+1 queries.
Move expensive personalization off the critical path.
Precompute popular pages.
Optimize database indexes.
Add application-level caching (Redis or similar).
D) Reduce payload and blocking work
Enable compression (Brotli/Gzip).
Remove unnecessary third-party scripts from content templates.
Lazy-load non-critical components.
Trim heavy fonts and hero media.
E) Tune timeouts intentionally
You can’t force a client not to cancel, but you can avoid long hangs, fail fast when upstream is unhealthy, and stop queueing requests until the client gives up. This is one of the places DevOps and SEO genuinely need to collaborate.
F) Make content easy to extract once fetched
This is the most genuinely LLM-flavored item on the list. Once an agent does fetch your page, give it the cleanest possible path to the answer:
Clear, descriptive headings.
Short answer blocks near the top of relevant sections.
Bullet lists where structure helps.
Definitions early in the document.
Appropriate schema markup.
A quick self-check
For your top retrieval-relevant pages, ask:
Can the main content be retrieved quickly without running a large JS bundle?
Is the first meaningful text visible early in the HTML?
Does the page depend on multiple upstream calls before content is available?
Do you see 499 (or equivalent) spikes during peaks?
If you hesitate on any of these, an audit is probably worth your time.
Closing: a hypothesis worth investigating
We’re not making the strong claim that 499s are a confirmed LLM ranking signal, or that every canceled request is a lost answer. The systems on the other end are too opaque, and our own evidence is too small, to talk like that.
What we are saying: real-time retrieval is part of how AI search works, we can’t assume an LLM will always answer from a pre-indexed copy of your page, and sustained 499 patterns on retrieval-relevant templates are a plausible visibility risk in that mode. The fixes are the same fixes that make your site better for everyone else, so the cost of investigating is low and the upside, if our reading is right, is real.
Treat this as an experiment to run on your own properties, not a checklist item to check off. And if your testing contradicts ours, we’d like to hear about it.
Months of trying to make a machine write something Tolstoy-tier. Here’s where it works, where it falls apart, and what surprised me.
Okay, so. For the past few months I’ve been doing something kind of ridiculous. I’ve been trying to use AI to write fiction. Not a parody, not a clone. Real literature. The kind of book that gets read in two hundred years, and could stand its own against the likes of Dostoevsky, Shakespeare, Tolstoy, Faulkner, Hemingway, and so on.
I know how that sounds. I’m not under the illusion that I cracked it. But the experiment was never really about producing the masterpiece on attempt one. It was about figuring out where exactly this stuff breaks. Where’s the seam? At what point does it stop being writing and start being a very smooth impression of writing?
This is just me sharing notes from the trenches. No hot takes, no manifestos. Just stuff I’ve actually noticed, in the order I noticed it. If you’re playing with this too, I’d love to hear what you’re seeing.
AI is a mirror, not a muse
This is the biggest thing, and honestly the thing I tell every writer friend who asks. AI doesn’t write your story. It writes your story back at you, in your voice, but only if you give it enough of your voice to work with.
First time I sat down with Claude Opus 4.7 and just asked it cold to write me an opening, the result was fine. Like, technically fine. Clean prose. It moved. It also belonged to nobody. It was the literary equivalent of stock photography. Looked good, said nothing, could’ve been written for anyone.
Then I tried something different. I wrote the first three to four pages myself first, and only then handed it over. Completely different result. Suddenly the model was picking up my rhythm, my weird pacing habits, the way I lean on certain kinds of clauses. It started feeling less like a co-author and more like a really attentive friend who’d been reading my drafts for years and was trying to keep me sounding like me while I stepped away.
Which reframes the whole thing for me. AI isn’t generating original literature. It’s amplifying voice you already have. If you don’t have a voice yet, it can’t help you find one. It’ll just give you back the average of every voice it has ever read, which is exactly what generic AI prose feels like.
So if you’re trying to write fiction with AI, my advice is dead simple: write the opening yourself. Write enough that the model has something real to grip onto. Then iterate. And expect to push back. A lot. Five, six rounds on tone, on a phrase that feels off, on a beat that came out wrong. The voice doesn’t survive on autopilot. You’re constantly correcting drift.
Where it absolutely shines: structural editing
If voice is where the model has gotten weirdly good, structure is where it’s just flat-out useful. Right now. Today. This is the unglamorous strength nobody writes essays about, but it’s probably the most valuable thing this tech does for a working writer.
Hand the model a messy draft. Tangled plot, two arcs that contradict each other, a subplot that goes nowhere, a scene you kept only because the dialogue was funny. It will find all of it in one read. It’ll tell you which threads are doing real work and which are decorative. It’ll suggest cuts that hurt to make and are basically always right. It catches the inconsistencies you’ve stopped seeing because you’ve read your own thing thirty times.
It’s not creative work in the romantic sense. It’s editorial work. But editorial work is what turns a draft into a book, and most of us don’t have access to a sharp, patient editor who’ll read the whole thing in one go and tell us the truth. The model is that editor. Three a.m., never tired, never offended.
One trap though. Don’t confuse “structurally cleaner” with “actually better.” The model can give you a more polished, more coherent version of your draft. That’s not always a better one. Some of the greatest novels in the canon are messy on purpose. Make sure you’re accepting cuts because they serve the story, not because the manuscript looks tidier afterward. Coherence isn’t the same as art.
Where it still struggles: actual human emotion
Now the harder part. The part I’m least optimistic about in the short term.
AI doesn’t really get human emotion the way a great novelist does. The newer models are surprisingly good at the surface of feeling. They can write grief, jealousy, longing, shame. The prose looks right. What they struggle with, and what every revision pass exposes, is the gravitational pull between two people.
Here’s the failure mode I keep hitting. The model writes two characters who are supposed to love each other, or be quietly destroying each other, and the words on the page are technically correct, but something is off. The dialogue is plausible. The interiority is plausible. The relationship isn’t. It feels like two people performing a relationship instead of being inside one. You can read it and tell.
When I push back, it usually improves on the second or third pass, but the improvement has a particular flavor. It starts borrowing moves from existing literature. The held silence, the betraying gesture, the line of dialogue that says one thing and means three. Sophisticated recombination of techniques rather than a fresh act of feeling. Sometimes that’s enough. A lot of the time, the seam shows if you’re paying attention.
The deeper version of this problem: AI is bad at subtext. It’s bad at what’s left unsaid. Half of what makes Tolstoy or Chekhov great lives in the gap between what a character says and what they mean. The silences. The misread look. The line that ends one beat too early. Models trained to be helpful and complete are basically biased against withholding. You have to fight them constantly to let a scene stay ambiguous, let a character stay unknowable, let a conversation just end without resolving.
Same problem shows up with endings. The model wants to close loops. It wants to land beats. It wants every chapter to feel earned. But great literature regularly refuses to do that. The Brothers Karamazov doesn’t really tie itself off. Hamlet leaves you with a corpse and a bunch of unresolved interior. Left alone, AI ties the bow every single time. You have to keep telling it not to.
The architecture problem
Here’s a failure that only shows up in long work, and it took me a while to catch.
The model can write a beautiful chapter. It can write a beautiful run of chapters. What it can’t reliably do, even with a ton of context, is the kind of architectural patience that defines a great novel. By that I mean the way Dostoevsky plants a tiny detail in chapter two that detonates in chapter forty. The way Tolstoy lets one image come back, transformed, three hundred pages later. The way you finish a novel and realize the whole thing was secretly about something the first chapter only hinted at.
That kind of long-range intentionality requires holding the entire book in your head as one object. The model holds context, sometimes a lot of it, but it doesn’t seem to hold the work as an object the way a writer does. It writes locally. It nails the next great paragraph. It doesn’t seem to know which paragraph is doing load-bearing work for a payoff three hundred pages out.
Until that changes, the architectural soul of a long novel has to come from a person. The model can help you execute. It can’t yet hold the whole thing.
Voice vs. worldview (these are not the same thing)
Quick distinction worth making, because people collapse these and they shouldn’t.
Voice, at the sentence level, is the texture. Diction, syntax, rhythm, the small recurring habits. Models pick this up fast. Three to five pages and Claude is producing sentences that sound like mine.
Worldview is something else. Worldview is the moral and philosophical lens that holds a body of work together. Tolstoy isn’t great because of his syntax. He’s great because of his lifelong obsession with moral awakening, with how to live, with the specific weight he gives to dying men and peasants and aristocrats. That worldview is inseparable from his actual life. His crises. His late-life religious turn.
Models don’t have a worldview. They have a statistical absorption of every worldview they’ve ever read. Ask one to write inside a worldview and it performs one. Sometimes the performance is excellent. It’s rarely the kind of unified moral vision that animates a real book from inside, because that vision tends to come from somebody who actually lived a life and arrived at convictions about it.
This isn’t obviously a problem you fix with bigger models or more data. It might be a different kind of limitation.
The “necessity” problem
There’s a thing critics call necessity. The feeling that a sentence had to exist exactly this way. That if you swapped it out, something would be lost.
AI prose, even the good stuff, mostly doesn’t have it. It’s well-formed. It’s appropriate. It moves the plot. But you can usually picture ten other versions of the same paragraph that would do basically the same job. The sentence doesn’t feel inevitable. It feels picked from a menu.
Great writers do something else. They write sentences that, once you’ve read them, you can’t unread. There’s almost a fingerprint at the level of word choice. AI tends to produce sentences that feel like a consensus of fingerprints. That gap, between inevitable and merely appropriate, is now the main thing I look for when I’m editing model output. I cut everything appropriate. I keep only what feels like it had to be there.
It always sounds like 2026
One more thing on style. AI defaults to contemporary literary fiction voice. Even when you ask it for nineteenth-century cadence or Elizabethan diction, the gravity pulls back toward something polished and present-day. You can win individual battles. You can feel it resisting the whole time. On its own, it doesn’t write like Melville or Faulkner or Woolf. It writes like a thoughtful contemporary writer doing an impression of them.
This matters more than it sounds. Part of why the canon is the canon is that those books are stylistically irreducible to any other era. They couldn’t have been written at any other time, by any other person. AI tends to produce work that feels like right now, which is a strange thing to say about a model trained on centuries of literature, but that’s what I see.
The friction question (this is the one that keeps me up)
Last thing, and the one I genuinely don’t have an answer for.
A lot of what we call great literature was made under absurd amounts of friction. Dostoevsky wrote in debt, in mourning, sometimes mid-seizure, sometimes for his life. Tolstoy revised War and Peace by hand for years. Shakespeare worked under commercial pressure inside an industry. The friction wasn’t incidental to the work. It shaped it.
AI removes friction. That’s the whole point. Blank page is less scary. Fifth draft arrives faster. Structural problems get diagnosed in minutes instead of months.
So here’s the question I keep circling: can art survive the removal of friction? Maybe friction was always romantic mythology and the work is what matters, full stop. Maybe shorter feedback loops just mean better art, faster, because we get more iterations per lifetime. That’s a totally defensible read.
But maybe not. Maybe some of what makes great literature great is the trace of a person who paid for every paragraph in years of their life. Maybe readers can feel that cost without being able to name it. Maybe a frictionless work, however polished, lacks the specific gravity that comes from being wrung out of someone.
I don’t know which is right. I suspect both are partly right. What I know is the question isn’t going away, and anyone using these tools seriously is going to have to come up with their own answer.
Where this leaves us
After all of it, here’s where I’ve landed for now.
AI cannot, today, write a novel that belongs next to Tolstoy. Voice is shallow without a writer behind it. Emotional connection between characters is performed, not felt. Subtext gets suppressed by training that wants clarity. Long-range architecture is beyond the model’s grip. Necessity is missing. Worldview is borrowed. The era keeps leaking through.
AI can, today, make a serious writer faster, sharper, less stuck, and more structurally rigorous than they would otherwise be. That’s not nothing. That’s actually a huge deal. It just turns out to be a different deal than producing literature that lasts.
My current bet: the great novels of the next twenty years will still be written by people. People who use AI heavily, in ways earlier writers didn’t have access to, but the central acts, the voice, the worldview, the emotional truth, the architectural intention, will stay stubbornly human. If that ever stops being true, that’s going to be one of the more interesting boundaries this technology crosses. I’m watching for it. I’ll keep poking at it.
Experiment continues. If you’re doing this too, message me. Genuinely curious what you’re seeing.
AI search didn’t just change SEO. It quietly removed the economic reason most informational content existed in the first place — and that should worry the LLMs more than it worries anyone else.
“There are some decades where nothing happens, and then there are weeks when decades happen.” — Lenin
Lenin was talking about revolutions. I think about that quote every time I open ChatGPT or Claude now.
For about twenty years, the open web ran on a deal that almost no one wrote down but everyone in marketing understood. A user types a question into Google. Google sends them to a page. The page answers the question, and on the way to the answer, the publisher gets a chance to show an ad, build brand affinity, capture an email, or pitch a product. Informational content was the entry ticket. Monetization came later in the funnel.
That deal is breaking in front of us, and most companies are still pretending it isn’t.
The trade that built the web
If you’ve ever been told to write a blog post titled “What is SEO?” or “What is an LLM?” or “How are footballs made?” , you have lived inside the trade. The post itself rarely paid for itself directly. Nobody buys software off a definitional explainer. The post existed because it ranked, and ranking pulled strangers into a property where the company could finally introduce itself.
The classic move was the listicle. “Best CRM software in 2024.” “Top 10 project management tools.” The company writing the post was, conveniently, also #1 on the list. It was a soft con, but a productive one: the user got a real comparison, the publisher got a real lead, the search engine got a real answer to index. Every party in the system had an incentive to keep producing more of it.
Multiply that across every B2B SaaS company, every DTC brand, every media site, every affiliate. That is most of the written internet of the last two decades. Top-of-funnel content was never really about education. It was a customer acquisition channel that happened to be educational.
What AI search actually broke
ChatGPT, Claude, Perplexity, Gemini — they all do the same thing from the user’s perspective. They answer the question. Inline. With citations sometimes, often without. The user never lands on the page that wrote the answer.
This sounds like a small UX change. It is not. It is a structural break in the economics of informational content.
If a user can ask “what’s the difference between RAG and fine-tuning” and get a competent two-paragraph answer in the chat window, the marginal value of writing the 47th blog post titled “RAG vs. Fine-Tuning Explained” collapses. The traffic doesn’t arrive. The lead doesn’t get captured. The product never gets pitched. The post becomes a write-only exercise, useful as training data for the model, useless as a customer acquisition channel for the author.
The incentive that funded the production of informational content for twenty years has been quietly removed. Not banned, not deprecated. Removed.
The competition layer disappears too
The thing people miss is that this isn’t just bad for individual publishers. It is bad for the long-run quality of the answers themselves.
The reason there are good explainers on the web is that ten companies fought to write the best one. Competition for the click is what produced the second draft, the better example, the clearer diagram, the updated 2024 numbers. When the click goes away, so does the competition. So does the second draft.
Informational content won’t die completely, universities will still publish, hobbyists will still write, a handful of companies with strong brand budgets will keep doing it for reasons unrelated to immediate ROI. But the volume and the rate of improvement both fall off a cliff. The web stops getting better at explaining itself.
The existential problem this poses for the LLMs
Here is the part that should be keeping the model labs up at night.
LLMs are, in a real sense, fine-tuned on the output of the very system they are dismantling. The reason a chatbot can give you a competent answer about RAG, or footballs, or the Treaty of Westphalia, is that thousands of humans were paid, directly or indirectly, to write good explainers, because the click economy made it worth their while. Take away the click economy and you take away the production line.
The first generation of frontier models was trained on a web that was being aggressively cultivated for human readers. The next generation will be trained on a web where the incentive to cultivate has been pulled. You can see where this goes. Models trained on models trained on models, with fewer and fewer fresh human explainers entering the corpus. The technical term is model collapse. The marketing term is “why do all the answers feel the same now.”
The chatbots need the open web more than the open web needs the chatbots, and the open web is figuring that out slowly.
What this means if you’re a marketer right now
If you’re running content for a company in 2026, the honest answer is that a lot of what worked in 2022 doesn’t anymore, and pretending otherwise is expensive. A few things I’d push on:
Stop measuring TOFU content by traffic alone. Generic explainers that used to pull search traffic will keep losing it. The pages that still earn their keep are the ones with a point of view, original data, or a perspective the model genuinely cannot synthesize from the rest of the web.
Take brand authority seriously as a distribution strategy. When a user asks an LLM “what’s the best tool for X,” the model surfaces the brands it has the strongest, most consistent signal about. Getting mentioned in the answer is the new ranking. That mention is earned through the same things brand has always been earned through — PR, original research, podcast appearances, partnerships, customers talking about you in places models read, just with a much sharper payoff than before.
Write less, and write things only you can write. A weekly explainer of yesterday’s news loses to the chatbot every time. A field report from your customer base, a benchmark you ran, a contrarian take from someone with skin in the game, those still work, because they’re primary sources. The model needs them. So do humans.
Treat being cited by LLMs as a real channel and instrument it. Track which models mention you, in what contexts, with what framing. This is the new SERP and almost no one is monitoring it yet.
The honest summary
Top-of-funnel content didn’t die because AI is better at writing it. It died because the click, the thing that paid for it, stopped arriving. The chatbots ate the entry ticket and left the rest of the funnel intact, which is the part nobody seems to be saying clearly.
It’s a strange moment. The companies that own the chatbots are also the ones quietly eroding the supply of training data they depend on. The companies that produce the training data are watching their distribution model evaporate. The users are getting faster answers and a thinner web. Nobody is obviously winning except, in the short term, the inference layer.
Lenin’s line keeps coming back to me because this really is one of those weeks-where-decades-happen moments. The mechanics of how content gets produced, distributed, and rewarded on the internet are being rewritten in real time, and the playbooks built for the previous regime aren’t going to survive contact with this one.
If you’re trying to figure out what your content and brand strategy should look like in the AI-search era — what to keep doing, what to kill, and how to actually get cited by the models that are now sitting between you and your customers — that’s the question we work on at Zeta Theorem. I’d be happy to talk.