Keyword Research Isn’t Dead, But the Keyword Might Be

Here’s a fun exercise. Open your keyword tool of choice and look up something like “best running shoes.” Thousands of searches a month. Solid difficulty score. A classic target.

Except nobody actually wants “best running shoes.” What they want is something like: “I’m training for my first half marathon, I overpronate, and my budget is about $120. What should I buy?”

For twenty years, that second sentence had nowhere to go. The search box rewarded short, robotic phrases, so we all learned to compress our messy human needs into three-word fragments. Keyword research was built on top of that compression. We counted the fragments, sorted them by volume, and wrote content for the winners.

AI search just removed the compression. People can ask the full question, add constraints, follow up, and get one synthesized answer. Which means the thing we’ve been counting all these years, the keyword, is dissolving back into what it always represented: intent.

That doesn’t make keyword research dead. It moves the job up a level, from counting strings to mapping intent. Let’s walk through what’s actually changing, what it does to your research process, and what it means for the content you publish.

The same need, two interfaces. Volume tools only ever saw the left side.

This was always an information retrieval game

Strip away the tools and dashboards, and search has always worked the same way. A person has a need, compresses it into a query, and a retrieval system decompresses that query into a ranked list of documents. Keyword research worked because everyone compressed the same way. Millions of different people, with millions of slightly different situations, all typed “best crm small business.” Demand pooled into countable strings, and an entire industry of volume and difficulty metrics grew around counting them.

LLMs didn’t change the game. It’s still information retrieval. They changed the interface, and the interface was the only reason keywords existed. Now the system retrieves passages from across the web, synthesizes them into an answer, and cites a handful of sources. Same game, new referee.

What actually changed: three shifts

1. Queries decompressed, and your metrics can’t see it

The data on this is striking. Queries in Google’s AI Mode run nearly twice the length of traditional searches (7.2 words versus 4, per Semrush). ChatGPT prompts run roughly 17 times longer than Google searches, according to SimilarWeb. And these aren’t one-shot queries anymore. They’re conversations, with follow-ups doing the work that five separate searches used to do.

Now here’s the uncomfortable part for those of us who grew up on keyword tools. Search volume and keyword difficulty are derived from traditional engine data. Prompts are private. They never enter those datasets. So the fastest-growing slice of demand is invisible to the exact metrics we use to decide what’s worth targeting.

Think about what each metric actually tells you now. Volume tells you how people used to compress this need on one platform. Difficulty tells you how hard it is to win a prize that’s shrinking. Neither is useless, but both got demoted: from decision gates to partial inputs.

Here’s the honest reassessment of the dashboard:

Metric	What it told you	What it misses now	Treat it as
Search volume	Pooled demand for one phrasing	Private prompts and infinite phrasings	A partial demand signal
Keyword difficulty	Competition for a ranking slot	Citation competition: entity authority, extractability	An input, not a gate
Rank tracking	Your position in ten blue links	Whether AI answers cite or name you	One scoreboard of two
Classic SERP analysis	Who ranks, with what content	The sub-queries, cited domains, and named brands inside AI answers	Replaced by fan-out analysis

2. Engines stopped ranking pages and started assembling answers

When someone asks an AI engine that half marathon question, the system doesn’t run one search. It quietly breaks the prompt into a swarm of sub-queries (shoes for overpronation, stability versus neutral shoes, best running shoes under $150, how shoe fit changes for long distances), retrieves content for all of them in parallel, and assembles an answer. Google calls this query fan-out, and it’s not unique to Google. ChatGPT runs a web search in about 31% of prompts, averaging two searches per prompt.

The new SERP: sub-queries you never see, passages instead of pages, citations instead of rankings.

Better yet, try it yourself with the interactive simulator above.

Two consequences worth sitting with. First, you no longer rank for what the user typed. You get retrieved for sub-queries you never see. Second, the unit of retrieval is the passage, not the page. Google’s own guidance confirms that AI features pull relevant chunks of pages into answers. A single well-structured section of your article can earn a citation even if the page as a whole would never have ranked for the original question.

3. The funnel collapsed into the conversation

Comparing, shortlisting, and “actually, what about…” now happen inside the answer instead of across ten tabs. The click data shows it. Pew Research found users click a result on 8% of searches with an AI summary, versus 15% without one. Ahrefs measured a 58% drop in click-through rate for the top result when an AI Overview is present.

Before you panic, look at the other side of the ledger. Semrush found the average AI search visitor is worth 4.4 times more than a traditional organic visitor. Seer Interactive watched ChatGPT referrals convert at 15.9% against 1.76% for organic search. The model did the comparison phase for the user, so the people who do click arrive pre-qualified. (One caveat: this skews toward considered purchases like software and services. Impulse ecommerce still behaves more traditionally.)

Four numbers that tell the whole trade: scarcer clicks, better clicks.

So queries are sorting into three buckets. Dying queries get answered inline, and most top-of-funnel informational content lives here. We’ve written before about how AI killed TOFU SEO content, and this is the mechanism. Surviving queries still earn the click: transactional, navigational, and anything people want to verify themselves. And newborn queries are the complex, multi-constraint prompts nobody bothered typing into a search box before. Your research process needs to classify demand by survivability, not just volume.

Query bucket	What’s happening	Example	Your move
Dying	Answered inline, the click rarely happens	“what is a crm”	Win the citation, not the visit
Surviving	A click is still needed to act or verify	“hubspot pricing”, “trail running shoes size 10”	Classic SEO still pays here
Newborn	Complex prompts nobody used to type	“best CRM for a 12-person agency that lives in Gmail”	Cover the question space

The new keyword research process, step by step

Here’s how the workflow changes in practice.

The new workflow at a glance. Details below.

Step 1: Start with people, not seed keywords. Your customers’ actual language is the new seed list. Mine sales call transcripts (Gong or Fireflies make this easy), support tickets, G2 and Trustpilot reviews, and the Reddit threads where your category gets discussed. You’re collecting questions, situations, and constraints, phrased the way real people phrase them. That’s what prompts look like.

Step 2: Keep your keyword tools, but change their job. Ahrefs, Semrush, and Google Search Console still map demand topology beautifully. The shift is in how you read them: treat each keyword as the head of an intent cluster, not a target. “Best CRM” isn’t one piece of content anymore. It’s the visible tip of fifty buyer situations, and your research should enumerate those situations.

Step 3: Simulate the fan-out. This is the new SERP analysis. Take your head intents and run them through ChatGPT, Perplexity, Gemini, and Google’s AI Mode. Record four things every time: the sub-questions the engine explores, the follow-ups it suggests, the domains it cites, and the brands it names. Tools like iPullRank’s Qforia can generate likely fan-out queries at scale, but even a manual afternoon of this is eye-opening. The gaps you find (questions where competitors get cited and you don’t exist) are your roadmap.

Step 4: Prioritize by coverage, not volume thresholds. Here’s a mindset shift that matters. Zero search volume no longer means zero demand. It usually means the demand is spread across infinite phrasings, or that it’s newly expressible now that the box accepts full sentences. If a phrase maps to a real pain point your customer has, it belongs in your research, volume or not, because any sub-question can be the hook that pulls you into a high-stakes answer. The metric to manage is coverage: the share of your category’s question space where you’re retrievable. One guardrail, though. Coverage means depth within clusters, sections and passages inside substantial resources. It does not mean a thin page for every permutation. That’s content farming with extra steps.

Step 5: Track share of answer, not just rank. Tools like Profound, Peec AI, Otterly, Semrush’s AI Toolkit, and Ahrefs’ Brand Radar now track whether you’re cited and mentioned across AI engines. Pair that with AI referral traffic in GA4 (and its conversion rate, which is the number your CFO will care about). Rankings still matter, but they’ve become an input, not the scoreboard.

And the whole process on one screen:

Step	Do this	Tools
1. Start with people	Mine real customer language for questions, situations, constraints	Gong, Fireflies, G2, Trustpilot, Reddit, support tickets
2. Re-task keyword tools	Treat each keyword as the head of an intent cluster	Ahrefs, Semrush, Google Search Console
3. Simulate the fan-out	Record sub-questions, follow-ups, cited domains, named brands	ChatGPT, Perplexity, Gemini, AI Mode, Qforia
4. Prioritize by coverage	Map every real pain point, volume or not, with depth per cluster	Your intent matrix, AlsoAsked
5. Track share of answer	Measure citations, mentions, AI referrals and their conversion	Profound, Peec AI, Otterly, Semrush AI Toolkit, GA4

What this does to the content you publish

The research shift drags content strategy along with it.

The page-per-keyword era is over; the resource-per-intent-cluster era has started. One substantial piece that covers a buying situation from every angle, built in extractable passages: question-shaped headings, the direct answer in the first sentence or two, then the supporting detail, data, and tables. You’re writing for a system that grabs self-contained chunks, so make your chunks self-contained. This is exactly the kind of architecture a good content strategy maps out before anyone writes a word.

Information gain stops being a nice-to-have. When an engine synthesizes from multiple sources, content that repeats what everyone else says contributes nothing quotable and gets averaged out. Original data, first-hand experience, and specific claims with numbers attached are what earn citations. Be the source the model quotes, not the summary it skips.

And your off-page presence is now part of your content strategy, whether you like it or not. Models triangulate from reviews, Reddit threads, and comparison posts. What the corpus says about you is what the answer says about you.

The bigger picture: you now have two audiences

Here’s the thing I most want non-SEO marketers to take away. Keyword research’s job description never changed. It was always about understanding how your audience seeks information, then making sure you show up at the moment of need. What changed is that an AI agent now sits between you and them, doing the finding, reading, and relaying.

That agent doesn’t behave like Google’s top ten. Ahrefs analyzed 15,000 queries and found that only 12% of URLs cited by AI tools overlap with Google’s top 10 results. The other 88% come from pages that don’t rank on page one at all. The engines go beyond the familiar index to find passages that actually answer the sub-question at hand. That’s a real opening for smaller brands who could never crack the top ten, and a real warning for big ones coasting on domain authority.

The retrieval pool is much bigger than the top 10, and it reads like a machine.

So your content has to work for the machine reader too. It needs to be retrievable (don’t block AI crawlers; we’ve even dug into oddities like whether 499 response codes affect AI SEO), parseable (clear structure, answer-first passages), and corroborated (consistent facts about your brand across the web, because models trust what multiple sources agree on).

Zoom out and the strategic picture looks like this. The KPI is shifting from rank to share of answer. Being the brand named in answers is the new position one, and being the brand named in prompts (“is X any good for Y”) is the new branded search. That pushes SEO, PR, and brand into one discipline, because influencing what models say about you means influencing the entire corpus they read. And through all of it, classic SEO remains the substrate rather than a casualty. These engines ground their answers in search indexes, so crawlability, authority, and rankings are the entry fee. Everything above is additive.

Where to start this week

Pick your five most important buyer intents. Run each through ChatGPT, Perplexity, and AI Mode, and write down who gets cited, who gets named, and which sub-questions you have no answer for. That one exercise will tell you more about your real visibility than any rank tracker currently can.

Then fix one cluster end to end: research the question space, build the resource, structure it for extraction, and track citations for a quarter. If you’d rather not figure out the machinery alone, that’s exactly what our AI/LLM optimization work is for. Either way, start now. The marketers who treat this as an evolution of information retrieval, rather than a brand-new dark art, are going to be very hard to catch.