The Naperville Test: What ChatGPT Actually Cites When It Recommends a Local Business

A viral thread says: prompt ChatGPT ten times, screenshot the sources, then go get profiles on whatever repeats. I ran it properly across 13 runs with SERP controls. It is half right, and the half it misses is the half that makes money.

The SEO Guy posted this on Friday the 14th and it went off. 475 likes and 15.6K views inside the first day. The method is four steps:

Ask Claude for 10 different prompts about finding a personal injury lawyer in Naperville. Run all 10 through ChatGPT. Screenshot the sources. Feed the screenshots back to Claude and ask what repeats. Whatever repeats is where you need a profile.

It is a good instinct. Go look at what the machine actually cites instead of guessing. I ran it the next morning and I would change three things before anyone builds a client deliverable on it.

Skip the screenshots. You can read the cited URLs straight out of the page. ChatGPT stamps every outbound citation with ?utm_source=chatgpt.com, so the sources are sitting right there as text. Screenshotting and re-uploading is a lossy way to do arithmetic you can just do.

Use a fresh temporary chat every time. Otherwise runs 2 through 10 are reading runs 1 through 9 and you are measuring your own session.

Run the same prompt more than once. This turned out to matter more than everything else, and I will get to why.

What I actually ran

Ten prompts, but ten different intents, not ten rewordings of “who is the best”. If all ten are paraphrases you have measured one query ten times and learned nothing about the shape of the space. So: a direct pick, a shortlist, a situational one, a vetting one, fees, reviews, credentials, near-me, a comparative, and a sub-niche.

Logged out, fresh temporary chat each time, citations read from the page source. Then three extra runs of the identical first prompt to test wobble, and two Google SERP pulls as a control.

Finding 01There is no stable list

This is the one that undercuts the whole premise. Nothing was cited in a majority of runs. The two most-cited sources, Avvo and Expertise, each showed up in 4 of 10.

You could screenshot one pass, see Avvo and Expertise, and walk into a client meeting saying “these are the two places you need to be.” You would be describing 40% of the picture.

Finding 02Over half the citations point at the firms own websites

Nine of the eighteen domains were law firm websites. Not directories. Their own sites. And they took 22 of the 39 citations, so 56% of everything ChatGPT cited was a firm publishing about itself.

The advice “those sources become the places you need profiles on” quietly assumes the sources are all third-party listings you can go sign up for. Half of them are the thing you already own.

Finding 03Query intent decides which kind of source you get

This is the part worth stealing. The source type was not random, it tracked the shape of the question.

Four of the ten runs cited no directory at all. Ask “I got rear-ended on Ogden Ave and my neck hurts, who do I call” and ChatGPT cites four law firm pages, all of them deep practice-area pages: /rear-end-crash/, /rear-end-collisions/, /naperville-rear-end-accident-lawyer/. An Avvo profile does nothing for you there. A page about rear-end collisions does.

Ask about contingency fees and you get FAQ and blog posts. /how-do-personal-injury-lawyers-get-paid/. /illinois-personal-injury-faqs/. That is content marketing, cited back to you as an answer.

And the credentials query is its own category, because you cannot buy your way in. Super Lawyers is peer-nominated. National Trial Lawyers is an admission. The IARDC lists you because the state says so. “Go make a profile” is not available as a move.

Finding 04On the head term it ignores Google. On the long tail it is Google.

I pulled the real SERP for two of these to see whether ChatGPT was just reading Google back to me.

For best personal injury lawyer naperville il, the overlap between ChatGPT cited domains and Google top 9 was zero. Avvo and Expertise, ChatGPT two favourites, do not rank in Google top 9 at all. Meanwhile Justia sits at Google #9 and was never cited once across thirteen runs.

Then I checked the long-tail one.

Four for four. It cited Google #3 result down to the exact URL. So the answer to “is AI search just SEO” is: on a broad superlative query, no, it is off doing its own thing with directories. On a specific query with real intent behind it, it is your organic rankings with a different font.

Ann Smarty said this to me on the show a while back and the data just agreed with her:

“SEO is still the most predictable path into LLMs. Not the only one.”

Ann Smarty — Unscripted SEO

Finding 05One prompt, three runs, four different answers

Here is the one that should stop you from screenshotting once. I ran the identical first prompt three times.

Four domains surfaced. Exactly one, Avvo, showed up every time. If you had screenshotted run A and stopped, you would have missed a firm site and you would have missed Bestlawyers.com completely.

Three runs is not a sample either. It is just enough to prove that one run definitely is not.

What Malte said about this exact method

I interviewed Malte Landwehr of Peec AI five days before this test, and his five-step fan-out method is basically this thread written by someone who does it for a living. The difference is one clause. He says mine the cited sources at both URL and domain level.

That clause is the whole finding above. At domain level, run 3 says “four law firm websites” and you shrug. At URL level it says “four pages specifically about rear-end collisions,” which is an actual instruction for what to publish on Monday.

His bigger point is why any of this works at all:

“If you only talk about your pricing on your pricing website, and then you change your pricing, and then there are five Reddit threads and two reviews on random blogs that still talk about your old pricing, ChatGPT will answer with your old pricing.”

Malte Landwehr, Peec AI — Unscripted SEO

LLMs assemble consensus. They do not rank pages. The directory profile matters not because the directory is magic but because it is another voice agreeing with you. Malte prescription is unglamorous and correct: publish the core message everywhere it can be found, then write the same story from several angles so the model finds agreement fast.

Dan Kurtz put the same idea more bluntly on the show: “Bots have a preferred content type. Technically, they are just an additional customer.”

The version I would actually run for a client

1. Write 10 prompts across 10 intents, not 10 rewordings. Direct, shortlist, situational, vetting, fees, reviews, credentials, near-me, comparative, sub-niche. The intent spread is where the information is.
2. Run each one at least three times in a fresh temporary chat. Anything appearing once is noise. Score by frequency, not by presence.
3. Record URLs, not just domains. The path is the deliverable. /rear-end-crash/ tells you what to publish; mathyslaw.com tells you nothing.
4. Split the results three ways. Sources you can join (directories, listings). Sources you must earn (Super Lawyers, bar associations, awards). Pages you should own and write yourself. Three different budgets, three different timelines.
5. Pull the SERP for the same queries. Where they overlap, it is an SEO job you already know how to do. Where they diverge, that is the actual AI-specific work and the only part worth a separate line item.
6. Check the map widget separately. On three of my ten runs ChatGPT rendered a Maps panel naming five firms with star ratings. That is a Google Business Profile surface, it is driving the named recommendations, and no screenshot of the citation list captures it.

What I would want before trusting any of this further

Being straight about the limits. Thirteen runs in one vertical in one city on one model on one morning, logged out. Logged-in ChatGPT with memory on will behave differently, and that is most real users. I did not test Perplexity, Gemini, or Claude, and there is no reason to assume the source mix transfers. Local queries also lean on geography in a way that national B2B queries will not.

The method holds up. The specific 18 domains are a snapshot of one Saturday morning in Naperville, and if you present them to a client as a stable target list you will be wrong within a month.

What actually transfers is the shape: intent decides source type, over half of it is your own site, the long tail is just SEO, and nothing is stable enough to measure once.