ChatGPT Reasoning Modes Share Only 25.6% of Cited Domains

Contents

1. What the study measured
2. How reasoning changes the search
3. Where the citations shift
4. What the study can’t tell you
5. Which prompts this matters for
6. What to change on your site
7. Model changes move citations too
8. How to test your own prompts
9. A worked example
10. Common mistakes
11. FAQ

What the study measured

Semrush and Kevin Indig of Growth Memo ran 100 prompts through GPT-5.2 twice, once at minimal reasoning and once at high, for 200 responses. Semrush maps those settings to ChatGPT’s Instant and Thinking modes. The prompts followed 20 buyer journeys in four categories. This is vendor data, collected with Semrush’s own AI Visibility toolkit.

Overlap of cited domains for the same prompt was 25.6%. Of the 173 domains cited at high reasoning, 99 never appeared at minimal reasoning. The net gain in unique domains is 46, so the bigger finding is the swap, not the growth.

Keep two readings of that overlap apart. By our arithmetic, 74 of the 173 high-reasoning domains (43%) also appeared somewhere at minimal reasoning. The 25.6% is described as overlap for the same prompts, which may be why it’s lower, though Semrush doesn’t publish the formula. A domain can drop out of one prompt and still show up on another.

How reasoning changes the search

The gap opens before any answer is written. ChatGPT turns your question into its own searches (called fan-out), retrieves pages, then picks which ones to cite. High reasoning ran 4.6 times as many searches, while responses grew only slightly, so the extra citations aren’t just a product of longer text.

Citations followed the searches. They peaked at Comparison, with 9.8 per response at high reasoning against 5.8 at minimal, and at Selection they were 4.7 against 2.6. Selection was the most erratic stage: Indig reports 0 to 40 queries per response, with tightly bounded prompts running none and open-ended product builds running 28 to 40.

A separate October 2026 preprint that recorded live ChatGPT web sessions found the same shape: GPT-5.4 Thinking often ran several retrieval rounds, while Instant settings mostly ran one. It isn’t peer reviewed, and it used different prompts.

More searching doesn’t always mean more variety. The same domain appeared more than once in 51 of 100 high-reasoning responses, against 26 of 100 at minimal. A site with deep coverage of a topic can take several citations in one careful answer.

Where the citations shift

The mix of sources moved toward official and institutional material. These are shares of citations by source type in the Semrush write-up. The minimal column adds to more than 100%, so the categories appear to overlap:

The citation rate rose most in Finance and Health and Lifestyle, less in B2B SaaS and barely in Consumer Tech. The gap also varied by buying stage: +35 points on Problem prompts and +5 on Validation prompts. Semrush doesn’t give the other stages.

A brand cited at the Problem stage was still cited at Selection in 4 of 20 journeys at high reasoning, and in none at minimal. Indig reports that all four were in Finance, and that brand mentions (rather than citations) persisted in 3 journeys at high and 2 at minimal. Treat persistence as a thin result that may be specific to Finance.

What the study can’t tell you

• How many people use each mode. Indig says he doesn’t know how many users turn reasoning on, so the real-world weight of each mode is unknown.
• Whether the settings match the app. The test used minimal and high reasoning settings. Semrush maps them to Instant and Thinking without saying how closely they correspond.
• How much is just variation. Each prompt ran once per setting, so the test can’t separate the effect of reasoning from ordinary run-to-run differences. In the preprint above, 40.8% to 44.5% of the pages ChatGPT retrieved for a question appeared in only one of 20 runs on the same Instant setting, so some of the gap between modes may be ordinary churn.
• Whether it holds on other models. It covers one model, and the prompt run dates aren’t given.
• Why a category moved. The numbers show that Finance rose more than Consumer Tech. They don’t explain why, and the 20 journeys are spread across just four categories.

Which prompts this matters for

Indig says ChatGPT routes hard prompts to reasoning automatically, and that free users can also use reasoning with limits. So the type of question predicts the mode better than the user’s plan does. He lists multi-criteria comparisons, evaluation frameworks, compliance questions and complex shopping builds as the likeliest triggers. Semrush puts definitions and single-factor questions in the simple group.

Semrush explains Consumer Tech’s small lift by saying the model already knows these topics. The test didn’t check that. Ecommerce isn’t covered as a category, so if you sell products, run the test below before changing anything.

What to change on your site

This is our reading of the data, not something the study tested. If high reasoning moves toward official and institutional sources, the pages to check first are the ones a careful researcher would want:

• Documentation and support pages. Make sure they are public and crawlable, and that they answer specific questions such as limits, integrations and security.
• Comparison and selection content. These stages drew the most fan-out, so publish plain pages on how you differ, who you’re wrong for, and what you cost where you can.
• Primary sources. Link the regulation, standard or study behind each claim, especially in finance and health.

Semrush proposes a two-track plan. For minimal reasoning: comparison content, review sites, Reddit and product pages. For high reasoning: early-stage education, official documentation and reference material at URLs that can be cited. Indig adds that the brand that wins is the one whose documentation surfaces for each sub-query. We read that as a reason to write for the narrow questions behind the main prompt.

Model changes move citations too

Reasoning settings aren’t the only switch. SE Ranking compared AI Overviews on 100,000 US keywords before and after Gemini 3 became the default on January 27, 2026. About 42.4% of previously cited domains (37,870 of 89,262) no longer appeared, almost all of them in the long tail. Among the 500 most cited domains, only one disappeared.

That is vendor data from a before-and-after comparison with no control group, so it can’t isolate the model as the cause. It does show why one baseline goes stale: re-measure after a major model change.

How to test your own prompts

1. Pick 10 prompts your buyers ask, with at least two each from early research, comparison and final selection.
2. Run each three times on Instant, in fresh chats, and record the cited domains.
3. Run each three times on the highest thinking level your plan offers, and record them separately.
4. Count your domain and your top three competitors in each set.
5. Compare the two sets. A gap that shows up across all three runs is worth acting on; a gap in one run is probably noise.
6. Repeat monthly, and after any major model release.

A worked example

The company and figures below are made up for illustration. Ledgerly, an accounting app, runs one prompt per buying stage three times in each mode and records how many runs cite ledgerly.example.

Three runs can only expose large gaps. SparkToro’s authors, who studied repeat runs, suggest at least 60 to 100 runs per prompt, averaged. By hand, act only on all-or-nothing splits like the Problem row, and use a tool if you need finer readings.

Common mistakes

• Blending modes in one report. Indig advises against aggregate views, because the two modes search differently.
• Testing only Thinking because it cites more. Semrush calls Instant the default experience, and it behaves differently.
• Reading a shift as a win or a loss. You need repeat runs in the same mode first.

You can run this test by hand. For an ongoing baseline, LLMagnet measures AI visibility across seven engines, and its free audit is one place to start.