Pressfront Research

How consistent are AI recommendations? A first look.

Published September 13, 2026 · An exploratory study by Pressfront

We asked ChatGPT and Gemini the same ten "best X for a small business" questions, three times each, within a single day. About a third of the businesses they recommended changed between identical runs. The most-named businesses stayed locked in every time; the rest of the shortlist shuffled underneath them.

~31%

of recommended businesses changed when the same question was asked again

56%

of businesses named were named in every run; a "locked" core

27%

were named in only one of the three runs; a shuffling tail

Why we ran this

People increasingly ask an AI, rather than a search engine, which company or product to use. So a fair question for any business owner is: if a customer asks ChatGPT or Gemini to name the best option in my category, how stable is that answer? If the same question produces a different shortlist each time, then "being recommended by AI" is less a fixed ranking and more a moving target. We wanted a number for it, so we measured it.

What we did

We chose ten open buying questions across ten categories (PR distribution, CRM, email marketing, project management, accounting, marketing agencies, website builders, live chat, e-commerce and scheduling), phrased the way a small-business owner would type them. We deliberately used open "best X" questions, not "X vs Y" comparisons, because a comparison can only return the two names in the question. We asked each question three separate times on both ChatGPT (with web search) and Google Gemini (with Search grounding), all within one day. For each answer we extracted the businesses it recommended, then measured how much the recommended set overlapped across the three identical runs. Full method and limitations are at the bottom.

The headline number

Averaged across every question and both engines, only about 69% of the recommended businesses stayed the same from one run to the next. Put the other way: roughly one in three names changed when we asked the exact same question again, minutes apart, with nothing changed but the click.

The leaders lock in. The tail shuffles.

The churn was not evenly spread. When we looked at every business named for a given question and counted how often it reappeared across the three runs, a clear pattern showed up: 56% of the businesses were named in all three runs, a stable core the model keeps returning to. But 27% were named in only one of the three runs, a rotating tail that changes each time you ask. The top two or three recommendations were remarkably steady; the fourth, fifth and sixth slots were where the shuffle happened, and that is exactly the space a smaller or newer business competes for.

ChatGPT was steadier than Gemini

Both engines searched the web on every run and nearly always produced a real shortlist, so this is a like-for-like comparison of what they recommend.

Engine	Avg. consistency	Gave a shortlist
ChatGPT (web search)	87%	100% of runs
Gemini (Search grounding)	52%	97% of runs

ChatGPT tended to repeat itself closely, its top recommendations were near-identical run to run (per-question consistency ranged from 59% to 100%). Gemini kept its leading picks but reshuffled the rest of the list more freely (33% to 67%), which is why its overlap score is lower. Neither is "wrong"; they simply treat the tail of a recommendation list as more negotiable than the head. With only ten questions per engine these averages carry no formal margin of error, so treat the gap as a clear direction rather than a precise figure.

Consistency per question

Question	ChatGPT	Gemini
Best PR distribution services	59%	50%
Top CRM platforms for startups	100%	42%
Best email marketing services	62%	63%
Leading project management software	100%	67%
Best accounting software for freelancers	100%	33%
Best digital marketing agencies	89%	61%
Top website builders	100%	62%
Best live chat software	100%	58%
Top e-commerce platforms	100%	33%*
Best scheduling software	62%	50%

Consistency = share of recommended businesses that overlapped across three identical runs. With three runs, per-question values fall on a coarse scale, so read these as directional, not precise. *The e-commerce cell for Gemini is based on two runs, not three: one of its three answers named only a single business and was excluded by the two-or-more rule.

The businesses named most often

Across all runs and both engines, and after excluding any name we had put in a question ourselves, the three most-named businesses were Shopify (12 mentions), Wix (8) and Squarespace (7). Behind them, twelve businesses tied at six mentions each: EIN Presswire, Pipedrive, Attio, Klaviyo, Trello, ClickUp, Monday.com, FreshBooks, Hostinger, Tidio, Tawk.to and Acuity Scheduling. These formed the stable core, recommended by both engines across most runs without prompting. The instability lived below them.

What this means if you run a business

Being named by AI is not a ranking you win once. On open "best X" questions the shortlist reshuffles, and the churn is worst in exactly the positions a challenger can realistically reach. The businesses that appear every single time are the ones described, by name, across many credible independent sources, so the model keeps arriving at the same answer. That consistency is the thing you actually build toward.

See where you stand, freeA live AI Visibility Report across ChatGPT, Gemini and Google AI. About a minute.

Method & limitations

Sample: 10 open "best/top X for a small business" questions, one per category. Comparison ("X vs Y") prompts were excluded on purpose, since they can only return the names already in the question.
Engines: ChatGPT via the OpenAI Responses API with the web_search tool; Google Gemini via the Generative Language API with Google Search grounding. US locale, English. Both were configured to search the web, and on this run both did so on every answer.
Trials: 3 independent runs per question per engine = 60 answers, all collected September 13, 2026, within one session to hold the models constant.
Measurement: for each answer we extracted the businesses recommended (a language-model extraction pass), counted only runs that produced a shortlist of two or more, then computed consistency as the average pairwise overlap (Jaccard similarity) of the recommended sets across the three runs. "Locked / tail" counts how many distinct businesses per question appeared in all three runs versus only one.
Extraction check: we verified all 290 extracted business names against their source answers. Every one appeared verbatim in the answer it was drawn from (no invented names), and a manual review of a sample confirmed the extractor captured the businesses each answer actually recommended while correctly skipping media outlets and services named only as context.
Limitations, stated plainly: this is a small exploratory study, not a comprehensive index. Ten questions and three runs are enough to show a pattern, not to publish precise per-category rates; with three runs the per-question overlap values are coarse. The brand-extraction step can miss or double-count a borderline name (for example counting "Wave" and "Wave Accounting" as two businesses), which tends to understate consistency rather than inflate it. Some answers were cut off in their trailing explanation after the recommendation list; in every such case the shortlist itself was intact and captured. Results reflect these two engines on one day; model behavior changes over time.
Reproducibility: the questions, run count and method above are enough to repeat the test. We used the same live query engine that powers our free AI Visibility Report.

Cite as: Pressfront Research, "How Consistent Are AI Recommendations? A First Look", September 2026, pressfront.co/ai-visibility-index.html

Pressfront LLC · Tampa, FL · pressfront.co