--- title: "I asked 7 AI engines the same question 3 hours apart. Half the sources changed." description: "Small test on my own site: two AI answers to the same question, 3 hours apart, shared only half their sources. One check tells you very little." canonical_url: "https://kinetixseo.com/articles/i-asked-7-ai-engines-the-same-question-3-hours-apart-half-the-sources-changed" published_at: "2026-11-09T08:26:42+00:00" updated_at: "2026-11-09T08:26:42+00:00" author: "Rogier Bruggeman" category: "GEO & AI Search" --- # I asked 7 AI engines the same question 3 hours apart. Half the sources changed. Small test on my own site: two AI answers to the same question, 3 hours apart, shared only half their sources. One check tells you very little. # I asked 7 AI engines the same question 3 hours apart. Half the sources changed. Last week I did something boring. I clicked one button twice. Same site (kinetixseo.com). Same 5 keywords. Same questions. Same 7 AI engines that answer each question: ChatGPT, Claude, Gemini, Grok, Perplexity, Meta AI and Google AI Mode. First click in the morning, second click about 3 hours later. (KinetixSEO also runs 3 search checks, Google [AI Overviews](https://kinetixseo.com/learn/google-ai-overviews) among them. Those run once a week, so they're not in this test.) Nothing on my site changed in between. No new article, no new link, nothing. So the answers should be the same, right? Wrong. ## What I measured Every time an AI engine answers with web search, it shows the pages it used. I stored those pages for every check and then compared each check with the one right before it: same keyword, same question, same engine. Three numbers per pair: - **Pages kept:** how many of the pages from the first check showed up again in the second. - **Domains kept:** the same, but per website instead of per page. - **Cited flipped:** did my site go from cited to not cited, or the other way around? I compared two kinds of pairs: 1. **Same day:** two checks about 3 hours apart. I did this on 3 different days (3, 5 and 7 November) to rule out one weird afternoon. 2. **A week apart:** two checks 3 to 10 days apart, from the weekly checks KinetixSEO already runs on my account. ## The result Measured with KinetixSEOPairsPages keptDomains keptCited flippedSame day (about 3 hours apart)13449%55%0%A week apart14535%44%8%Read that first row again. Two answers to the exact same question, 3 hours apart, share only half their sources. And it wasn't a fluke. I checked it on 3 test days: 53%, 48% and 45% of pages kept. Every day roughly half. Over a week it drops to about a third. And in 8% of the weekly pairs my site went from cited to not cited (or back). Within the same day that never happened. ## Not every engine moves the same This surprised me the most. Engine (measured with KinetixSEO)Pages kept, same dayPages kept, a week apartPerplexity76%36%Google AI Mode45%19%Grok51%35%Claude46%49%ChatGPT33%28%Perplexity is quite stable within a day but changes a lot over a week. Google AI Mode the same. Claude and ChatGPT? They change just as much in 3 hours as in a week. Ask ChatGPT the same thing twice in one morning and two thirds of the pages can be different. (Gemini only gives the domains, not the pages, so it's not in this table. Domains kept for Gemini in my KinetixSEO checks: 42% same day, 34% a week apart. Meta AI had too few pairs to say anything.) ## So what does this mean for you? If you check your [AI visibility](https://kinetixseo.com/ai-visibility-checker) once and see your site cited: nice. But you may be looking at a lucky roll of the dice. If you check once and you're not cited: don't panic either. Ask again tomorrow and the list can look very different. One check tells you very little. A trend over many checks tells you something. That's also why I don't trust "you are cited in ChatGPT" screenshots. A screenshot is one roll. What you want to know is how often you're cited, across many questions and many days. I wrote about that in [AI selection rate: a better metric than mention count](https://kinetixseo.com/articles/ai-selection-rate-a-better-metric-than-mention-count). It's also the reason [AI visibility tracking works differently from rank tracking](https://kinetixseo.com/articles/ai-visibility-tracking-what-it-is-and-why-it-differs-from-rank-tracking): a Google ranking stays put for days, an AI answer can change before lunch. ## What I would do 1. **Never judge on one check.** Look at weeks, not at a single day. 2. **Track the same questions over time.** Changing your questions every week makes the noise worse. Keep a fixed set. 3. **Look per engine.** An average over 7 engines hides that Perplexity and ChatGPT behave completely differently. 4. **Watch the domains, not only the pages.** Domains are a bit more stable (55% vs 49%). If the same competitor domain keeps showing up, that's a real signal. 5. **Be careful with before/after claims.** Changed your page on Monday and got cited on Tuesday? With this much noise that can be pure coincidence. Want to know [how AI citation tracking actually works](https://kinetixseo.com/articles/ai-citation-tracking-how-it-actually-works) and why it needs many checks? That article goes deeper. ## The honest small print This is a small test. One site (mine), 5 keywords, 7 engines, 134 same-day pairs. Not a study of the whole internet. In my KinetixSEO checks, two of the 5 keywords showed no difference between same day and a week apart. So it's not the same for every topic. And the weekly checks run at night, mostly from September and October, while my same-day clicks were in the daytime, in November. So "a week apart" also means "a different time of day and a different month". I can't pull those apart with this data. Is the conclusion still solid? I think yes. Even in the best case I measured (Perplexity, same day) a quarter of the sources changed in 3 hours. Most engines changed half or more. AI answers are not a fixed list. They're a roll of the dice with loaded dice. Your job is to load them in your favour, and to count enough rolls to know if it's working :-) Want to see where you stand today? The [free ChatGPT visibility checker](https://kinetixseo.com/chatgpt-visibility-checker) gives you a first roll. Just don't stop at one. ## Sources - KinetixSEO — Same day, about 3 hours apart, 134 pairs: 49% of pages kept, 55% of domains kept, 0% cited flipped. A week apart, 3 to 10 days, 145 pairs: 35% of pages kept, 44% of domains kept, 8% cited flipped. Pages kept per test day: 53%, 48% and 45%. Domains 55% vs 49% pages. (November 2026) — KinetixSEO's own AI visibility checks on kinetixseo.com, export of 9 November 2026. Each check compared with the previous check of the same keyword, question and engine; share of cited pages and domains from the first check that appear again in the second. - KinetixSEO — ages kept same day vs a week apart: Perplexity 76% vs 36%, Google AI Mode 45% vs 19%, Grok 51% vs 35%, Claude 46% vs 49%, ChatGPT 33% vs 28%. Gemini domains kept 42% same day, 34% a week apart. Best case a quarter of the sources changed; ChatGPT up to two thirds of the pages different. (November 2026) — KinetixSEO's own AI visibility checks on kinetixseo.com, export of 9 November 2026. Each check compared with the previous check of the same keyword, question and engine; share of cited pages and domains from the first check that appear again in the second. - KinetixSEO — One site, 5 keywords, 7 AI engines, 3 search checks weekly, 3 test days (3, 5 and 7 November), checks 3 hours apart; 2 of the 5 keywords showed no difference. (November 2026) — KinetixSEO's own AI visibility checks on kinetixseo.com, export of 9 November 2026. Each check compared with the previous check of the same keyword, question and engine; share of cited pages and domains from the first check that appear again in the second.