KinetixSEO
GEO & AI Search

Does GEO Work for LLMs? What C-SEO Bench Found

Mostly no — C-SEO Bench finds most GEO/C-SEO tactics are largely ineffective and often lower a document's citation rank in LLMs.

· By Rogier Bruggeman, Founder of KinetixSEO

RB
Rogier BruggemanFounder of KinetixSEO · 8 min read

What C-SEO Bench is

Mostly no — C-SEO Bench, the first benchmark to test conversational SEO across multiple tasks, domains and numbers of competing sites, finds that most C-SEO methods are largely ineffective and often lower a document's citation rank, while getting the document into a better position in what the LLM reads helps far more. Puerto et al. (2025) built the benchmark around two tasks, question answering and product recommendation, each covering three domains, for six domains in total: Retail, Video Games, Books, Web, News and Debate. It draws on public datasets including Amazon Shopping Queries, Steam Games, Natural Questions and Multi News, and contains more than 1.9k queries and 16k documents. Dataset size varies by domain — Retail alone has 500 queries and 5,000 documents, while Debate has 142 queries and 880 documents. The authors measure effectiveness as citation rank: whether an LLM cites the modified version of a document earlier than it cited the original, not whether the document merely gets mentioned more.

Measure actual visibility

Want to see this on your own site?

Use your own site as the evidence. Get a free SEO and AI-citation readiness baseline, then monitor what changes.

Check my AI visibilityStart monitoring

What the paper tested

The researchers benchmarked ten C-SEO methods in total — eight drawn from earlier work by Aggarwal et al. (2024) plus two the authors proposed themselves — against a baseline called Best SEO, which simply moves a document to the first position in the list the LLM reads, as described by Puerto et al. (2025). In the product recommendation task, the system retrieves 10 unsorted product descriptions and the model has to recommend the best five using only that content, so where a document sits and how it's written both plausibly matter. Rank improvement for each document is defined as the original citation rank minus the rank after a method is applied, and the authors checked whether that improvement was statistically real using a right-tailed Wilcoxon signed-rank test with Holm–Bonferroni correction.

What the paper found

Out of 54 tested cases, the authors found only three statistically significant ranking improvements, and most of the rest either did nothing or actively hurt rank — the opposite of the result marketers assume, according to Puerto et al. (2025). The Statistics method, which adds statistical elements to a document, lowered rank in 19 of 24 settings. Product recommendation on Claude 3.5 Haiku produced significant negative effects in 26 of 30 cases, and question answering on o4-mini in 19 of 30; on Haiku, Statistics made things worse in every single domain, from -0.10 on Games to -0.85 on Debate. On OpenAI's o3 and o4-mini models, no content-optimization method reached statistical significance at all — only Best SEO did, in every test.

The two methods that did show gains were LLM Guidance, in retail and video games, and Content Improvement, in retail only — and only on GPT-4o-mini, never for question answering and never on Haiku 3.5, o3 or o4-mini. Even the retail win looks modest up close: the paper reports that 61.0% of ranks stayed exactly the same after LLM Guidance was applied, 26.2% improved, and 12.8% got worse, so the average effect sits close to zero with high variance. Compare that to Best SEO on the same model: placing a document first improved rank by 2.77 in Retail, 1.89 in Games, 1.60 in Books, 0.87 in Web, 0.70 in News and 1.54 in Debate. LLM Guidance moved rank by 0.36, 0.24, 0.14, 0.10, 0.00 and 0.15 across the same domains, so Best SEO's gain was roughly 8 to 11 times larger in five domains, and in News LLM Guidance gained nothing at all. The authors conclude that a document's position in what the LLM reads has a dominant effect on what gets cited, which is why they frame "Best SEO" as a stand-in for ranking well in the search results an AI assistant pulls from — a simulation of search position, not a measurement of real Google rankings.

The claim that rewriting content for AI makes an LLM cite you more does not hold up as a general rule in this benchmark. It held for exactly two methods, in two of six domains, on one of the models tested — for every other combination of method, domain and model, rewriting did nothing or made rank worse. That is worth saying plainly, without treating anyone who tried it as foolish: the confusion traces back to earlier GEO research by Aggarwal et al. (2024), which the C-SEO Bench authors note measured something different — how much of an LLM's answer talks about a document (word count), not whether that document gets cited earlier (rank). The two sets of findings don't contradict each other; they answer different questions, and "visibility" in the earlier sense isn't the same as "cited first" in this one. For a fuller look at what that earlier research actually claimed, see The GEO Paper: What the Research Actually Found. The paper also finds that C-SEO gains shrink as adoption spreads: as more sites adopt the same method, the gain per adopter falls, and the authors expect the advantage to approach zero once everyone does it, describing it as a congested, zero-sum dynamic. Early movers get the most out of these tactics; a lever every competitor can pull is not much of a lever.

What our data shows about llms.txt

Only 16% of the 32 responding sites among the top 50 domains on Tranco list 64VVX publish an llms.txt file, according to a KinetixSEO crawl study run on 17 October 2026 (sample: 32 observations). The study requested /llms.txt from each domain and counted only sites returning a non-empty plain-text file — not a 404, a homepage redirect or an HTML page standing in for it. This is context, not a verdict: the paper's best-performing C-SEO method, LLM Guidance, was inspired by the idea of prepending a markdown summary to a document, the same concept behind an llms.txt file, so the adoption figure shows where the largest sites currently stand on that idea, not whether it works. Separately, when Gemini's web search answered a prompt asking which tools track brand mentions across AI assistants, it cited 14 sources including viali.ai, mersel.ai, noqode.fr, tryprofound.com and ziptie.ai on 17 October 2026, and kinetixseo.com was not among them. One answer proves nothing about C-SEO methods, but it shows how a relevant, on-topic page can be left out of a live citation list entirely.

What to do with C-SEO and GEO tactics

Keep investing in classic SEO — content that ranks well in the search results AI assistants retrieve from — because that's the effect the paper found to dominate everything else it tested. If you do test a C-SEO rewrite, measure citation rank before and after rather than trusting intuition, and watch specifically for drops, since Statistics-style additions lowered rank in the majority of settings tested. One structural check that does matter regardless: on 17 October 2026 a KinetixSEO answer-block check confirmed that our article on pages AI crawls but does not cite opens with a short first paragraph that echoes a distinctive word from its own heading — a basic citability trait that's cheap to verify and doesn't depend on any of the unproven rewriting tactics. Treat llms.txt files and AI-targeted rewrites as experiments, not reliable levers, and remember the paper's own limitations: it tested white-hat methods only, used commercial proprietary models, covered English content only, skipped documents too large for a model's context window, and didn't study how classic SEO and C-SEO interact together. For a broader rundown of what to verify, see our GEO checklist or the practical GEO best practices checklist.

Frequently asked questions

Does llms.txt actually help?

The paper didn't test an llms.txt file published on a website — it tested a related idea: a markdown summary added to the top of the document itself. That method, called LLM Guidance, gave small, statistically significant gains, but only in two product domains and on one model, GPT-4o-mini.

Does GEO work at all?

GEO mostly does not work, when effectiveness is measured by citation rank, which is the standard C-SEO Bench uses. Earlier claims that "GEO works" often trace back to Aggarwal et al. (2024), which measured something different — how many words of an LLM's answer reference a document — so the two findings aren't really in conflict, they're answering different questions.

What is C-SEO Bench?

C-SEO Bench is the first benchmark built to test conversational SEO methods across multiple tasks, domains and numbers of competing documents, covering question answering and product recommendation across six domains with more than 1.9k queries and 16k documents.

Why do C-SEO gains disappear as more sites adopt the same tactic?

C-SEO gains disappear with wider adoption because one document's better citation rank necessarily comes at the expense of others. In the paper's simulation, run for retail and video games on GPT-4o-mini, the average gain per adopter fell steadily as more documents used the same method, converging toward zero at full adoption. The authors describe it as a congested, zero-sum game in which early adopters benefit most.

Is traditional SEO still the better investment?

Traditional SEO is the better investment by a wide margin in this benchmark. On GPT-4o-mini, Best SEO, the paper's simulation of ranking a document first in what the LLM reads, gained roughly 8 to 11 times more than LLM Guidance in five of six domains, and in News LLM Guidance gained nothing at all.

Sources

Sources

  1. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The benchmark contains more than 1.9k queries and 16k documents across six domains. Paper: "In total, our C-SEO benchmark contains more than 1.9k queries and 16k documents." (Section 4.1, p. 6).

  2. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    C-SEO Bench covers 2 tasks and 6 domains, uses real data, and contains 16.3k documents with many adopters. Paper: "C-SEO Bench (Ours) 2 6 ✓ 16.3k Many" (Table 1, p. 3).

  3. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Per-domain dataset sizes are: Retail 500 queries / 5000 docs, Video Games 436 / 4360, Books 249 / 2245, Web 300 / 1500, News 294 / 2375, Debate 142 / 880. Paper: "Retail 500 5000 Video Games 436 4360 Books 249 2245 Web 300 1500 News 294 2375 Debate 142 880" (Table 2, p. 5).

  4. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    After applying LLM guidance, 61.0% of retail product ranks stayed the same. Paper: "For example, 61.0% of the retail product ranks are the same after applying the LLM guidance." (Section 6.2, p. 8).

  5. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    LLM guidance on retail data produced a positive rank boost in 26.2% of cases and a negative boost in 12.8%, so the average effect was near zero with high variance. Paper: "guidance on the retail data has a positive boost in 26.2% of the cases and a negative boost in 12.8% of the cases. As a result, the overall average effect is close to zero, but with high variance." (Section 6.2, p. 9).

  6. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The authors report only three statistically significant ranking improvements out of 54 cases. Paper: "Out of 54 cases, we uncover only three where the ranking improvements are statistically significant." (Section 6.2, p. 9).

  7. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Left-tailed tests showed many C-SEO methods significantly reduce rankings: the Statistics method lowered rankings in 19 of 24 settings, product recommendation on Haiku 3.5 showed 26 of 30 significant negative cases, and question answering on o4-mini 19 of 30. Paper: "for example, the Statistics method decreases rankings in 19 out of 24 evaluated settings. In product recommendation tasks on Haiku 3.5, 26 out of 30 cases show significant negative effects, and in question answering on o4-mini, 19 out of 30 cases do so." (Section 6.2, p. 9).

  8. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    On GPT-4o-mini, LLM Guidance improved rank by 0.36 on Retail, 0.24 on Games, 0.14 on Books, 0.10 on Web, 0.00 on News and 0.15 on Debate. Paper: "LLM Guid. 0.36 ±1.47 0.24 ±1.05 0.14 ±1.07 0.10 ±0.99 0.00 ±1.10 0.15 ±1.65" (Table 3, p. 8).

  9. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Best SEO on GPT-4o-mini gave rank improvements of 2.77 (Retail), 1.89 (Games), 1.60 (Books), 0.87 (Web), 0.70 (News) and 1.54 (Debate). Paper: "Best SEO 2.77 ±2.31 1.89 ±2.32 1.60 ±2.04 0.87 ±1.35 0.70 ±1.64 1.54 ±2.07" (Table 3, p. 8).

  10. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    On Claude 3.5 Haiku, the Statistics method lowered rank in every domain: -0.82 on Retail, -0.10 on Games, -0.60 on Books, -0.58 on Web, -0.12 on News and -0.85 on Debate. Paper: "Statistics -0.82 ±1.47 -0.10 ±1.01 -0.60 ±1.34 -0.58 ±1.18 -0.12 ±0.91 -0.85 ±1.66" (Table 4, p. 8).

  11. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Placing the document first in the GPT-4o-mini context improved its rank by 2.77 on Retail, 1.89 on Games, 1.60 on Books, 0.87 on Web, 0.70 on News and 1.54 on Debate. Paper: "1 2.77 ±2.31 1.89 ±2.32 1.60 ±2.04 0.87 ±1.35 0.70 ±1.64 1.54 ±2.07" (Table 5, p. 14).

  12. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    In the multi-actor simulation, LLM Guidance reached AUC 1.88 on retail and 1.58 on games, while Content Improvement reached 1.05 and 0.74. Paper: "LLM Guid. [Retail] (AUC = 1.88) LLM Guid. [Games] (AUC = 1.58) Content Impr. [Retail] (AUC = 1.05) Content Impr. [Games] (AUC = 0.74)" (Figure 5, p. 10).

  13. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The paper presents C-SEO Bench, described as the first benchmark to evaluate conversational SEO methods across multiple tasks, domains and numbers of competing actors. Paper: "We present C-SEO Bench, the first benchmark designed to evaluate C-SEO methods across multiple tasks, domains, and number of actors." (Abstract).

  14. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The benchmark covers two search tasks — question answering and product recommendation — with three domains each. Paper: "We consider two search tasks, question answering and product recommendation, with three domains each." (Abstract).

  15. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The headline finding is that most current C-SEO methods are largely ineffective and often hurt document ranking, the opposite of what is expected. Paper: "Our experiments reveal that most current C-SEO methods are not only largely ineffective but also frequently have a negative impact on document ranking, which is opposite to what is expected." (Abstract).

  16. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Traditional SEO strategies, which improve the ranking of the source in the LLM context, were significantly more effective than C-SEO methods. Paper: "Instead, traditional SEO strategies, those aiming to improve the ranking of the source in the LLM context, are significantly more effective." (Abstract).

  17. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    As more actors adopt C-SEO, the overall gains decrease, which the authors describe as a congested, zero-sum problem. Paper: "We also observe that as we increase the number of C-SEO adopters, the overall gains decrease, depicting a congested and zero-sum nature of the problem." (Abstract).

  18. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The authors conclude C-SEO will complement rather than replace traditional SEO. Paper: "Hence, we believe C-SEO will not replace SEO, but will complement it." (Section 1, p. 2).

  19. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Effectiveness is measured as improvement in citation ranking: an effective method should make the LLM cite the modified document earlier than the original. Paper: "We propose to measure the effectiveness of C-SEO methods by the improvement in the citation ranking, i.e., an effective method should lead the LLM to cite the modified document earlier than it cited the original document." (Section 1, p. 2).

  20. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Rank improvement (gain) is defined as the baseline citation rank minus the citation rank after C-SEO optimisation at adoption rate alpha. Paper: "We define the rank improvement (i.e., gain) for the document i for query q as: ∆q,d = rank0 d − rankαd" (Section 4.2, p. 6).

  21. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Statistical significance of rank improvements was assessed with a right-tailed Wilcoxon signed-rank test with Holm–Bonferroni correction for multiple comparisons. Paper: "We access the statistical significance of the rank improvement using the right-tailed Wilcoxon signed-rank test. This test allows us to evaluate whether the rank after applying a C-SEO method is statistically significantly smaller than the original rank. We adjust the p-values using the HolmBonferroni correction [Holm, 1979]" (Section 4.2, p. 6).

  22. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The authors introduce an AUC metric: the area under the curve of average gains across adoption rates from 0% to 100%. Paper: "We propose to report the Area Under the Curve (AUC) of the overall gains computed in the range of adoption rate from 0% to 100%" (Section 4.2, p. 6).

  23. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    The datasets used include Amazon Shopping Queries, Steam Games from FronkonGames, Natural Questions and Multi News, all licensed for research use. Paper: "We used the following datasets with the corresponding licenses. In all cases, they allow the use of their data for research purposes, so we agree to their terms of use. • Amazon Shopping Queries [Reddy et al., 2022]. License: Apache 2.0 • Steam Games from FronkonGames. License: CC-by-4.0. • Natural Questions [Kwiatkowski et al., 2019]. License: CC-by-3.0. • Multi News [Fabbri et al., 2019]." (Appendix B, p. 13).

  24. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    In the product recommendation task the system retrieves 10 unsorted documents and the LLM must recommend the best five products using only the retrieved content. Paper: "The user submits a query, and the system retrieves 10 documents (product descriptions), which are presented in an unsorted order. The LLM must then recommend the best five products based on the provided descriptions, using only the content of the retrieved documents." (Section 4, p. 5).

  25. Puerto et al. (2025), "C-SEO Bench: Does Conversational SEO Work?" ()

    Ten C-SEO methods were benchmarked in total: eight from Aggarwal et al. 2024 plus two new ones proposed by the authors. Paper: "Overall, we benchmark a total of ten methods." (Section 5, p. 7).

  26. KinetixSEO crawl study ()

    Only 16% of the 32 the top 50 sites of the Tranco ranking (list 64VVX) publish an llms.txt file at all. Sample: 32 observations, measured Oct 17, 2026. Methodology: Requested /llms.txt on each of the 32 the top 50 sites of the Tranco ranking (list 64VVX) and counted the sites returning a non-empty plain-text file, not a 404 and not an HTML page served in its place (a redirect to the homepage or a soft 404).

  27. KinetixSEO GEO citation tracking ()

    On 17 October 2026, Gemini (web search) answered the tracked prompt "I'm an SEO consultant. Clients keep asking how visible they are in AI answers. What tools track brand mentions and citations across AI assistants like ChatGPT and Perplexity? Name specific products." and cited 14 sources: viali.ai, mersel.ai, noqode.fr, tryprofound.com, ziptie.ai and 9 more. kinetixseo.com was not among them.

  28. KinetixSEO crawl check ()

    On 17 October 2026, KinetixSEO ran its "answer block shape" check against https://kinetixseo.com/articles/ai-crawls-my-site-but-does-not-cite-it. What it tests: Whether the page's first body paragraph is short and repeats a distinctive word from its own H1. Result: Answer up front.

How this was checked: the full method, with the paper’s own words behind every claim.

Measure actual visibility

See how your own site scores on SEO and AI-search visibility — free report, no signup.

Use your own site as the evidence. Get a free SEO and AI-citation readiness baseline, then monitor what changes.

Check my AI visibilityStart monitoring

More in Articles →