Original research
Crawl Studies: What Real Websites Actually Do
Each study runs one of KinetixSEO's own checks across a named, hand-curated list of public websites on a stated date, and publishes the count, the sample size and the method. Only studies with at least 30 observed sites, every listed URL attempted, and a method sentence that matches the check actually run are shown here.
Published studies
The findings
AI search crawler blocking among public B2B SaaS homepages
0% of the 45 public B2B SaaS homepages still block at least one AI search crawler in robots.txt.
· n = 45 · 0 of 45 (0.0%)
Method, limits and data →llms.txt adoption among public SEO and content-marketing tool vendor sites
Only 38% of the 42 public SEO and content-marketing tool vendor sites publish an llms.txt file at all.
· n = 42 · 16 of 42 (38.1%)
Method, limits and data →Answer-first opening paragraphs among public SEO and web-performance guide articles from software vendors and developer publications
Only 31% of the 35 public SEO and web-performance guide articles from software vendors and developer publications answer their own headline question in the first paragraph.
· n = 35 · 11 of 35 (31.4%)
Method, limits and data →Article and FAQPage structured data among public B2B SaaS homepages
5% of the 44 public B2B SaaS homepages publish Article or FAQPage structured data on the page itself.
· n = 44 · 2 of 44 (4.5%)
Method, limits and data →
Reuse
Free to cite
Every study is published under a CC BY 4.0 license. Each study page has a suggested citation line and a CSV of its aggregate results. No per-site result is published. For our AI citation benchmark, see the AI search citation statistics.