What 1.23M Edge Requests Taught Us About AI Crawlers (and Our Own Analytics)

What 1.23M Edge Requests Taught Us About AI Crawlers (and Our Own Analytics)

Leader ●1 ●12 ●80
calendar_today ago • schedule1 min read
— Originally published at www.seosiri.com

Originally published on SEOSiri.

We read 90 days of Cloudflare edge logs through Microsoft Clarity's bot analytics (~1.23M requests) and compared them with GA4 and Blogger stats. Here's what stood out, with the caveats.

The split

  • About 70% of requests were likely human, about 30% automated.
  • Clarity classifies roughly 200K requests as AI crawler, AI search or AI assistant activity.

Selected AI operators (requests in 90 days)

Operator Requests
Microsoft 43,380
Google 42,050
Meta 35,690
Anthropic 19,730
OpenAI 14,900

Clarity also lists Huawei and Amazon above OpenAI. Logs show who asked for a page, not how the data is used afterward.

robots.txt compliance

Clarity's Violations card showed 0.98%: requests to URLs our robots.txt disallows. That's a compliance metric, not a security one. If you set per-operator rules, they look like this (choose Allow or Disallow for your own policy):

User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

Status codes

Around 48% of requests showed as unsuccessful in Clarity. We're breaking them down by path and status code next.

Don't mix units

Blogger showed 64.7K pageviews from the Netherlands. GA4 showed 102 active users there. Same site, same 90 days, different measures.

How do you separate bots from humans in your analytics?

Full write-up with tables and methodology: SEOSiri edge telemetry audit

SEOSiri is my own property

2 Comments

1 vote
0
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

SEO Rates Philippines 2026: Costs & Packages | FloxoLab

floxolab - Oct 18

Telemetry: Shared Analytics and Crash Reporting in Kotlin Multiplatform

kmp-bits - Oct 9

What Developers Already Know About Data Center Delays

Tom Smithverified - Sep 28

I’m a Senior Dev and I’ve Forgotten How to Think Without a Prompt

Karol Modelski - Mar 19

Topic Clusters for Small Sites: Avoid Cannibalization

floxolab - Sep 29
chevron_left
4k Points • 93 Badges
Bangladesh • seosiri.com
47Posts
25Comments
9Connections
I don’t come from a traditional Computer Science background. I spent years in high-level digital mar... Show more

Commenters (This Week)

9 comments
2 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!