welcome
Verge

Verge

Technology

Technology

Cloudflare will now block AI crawlers by default

Verge
Summary
Nutrition label

72% Informative

Cloudflare will now block known AI web crawlers by default by default.

The internet architecture provider will also let some publishers make known AI scrapers pay to crawl their sites.

The company started letting websites block AI crawlers in 2023 , but it only applied to ones that abide by a site’s robots.txt file.

VR Score

64

Informative language

58

Neutral language

49

Article tone

semi-formal

Language

English

Language complexity

50

Offensive language

not offensive

Hate speech

not hateful

Attention-grabbing headline

not detected

Known propaganda techniques

not detected

Time-value

short-lived

Affiliate links

no affiliate links