232,064,768
Domains with an a3 crawl-status value on file
Domains Indexed
4,615,606
Newly indexed robots.txt status
New · Last 30 Days
30%
Share of domains serving a robots.txt file
Have a robots.txt
6,691,131
Domains whose robots.txt disallows crawling
Disallow / Nofollow
robots.txt Status Breakdown