WebIgnite

Robots.txt Tester

Read a site's robots.txt and test whether a path is allowed for Googlebot or any crawler.

Loading the tool…

We fetch only the address you enter, the same way a browser or Google would. Nothing is saved.

Found problems you'd rather not fix yourself?

We fix SEO, speed and security issues — or rebuild the site properly. Itemised quote within a day.

Get it fixed

About the Robots.txt Tester

Search engines reward pages that get the basics right — titles, headings, structured data, a sitemap that actually lists every page. The SEO tools here check one of those basics at a time, against the same page a crawler sees, with no account and no limit on how many times a day you run them.

Common questions

Does this affect my search rankings just by running it?

No. It only reads the page the same way a crawler would; it doesn't change anything on your site or submit anything to a search engine.

How often can I run it?

As often as you like — there's a light per-IP rate limit to keep the service fair to everyone, but no daily cap and no account required.

Will it work on a page that needs a login?

No — it fetches the page the way a logged-out visitor or a search crawler would, so anything behind a login wall won't be visible to it.

Related tools