Robots.txt Rule Lister – Website Audit

Robots.txt Rule Lister – Website Audit: How it works

Show user-agent and disallow rules for website audit.

This utility belongs to the Website Audit collection and performs the Robots.txt Rule Lister task locally.

1. Paste the source text or data into the input panel.
2. Run the processing step and review the result.
3. Download or copy the output if it matches your needs.

Important limitations

This tool analyzes text or source code you supply. It does not remotely crawl arbitrary sites.

Privacy

Your submitted text or media stays in the browser for these tools. Only the normal tool-open analytics event is recorded by the website.