Robots.txt Rule Lister – Website Performance

Robots.txt Rule Lister – Website Performance: How it works

Show user-agent and disallow rules for website performance.

This utility belongs to the Website Performance collection and performs the Robots.txt Rule Lister task locally.

1. Paste the source text or data into the input panel.
2. Run the processing step and review the result.
3. Download or copy the output if it matches your needs.

Important limitations

This tool analyzes text or source code you supply. It does not remotely crawl arbitrary sites.

Privacy

Your submitted text or media stays in the browser for these tools. Only the normal tool-open analytics event is recorded by the website.