Directive Tests

Controlled pages for measuring robots and meta directive behavior.

Use a unique attempt id in the URL, for example /lab/directives/meta-noindex?id=directive-001.

• Robots Allowed Fixture - Path is explicitly allowed in robots.txt.
• Robots Disallowed Fixture - Path is explicitly disallowed in robots.txt.
• Robots Crawl Delay Fixture - Path is fetchable, while robots.txt advertises Crawl-delay: 7.
• Meta Noindex Fixture - Page is fetchable but asks compliant crawlers not to index it.
• Meta Nosnippet Fixture - Page is fetchable but asks compliant crawlers not to show snippets.
• Meta Nofollow Fixture - Page is fetchable but asks compliant crawlers not to follow links.