# robots.txt - scraper compliance test site # /events/ is intentionally disallowed. If a scraper returns content from it, # it is not honoring this file. /locations/ is intentionally allowed. # # This is a REAL file on disk. Caddy's try_files serves it directly, so # WordPress's virtual robots.txt never applies. User-agent: * Disallow: /events/ Sitemap: https://scrapy.coe-ding.com/wp-sitemap.xml