# Sinker — crawl policy, version 1. # Scope: this hostname only. Serve this file at /robots.txt. # This is voluntary crawler guidance, NOT authentication or a firewall. # It does not classify visitors, stop malicious bots, or prevent indexing. # Do not list secret locations here: this file is public. # Default: ask every crawler to avoid every path unless allowed below. User-agent: * Disallow: / # Exact public homepage and the files needed to render the observatory. Allow: /$ Allow: /index.html$ Allow: /app.js$ Allow: /swim-motion.mjs$ Allow: /fish.css$ Allow: /feed-ui.mjs$ Allow: /bait-content.mjs$ Allow: /robots.txt$ Allow: /assets/ # Observation API, reserved areas, and real decoy request paths. # These entries also document policy; Disallow: / already covers them. Disallow: /api Disallow: /admin Disallow: /internal Disallow: /telemetry Disallow: /decoy Disallow: /bait Disallow: /hooks Disallow: /canary Disallow: /storage Disallow: /files Disallow: /resources Disallow: /store # Query URLs remain disallowed: the public exact-match rules end with $. # No Crawl-delay: not consistently supported across crawlers. # No sitemap is advertised until a real public domain is configured. # For repository-subpath hosting, place policy at the hostname root and # prefix public allow rules with the deployment path. A robots.txt inside # /Chumbucket/ alone is not a hostname-wide crawl policy.