ShelfglanceBot reads public pages of a store to work out what an AI shopping assistant would find there. It requests at most a dozen URLs per scan and keeps no copy of page content, only the findings.
Most scans happen because someone typed a domain into the box on this site. That request is theirs, so the scan fetches the pages they pointed it at without checking robots.txt first, the same way your browser does when you open a link.
A store owner can ask to hear from us if their score changes. Those stores are
re-scanned about once a month. Nobody is standing there asking for that one, so it
is bound by your robots.txt: before a scheduled re-scan we fetch
/robots.txt and stop if ShelfglanceBot is disallowed.
Add this to your robots.txt and no scheduled re-scan will fetch your pages:
User-agent: ShelfglanceBot
Disallow: /That also takes an existing report down. A report is deleted a month after the scan that produced it, and a disallow stops it being renewed, so the store leaves the directory and does not come back.
Your homepage, one product page, robots.txt, sitemap.xml
and llms.txt. It does not log in, does not submit forms, and never
touches customer names, emails, phone numbers, addresses or orders.