Our reader
How we read a shop’s website
mom N’ pop lists what Seattle’s independent shops sell by reading what their own websites already publish to any visitor. This page says exactly what that reading does, and how to stop it.
What it reads
- The homepage, once, to tell which platform runs the store.
- The public catalog that platform offers to any visitor: Shopify’s product feed, WooCommerce’s Store API, Square Online’s storefront catalog, and a Wix store’s product sitemap with the product pages it lists. Titles, prices, links and the addresses of pictures. Never the pictures themselves.
- One item again when a buyer here clicks through to it and its price is more than an hour old, so the price we show is the one you are asking.
It never logs in, never adds to a cart, never fills a form, and never reads a page your robots.txt keeps it out of.
How it behaves
- At most one request a second to any site, with a longer pause between pages of a catalog, and a whole catalog no more than once every six hours.
- It honours robots.txt, both a rule for everyone and a rule for it by name, and it identifies itself on every request as
momnpop-bot/1.0 (+https://momnpop.vercel.app/bot). - Every buy link on this site goes to the shop’s own store. We do not take the order, and we take no cut of it today.
To opt out
Two ways, and either one works on its own:
- Tell us. On the contact page, pick the listing and ask for it to come down. It comes down within a day, no reason needed, and it stays down: we hide the listing and its items rather than delete them, so the next import from OpenStreetMap does not put it back.
- Say it in robots.txt. Add these two lines and the reader stops, since it checks robots.txt before every visit:
User-agent: momnpop-bot Disallow: /
Who we are and how a shop gets listed is on the about page; what we keep while people browse is on the privacy page.