OctoCartBot, our flyer reader

What it is

OctoCart shows shoppers this week's flyers from neighbourhood grocers. Most of those flyers come from the stores' own websites, and OctoCartBot is the program that reads them. If you saw this in your server's logs, it was us:

OctoCartBot/3.0 (+https://octocart.app/bot; grocery flyer aggregator)

What it does

• It reads the public pages where a store posts its weekly flyer for shoppers, and the flyer files on them. Nothing behind a sign-in.
• It reads your robots.txt first, and again at least once a day, for its Crawl-delay, and keeps to one of up to 30 seconds. It doesn't follow robots.txt's Disallow lines: to stop it, write to us (below).
• It fetches one file at a time, at most one a second from each website. A 429 or 503 answer, your site saying it's busy, makes it wait longer before its next look.
• A 401 or 403 answer, or a page that checks for robots, stops it there. It never tries to get past one, and always says who it is.
• Some flyer pages only work in a browser, so it sometimes opens them in one (Chromium), under the same name. The page then loads its own images and scripts, as it would for one shopper.
• Some flyer pages ask you to pick a store first. OctoCartBot picks one the way a shopper would: it types the store's postal code into the page's own store finder, or chooses the store from its list. It never signs in, makes an account or sends anything else.
• It checks a store's flyer page around the day the store's new flyer usually comes out: once a few days before, then every six hours or so until the new flyer is there. Then it leaves your site alone until the next week.

To stop it, or ask us anything

Write to support@chekinfotech.com with your website's address. A person writes back, stops OctoCartBot reading your site, and takes down anything of yours we show if you ask.

Run the store? You can also send us your flyer yourself.