Competitor Price Mapping in Production

The Problem

Pricing decisions at a supermarket in the group depended on knowing what competitors charged for equivalent products. The hard part isn’t scraping the prices — it’s deciding that this product on our shelf and that product on a competitor’s site are the same thing, when the names disagree, the pack sizes are written differently, and the only reliable signal is often the photograph.

The approach

I led a team of three engineers on this. Matching is multimodal: we used Google SigLIP to embed both the product text and the product image, so a match could draw on either signal rather than relying on string similarity that breaks the moment a retailer renames something. Collection ran on scheduled scraping, including the cookie-handling needed to keep it working, and the whole thing ran daily and unattended on GCP with BigQuery underneath.

The results

The system matched 6,000 SKUs across six competitor retailers with automated daily updates, and it stayed in production for more than two years as a core input to pricing decisions.

This is one of two systems in my career that genuinely ran in production for years and had people depending on its output daily — the other being a BigQuery analytics platform serving marketing, operations and management dashboards over millions of daily transactions, where I was a team member rather than the lead. I’d rather name those two precisely than imply everything I’ve built was deployed, because most R&D work isn’t, and the difference is the whole game.