Argus: measuring whether on-device logo recognition is good enough to ship

Most AI feasibility work ends with a demo that looks great on the images it was tuned on. Argus was built to answer a harder question: what are the real numbers, what do they cost, and what would change them?

The pipeline. An open-vocabulary detector proposes regions that might be a badge. A self-supervised vision model embeds each region and compares it with a gallery of verified reference crops using nearest-neighbour matching. A margin score penalises matches that are almost as close to a known distractor, which is what stopped other marques’ chrome grille badges being counted as hits. A match floor rejects headlights, stickers and reflections outright. Only permissively licensed models were allowed, so nothing in the stack carries a copyleft or commercial restriction, and the whole thing runs locally on a laptop’s GPU.

The data discipline. Every image has a licence, a source and an annotation in a provenance manifest. The evaluation set was locked before tuning began. A fresh negative set, screened by hand and checked for overlap by hash, exists only for calibration. Distractor crops come from a third, separate source. Mixing any of these would have inflated the numbers, and the tooling refuses to run a calibration against a mismatched gallery or configuration.

The tooling. A benchmarking UI lets a reviewer flip scoring modes, inspect the gallery and distractors, and annotate results. A calibration command chooses operating thresholds on the negative set alone, then reports precision and recall at each on the held-out positives, broken down by scene type. Each run writes a report with a configuration digest and gallery hash, so any number in any conversation can be reproduced to the run. Architecture decisions are recorded as immutable ADRs, each module has a specification, and CI runs a spec gate plus a test suite with a light tier that needs no model weights and a model tier that does.

What the measurement showed. The headline accuracy in the pitch required a false-positive rate no event team would accept. At an acceptable rate the system is precise but misses small or distant badges. The residual confusion is one rival marque’s emblem on a chrome grille. An ablation of neighbour count showed the apparent gain from a larger k was a score-scale shift, not better separation, which saved a week of tuning in the wrong place. That turns a vague “it mostly works” into three specific levers, each with an estimated cost: a stronger embedder, more reference crops, and a small-badge pass.

Why it matters. The same method applies to any “can AI do X for us” question: build the smallest honest instrument, keep the test data clean, measure on images the model has never seen, and report the trade-off rather than the best case.