Corvus ISR Publishes Transparent Benchmark Comparing Tracker Models

AIThis post was created with the assistance of artificial intelligence (AI).

Corvus ISR tracker benchmark matrix (seed 1337) — The published matrix — every row reproducible. Source: corvusisr.com/benchmark

Corvus ISR’s latest release offers a detailed look into the performance of its wide-area motion imagery (WAMI) exploitation product through a public tracker benchmark. This benchmark compares two distinct tracker models on a synthetic scene with perfect ground truth, ensuring an objective measurement environment. The scene is fixed-seed, with seed 1337, a 20-second warm-up, and 120 seconds of measured data per row, all using identical sensor models, detection generation, and metric definitions. This rigorous setup allows for direct, apples-to-apples comparison of tracker algorithms in a controlled environment.

The first model, v1, employs a simple greedy nearest-neighbour approach with two-pass greedy association, constant-velocity prediction, and fixed 2-second coasting. Despite its simplicity, it serves as a baseline for performance. The second, v2, incorporates a more sophisticated confirmed-track auction system with three-tier auction association, velocity-consistency gating, noise-scaled reservation prices, and confidence-decayed coasting. These enhancements aim to reduce identity errors, particularly ID switches, under challenging conditions.

The benchmark results are compelling. In scenarios with 150 movers at 2fps, the ID switches per minute dropped from 2,042 to 1,183—a 42.1% improvement. For a denser scene with 400 movers, the switch rate decreased from 14,032 to 8,040, a 42.7% reduction. Additional stressors such as frame starvation at 0.5fps, occlusion at 20%, and degraded conditions with jitter and low contrast further lowered identity errors by approximately 18-19%. The detection rate remains consistent between models, as it depends solely on sensor properties.

Notably, the metric honesty of the benchmark is emphasized. The ID switch metric counts every change of track identity assigned to a ground-truth object, making it more stringent than standard MOT challenge definitions. This approach ensures a comprehensive measure of tracker reliability, capturing all types of identity errors, including fragmentations and re-acquisitions. Publishing these failure numbers publicly underscores the importance of transparency in evaluating tracking algorithms.

Corvus ISR live demo — The live demo — press “Run benchmark” to reproduce the numbers. Source: corvusisr.com/demo

Corvus ISR intentionally publishes these failure metrics because both models still make thousands of identity errors per minute under stress. Since the scenes are synthetic with perfect ground truth, these numbers reflect measurements, not marketing. Future tracker development will be demonstrated against these fixed seed results, making progress quantifiable and repeatable. As the company states, “Vendors who show only successes ask for faith; a published failure matrix asks for measurement.”

From an engineering perspective, the v2 tracker achieves real-time performance, averaging about 1.2 milliseconds per sensor tick at a density of 400 objects—well within a 10-millisecond budget. Even at worst-case scenarios (~5ms), the system remains suitable for live applications. Interested users can easily reproduce these results by visiting the live demo and clicking “Run benchmark”—no signup or NDA required. This transparency demonstrates the robustness of the methodology and the importance of testing algorithms against well-defined, synthetic benchmarks.

Ultimately, Corvus ISR’s synthetic scene approach, with pixel-perfect ground truth, allows for objective evaluation free from real-world noise and ambiguity. By openly sharing failure metrics and providing accessible tools for reproduction, the company fosters a scientific environment where progress in tracking technology can be reliably measured. We invite curious readers to explore the public benchmark themselves and see the performance differences firsthand.