Front2Back-ReID

ACCV 2026 Workshop · CV4DC · under review

Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification

Moseli Mots’oehli1,2Thulani Babeli1

1MindForge AI, Johannesburg, South Africa2University of Hawai‘i at Mānoa, Honolulu, HI, USA

TL;DR 500 real front-to-rear vehicle handovers from South African roads. Zero-shot VLMs don’t beat a frozen SigLIP2 encoder, and people still lead the best model by 16–31 points.

Code Paper Download dataset BibTeX Try the task
A vehicle highlighted in the front-left camera disappears through a blind interval and reappears among labelled candidates C1 to C6 in the rear-left camera. Front evidence can be the full RGB scene, a target crop, or a binary silhouette.
The task. A highlighted front-view vehicle disappears through a blind interval and reappears in a closed rear gallery. The matcher must select its unique match given full-RGB, crop, or silhouette front evidence.
Human participants, full RGB94.0%25 people · 92.2% on crops
Best VLM, target crop76.6%GPT-5.5 with reasoning
Frozen SigLIP2, target crop74.0%difference not significant
Verified handovers50020 South African sequences

Abstract

Matching the same vehicle across front and rear cameras is difficult because the cameras do not share a view and the vehicle’s appearance changes substantially. We introduce Front2Back-ReID, a benchmark of 500 manually verified vehicle handovers from 20 recording sequences in South Africa. Each example asks a model to match a vehicle highlighted in a front-camera image to the same vehicle among at least three candidates in a later rear-camera image. We evaluate seven zero-shot vision-language models, four image-retrieval baselines, and 25 human participants. Models are tested using full front RGB images, cropped target vehicles, and binary silhouettes. The strongest VLM achieved 76.6% Rank-1 accuracy on target crops, compared with 74.0% for the frozen SigLIP2 baseline; this difference was not statistically clear. Human participants achieved 94.0% accuracy with full images and 92.2% with target crops. Under our evaluation setup, enabling reasoning improved accuracy across all three input conditions for every model evaluated in both modes. We also found that VLMs generally performed worse on full scenes than on target crops. These results show that general-purpose VLMs do not yet consistently outperform strong visual retrieval for front-to-rear vehicle matching, while humans remain substantially more reliable.

01

Key findings

  1. 1

    VLMs don’t beat frozen retrieval

    The best VLM, GPT-5.5 with reasoning, scores 76.6% on crops against 74.0% for SigLIP2. The gap isn’t significant.

  2. 2

    Context hurts models, not people

    VLMs lose 6.6–16.4 points on full scenes, while humans gain 1.8.

  3. 3

    Small targets break models

    Humans hold 93.2% on the smallest rear targets, while Llama 4 Scout drops to 38.7%.

02

The task

  • Find a vehicle highlighted in the front-left camera
  • among the K ≥ 3 labelled candidates in a later rear-left frame,
  • with no video in between, no fixed time gap and no camera calibration.
Five example pairs, each shown as the marked full-RGB front query, the RGB target crop, the binary silhouette, and the rear candidate gallery.
Examples. Each row shows the marked front query, the target crop, the silhouette and the rear gallery. The match is green and the distractors red, and faces are blurred.
03

The benchmark

Handovers
500
Recordings
20 · 5 cities
Driving
≈1.9 h
Frames
844
Gallery size
3–15
Cameras
2 × <US$100 stereo

Composition

Overview diagram in three panels: benchmark construction (front and rear camera recordings, YOLOE-26 proposals, curator correction, front-target and rear-match verification, S2M2 stereo depth, 500 verified handovers); task and evidence conditions (front-left target, blind interval, rear-left candidate gallery with one true match, front evidence as full RGB, target crop or binary silhouette); evaluation protocol (zero-shot VLMs with reasoning disabled or enabled, frozen retrieval baselines, human reference with 25 participants, Rank-1 accuracy with 95% BCa intervals).
Pipeline. Curators verify every handover, and the rear gallery stays fixed across the three front inputs.
04

Evaluation

Seven zero-shot VLMs, four frozen retrieval baselines and 25 people make the same 500 decisions. The metric is Rank-1, with 95% BCa intervals. See the results →

Prompt used for every VLM

Shared task and decision rules

You are a careful evaluator for an asymmetric front-to-rear vehicle re-identification benchmark. Match the front target to exactly one rear candidate using only the permitted evidence below. Allow for viewpoint, illumination, scale, blur, pose, truncation, and partial occlusion; treat a hidden cue or contour as unknown, not as a mismatch. Never use candidate order, image position, bounding-box size, or road plausibility as identity evidence. Detector classes, detector confidence, and numeric box coordinates are withheld. Rear candidates: [C1, C2, ...].

RGB cues: body configuration, silhouette, color or livery, roofline, windows, lights, wheels, cargo structure, markings, and accessories. Mask cues: silhouette, body proportions, roofline, wheel placement, cargo structure, and visible accessories; never infer color or texture.

Full RGB

The front input is the full RGB scene with target T highlighted. Apply the RGB cues to the target. Scene context may help interpret viewpoint, visibility, motion, and surrounding relationships, but context alone is not sufficient evidence of identity.

Target crop

The front input is a crop of target T. Apply the RGB cues using only appearance visible inside the crop; no wider scene context is provided.

Target mask

The front input is target T’s binary segmentation mask. Apply only the mask cues; target color and texture are unavailable.

Shared structured-output contract

candidate_id: one listed alias; candidate_probabilities: every alias exactly once, values in [0, 1] summing to 1; confidence: candidate_probabilities[candidate_id]; reasoning: one short permitted-evidence sentence. Return JSON only; no Markdown, preamble, hidden reasoning, detector labels, or detector confidence scores.

05

Get the data

Download

Download dataset (ZIP)

  • annotations/pairs.jsonlpairs, image paths, target and candidate boxes
  • annotations/ground_truth.jsonlcorrect candidate per pair. Keep it out of prompts.
  • splits/test500.txtthe fixed evaluation order
  • data/844 original frames and target masks
  • inputs/marked front, labelled rear gallery, target crop
  • evaluate.py · verify.pyscoring and integrity checks

Quick start

git clone https://github.com/Mind-Forge-AI/Front2Back-ReID
cd Front2Back-ReID && pip install -e ".[baselines]"
python scripts/download_data.py
python scripts/run_baselines.py --data data/front2back-reid-v1.1
python scripts/evaluate.py --data data/front2back-reid-v1.1 predictions.jsonl

Predictions: one {"pair_id": "P0001", "candidate_id": "C2"} per line, scored as Rank-1 over all 500 pairs. Reproduce the paper →

Code: MIT License. Using the benchmark? Please cite the paper.

Citation

Cite as: Moseli Mots’oehli and Thulani Babeli. Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification. arXiv:2609.39492, 2026.

@article{motsoehli2026front2back,
  title         = {Front-to-Back: Benchmarking Vision-Language Models for Asymmetric Cross-View Vehicle Re-Identification},
  author        = {Mots'oehli, Moseli and Babeli, Thulani},
  journal       = {arXiv preprint arXiv:2609.39492},
  year          = {2026},
  eprint        = {2609.39492},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2609.39492}
}

Acknowledgements

We thank the 25 participants in the human evaluation for their time, and Dalitso Chomey for valuable discussions on the paper.