[Benchmark] Add support for MultihopSpatial benchmark

[youngwanLEE]

Summary

This PR adds MultihopSpatial, an ECCV 2026 benchmark for multi-hop compositional spatial reasoning with visual grounding, to VLMEvalKit.

Each of the 4,500 test samples is a multiple-choice question paired with a ground-truth bounding box: a model must both pick the correct choice and localize the answer object. This exposes the "lucky guess" gap where MCQ accuracy is high but grounding is weak. Questions span 1/2/3 reasoning hops over Attribute / Position / Relation, in both ego- and exo-centric views.

• Project page: https://youngwanlee.github.io/multihopspatial
• Dataset: etri-vilab/MultihopSpatial

Supported entry:

• MultihopSpatial

Usage

python run.py --data MultihopSpatial --model <your_model>

The benchmark TSV (images embedded as base64) auto-downloads from Hugging Face to $LMUData on first run, with an MD5 integrity check.

Implementation

This PR follows the VLMEvalKit benchmark contribution pattern by adding a dataset class with:

• build_prompt(self, line)
• evaluate(self, eval_file, **judge_kwargs)

The implementation:

• adds vlmeval/dataset/multihopspatial.py (MultihopSpatial(ImageBaseDataset))
• registers MultihopSpatial in vlmeval/dataset/__init__.py
• loads the dataset from Hugging Face: etri-vilab/MultihopSpatial
• is fully rule-based (regex MCQ parsing + IoU) — no external LLM judge / API key required, so no run.py change is needed
• parses boxes uniformly for every model (xyxy as prompted; a 0–1000-scale box is divided by 1000), so the protocol favors no model
• reports three metrics (all ×100), with per-hop and per-hop×view breakdowns:
  • mcq_acc — fraction with the correct choice
  • avg_iou — mean IoU of the predicted box over MCQ-correct samples
  • acc@50iou — fraction that are both MCQ-correct and IoU ≥ 0.5

Validation

Ran end-to-end via the local vLLM backend (0.00% infer-fail, 0.00% eval-fail). Numbers reproduce our reference implementation within ~1 point:

Model	mcq_acc	acc@50iou	avg_iou
MultiHopSpatial-Qwen3-VL-4B-Instruct	63.93	49.91	68.30
Qwen/Qwen3-VL-4B-Instruct (official)	38.04	25.27	60.56

Checklist

• pre-commit run --all-files passes (flake8 / isort / yapf).
• Dataset auto-registers and is listed in SUPPORTED_DATASETS.
• No new required dependencies.

[Copilot]

Pull request overview

This PR adds the MultihopSpatial benchmark to VLMEvalKit, enabling evaluation of multi-hop spatial reasoning with both MCQ accuracy and visual grounding (IoU-based) metrics.

Changes:

• Added a new dataset implementation MultihopSpatial(ImageBaseDataset) with prompt construction and rule-based evaluation (MCQ parsing + IoU).
• Registered MultihopSpatial in the dataset package so it appears in SUPPORTED_DATASETS and can be invoked via --data MultihopSpatial.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
vlmeval/dataset/multihopspatial.py	Implements prompt format, answer/box parsing, IoU computation, and metric aggregation for MultihopSpatial.
vlmeval/dataset/__init__.py	Registers the new dataset class and adds it to the image dataset registry list.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[youngwanLEE]

Good catch — fixed in the latest commit. load() can indeed return a list (e.g. PRED_FORMAT=json), so evaluate now converts to a DataFrame before indexing:

data = load(eval_file)
if isinstance(data, list):
    data = pd.DataFrame(data)

Verified the json-backed path produces results identical to the xlsx path (mcq 63.93 / acc@50iou 49.91 / avg_iou 68.30).

[youngwanLEE]

Fixed — the per-hop×view loop now also records avg_iou, so all three metrics are reported at the top level, per hop, and per hop×view consistently:

row[f"{hop}_{view}_mcq_acc"] = m2
row[f"{hop}_{view}_acc@50iou"] = a2
row[f"{hop}_{view}_avg_iou"] = v2

Confirmed the score CSV now contains the 6 {hop}_{view}_avg_iou columns.