Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?

Zihao Zhang, Haochen Tian, Tianyu Li, Changhui Jing, Jingliang He, Naisheng Ye, Ziyuan Pu, Zhenjie Yang

• 📧 Primary Contact: Zihao Zhang (zihao.zhang@opendrivelab.com)
• 🖊️ Joint effort by SEU, OpenDriveLab at HKU, and SLAI.

---

Highlights

• 🚗 Planner-agnostic alignment: ViRA improves driving across diverse end-to-end planners without changing their deployed architecture or inference cost.
• 🔍 Target selection and supervision matter: VFM target choice affects planning gains, while auxiliary perception supervision narrows performance differences across targets.
• 📈 ViRA-Diffusion: DINOv3 alignment enables 92.3 EPDMS on NAVSIM v2 navtest without auxiliary perception supervision.

News

• [2026/10/23] As of October 23, 2026, ViRA-Diffusion ranks first on the Papers with Code NAVSIM v2 navtest benchmark.

• [2026/10/22] We released the code and checkpoints of ViRA-Diffusion.

• [2026/10/7] We released our paper on arXiv.

TODO List

• Results and Demo release.
• ViRA-Diffusion inference code and checkpoints.
• All Code release.
• All Checkpoints release.

Getting Started

1. Clone ViRA Repo

git clone https://github.com/OpenDriveLab/ViRA
cd ViRA

2. Create Environment

conda env create --name vira -f environment.yml
conda activate vira
pip install -e .

3. Data Preparation

Prepare the NAVSIM OpenScene data as described in NAVSIM. Download the ViRA-Diffusion checkpoints from Hugging Face into ckpts/. Generate the navtest metric cache and place it at the path below. The trajectory anchor is already in the repository.

ViRA/
├── datasets/
│   ├── maps/                  # NUPLAN_MAPS_ROOT, nuplan-maps-v1.0
│   ├── navsim_logs/
│   ├── sensor_blobs/
│   ├── warmup_two_stage/
│   └── navhard_two_stage/
├── navsim/traj_final/kmeans_navsim_traj_20.npy
├── exp/cache/navsim_v2_eval_cache/navtest_metric_cache/
└── ckpts/
    ├── resnet34.bin
    └── vira_diffusion.ckpt

4. Inference

bash ./scripts/evaluation_navtest/run_vira_diffusion_evaluation.sh

Results

Camera-only. Rap* is our reimplementation with a different backbone and without the original data augmentation. HUGSIM is zero-shot: planners are trained only on NAVSIM.

Method	Decoder	NAVSIM v2 navtest	NAVSIM v2 navhard	HUGSIM
		EPDMS	EPDMS	HD-Score
Perception-based
TransFuser	Regression	83.6	27.6	22.1
TransFuser-ViRA		89.0 | +5.4	31.7 | +4.1	24.8 | +2.7
DiffusionDrive	Diffusion	84.5	30.5	22.2
DiffusionDrive-ViRA		91.9 | +7.4	35.8 | +5.3	28.6 | +6.4
Perception-free
Rap*	Scoring	72.8	32.8	7.3
Rap*-ViRA		83.7 | +10.9	49.2 | +16.4	10.3 | +3.0

ViRA-Diffusion is DiffusionDrive trained with DINOv3 alignment and without auxiliary perception supervision.

Method	NAVSIM v2 navtest EPDMS
Epona	85.1
DiffusionDriveV2	87.5
Latent-WAM	89.3
DriveFuture	89.9
SparseDriveV2	90.1
Discrete-WAM	90.4
ViRA-Diffusion	92.3

🏆 Papers with Code Leaderboard

As of October 23, 2026, ViRA-Diffusion ranks #1 on the Papers with Code NAVSIM v2 navtest leaderboard with 92.3 EPDMS.

Visualization of HUGSIM

(a) Car following: the lead vehicle brakes suddenly

(b) Oncoming traffic: other vehicles enter the current driving area

Acknowledgements

We acknowledge all the open-source contributors for the following projects to make this work possible:

• NAVSIM | HUGSIM | SimScale | DiffusionDrive | Rap
• DVGT | VGGT | DINOv3 | SAM3 | DA3

License and Citation

All content in this repository is under the Apache-2.0 license.

If any parts of our paper and code help your research, please consider citing us and giving a star to our repository.

@misc{zhang2026vira,
  title = {Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?},
  author = {Zhang, Zihao and Tian, Haochen and Li, Tianyu and Jing, Changhui and He, Jingliang and Ye, Naisheng and Pu, Ziyuan and Yang, Zhenjie},
  year = {2026},
  eprint = {2610.09695},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.09695}
}