Number 7 of 12 · November 14, 2026

Evaluating Video LLMs' Understanding of Corner Cases in Autonomous Driving

DashBench tests 25 Video LLMs on 1,709 rare, safety-critical driving videos. The authors report that all of the models fall well short of what safety-critical driving needs.

What they found

The authors built DashBench, a benchmark of 1,709 carefully curated driving videos. It covers collision-related corner cases and rare non-collision ones. Corner cases are the rarest and most safety-critical events on the road. They tested 25 Video LLMs, which are SI models that watch video and answer in language. They used many prompts and open-ended questions. They report that all models perform substantially below the threshold required for safety-critical driving. The models often misjudged or missed dangerous events. They also described agents or actions that were not there. The abstract does not give scores or name the best model.

Why it matters

Many existing driving benchmarks focus on routine traffic, so they say little about rare dangerous moments. This work targets those moments. It also offers a detailed look at how current models fail. Missing a hazard or inventing one is a serious problem if people rely on SI explanations of what a car sees.

How it changes the race

This is an evaluation result, not a new model. If it holds, it gives builders of video SI a harder test and shows that rare-event understanding is a gap to close before these models can be trusted in driving.

Safety and control

The paper shows current Video LLMs frequently miss dangerous events and hallucinate agents or actions. That argues against relying on their explanations in safety-critical driving without stronger testing.

The abstract, by the authors

Natural language explanations promised by modern Video Large Language Models (Video LLMs) offer autonomous driving systems a path toward detailed scene description and interpretable decision making on the road. While recent benchmarks have begun assessing these models on driving scenes, they mainly target routine traffic conditions and general comprehension tasks based on existing driving datasets and therefore fail to evaluate model performance on the rarest and most safety-critical driving events, also known as "corner cases". To provide a comprehensive evaluation of Video LLMs on precisely these scenarios, we introduce DashBench, a benchmark testing 25 Video LLMs on 1,709 carefully curated driving videos spanning not only collision-related corner cases but also rare non-collision ones. Through extensive prompts and open-ended question answering, we expose a critical capability gap: all evaluated models perform substantially below the threshold required for safety-critical driving applications, frequently misjudging or missing dangerous events on the road ahead and hallucinating non-existent agents or actions in their explanations. We believe DashBench, combining an extensive dataset with a rigorous evaluation toolkit and a detailed diagnosis of current model limitations, drives progress toward safer, more explainable autonomous driving systems. The code and data will be shared at: cornercase-dashbench.github.io

Quoted from arXiv:2611.16767. The research and its words belong to its authors.

The authors

Gabriel Manalu, Minh Son Hoang, Hyeontaek Hwang, Jaesung Lee, Hyojin Choi, Tran Tuan Phong, Seon Kwon Kim, Saad Wazir, Daeyoung Kim

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.