What they found
The authors built SixthView, a benchmark where a model sees five flat views of a 3D assembly and must predict the sixth, hidden view. It has 602 synthetic assemblies made of cube-based parts, with 2-10 parts each. For every item, the authors checked all legal assemblies that fit the inputs and confirmed they give the same hidden view, so each answer is unambiguous. They tested three vision-language models under four input conditions. With images plus exact text, exact-grid accuracy ranged from 33.2% to 85.4% across 552 items with complete evaluations. Adding exact text to separate images raised accuracy by 19.7-63.0 percentage points. Adding images to exact text changed it by only -0.5 to +2.1 points. Separate image panels beat a merged sheet by 6.7-11.1 points.
Why it matters
Many real tasks, such as engineering, robotics and design, need a system to reason about parts that cannot be seen. This benchmark separates two skills: reading shapes from pictures and reasoning about where parts go. The results suggest that, for these models, the weak point is getting exact shape information from images, not the reasoning once the shapes are given. The abstract does not say why the models fail, so that reading is a suggestion, not a proven cause.
How it changes the race
This is a benchmark, not a new method. It gives developers of vision-language models a clean, verifiable test that points at visual shape recovery as a gap to close.
The abstract, by the authors
Inferring the unseen side of a 3D assembly requires reasoning about part placement and occlusion. Different assemblies can match the observed views while producing different hidden views, leaving the target ambiguous. We introduce SixthView, a benchmark in which models predict a sixth orthographic view from five observed views and information about the component shapes. The benchmark contains 602 synthetic polycube assemblies with 2-10 parts and requires a dense grid of part labels for the missing view, allowing both exact-view and cell-level evaluation. For each item, we exhaustively verify that all legal assemblies consistent with the exact component geometry and observed label grids yield the same target view. With these inputs, the task tests symbolic assembly and coordinate reasoning. The image-only variants additionally require shape recovery from rendered views, where occlusion can make different 3D shapes look identical. We evaluate three vision-language models under four input conditions. Across 552 items with complete evaluations, exact-grid accuracy with images plus exact text ranges from 33.2% to 85.4%. Adding exact text to separate images improves accuracy by 19.7-63.0 percentage points, whereas adding images to exact text changes it by only -0.5 to +2.1 points. Separate image panels outperform a merged sheet by 6.7-11.1 points. The results show that performance depends strongly on access to exact symbolic information, with little further improvement from images once the geometry is specified. GitHub: https://github.com/facebookresearch/sixthview_benchmark
Quoted from arXiv:2611.23891. The research and its words belong to its authors.
