Number 9 of 12 · November 19, 2026

Manufacturing Differential Sparsity: Planner-Hardware Co-Design for Efficient World Model Planning

The authors speed up world-model planning for robots by designing the planner and the chip together, so the planner creates repeated, similar work that the hardware can skip.

What they found

The authors study JEPA-style world models, which let a robot plan toward new goals by trying many candidate action sequences. This is slow because the model is run tens of thousands of times per plan, and one planning step can take tens of seconds on high-end GPUs. They notice that similar actions give similar internal states, which shows up as differential bit-level sparsity (many bits that do not change between runs, so the work can be skipped) in a quantized model. Their planner, using reference forest sampling, ties groups of candidate action sequences to a shared root to create more of this sparsity, with competitive planning quality. On three JEPA world models, their co-design is 1.28-1.80x faster than a dense accelerator and uses 38-52% less energy. It is 1.22-1.58x faster than a prior bit-sparse design and uses 16-36% less energy, with the same area.

Why it matters

Planning that takes tens of seconds is too slow for real robots acting in the real world. This work shows one route to cutting that cost: shape the algorithm so the hardware can do less work, instead of only building faster hardware. The abstract reports speed and energy numbers for three models, but it does not say how these results translate to a real robot.

How it changes the race

If it holds, this makes planning with world models cheaper and faster on specialized chips, which helps move such SI systems toward real-time robot control. The gains are moderate, not orders of magnitude, so it is an incremental step.

The abstract, by the authors

JEPA-style latent world models let embodied agents plan toward new goals with model predictive control, but they are still too slow for real-life embodied control: sampling-based MPC repeats predictor inference tens of thousands of times per plan, and even on high-end GPUs one planning step can take tens of seconds. We observe substantial latent locality in this repeated-inference workload: the predictor maps close actions to close latents, and inside a quantized predictor this closeness takes a form that hardware can exploit, differential bit-level sparsity. Unlike an input stream, whose temporal similarity is given, a planner itself decides which candidates it evaluates together, so the planner can choose its own references and, with reference forest sampling, manufacture more sparsity: it couples the sampled action sequences of each group to a shared root, with competitive planning quality. We build the accelerator on the insights of the rich line of bit-level sparsity accelerators and design it and the planner tightly coupled as a single co-design: the planner increases sparsity and exposes structure for hardware scheduling, and the bit-exact hardware effectively uses the sparsity that the planner creates. On three JEPA world models, our co-design is 1.28-1.80x faster than a dense accelerator and uses 38-52% less energy, and it is 1.22-1.58x faster than prior bit-sparse design and uses 16-36% less energy, with the same area. To our knowledge, our planner is the first planner that manufactures differential bit-level sparsity across its candidates and exposes its structure for hardware scheduling, and our co-design is the first algorithm-hardware co-design for JEPA-style world-model planning.

Quoted from arXiv:2611.22483. The research and its words belong to its authors.

The authors

Wenbo Zhu, Paul Yi-Chia Chen, Yuanhan Li, Zishen Wan, Christopher Torng

Credit for this research belongs to its authors. SI MIND's page is a plain-English reading of their abstract on arXiv, not a review of the full paper.