DeepJEPA

Scaling World Models from Within

Zijian Jin1,†, Yunbei Zhang2,†, Yuanzhe Liu3, Ming Liu4, Baian Chen5, Weirui Ye5, Shilong Liu4,6, Marco Pavone7

1New York University   2Tulane University   3UIUC   4Princeton University   5MIT   6Columbia University   7Stanford University

†Equal contribution

DeepJEPA keeps most imagined transitions shallow and refines decision-critical transitions before selecting an action.
Think before you roll. DeepJEPA scales the world model from within while leaving the outer planner unchanged.

Overview

World-model planners usually scale outward by rolling farther, sampling more trajectories, or optimizing longer. DeepJEPA introduces a complementary test-time scaling axis: adaptive recurrent depth inside each imagined transition.

A weight-tied joint-embedding predictive world model learns whether another recurrent update is worth computing for each candidate and rollout step. Most transitions remain shallow, while additional computation concentrates at decision-critical events that can change CEM rankings and the selected action.

Method

1. Imagine

Roll candidate actions through latent space using the same outer planning budget.

2. Refine selectively

Spend recurrent depth only when the continue head predicts that another update is worthwhile.

3. Rank and act

Return the selected latent predictions to CEM, where changed costs can alter the executed action.

DeepJEPA inference, adaptive stopping, and training mechanism.
DeepJEPA mechanism. Each imagined transition becomes a budgeted computation process whose selected prediction is returned to the unchanged outer planner.

Results

5 visual-control settings  ·  1.00–1.26 mean selected updates  ·  +3.3 pp largest gain over the best fixed depth

DeepJEPA success gains, mean selected depth, and performance relative to fixed-depth baselines across five tasks.
DeepJEPA improves or matches the fixed-depth frontier while remaining shallow on average. Values are means over three training and evaluation seed pairs.
TaskBest fixedDeepJEPAGainMean depth
Reacher84.085.3+1.31.03
Cube Single79.379.30.01.00
Cube Double72.774.0+1.31.26
Cube Triple74.077.3+3.31.22
PushT10.711.6+0.91.02

Where computation goes

Refinement rises at PushT contact onset and during sustained Cube Triple contact, despite training without contact labels. The learned computation policy tracks changes in local dynamics rather than rollout position alone.

Allocate internal computation where it can change the planner's decision.
Adaptive computation rises around contact onset and sustained object contact.
Adaptive computation is interaction aligned. Refinement increases near contact and falls after release.

Citation

@article{jin2026deepjepa,
  title={DeepJEPA: Scaling World Models from Within},
  author={Jin, Zijian and Zhang, Yunbei and Liu, Yuanzhe and Liu, Ming and Chen, Baian and Ye, Weirui and Liu, Shilong and Pavone, Marco},
  journal={arXiv preprint arXiv:2610.00368},
  year={2026}
}