Researchers at LingBot World have released an open-weights interactive world model that uses a MoBA attention mask (mixing bidirectional and autoregressive attention) and post-training distillation over long self-rollout trajectories to significantly reduce drift, demonstrated by a continuous 60-minute stable rollout. This work addresses a critical challenge in interactive world models—drift accumulation during long rollouts—which has limited their practical usability. The combination of MoBA attention and long-rollout distillation could enable more stable and reliable interactive simulations for applications like video games, robotics, and virtual environments. The MoBA attention mask dynamically schedules KV-cache to keep long rollouts tractable, and the model uses Plücker embeddings plus AdaLN for camera control. A limitation is that persistence is only in appearance, not identity, meaning regions leaving the context window are regenerated on revisit, not recalled. The weights are released under CC-BY-NC-SA license.
Background
Interactive world models generate video frames conditioned on user input, essentially serving as a learned simulator. A common problem is drift: as the model autoregressively generates new frames based on its own previous outputs, small errors accumulate, causing the simulation to diverge from plausible reality over time. Existing methods often use teacher forcing during training, where the model sees ground-truth frames, but this does not translate to stable self-rollouts during inference.