DVG-WM: Disentangled Video Generation Enables Efficient Embodied World Model for Robotic Manipulation

Abstract

Disentangling dynamics learning
from visual synthesis

DVG-WM teaser: given an initial observation and a language instruction, the model previews a fast low-resolution trajectory, then reconstructs a high-fidelity video for real-world execution

DVG-WM at a glance. Given an initial observation and a language instruction, DVG-WM first generates a low-resolution imagined trajectory to preview the physical interaction (50 NFEs), then reconstructs a high-fidelity video with only 4 NFEs. The disentangled design yields both faster inference and stronger real-world manipulation performance.

Video-based embodied world models provide an appealing substrate for robotic manipulation by predicting future states, yet current approaches remain limited by a fundamental entanglement: accurately modeling dynamics typically requires low-level temporal reasoning, while producing high-resolution frames demands expansive visual synthesis according to high-level semantics. This entanglement results in slow inference speed for iterative planning or too coarse predictions to retain contact-rich details. To solve this dilemma, we present the Disentangled Video Generation World Model (DVG-WM), an efficient framework that explicitly decomposes world modeling into dynamics learning and visual synthesis. Conditioned on an initial observation and a language instruction, our model first generates a plausible sequence of intermediate visual states to preview the physical interaction and refines them to obtain high-fidelity videos. Furthermore, an efficient cascading mechanism is proposed, where DVG-WM leverages flow matching to directly map the dynamics to video latents, and introduces a latent degradation mechanism to enable the capability of regenerating contact-rich details. Experiments on LIBERO and real-world platforms demonstrate improved video quality with up to 3.97× acceleration, validating that disentangled video generation can be an efficient embodied world model for robotic manipulation.

Two-stage disentanglement

A low-resolution preview stage learns dynamics; a high-resolution refinement stage synthesizes appearance—each optimized at its ideal resolution.

Flow-matching cascading

Flow matching directly maps low-resolution dynamics to high-resolution video latents, restoring detail in just 4 function evaluations.

Latent degradation

Perturbing latents instead of pixels lets the model regenerate contact-rich structures rather than take shortcut upscaling.

Method

A two-stage disentangled pipeline

DVG-WM decomposes video-based world modeling into a fast dynamics preview and a high-fidelity refinement, connected by an efficient cascading mechanism. An optional action expert turns the imagined future into executable actions.

DVG-WM pipeline: preview stage with LoRA-adapted diffusion transformer producing low-resolution latents, upscale and latent degradation, refinement stage producing high-resolution frames, then action expert

Pipeline overview. (1) A preview stage learns dynamics and generates low-resolution trajectories conditioned on the language instruction and initial frame. (2) A refinement stage upsamples the latent dynamics to produce high-fidelity video predictions with minimal additional computation. Finally, an action expert extracts the action sequence from the predicted video for manipulation.

Stage 01

Low-Resolution Dynamics Preview

A CogVideoX-5B backbone, adapted with LoRA and flexible 3D rotary position embeddings, generates low-resolution video latents that capture the essential interaction dynamics. Operating at low resolution makes each denoising step cheap, providing a fast and physically meaningful preview for iterative planning.

Stage 02

Flow-Matching Refinement

A compact CogVideoX-2B refiner uses flow matching to map the upscaled low-resolution dynamics directly to high-resolution video latents along a straight-line probability path—restoring high-frequency detail and contact-sensitive cues in only 4 function evaluations instead of generating from noise.

Stage 03

Action Expert for Imitation

A vision-only Diffusion Policy is attached to convert imagined futures into executable actions. The imitation-learning gradients propagate through both the action expert and the world model, aligning visual prediction with downstream action planning—no robot state required.

Comparison of pixel-level degradation versus latent degradation for training the refinement stage

Latent degradation for contact-rich manipulation. Instead of degrading high-resolution video at the pixel level (which leaves the source and target tightly correlated and causes shortcut upscaling), DVG-WM perturbs the upscaled preview latents. This lets the refiner regenerate fine-grained contact details—preserving the structure of the gripper and target object in the zoomed-in view—rather than merely sharpening blurry pixels.

Results

Efficient, high-fidelity
video prediction

DVG-WM is evaluated on the LIBERO simulation benchmark for video quality and on a real-world Galaxea A1 platform for action planning.

3.97×

Faster inference
vs. LVP-14B

89%

Object-level accuracy
on LIBERO

88.7s

Total inference time
vs. 354.2s for LVP-14B

+25pp

Real-world success rate
over the vision-only policy

Video Quality on LIBERO

Quantitative comparison of video quality and object-level accuracy. Best values in each column are highlighted. LongScape reports original metrics; object-level accuracy is unavailable.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	FVD ↓	Object ↑
CogVideoX-5B	19.286	0.761	0.138	171.24	76%
Wan2.1-14B	18.964	0.732	0.162	198.54	68%
LongScape	19.977	0.788	0.123	153.72	–
LVP-14B	19.582	0.765	0.134	187.69	80%
DVG-WM (Ours)	20.019	0.783	0.120	152.36	89%

Inference Efficiency

Inference time for generating a full video prediction. DVG-WM runs its 50 denoising steps at low resolution and refines with only 4 steps, yielding up to a 3.97× speedup.

Inference time comparison: CogVideoX-5B 236.8s, Wan2.1-14B 312.0s, LVP-14B 354.2s, DVG-WM 88.7s

Up to 3.97× faster. DVG-WM completes a full prediction in 88.7s, versus 236.8s (CogVideoX-5B), 312.0s (Wan2.1-14B) and 354.2s (LVP-14B). The 50 denoising steps run at low resolution and refinement needs only 4 steps—crucial for real-world iterative planning.

Qualitative Comparison

LIBERO open-drawer task. CogVideoX shows the intent to interact with the wrong object (the bowl) at t=10 and misses the final drawer-pulling stage at t=40 (red circles). DVG-WM faithfully captures the contact-rich details—the gripper contacting the drawer and the subsequent drawer motion—while following the instruction accurately.

Ablation: Dissecting the Disentangled Design

Component analysis on LIBERO. Both stages and the proposed cascading choices are necessary for the best video quality.

#Stages	Method	PSNR ↑	SSIM ↑	LPIPS ↓	FVD ↓
Two-Stage
Two	DVG-WM (Ours)	20.019	0.783	0.120	152.36
Two	Pixel Degradation	19.421	0.768	0.128	168.54
Two	Naive Cascading	19.580	0.765	0.134	187.69
Two	360p Preview + Refine	18.812	0.756	0.146	188.47
One-Stage
One	Only Preview	14.714	0.721	0.341	238.72
One	Only Refine	19.277	0.760	0.135	169.29

Preview-stage videos at different LoRA fine-tuning steps; blue circles highlight bowl texture, red circles highlight gripper-drawer contact

Inside the preview stage. As LoRA fine-tuning proceeds, the preview stage trades fine texture (blue circles) for accurate interaction dynamics (red circles): by step 5K it correctly models the gripper–drawer contact. This confirms the preview stage specializes in dynamics, supplying a meaningful initialization that the refinement stage turns into high-fidelity video.

Real-World Manipulation

Real-world setup: Galaxea A1 robotic arm with parallel gripper and RealSense L515 RGBD camera

Platform setup. Experiments run on a Galaxea A1 robotic arm with a parallel gripper, observed by a RealSense L515 RGBD camera. We evaluate two tasks—moving a banana and placing bread into a gray bowl—using DVG-WM as a language-guided video planner equipped with an action expert.

Video quality on real-world scenarios. Mask-IoU measures overlap of the embodiment region between generated and ground-truth frames.

Method	PSNR ↑	SSIM ↑	LPIPS ↓	FVD ↓	Mask-IoU ↑
CogVideoX-5B	18.642	0.745	0.151	189.32	0.54
DVG-WM (Ours)	19.287	0.771	0.128	164.78	0.57

Task success rate over 10 rollouts: SR1 = moving banana, SR2 = placing bread into the gray bowl. DVG-WM with an action expert improves the average success rate by 25 points over the vision-only policy.

Method	SR1 %	SR2 %
DP (vision-only)	30 ± 15	20 ± 10
CogVideoX	50 ± 10	45 ± 5
DVG-WM (Ours)	75 ± 10	70 ± 0

Qualitative results on the real-world task of placing bread into bowl, comparing ground truth, CogVideoX and DVG-WM

Real-world placing-bread task. DVG-WM predicts realistic future interactions with fine-grained contact details between the gripper and objects, even preserving the contact area and texture of the bread. CogVideoX exhibits ghosting and motion blur (zoomed-in view), whereas DVG-WM completes the pick-and-place sequence faithfully.

DVG-WM: Disentangled Video Generation Enables Efficient Embodied World Model for Robotic Manipulation

Disentangling dynamics learningfrom visual synthesis