# VOID from Netflix: Open AI Model for Removing Objects from Videos with Physics Simulation
Netflix has released the open-source VOID model (Video Object and Interaction Deletion), designed for removing objects from videos while automatically reconstructing physical interactions. When an object is removed, the model recalculates trajectories, falls, and supports, preventing unrealistic artifacts like floating objects. Available on Hugging Face and GitHub under Apache 2.0.
Key Differences from Standard Methods
VOID goes beyond simple inpainting, focusing on cause-and-effect relationships. Traditional tools mask the removed area, correct shadows and reflections, but ignore interaction dynamics.
The model generates a quadmask — a four-component mask:
- Object to remove.
- Direct influence zone (contact surfaces).
- Indirect effects (shadows, reflections).
- Background compensation.
To create the quadmask, VOID integrates multimodal AI Gemini and segmenter SAM2. This allows precise identification of zones where the object supported other items or influenced motion.
Architecture and Training
Base: CogVideoX-Fun, a 5B-parameter video diffusion transformer from Alibaba PAI. Trained on synthetic datasets:
- HUMOTO: Blender simulations with physics (with/without object).
- Kubric (Google): Monte Carlo rendering of scenes with physics control.
Synthetic data provides perfect video pairs where removal consequences are physically accurate. Training on 8×A100 80GB. Inference requires GPU with 40+ GB VRAM — suitable for server A100/H100 or high-end workstations.
# Example inferensa (uproschennyy)
import torch
from void.model import VOID
from void.mask import QuadMaskGenerator
video = load_video('input.mp4')
quadmask = QuadMaskGenerator(gemini_model, sam2).generate(video, bbox)
output = void_model.infer(video, quadmask)
save_video(output, 'output.mp4')Inference code and GUI for masks included in the repository.
Performance Comparison
In a blind test with 25 developers, VOID was preferred in 64.8% of cases over Runway's 18.4%. Benchmarks on synthetic and real videos:
| Model | Synthetic (PSNR/SSIM) | Real Videos (Clipping Score) |
|-------------|-----------------------|------------------------------|
| VOID | 32.4 / 0.94 | 8.2 |
| Runway | 28.1 / 0.89 | 6.5 |
| ProPainter | 27.9 / 0.87 | 5.9 |
| DiffuEraser | 26.5 / 0.85 | 5.2 |
| MiniMax | 25.8 / 0.83 | 4.8 |
VOID leads in realism metrics, especially in dynamic scenes with gravity and collisions.
Practical Applications for Developers
For mid/senior specialists:
- Post-production: Automatic removal of boom mics and props without manual animation.
- VFX pipelines: Integration into Nuke/After Effects via Python API.
- Research: Baseline pipeline for data generation to fine-tune on custom domains (medicine, AR).
- Optimization: LoRA adaptation to reduce VRAM to 24 GB on RTX 4090.
Netflix has open-sourced not only the weights but also the dataset generation scripts — fully reproducible from scratch.
Key Takeaways
- Netflix's first open-source model with a full stack (data + code + GUI).
- Quadmask + Gemini/SAM2 solves physical interaction problems.
- 5B CogVideoX-Fun: balance of quality and resources (inference ~40GB VRAM).
- Benchmarks confirm 20–40% superiority over Runway/ProPainter.
- Apache 2.0: commercial use without restrictions.
— Editorial Team
No comments yet.