Sony Integrates AI Technologies from Cinemersive Labs for 3D Conversion in Games
Sony Interactive Entertainment has acquired the British startup Cinemersive Labs, which specializes in AI tools for converting 2D photos and videos into 3D. The startup's team will join SIE's Visual Computing Group, which focuses on game rendering, video encoding, and generative AI models. This will boost Sony's visual computing capabilities for PlayStation.
Cinemersive Technologies for 3D Content
Cinemersive's latest development is the Parallax app for VR, designed for viewing parallax photographs. These images are captured on smartphones or professional cameras with stereo lenses and let you change the viewing angle with natural head movements.
Cinemersive's AI algorithms convert standard 2D content into 3D formats. The process involves depth analysis, stereo pair generation, and parallax calculation—no specialized hardware required. These tools work not just for photos but also for videos, opening up exciting possibilities for gaming content.
In games, this means quickly creating immersive environments from existing assets, cutting down on 3D modeling time.
Integration into the PlayStation Ecosystem
SIE's Visual Computing Group will leverage Cinemersive's expertise to push ML forward in graphics. Key focus areas:
- Enhancing real-time rendering with generative models.
- Optimizing textures and lighting via AI analysis.
- Generating dynamic 3D scenes from 2D sources for procedural generation.
Sony already employs machine learning in the PlayStation 5 Pro. PlayStation Spectral Super Resolution (PSSR) upscales low-resolution images to 4K using neural networks for pixel interpolation and detail reconstruction.
PSSR runs on a custom GPU with ML acceleration, delivering stable FPS at top quality. For developers, it simplifies porting games across resolutions without sacrificing performance.
Prospects for Game Development
AI-powered 2D-to-3D conversion will enable:
- Automating VR content from flat textures.
- Real-time stereoscopic effects for mono video.
- Faster environment prototyping in engines like Unreal Engine or Sony's proprietary tools.
- Community modding support via accessible 2D assets.
On PlayStation 5 Pro, PSSR proves its worth: games render at 1080p-1440p and upscale to 4K with minimal artifacts. Cinemersive's approaches will extend this to full 3D content.
Technical hurdles include precise depth estimation in dynamic scenes and minimizing VR latency. Sony could pair this with ray tracing for photorealistic parallax.
Key Takeaways
- Cinemersive Labs bolsters SIE's Visual Computing Group with AI for 2D-to-3D conversion.
- Parallax: VR app for parallax photos with head-tracking.
- PSSR in PS5 Pro: AI upscaling to 4K for performance gains.
- Game dev applications: procedural 3D, VR content, ML rendering.
- Emphasis on real-time processing without specialized gear.
— Editorial Team
No comments yet.