May 30, 2018 at 16:38Will voxels become a new breakthrough technology?
- Transfer
We talked to the amazing Atomontage developers , trying to figure out if the voxels could return and defeat the pixels.
Branislav: in 2000-2002 I participated in the European demoscene competitions. I wrote some 256-byte demos (also called intro) under the nickname Silique / Bizzare Devs (see “Njufnjuf”, “Oxlpka”, “I like ya, Tweety” and “Comatose”). Each of the intro generated real-time voxels or graphics from a point cloud. Both voxels and point clouds are examples of sampled geometry.
Intro performed their task in only 100 processor instructions, such as ADD, MUL, STOSB, PUSH and the like. However, due to the very nature of this type of program, in fact, dozens of instructions were used simply for proper configuration, and not for generating the graphics themselves. Nevertheless, these more than 50 instructions, which in essence were elementary mathematical operations or memory operations, were enough to generate pretty beautiful moving 3D-graphics in real time. All these 256-byte intros won from first to third place. This made me realize that if it is possible to create such 3D graphics without polygons, then in games and other applications much more can be achieved using the same principle: using sampled geometry instead of polygonal meshes. The solution is simplicity. I understood, that the then dominant paradigm, based on a complex and fundamentally limited (voluminous) representation of data, was ready to run into a ceiling of possibilities. That is, it’s the right time to try out this “new”, simpler paradigm: volumetric sampled geometry.
Dan: While still in high school in Sweden, I started programming a side-scrolling 2D engine, on which I created an indie game called “Cortex Command” as a result. It was like Worms or Liero, but with the gameplay it was more in real time and with RTS elements. The game also used a more detailed simulation of the different materials of each pixel of the relief. In a side view, similar to an “ant farm”, the player’s characters could dig gold in soft earth and build protective bunkers with solid concrete and metal walls. In 2009, the Cortex Command won the Technical Excellence Award and Audience Award at the Independent Games Festival. Since that time, I dreamed of creating a completely three-dimensional version of the game, and this was possible only with the help of volumetric simulation and graphics.
About six years ago, I looked for ready-made voxel solutions and found Branislav’s work on his website and in a video in which he talked about the inevitable transition from polygonal 3D graphics to something that resembled what I did in 2D: to simulate the entire virtual world like small atomic blocks with material properties. Not only did his statement seem right to me - his technology, judging by the simple but impressive videos, turned out to be the best and most convincing of those that existed. I began to sponsor his project through his website and communicate with him, which led to the beginning of many years of friendship, and now co-financing of our company. It’s amazing to feel part of this turning point in such an epic project,
We believe that many major players have realized that polygon technologies have hit a ceiling of complexity more than a decade ago. This problem manifests itself in a variety of ways: in complex toolchains, in tricky hacks that allow you to implement interactions and damage simulation, in a complex representation of geometry (polygonal surface model + collision models + other models for representing the internal structure, if any), in redesigned approaches to volumetric video, hacks and huge code bases, etc. Because of these problems, progress depends almost entirely on the power of the video processors, and some aspects are generally unattainable. This is a battle that cannot be won. This is characteristic of the nature of large companies: often they don’t even try to spend a lot of time and resources on developing risky and changing game conditions;
There are a set of techniques that people usually consider voxel-based. The oldest of these were used in altitude map games, where the renderer interpreted a 2D height map to calculate the boundaries between the air and the ground of the scene. This is not quite a voxel approach, because it does not use a volumetric dataset (examples: Delta Force 1, Comanche, Outcast, and others).
Some engines and games use large blocks with their own internal structure that make up the virtual world (example: Minecraft). These blocks are usually rendered using polygons, meaning the smallest elements are triangles and texels, not voxels. Such geometry is simply ordered into a grid of larger blocks, but this, strictly speaking, does not make them voxels.
Some games use relatively large voxels, or SDF elements (signed distance fields, fields with sign distance), which still do not provide realism, but will already allow you to create interesting gameplay (examples: Voxelstein, Voxelnauts, Staxel). There are also SDF-based projects that provide great interactions and simulations and have the potential to create high realism (example: Claybook). However, so far we have not seen attempts to develop a solution for simulating and rendering realistic large scenes, similar to those that our technology is capable of.
In AtomontageVoxels are used as the basic building blocks of scenes. Individual voxels in our technology have virtually no structure. This approach provides simplicity that greatly helps in simulations, interactions, content generation, as well as in data compression, rendering, encoding of surround video, etc.
Our voxel-based solution eliminates the many complexities people face with polygon-based technologies, including the very concept of limiting the number of polygons and the many hacks used to circumvent this limitation. Our technology is voluminous by its very nature and therefore does not require any additional representation for modeling inside objects.
In this approach, there is a powerful and integral system of LOD (level of detail, level of detail), which allows technology to balance performance and quality in traditionally difficult situations. One of the many benefits is granular control over LOD, foveated rendering and processing, which require almost no unnecessary costs.
Voxel geometry eliminates the burden of a complex structure: it is based on sampling, and therefore it is easy to work with (a simple and universal data model for any geometry, in contrast to a complex resource data model based on polygons). This allows you to iterate faster when developing powerful compression methods, interaction tools, content generators, physics simulators, etc. This does not happen in polygonal technologies, because they have been struggling for a ceiling of complexity for at least a dozen years, and their progress depends heavily on the exponential growth in the power of video processors.
The voxel approach is effective because it does not spend resources or bandwidth on the poorly compressible components of vectors or vector representations of data (polygons, point clouds, particles), i.e. values that have an almost random location. In the case of voxels, it is mainly possible to encode useful information (color, material information, etc.), rather than redundant data, simply by placing this information in the right place in space. You can compare it with JPEG and some two-dimensional vector format encoding a large and complex image. JPEG coding is predictable and can be fine-tuned for optimal quality and small volume, and a vector image will spend most of the space on vector information instead of the color samples themselves.
Our approach will allow ordinary people to unleash their creative talent without the need to learn and understand internal technology and its limitations. The skills that we received, growing up in the real world, will be enough for interacting with virtual environments in a useful and realistic way.
Thanks to the LOD system, which is an integral part of the technology, and the simple spatial structure of the voxel resources, it is easy to obtain large-scale high-resolution video. Our voxel-based rendering does not reduce the quality of the geometry due to the simplification of the mesh, and the performance achieved with standard modern equipment is incomparable.
Now we are at the stage when we can voxelize not only one large high-poly model, but also a series of such resources with the assembly of three-dimensional video from them. We can also do this with whole environments: imagine a cinematic quality scene with characters that can be used in a science fiction or animated film. We can turn the environment into a VR movie in which the user can be in the same space with the characters, freely moving the viewpoint (and not just looking around, like in a virtual video) in a scene the size of a room or even larger. The user can perceive the process, as is usual for VR games, with the exception of interactivity. Now we are looking for partners who would help us make the first test short film with high-quality graphic data.
Our technology already allows the use of various transformations that affect any voxel model, creating convincing effects of deformation of soft bodies. Although we do not yet have a demo of character animation with full skinning and other functions, our technology is certainly capable of this if you attach it to any traditional rigging system. This is already evident in our videos with soft bodies: tires on the wheels of a car are compressed in response to simulated forces, and this deformation is similar to what happens in the character animation under the influence of attached bones. In the early stages, rigging itself can even be performed in some of the existing tools even before voxelization, and over time we will implement it in our own tools.
Finished texture resources can be voxelized in various ways. We get the best results with two of them - with a ray-based voxelizer and projection-based voxelizer. The first one emits rays through the polygonal model, recognizes the intersections between each of the rays and the mesh, calculates the relative positions on the attached texture, reads texels and bakes them into the corresponding voxels applied to the model.
A voxelizer based on projection renders a model from several viewpoints into maps, including depth maps. The intersection of volumes defined by depth maps provides information on the voxels that need to be created. Other generated maps provide the rest of the surface information (color, normals, etc.), which is also baked into the surface voxels.
There are also other ways to create properly painted surfaces - voxelization of data from a point cloud or procedural generation of content / surface properties of finished models.
These are two separate issues. Generating content using voxels is simple, because you just need to add a large number of samples with some useful properties (color, material information, etc.) into a uniform grid with a given spatial resolution, or with several resolutions in the case of data with a variable LOD . This is easily implemented using voxels, because we do not face restrictions on the number of polygons. In addition, there are no textures either, so we have no problem resolving textures.
Rendering large scenes is also quite simple thanks to the powerful LOD system. The renderer can use the most optimal LOD combination of small geometry segments to render the entire scene with the greatest detail possible, while maintaining a high FPS. Voxel LODs are very inexpensive and well suited for keeping voxel sizes (and therefore shape errors) smaller than the pixel size on the screen.
Our voxelizers are already powerful enough to voxelize very high poly mesh and point clouds using the methods mentioned above. We demonstrated the first results with photogrammetry data at the beginning of 2013, voxelizing a mesh of 150 million polygons, which we rendered and changed in real time on an average gaming laptop in 2008. After voxelization, the number of polygons of the original geometry becomes unimportant, and changing the model is a simple task. This can be seen in our videos; performance is more dependent on the number of pixels than the number of polygons of the source data. All this is necessary to provide users with a comfortable workflow while cleaning up huge scanned models. In addition, voxels are in some ways similar to pixels,
Representation of space in the form of voxels is a necessary requirement for creating huge networked interactive virtual environments that at some point will begin to resemble the real world. The key point is that such environments are not static and allow users to interact with them in a familiar way in the real world. That is, such virtual environments should be fully simulated and with convincing physics. Simulation and rendering of a complex, fully dynamic world requires such worlds to be volumetric, and all processing, synchronization and rendering of the simulated geometry must be effective. The volumetric nature of such a process precludes the use of polygons. In addition, as mentioned above, other vector forms of representation are not effective enough to create high quality at low cost.
We expect that such huge simulated environments will consist mainly of voxels, and only a small fraction will be simulated using particles to create some effects. Remarkably, both of these representations are simple and sampled, and that the conversion from one to the other will be trivial and cost-effective in terms of computation.
These are the huge advantages of voxels over other presentation methods, making them an ideal solution for describing such gigantic virtual environments. The LOD system is great for optimizing processing and rendering on the fly depending on any combination of parameters (distance to the user, importance of the simulated process, accuracy / cost ratio, etc.). Also thanks to this, they are ideal for foveated rendering and very efficient streaming. They can also be specified with a larger dimension, which is necessary to perform large-scale distributed physical simulation. Such simulations cannot be performed in only three dimensions due to network delays and the inability to process a large model on one PC and a one-time transfer of a large part of the geometry. Segmented a variable voxel geometry LOD system will help a lot here. When changing a small part or several parts of a large voxel model, there is no need to recalculate its large mesh and textures, as well as synchronize the entire model over the network - only the involved parts are important, which can be synchronized with the most suitable for transmission over the LOD network.
All these requirements for the huge virtual environments of the future lead to the necessity and inevitability of a paradigm shift in the direction of volumetric sampled geometry. The properties of our voxel technology make it the best and only candidate for a paradigm shift in the near future. Atomontage
Development Team . Interview conducted by Kirill Tokarev.
Voxel development
Branislav: in 2000-2002 I participated in the European demoscene competitions. I wrote some 256-byte demos (also called intro) under the nickname Silique / Bizzare Devs (see “Njufnjuf”, “Oxlpka”, “I like ya, Tweety” and “Comatose”). Each of the intro generated real-time voxels or graphics from a point cloud. Both voxels and point clouds are examples of sampled geometry.
Intro performed their task in only 100 processor instructions, such as ADD, MUL, STOSB, PUSH and the like. However, due to the very nature of this type of program, in fact, dozens of instructions were used simply for proper configuration, and not for generating the graphics themselves. Nevertheless, these more than 50 instructions, which in essence were elementary mathematical operations or memory operations, were enough to generate pretty beautiful moving 3D-graphics in real time. All these 256-byte intros won from first to third place. This made me realize that if it is possible to create such 3D graphics without polygons, then in games and other applications much more can be achieved using the same principle: using sampled geometry instead of polygonal meshes. The solution is simplicity. I understood, that the then dominant paradigm, based on a complex and fundamentally limited (voluminous) representation of data, was ready to run into a ceiling of possibilities. That is, it’s the right time to try out this “new”, simpler paradigm: volumetric sampled geometry.
Dan: While still in high school in Sweden, I started programming a side-scrolling 2D engine, on which I created an indie game called “Cortex Command” as a result. It was like Worms or Liero, but with the gameplay it was more in real time and with RTS elements. The game also used a more detailed simulation of the different materials of each pixel of the relief. In a side view, similar to an “ant farm”, the player’s characters could dig gold in soft earth and build protective bunkers with solid concrete and metal walls. In 2009, the Cortex Command won the Technical Excellence Award and Audience Award at the Independent Games Festival. Since that time, I dreamed of creating a completely three-dimensional version of the game, and this was possible only with the help of volumetric simulation and graphics.
About six years ago, I looked for ready-made voxel solutions and found Branislav’s work on his website and in a video in which he talked about the inevitable transition from polygonal 3D graphics to something that resembled what I did in 2D: to simulate the entire virtual world like small atomic blocks with material properties. Not only did his statement seem right to me - his technology, judging by the simple but impressive videos, turned out to be the best and most convincing of those that existed. I began to sponsor his project through his website and communicate with him, which led to the beginning of many years of friendship, and now co-financing of our company. It’s amazing to feel part of this turning point in such an epic project,
Growing interest
We believe that many major players have realized that polygon technologies have hit a ceiling of complexity more than a decade ago. This problem manifests itself in a variety of ways: in complex toolchains, in tricky hacks that allow you to implement interactions and damage simulation, in a complex representation of geometry (polygonal surface model + collision models + other models for representing the internal structure, if any), in redesigned approaches to volumetric video, hacks and huge code bases, etc. Because of these problems, progress depends almost entirely on the power of the video processors, and some aspects are generally unattainable. This is a battle that cannot be won. This is characteristic of the nature of large companies: often they don’t even try to spend a lot of time and resources on developing risky and changing game conditions;
Technology
There are a set of techniques that people usually consider voxel-based. The oldest of these were used in altitude map games, where the renderer interpreted a 2D height map to calculate the boundaries between the air and the ground of the scene. This is not quite a voxel approach, because it does not use a volumetric dataset (examples: Delta Force 1, Comanche, Outcast, and others).
Some engines and games use large blocks with their own internal structure that make up the virtual world (example: Minecraft). These blocks are usually rendered using polygons, meaning the smallest elements are triangles and texels, not voxels. Such geometry is simply ordered into a grid of larger blocks, but this, strictly speaking, does not make them voxels.
Some games use relatively large voxels, or SDF elements (signed distance fields, fields with sign distance), which still do not provide realism, but will already allow you to create interesting gameplay (examples: Voxelstein, Voxelnauts, Staxel). There are also SDF-based projects that provide great interactions and simulations and have the potential to create high realism (example: Claybook). However, so far we have not seen attempts to develop a solution for simulating and rendering realistic large scenes, similar to those that our technology is capable of.
In AtomontageVoxels are used as the basic building blocks of scenes. Individual voxels in our technology have virtually no structure. This approach provides simplicity that greatly helps in simulations, interactions, content generation, as well as in data compression, rendering, encoding of surround video, etc.
Benefits
Our voxel-based solution eliminates the many complexities people face with polygon-based technologies, including the very concept of limiting the number of polygons and the many hacks used to circumvent this limitation. Our technology is voluminous by its very nature and therefore does not require any additional representation for modeling inside objects.
In this approach, there is a powerful and integral system of LOD (level of detail, level of detail), which allows technology to balance performance and quality in traditionally difficult situations. One of the many benefits is granular control over LOD, foveated rendering and processing, which require almost no unnecessary costs.
Voxel geometry eliminates the burden of a complex structure: it is based on sampling, and therefore it is easy to work with (a simple and universal data model for any geometry, in contrast to a complex resource data model based on polygons). This allows you to iterate faster when developing powerful compression methods, interaction tools, content generators, physics simulators, etc. This does not happen in polygonal technologies, because they have been struggling for a ceiling of complexity for at least a dozen years, and their progress depends heavily on the exponential growth in the power of video processors.
The voxel approach is effective because it does not spend resources or bandwidth on the poorly compressible components of vectors or vector representations of data (polygons, point clouds, particles), i.e. values that have an almost random location. In the case of voxels, it is mainly possible to encode useful information (color, material information, etc.), rather than redundant data, simply by placing this information in the right place in space. You can compare it with JPEG and some two-dimensional vector format encoding a large and complex image. JPEG coding is predictable and can be fine-tuned for optimal quality and small volume, and a vector image will spend most of the space on vector information instead of the color samples themselves.
Our approach will allow ordinary people to unleash their creative talent without the need to learn and understand internal technology and its limitations. The skills that we received, growing up in the real world, will be enough for interacting with virtual environments in a useful and realistic way.
Thanks to the LOD system, which is an integral part of the technology, and the simple spatial structure of the voxel resources, it is easy to obtain large-scale high-resolution video. Our voxel-based rendering does not reduce the quality of the geometry due to the simplification of the mesh, and the performance achieved with standard modern equipment is incomparable.
Now we are at the stage when we can voxelize not only one large high-poly model, but also a series of such resources with the assembly of three-dimensional video from them. We can also do this with whole environments: imagine a cinematic quality scene with characters that can be used in a science fiction or animated film. We can turn the environment into a VR movie in which the user can be in the same space with the characters, freely moving the viewpoint (and not just looking around, like in a virtual video) in a scene the size of a room or even larger. The user can perceive the process, as is usual for VR games, with the exception of interactivity. Now we are looking for partners who would help us make the first test short film with high-quality graphic data.
GIF
Rigging
Our technology already allows the use of various transformations that affect any voxel model, creating convincing effects of deformation of soft bodies. Although we do not yet have a demo of character animation with full skinning and other functions, our technology is certainly capable of this if you attach it to any traditional rigging system. This is already evident in our videos with soft bodies: tires on the wheels of a car are compressed in response to simulated forces, and this deformation is similar to what happens in the character animation under the influence of attached bones. In the early stages, rigging itself can even be performed in some of the existing tools even before voxelization, and over time we will implement it in our own tools.
Texturing
Finished texture resources can be voxelized in various ways. We get the best results with two of them - with a ray-based voxelizer and projection-based voxelizer. The first one emits rays through the polygonal model, recognizes the intersections between each of the rays and the mesh, calculates the relative positions on the attached texture, reads texels and bakes them into the corresponding voxels applied to the model.
A voxelizer based on projection renders a model from several viewpoints into maps, including depth maps. The intersection of volumes defined by depth maps provides information on the voxels that need to be created. Other generated maps provide the rest of the surface information (color, normals, etc.), which is also baked into the surface voxels.
There are also other ways to create properly painted surfaces - voxelization of data from a point cloud or procedural generation of content / surface properties of finished models.
Large-scale landscapes
These are two separate issues. Generating content using voxels is simple, because you just need to add a large number of samples with some useful properties (color, material information, etc.) into a uniform grid with a given spatial resolution, or with several resolutions in the case of data with a variable LOD . This is easily implemented using voxels, because we do not face restrictions on the number of polygons. In addition, there are no textures either, so we have no problem resolving textures.
Rendering large scenes is also quite simple thanks to the powerful LOD system. The renderer can use the most optimal LOD combination of small geometry segments to render the entire scene with the greatest detail possible, while maintaining a high FPS. Voxel LODs are very inexpensive and well suited for keeping voxel sizes (and therefore shape errors) smaller than the pixel size on the screen.
Work with scans
Our voxelizers are already powerful enough to voxelize very high poly mesh and point clouds using the methods mentioned above. We demonstrated the first results with photogrammetry data at the beginning of 2013, voxelizing a mesh of 150 million polygons, which we rendered and changed in real time on an average gaming laptop in 2008. After voxelization, the number of polygons of the original geometry becomes unimportant, and changing the model is a simple task. This can be seen in our videos; performance is more dependent on the number of pixels than the number of polygons of the source data. All this is necessary to provide users with a comfortable workflow while cleaning up huge scanned models. In addition, voxels are in some ways similar to pixels,
Будущее VR и стриминга
GIF
Representation of space in the form of voxels is a necessary requirement for creating huge networked interactive virtual environments that at some point will begin to resemble the real world. The key point is that such environments are not static and allow users to interact with them in a familiar way in the real world. That is, such virtual environments should be fully simulated and with convincing physics. Simulation and rendering of a complex, fully dynamic world requires such worlds to be volumetric, and all processing, synchronization and rendering of the simulated geometry must be effective. The volumetric nature of such a process precludes the use of polygons. In addition, as mentioned above, other vector forms of representation are not effective enough to create high quality at low cost.
We expect that such huge simulated environments will consist mainly of voxels, and only a small fraction will be simulated using particles to create some effects. Remarkably, both of these representations are simple and sampled, and that the conversion from one to the other will be trivial and cost-effective in terms of computation.
GIF
These are the huge advantages of voxels over other presentation methods, making them an ideal solution for describing such gigantic virtual environments. The LOD system is great for optimizing processing and rendering on the fly depending on any combination of parameters (distance to the user, importance of the simulated process, accuracy / cost ratio, etc.). Also thanks to this, they are ideal for foveated rendering and very efficient streaming. They can also be specified with a larger dimension, which is necessary to perform large-scale distributed physical simulation. Such simulations cannot be performed in only three dimensions due to network delays and the inability to process a large model on one PC and a one-time transfer of a large part of the geometry. Segmented a variable voxel geometry LOD system will help a lot here. When changing a small part or several parts of a large voxel model, there is no need to recalculate its large mesh and textures, as well as synchronize the entire model over the network - only the involved parts are important, which can be synchronized with the most suitable for transmission over the LOD network.
All these requirements for the huge virtual environments of the future lead to the necessity and inevitability of a paradigm shift in the direction of volumetric sampled geometry. The properties of our voxel technology make it the best and only candidate for a paradigm shift in the near future. Atomontage
Development Team . Interview conducted by Kirill Tokarev.