The Hermitage Virtual Hall is the first step towards the future according to Pelevin
We are stuck with digitizing the Hermitage hall for about a month. Initially, we thought that the existing solutions for the transfer of objects in 3D (such as laser scanners) would work fine and the question is purely technical. In fact, it turned out that this area is not very well developed and the pitfalls there the sea.
For the project, they got to the FMX conference on immersion technologies and graphics. Russian developers come to this conference for the first time. We stand together with Pixar and the guys who made Game of Thrones. Not that we are as cool as Pixar, just aggregate all the best that was on the market, the best practices in photogrammetry and experienced it all applied. It turned out that no one else had done so.
But let's start with the fact that it can be difficult to digitize a hall with a statue. Business for a couple of days, right?
Task
There is the Hermitage Museum in St. Petersburg.
They had an idea to make digital copies of objects 3-4 years ago. And the objects themselves ruthlessly put in the hall. From the point of view of a museum worker, this is almost the same as to destroy. Just the process is delayed. Actually, no, but the understanding that you need to copy objects to preserve their images is there.
We drove to them for the first time, shot statues on cameras to test the photogrammetry technique. This type is like a 3D scanner, only photos are combined from all sides of the object (about 400 photos per statue), then the software searches for similar nodes, and then a cloud of points is built.
It turned out crooked and askew.
In general, we have postponed the idea of photos to the stage “we will return when there will be normal Data Mining in about five years.” Two passed, and the Hermitage needed a fully digital hall.
Why does the Hermitage need a virtual room?
To brightly show themselves on the world stage. Seriously, we were explained many clever things about the role of technology in a modern museum and so on, but we understood only that no one can properly solve this problem properly. And I want to make sure that any student of a university somewhere in the Far East or in Italy can look at the beautiful. For this, he will be given virtual reality glasses and sent to the Hermitage. Virtually. First of all - in the offices of the Hermitage. Already shown in Omsk and Vladivostok.
It is important that this is interactive - in reality, you will not come close to the statue of Jupiter, and now art critics will not be able to lick it in VR. You don't look at the uncle from above, but now you can. And so on. And, of course, those who cannot reach the Hermitage themselves will now arrive in virtual reality.
The complete task is to digitize the entire museum in general. But we must begin with one hall. It must be so that it is real.
We began to use other approaches.
At first they offered a render, like in games. The artist draws, the art worker spits. Modeling from scratch does not fit, because you need a work of art by a Greek sculptor, not a modern modeler. Clear.
We tried a laser scanner. Geometry he gives a good, but the color does not fall at all.
Removed the texture on the camera, combined - it seems normal.
Art critics were extremely critical to convey the smallest nuances of the form of the statue. We wrote it ourselves: you need very high resolution or very high polygonal accuracy.
In the end, they decided to use reference measurements and photogrammetry. They began experiments in the office with different cameras: they tried SLRs and even rented a medium-format PhaseOne with a matrix of 100 megapixels. It turned out that the problem was not in the camera, but in the accuracy of the choice of points. The slightest wavering led to a blurring of the picture, and this caused distortion on the models. We shot on aperture 11, tried to close more - diffraction and blurring. The problem with the wiggler was that we were shooting the statue from the top of a 4-meter-high ladder. And she rocked.
We made almost 17 thousand photos. Began to count.
Our workstations are powerful, like a household heater, but still there is a huge calculation time for them. I had to deploy a separate infrastructure for the point cloud and get the model. We bought a GPU server, deployed the infrastructure on it.
It turned out a cloud that had to be optimized.
Hall optimized four people for two months.
Why optimized?
VR is a thing that should not be played on super powerful devices. There are three categories of devices for this project:
- Gaming home PCs (maximum resolution and quality) and museum VR installations.
- School computers. Under them already had to optimize so that the quality is not lost. This is handmade, time consuming.
- Mobile devices. Under them it took to make the most optimized models.
Optimization is also related to the fact that from a distance the plane is seen as bends. We needed to achieve optimal perception - “as in the real world.” This was followed by art critics and curators: everything in the art world is evaluated by the degree of “yes” or “no” from them.
What happened?
The Hermitage was able to show its hall with its statue abroad at important exhibitions and give students the opportunity to join the nude nature. By the way, jokes about how we optimized a month, well ... back of Jupiter, immediately set aside. And in another place he has a bandage, and this greatly saved our psyche.
The hall and the statues are not transportable (more precisely, it is incredibly undesirable to carry the statue). Now you can virtually carry around the world. The Hermitage is now competing with the Louvre, because the museum is for everyone - and this is very cool.
After optimization, the high-field model consists of 120 thousand polygons (it has 3 levels of detail depending on the distance to the viewer, at least 12 thousand polygons). Next - rougher, closer - better.
As a result, it turned out that there are no other projects where the whole hall would be optimized for BP. There are snot, clean scanned without optimization, curves and oblique in different places. But there is no one who works. In general, we have the highest quality in the world today. Because the only such full.
In the real world, with trembling people and swaying elevators for cameras without billions of costs for it - very cool. Yes, the first hall was very long, more than 4 months. But we stepped on all the rakes, worked out the methodology and are ready to make the hall in about 1.5–2 months.
Oh, and a moment from the Russian reality. After the Hermitage began to distribute a VR tour (free), in Vladivostok, some clever people started selling tickets to the Hermitage hall in virtual reality.
For any questions you can write in the comments or email - infoVR@croc.ru