Nvidia Neural Network Turns Simple Sketches Into Beautiful Landscapes


    Smoker's Falls and the Falls of a Healthy Person

    We all know how to draw an owl. First you need to draw an oval, then another circle, and then you get a gorgeous owl. Of course, this is a joke, and it’s very old, but the Nvidia engineers tried to make the fantasy a reality.

    The new development , called GauGAN, creates gorgeous landscapes from very simple sketches (really simple ones - circles, lines and all). Of course, this development is based on modern technologies - namely, generative adversarial neural networks.

    GauGAN allows you to create colorful virtual worlds - and not only for entertainment, but also for work. So, architects, landscape designers, game developers - all of them can learn something useful. Artificial intelligence immediately “understands” what a person wants and complements the original idea with a huge amount of detail.

    “Brainstorming in terms of design development is made much easier with the help of GauGAN, since a smart brush can complement the initial sketch by adding high-quality images,” said one of the developers of GauGAN.

    Users of this tool can change the initial idea, modify the landscape or other image, add the sky, sands, sea, etc. Everything that your heart desires, and the addition takes place in just a couple of seconds.

    The neural network was trained using a database of millions of images. Thanks to this, the system can understand what a person wants and how to achieve what he wants. Moreover, the neural network does not forget about the smallest details. So, if you sketch the pond and some trees next to it, then after the landscape is revived, all nearby objects will be reflected in the mirror of the pond water.

    The system can be specified what the visible surface should be - it can be covered with grass, snow, water or sand. All this can be transformed in a second, so that the snow becomes sand and instead of a snowy wasteland the artist will receive a desert landscape.

    “It's like a coloring book that says where to place a tree, where is the sun, and where is the sky. Then, after the initial task, the neural network enlivens the picture, adds the necessary details and textures, draws reflections. All this is based on real images, ”says one of the developers.


    Despite the fact that the system lacks an “understanding” of the real world, the system creates impressive landscapes. This is because two neural networks, a generator and a discriminator, are used here. The generator creates an image and shows it to the discriminator. That, based on millions of previously seen images, chooses the most realistic options.

    That is why the generator “knows” where the reflections should be. It is worth noting that the tool is very flexible and equipped with a large number of settings. So, with its help you can paint pictures, adjusting to the style of a certain artist, or just indulge in the quick addition of sunrise or sunset.

    The developers claim that the system does not just take images from somewhere, adds them together and gets the result. No, all received “pictures” are generated. That is, the neural network "creates" as a real artist (or even better).

    So far, the program is not in the public domain, but soon it will be possible to test it in work. This can be done at the GPU Technology Conference 2019, which is being held in California right now. The lucky ones who were able to visit the exhibition can already test GauGAN.

    Neural networks have long been taught to take part in the creative process. For example, last year, some of them could create 3D models. In addition, developers from DeepMind trained the neural network to restore three-dimensional spaces and objects from drawings, photographs, sketches. In order to recreate a simple figure, a neural network suffices with one image; to create more complex objects, five images are needed for “dragging”.

    As for GauGAN, this tool will obviously find worthy commercial application - many areas of business and science have a need for such services.

    Also popular now: