We draw the rest of the owl on the basis of neural networks
The idea to draw an owl from circles is implemented using neural networks.
Based on machine learning and a neural network, developer Christopher Hesse created an online service that can "finish" sketches of full color photographs.
On the website http://affinelayer.com/pixsrv/index.html Christopher Hesse presents a neural network capable of drawing cats.
The user is invited to draw a sketch of a cat in the left window, click on “process” and see how the neural network completes the illustration. The page shows such a beautiful example:
Apparently, the more lines used in the sketch and the more the outline resembles a cat from a training sample, the better the neural network will create a full-fledged drawing based on it.
Here is what the author himself writes:
Recently, I made the Tensorflow port pix2pix on Isola , etc., discussed in the article Image-to-Image Translation in Tensorflow . I took some previously created models and made an interactive web page for testing. Chrome browser is recommended for viewing.
The pix2pix model works by training on pairs of images, such as a sketch of a building's facade and a full image of building facades, and then trying to generate a corresponding output image for any input transmitted image. This idea originates fromarticle pix2pix , which is recommended for reading.
Facades The
model was trained on samples of sketches of facades of buildings for full-fledged facades. It is unlikely that this model will work on sketches with a large empty area, but if you draw a sufficient number of windows in the outline, the model often gives good results. In the sketch picture, the elements of the facade are drawn with colored rectangles to indicate not only the borders, but also the entire elements.
I did not have the names of the various parts of the facades of the buildings, so I just roughly outlined them.
edges2cats
The training took place on approximately 2000 photographs of cats and sketches with automatically generated borders for these photographs. The model creates color images of cats from sketches, but some are confronted with nightmarish results. One of those I saw here .
Some pictures look especially creepy, I think, because cats were painted wrong, especially from behind the eyes. The procedure for autogeneration of boundaries was not very high-quality, and in many cases did not detect the cat's eyes, which degrades the quality of the prepared image base for training the model.
edges2shoes The
training was conducted on the basis of approximately 50 thousand pictures of shoes collected from Zappos, as well as automatically generated border sketches for these images. If you’re really good at drawing the boundaries of your shoes, you can try creating some new designs. Keep in mind, the model was trained on real objects, so if you can draw a better 3D sketch, the result will look better.
edges2handbags
By analogy with the previous ones, the training was conducted on the basis of approximately 137 thousand photographs of bags collected from Amazon, with automatically generated border sketches for these photographs. If you draw a shoe here instead of a handbag, you will get a very strange texture of the shoe.
Implementation
Training and export of models was performed using pix2pix.py script from pix2pix-tensorflow. An interactive demo is made based on JavaScript using the Canvas API, which interacts with the server, which sends images to Tensorflow. The server can run Tensorflow itself or forward requests to Google Cloud ML with Tensorflow.
Trained models are available in the Datasets section on GitHub. Also should be available models coming with the original implementation of pix2pix. Models can be exported from trained examples using the pix2pix.py script and links to exported models are in the README file on the GitHub server.
Borders for cat photos were obtained using the Holistically-Nested Edge Detection algorithm and this functionality was added to the script.process.py and corresponding dependencies have been added to the Docker image .
Based on machine learning and a neural network, developer Christopher Hesse created an online service that can "finish" sketches of full color photographs.
On the website http://affinelayer.com/pixsrv/index.html Christopher Hesse presents a neural network capable of drawing cats.
The user is invited to draw a sketch of a cat in the left window, click on “process” and see how the neural network completes the illustration. The page shows such a beautiful example:
Apparently, the more lines used in the sketch and the more the outline resembles a cat from a training sample, the better the neural network will create a full-fledged drawing based on it.
Here is what the author himself writes:
Recently, I made the Tensorflow port pix2pix on Isola , etc., discussed in the article Image-to-Image Translation in Tensorflow . I took some previously created models and made an interactive web page for testing. Chrome browser is recommended for viewing.
The pix2pix model works by training on pairs of images, such as a sketch of a building's facade and a full image of building facades, and then trying to generate a corresponding output image for any input transmitted image. This idea originates fromarticle pix2pix , which is recommended for reading.
Facades The
model was trained on samples of sketches of facades of buildings for full-fledged facades. It is unlikely that this model will work on sketches with a large empty area, but if you draw a sufficient number of windows in the outline, the model often gives good results. In the sketch picture, the elements of the facade are drawn with colored rectangles to indicate not only the borders, but also the entire elements.
I did not have the names of the various parts of the facades of the buildings, so I just roughly outlined them.
edges2cats
The training took place on approximately 2000 photographs of cats and sketches with automatically generated borders for these photographs. The model creates color images of cats from sketches, but some are confronted with nightmarish results. One of those I saw here .
Some pictures look especially creepy, I think, because cats were painted wrong, especially from behind the eyes. The procedure for autogeneration of boundaries was not very high-quality, and in many cases did not detect the cat's eyes, which degrades the quality of the prepared image base for training the model.
edges2shoes The
training was conducted on the basis of approximately 50 thousand pictures of shoes collected from Zappos, as well as automatically generated border sketches for these images. If you’re really good at drawing the boundaries of your shoes, you can try creating some new designs. Keep in mind, the model was trained on real objects, so if you can draw a better 3D sketch, the result will look better.
edges2handbags
By analogy with the previous ones, the training was conducted on the basis of approximately 137 thousand photographs of bags collected from Amazon, with automatically generated border sketches for these photographs. If you draw a shoe here instead of a handbag, you will get a very strange texture of the shoe.
Implementation
Training and export of models was performed using pix2pix.py script from pix2pix-tensorflow. An interactive demo is made based on JavaScript using the Canvas API, which interacts with the server, which sends images to Tensorflow. The server can run Tensorflow itself or forward requests to Google Cloud ML with Tensorflow.
Trained models are available in the Datasets section on GitHub. Also should be available models coming with the original implementation of pix2pix. Models can be exported from trained examples using the pix2pix.py script and links to exported models are in the README file on the GitHub server.
Borders for cat photos were obtained using the Holistically-Nested Edge Detection algorithm and this functionality was added to the script.process.py and corresponding dependencies have been added to the Docker image .
