Neural networks have learned to color black and white videos



    Couple of weeks ago on Geektimes the material was published that neural networks learned to paint black and white photos. Neural networks have long been able to do a lot, not only work with photos. But this is probably one of the most illustrative demonstrations of network capabilities. Even more significant is the “customization of images” of AI DeepDream from Google, although the resulting images are somewhat ... psychedelic , so to speak .

    Now the neural network has been taught to color black and white films. This was reported by the user mar_cnu . The appropriate tool for colorization of black and white films created by his efforts. And although the result can not be called perfect, everything is pretty good.

    He chose the trailer of the old French film from 1964 as the source material. Here is the source fragment:



    The developer explained his choice to the fact that the trailer has everything you need - jungle, river, people, houses, roads. Here's what happened after video processing:



    As you can see, the result is not perfect, as mentioned above. But one of the reasons for this may be the original video itself. It is fuzzy, like all the films of those times, often on the “picture” everything merges, so it is logical that the program has difficulties.

    All this was done as follows:

    1. Found the original video;
    2. The source is converted using this service here ;
    3. Run the script

    for file in video / *
    do
    echo "$ file"
    th colorize.lua "$ file" "color - $ {file% .jpg} .png"
    done

    4. ffmpeg -i "color-frame% 04d. png "-c: v libx264 colorvideo.mp4
    5. ffmpeg -i blackandwhitevideo.webm audio.mp3
    6. 6) ffmpeg -i colorvideo.mp4 -i audio.mp3 -c: v libx264 -c: a copy -strict

    For the service, as users have rightly noted, the most difficult task is to implement a smooth transition between individual frames. If you put the video on pause, it is often possible to “catch” a frame with almost perfect colors. But to make a smooth transition between frames is a separate complex task.
    Tags:

    Also popular now: