Tool to quickly find scenes in video files

    Background


    image
    Once I argued with my friend about one movie. He claimed that the main character says one thing, but I thought that he says the exact opposite. As a result, we bet on the N-th number of alcohol-soft drinks and went to a mutual friend who was the happy owner of a VHS video player and who should have had this same movie. I lost this argument, but I remember very well that we spent a lot of time searching for this particular scene. Years later, these days, another friend of mine wrote me a video from the recorder to a flash drive saying that it’s there, after some kind of turn to the left our common acquaintance (an unpleasant type) was supposed to get into the device’s lens, who was in a wonderful condition and suggested that I joke about this. To look for someone’s physiognomy in a couple of hours long video, of course, not just looking for a needle in a haystack, but, too, an amateur pleasure. It seems that it was then that an idea began to ripen in my head and I wondered:How can I watch several hours of video recordings to find what you need there and at the same time not really limit yourself in a dream?

    main idea


    As a result of reflection, I came to the conclusion that the easiest way to search is to use the tabular presentation of the video: each cell has its own time interval.
    image

    Tabular presentation of the video: what kind of animal is it and what does it eat with?


    Video examples in tabular format

    Candies
    parking
    road

    Table Sample Images

    Candies
    parking
    road

    Actually, the tool itself


    After some torment, I wrote the program Table View Video Player .image

    The main features of the program


    • Play video in tabular form
    • The ability to save video in a tabular form (for viewing on a tablet or simply on another computer)
    • Customizable number of rows and columns in the table, as well as the size of the resulting frame in pixels
    • List of files (for quick switching between videos, and also if the event consists of several files, for example, VOB-files on DVD or recording from a video recorder)
    • Saving an index image
    • Batch create video and index images for all files in a list
    • Some transformations of the original video during playback (rotation of 90, 180, 270 degrees, reflection vertically and horizontally)
    • Ability to select a specific area of ​​the source video
    • The ability to play the source video in normal mode from a given position in the built-in player or in another program

    And what's under the hood?


    The program is written in C ++, MS Visual Studio 2015 was used. FFMPEG could be limited to video libraries, but I used OpenCV (I plan to use other features of this library).

    Scope of this tool


    I will try to show everything with an example. I put a video camera in the kitchen and hid the candies on the kitchen wall cabinet. Video duration 2 hours. Now look at the picture and tell me who ate them: dad, mom or one of the children? Even if there were no selection of the necessary areas, everything would become clear to everyone after 2 minutes of viewing, or maybe earlier. imageIf someone wants to conduct an experiment, then here is the link to the source video , and here is the link to the video in tabular form . So, the main purpose of the program is to find out who stole candy visual analysis of the video .

    Some more practice


    On holidays, I shoot videos in kindergarten, as well as family anniversaries, weddings of friends and relatives. Pinnacle Studio is a great video editing program! They even say that housewives easily understand it! I don’t understand the truth, where do housewives get so much perseverance and free time to spend hours trying repeatedly to export the mounted video to a file or to a DVD-ROM? I myself have been using this program since the 8th version (now I use the licensed 17, and the last released is the 19th version), but, in spite of my experience, it rarely turns out that the application does not hang during editing or export. But even if the export was successful (according to Pinnacle Studio), when watching a video in a VLC player or Windows-Media player, sometimes artifacts are found that look confusing to users of my products. Well! Table View Video Player can also be used here! We start the program, open our file and without concentrating on any fragment, we are just trying to feel where a frieze or some kind of junk suddenly appears. This type.imageFive minutes to search for video artifacts is not so much! In this case, re-open the project and try to fix the problem. And we are a little happy that we discovered this ourselves, and not “ happy parents ” during a family viewing.

    Step back


    Some time ago, I boasted of my program to a friend, to which I received a comment that the program was too complicated and personally he would never have figured it out. Thinking, but I thought ... Considering constructive criticism, I realized that perhaps he was right. So there was a need to create a simpler (Lite) version.image

    The main features of the program


    • Play video in tabular form
    • Resizable table
    • List of files (for quick switching between videos, and also if the event consists of several files, for example, VOB-files on DVD or recording from a video recorder)
    • Ability to play the original video in normal mode from a given position in the built-in player.
    Compared to the standard version, many features have been removed, but the program has become simple as 2 pennies !

    Scene and Object Search Theory


    Existing automatic solutions for searching a video resource-intensive archive, in addition, it is not always possible to clearly state what you are looking for. In this case, the use of manual search (I mean using the operator), if you accelerate it using table playback, can be very effective.

    Strengths


    • Search for some events in a relatively static picture. For example, recording visitors in the lobby. If there are not many, then finding the right person is pretty easy. Here, the search can be accelerated dozens of times!
    • Search for scenes that have a contrasting color gamut or illumination in a dynamic video. For example, there is a film shot mainly in the room, but there is one or more scenes shot on the street on a clear sunny day. It is required to find them. In this case, they are very quickly and in person.

    Weaknesses


    • Search for specific people in the crowd. For example, in the subway at rush hour. Here, the table view will not help, it is better to watch the video in normal mode (less likely to miss the search) or use specially sharpened automatic tools.


    Competitors or programs to search for scenes and objects


    I can’t name direct competitors, but there are several programs that do something similar. The list is far from complete, some of the links I can not find. But I would be grateful for the addition ...

    Future plans


    • Speeding up video playback. Already now, video in VGA resolution on a modern computer can be viewed in a 10x10 table with a frequency of 20-30 FPS. However, video from modern photo and video cameras, action cameras and video recorders has a resolution of Full HD or even higher and just around the corner 4K. Therefore, you can’t do without using a GPU.
    • The ability to save the information found in the form of images (freeze frame), audio or video fragments of points of interest for later use. For example, for editing a video (we collect funny stories from a party) or for presenting evidence in court (if the video surveillance camera captured the moment of the theft).


    conclusions


    • There are practically no tools for searching scenes and objects in video for the average user today.
    • The applications Table View Video Player and Table View Video Player Lite are trying to at least fill this niche.
    • For certain types of scene search in video, the table view method is very effective.
    • Despite the fact that simultaneous playback of several dozens of video streams requires high computer performance, with low resolution (SD quality) of the original video, the task is successfully solved by modern PCs.

    Only registered users can participate in the survey. Please come in.

    Do you think that programs for searching for scenes and objects in a video could be useful in your work?

    • 71% Yes 140
    • 28.9% No 57
    • 0% already using 0

    Would you like to continue the series of articles on this topic?

    • 80.7% Yes 176
    • 3.2% No 7
    • 16% I don't care 35

    Also popular now: