Script collector quotes and recognition of text from video to Python

A post about a script that downloads videos from youtube and recognizes text in it. I decided to start right away with practical implementation. "Vdudictionary" - Script collector quotes collection of characters "Vood" in Python. Yury Dud and his project “Vood” need no introduction. The hottest interviews that are fun to watch. Yuri Alexandrovich is able to make an interesting show, regardless of whether you know the hero of a particular release, are you a fan, or hear this name for the first time.

How many cm do you have? What do you say to Putin? Do you listen to OXY?

These and many other questions are now associated with Dud. When an interviewee utters a phrase full of wisdom, caring editors carefully display it on the screen of our monitors with you in order to convey the whole point to us. My goal was to crystallize this wisdom of generations and create the dictionary “Vududexicon” or “Vdudictionary”.

Naturally, a person, even if he is not burdened with a specific IT background, should not collect these sayings with his hands. To do this, I sketched a script in python.



First of all, we need a file that we will process. For downloading videos from Youtube, I used the pytube module.

pip install pytube

An example of downloading a file from youtube

from pytube import YouTube
a=YouTube('https://www.youtube.com/watch?v=RNbXm8WKmow')
a.streams.first().download()

File downloaded. Now we will begin to methodically identify frames with a concentrated meaning, with the wisdom of contemporaries, with the sayings of the heroes of our time.



In the old issues there was no rectangular plate, so we can search for text at the bottom of the screen. In new releases, you can use the same good old OpenCV to search for a rectangle, which will allow you to get a frame from the video.

a,contours,h = cv2.findContours(gray3, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
for i in contours:
    cv2.drawContours(gray3,[i],0,(0,0,255),1)

To install cv2 for python3 on raspberry3 I had to install many packages due to dependencies. Perhaps some packages are redundant, this is because of my inexperience, this is how it all started.

sudo apt-get install build-essential cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get -y install libopencv-dev
sudo apt-get -y install build-essential checkinstall cmake pkg-config yasm
sudo apt-get -y install libtiff4-dev libjpeg-dev libjasper-dev
sudo apt-get -y install libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev
sudo apt-get -y install python-dev python-numpy
sudo apt-get -y install libtbb-dev
sudo apt-get -y install libqt4-dev libgtk2.0-dev
sudo apt-get -y install libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev
pip install opencv-python

We put tesseract - this is an OCR engine, necessary for optical text recognition.

sudo apt-get install tesseract-ocr
sudo pip3 install pytesseract
sudo pip3 install tesseract

The releases use a very specific headset, which makes it difficult to recognize. In general, to install Cyrillic Dictionary . Download and transfer to / usr / share / tesseract-ocr / tessdata.



The input script gets the clip address on youtube. Downloads a file, starts processing one frame at a time in 5 seconds. If there is a square in the frame, cut it out, discolor it, increase the contrast and recognize it. If the string is less than 15 characters, do not consider it. You can, of course, use a string of less than 15 characters, but as one of the heroines of the program said: “
I don't know, boys, how you live with small strings.

Log in the log file text, time and a link at the time on youtube. We skip 5 seconds (do not ask why it was this figure that first came to mind, checking did not find the overlay of two quotes within this time). You can delete the video file and proceed to the next release.

Full script code:



The script obviously has some problems with the recognition of the “specific” WILD-font. I see the solution to this problem in the finalization of the dictionary file for OCR and in the post-processing of text through PyEnchant.

With a slight refinement, this script can be used to search for embedded subtitles, their recognition and automatic translation into another language.

If you can contribute to Yuri finding out about this experiment, please do so without delay. #habr #vdudictionary VK , FB .

Thanks for attention! The script and the post came out as a result of the flight of fancy when exploring OpenCV for my robot project for collecting golf balls .