OpenCV: setting a timeout to wait for a frame in the VideoCapture class

Good day to all! Somehow a task appeared: to reproduce the RTSP video stream from the camera. Because I am familiar with the OpenCV API, it was decided to use it. OpenCV uses the VideoCapture class to capture a video stream. Unfortunately, the network often breaks off with us, and this problem cannot be solved at my level, therefore, a rather quick reaction to a drop in the video stream became a prerequisite for comfortable work - the standard timeout for connecting and waiting for the next frame is 30 seconds, and inside VideoCapture, calls are open ( ) and read () are blocking, which forces you to write various wrappers around code that is actually simple, like calling them in a separate thread and waiting for the result to be received in asynchronous mode. Naturally, I didn’t feel any joy about this - all these are resources, which the program should have gone to other purposes, not to mention the complication of the code. It was decided: to change the standard timeout, or add the ability to install it externally. It turned out to be a rather dirty hack, which, however, can be useful to someone. Perhaps if there is a better way - if there is one - I would really like to know it, so I ask for comments. Ideally - maybe among the readers of Habr there are OpenCV developers who will pay attention to this problem. The goal was to make the code "work as it should under Windows x64." if there is a better way - if there is one - I would really like to know it, so I ask for comments. Ideally - maybe among the readers of Habr there are OpenCV developers who will pay attention to this problem. The goal was to make the code "work as it should under Windows x64." if there is a better way - if there is one - I would really like to know it, so I ask for comments. Ideally - maybe among the readers of Habr there are OpenCV developers who will pay attention to this problem. The goal was to make the code "work as it should under Windows x64."

Who cares - I ask for a cat.

Research

With the help of Google, it was found that this problem rests on ffmpeg - there are callbacks that provide information about disconnecting. A timeout of 30 seconds was set in pull request # 6053. The problem was added as follows: at the moment, the cmake-collector downloads the opencv_ffmpeg.dll file instead of assembling it in place, and the assembly instructions with ffmpeg have disappeared. Code with timeout constants (which, at least on Windows does not compile in any way) is located in the modules / videoio / src / cap_ffmpeg_impl.hpp file:

#define LIBAVFORMAT_INTERRUPT_OPEN_TIMEOUT_MS 30000#define LIBAVFORMAT_INTERRUPT_READ_TIMEOUT_MS 30000

The task, thus, was formed as follows:

1. Make this file build under Windows;
2. Change constants in it;
3. Make sure there are no problems at work.

Without thinking twice, I will talk about how it was solved. First you need to download the latest development version of ffmpeg with the header files, dll, and lib, and put it all in the right places in source / 3rdparty - for the sake of convenience, you can actually put it anywhere. Next, you need to make the following changes to the OpenCV source files:

modules / videoio / src / cap_ffmpeg.cpp:

Comment out lines in the inclusions regarding the potential inclusion of cap_ffmpeg_api.hpp

//#if defined HAVE_FFMPEG && !defined WIN32#include"cap_ffmpeg_impl.hpp"//#else//#include "cap_ffmpeg_api.hpp"//#endif

In icvInitFFMPEG () function

...
#ifdef WIN32...//все закомментировать, включая директивы препроцессора#elif defined HAVE_FFMPEG//оставить только то, что есть внутри#endif
...

What happened here - we abandoned the use of ffmpeg functionality loading from opencv_ffmpeg.dll using LoadLibrary, and switched to the internal implementation, which is located inside the cap_ffmpeg_impl.hpp file.

modules / videoio / src / cap_ffmpeg_impl.hpp:

Find the above constants, change them to the desired ones. In my case -

#define LIBAVFORMAT_INTERRUPT_OPEN_TIMEOUT_MS 2000#define LIBAVFORMAT_INTERRUPT_READ_TIMEOUT_MS 1000

This file, in general, was not created for assembly under Windows, therefore, some of the functionality required to compile it is missing in it - we are talking about snprintf. The code was pulled from somewhere with StackOverflow. Paste in any convenient place.

#if defined(_MSC_VER) && _MSC_VER < 1900#define snprintf c99_snprintf#define vsnprintf c99_vsnprintf
__inlineintc99_vsnprintf(char *outBuf, size_t size, constchar *format, va_list ap){
	int count = -1;
	if (size != 0)
		count = _vsnprintf_s(outBuf, size, _TRUNCATE, format, ap);
	if (count == -1)
		count = _vscprintf(format, ap);
	return count;
}
__inlineintc99_snprintf(char *outBuf, size_t size, constchar *format, ...){
	int count;
	va_list ap;
	va_start(ap, format);
	count = c99_vsnprintf(outBuf, size, format, ap);
	va_end(ap);
	return count;
}
#endif

After making the changes, you need to force all this stuff to compile. To get started, you need to generate * .sln for OpenCV using CMake. An attempt to build opencv_videoio at the moment will not work. Correctable. Open the Visual Studio project for the videoio module: build / modules / videoio / opencv_videoio.sln. Add ffmpeg header files to include directories, add it * .lib to the link.

As soon as opencv_videoio is built, this dll can be easily laid in place of the "standard" one. Remember that now all the dlls from the ffmpeg delivery will also be needed to work - the application will not work without them.

An important point: now, for this to work, cv :: CAP_FFMPEG must be specified as the VideoCapture backend.

The result at the moment is the flight is normal, I did not notice any bugs for the month. However, given all of the above, they are more than possible, so use only at your own peril and risk. If there are other ways to achieve the desired, as I said, I will be very glad to listen.

Thanks for attention.