May 9, 2013 at 15:36Introducing GStreamer: Data Sources
- Tutorial
Hello, a habrayuzer who is interested in the GStreamer framework! Today we’ll talk about sources of media data (sources), and thereby continue the course of articles on GStreamer.
Previous article: Introducing GStreamer: Introduction .
Data Sources is a class of GStreamer plugins that allows you to read media data from various sources, such as the file system or audio inputs of a sound card. Also, they allow you to receive media stream from various streaming servers, such as HTTP (ICECast, ShoutCast), RTSP, RTMP, TCP and UDP. And there is also the opportunity to read data from DVB cards, CDDA-disks (popularly known simply as "CDs"), and much more, with the help of various plug-ins, which are currently about 30.
Note: as mentioned in a previous article, data sources have only one pad called src, since it can be connected to another element, but nothing can be connected to it.
In this article we will analyze some (perhaps the most popular) data sources, write some Python code and learn a lot.
The gst-launch-1.0 utility allows you to start the GStreamer pipeline without writing a single line of code. I will use it to run small examples, and what we need to know today is that the pipeline launch looks something like this:
and the pipeline description, in turn, is divided into a description of elements of the form
It seems that everything is clear in this scheme; there is an element of type element1 with properties property1 and property2 which have values value1 and value2 respectively, and there is an element of type element2. The “!” Symbol indicates that the output of element1 must be connected to the input of element2.
The possibilities of gst-launch are not limited to this, they simply will not be useful to us in today's article and will be considered in the future.
filesrc - in my opinion, this is the most commonly used source of multimedia data. As its name implies, it is an element for reading data from files. In the last article, we looked at the scheme of a primitive player, in which filesrc appeared as a data source.
The filesrc element has several parameters. Let's consider some of them.
The location property must contain the path to the file from which to read.
Example: /foo/bar.mp3
This parameter sets the size of the read buffer. The value must be described with an unsigned number.
The number of blocks after reading which an EOS message will be sent.
The blockize and num-buffers properties can be omitted. In this case, the default values will be used. And the indicated properties are available at all data sources, and will not be considered further.
An example of the use of the filesrc source is taken from the previous article:
The souphttpsrc element provides the ability to read data over the HTTP (HTTPS) protocol. Reading can be done either from a regular file accessible via the HTTP protocol, or from an ICECast / ShoutCast server. This element also allows reading data over the ICY and ICYX protocols. According to these protocols, I could not find any information.
souphttpsrc has slightly more parameters than filesrc, let's consider the most important ones.
This parameter is similar to the filesrc element parameter of the same name, except that only http, https, icy or icyx URI schemes are supported.
The name speaks for itself - this parameter sets which User-Agent to use when sending HTTP requests.
Sets the automatic transition in case the server replies that the resource has been moved (statuses 301 and 302).
These properties set the username and password, in case the resource requires authorization (HTTP Basic Authentication).
These properties set the proxy server address, username and password for authorization (if necessary). HTTP and HTTPS proxies are supported.
Using the souphttpsrc source is no different than using the filesrc source:
The GStreamer plugin suite has an interesting giosrc element. This is almost the same as filesrc, it only works through the GIO library , and allows you to receive data from various sources with which GIO can work. There are 22 such sources, from file to mtp. Also, this element supports ftp, sftp, smb and webdav protocols.
This element has parameters similar to the filesrc element. An example of using this source is similar to using the previous ones:
As the name implies, this element allows you to read data from the RTSP server. We will not consider in detail the properties of this element, I will only say that it has a location property for specifying the address of the data source, proxy, proxy-id and proxy-pw for specifying proxy settings, and with dozens of properties specific to the RTSP protocol, the consideration of which is not included into our plans.
But, rtspsrc has one feature, it has several pad-s, and they have names suitable for the source_% u pattern. This is due to the fact that the server can broadcast several streams simultaneously, for example, audio and video can be scattered across two streams.
Note:
I am not familiar with the RTSP protocol, so I could be mistaken in the assumption above.
An example of using rtspsrc is similar to the previous elements:
The essence of the multifilesrc element is the same as the filerc element - to read files from disk, but there is one difference - multifilesrc reads files sequentially (1.mp3, 2.mp3 ...), while filesrc can read only one file . This element has one drawback - file names should differ only in number, for example, file-1.mp3, file-2.mp3 and so on. But such a flaw can be attributed to the category of features, because with the help of this “flaw” you can easily create a video clip from images 0000.png-9999.png from photos, or a melody from samples, or a mix from tracks.
The multifilesrc element has few parameters, and oddly enough - they are all useful. Let's consider them in more detail.
The file path can (and should) include the % d escape sequence , which will be automatically replaced using the sprintf () function with a number equal to the current index.
Example: /foo/bar-%04d.mp3
These properties contain the current index, start index, and final index. These properties contain an integer.
This property may contain a Boolean value, and if it is true, the reading of the files will be looped. Roughly speaking, this is an analogue of the “Repeat All” function.
An example of using the multifilesrc element that reads files with names starting from bar-1.mp3 with the loop option enabled:
In my opinion, the most useless element. Reads data from a file descriptor. It can be used when too lazy to describe filesrc for gst-launch.
This element has one parameter:
This parameter must contain a number equal to the open file descriptor. The default is zero.
Usage example:
As the name implies, this element does nothing - it is simply a dummy source of empty buffers.
These elements are designed to receive the stream from the microphone input of the sound card. From the names it follows that they are used to work with various sound subsystems, such as Alsa, PulseAudio, OSS, Jack. The autoaudiosrc element is a “wrapper” over the rest of the elements, and simplifies development by taking all the work of choosing a sound source onto your shoulders.
The properties of these elements are not very many. The main property is device , it is present in the first three elements and should contain the name of the device (hw: 0, hw: 1 ...). The remaining properties are used specific for each sound subsystem, and we will not consider them.
This element is intended for opening a video stream through the Video4Linux2 interface . The V4L2 interface supports most webcams, TV tuners, DVB cards and video capture cards.
Consider the most important parameters:
Indicates the name of the device from which to capture. Find out the list of available devices using
The value of this parameter must contain the full path to the device.
Example: device = / dev / video0
These parameters are responsible for color balance, saturation, contrast and brightness, respectively. Of course, if the video capture device does not support these parameters, then the image will be what the device gives it. The values of these properties must be in the range from -2147483648 to 2147483647.
An example of the use of an element:
This element is similar to the fakesrc element, but differs in that it generates a real audio signal. It can be used both for testing complex pipelines launched through gst-launch, for testing the pipeline in a written application, and it can also be used as a reference generator for measuring the frequency response and distortions introduced by various filters.
Consider the most interesting parameters of this element.
This parameter sets the waveform. 13 values are available: sine, square, saw, triangle, silence, white-noise, pink-noise, sine-table, ticks, gaussian-noise, red-noise, blue-noise, violet-noise .
It is not difficult to guess that this parameter sets the signal level (volume). Valid values range from 0 to 1.
This property sets the frequency of the signal, and the value can range from 0 to 20,000.
An example of using the audiotestsrc element:
This element, like audiotestsrc, is used to generate test data, namely, a video stream. It can be used either to check the operability of the entire pipeline, or to check for distortions that are introduced by filters.
This element has a lot of parameters, we will consider only one, the most interesting in my opinion.
This parameter sets the video signal pattern. The valid value can be one of 21. Full list: smpte, snow, black, white, red, green, blue, checkers-1, checkers-2, checkers-4, checkers-8, circular, blink, smpte75, zone-plate , gamut, chroma-zone-plate, solid-color, ball, smpte100, bar .
I will not dwell on all the values, I will only say that the values smpte, smpte75 and smpte100 are SMPTE settings tables that are used in the settings of the TV / video equipment. Patterns checkers- * is a grid of NxN pixels, where N equals the number in the pattern name. The remaining values set a template with a monotonous image, for example, a color fill.
Usage example:
Above we examined 15 main data sources, this is about half of all available plug-ins from the source category. The remaining plugins were not considered for two reasons:
When considering plugins, many parameters were omitted, since basically they are all used for "fine tuning", and as you remember, we are getting to know GStreamer. Or, these options are read-only. To get a list of all available properties, as well as values that are valid for them, it is enough to perform
and in the "Element Properties" section you can observe all properties (including those inherited).
Today we will look at one simple example of using GStreamer. In the example, I will use Python 2.7, GStreamer 1.0, and GObject Introspection.
I’ll make a reservation right away, the code may not be ideal, and it may even be the so-called “shit code,” but its only purpose is to show how to use GStreamer, no more.
This example shows how elements are linked, how messages are processed, and the properties of elements are changed.
Examples of running this script:
In case everything goes as it should - you will hear the track you need and see a list of ID3 tags, if any.
An example is available on GitHub .
Today we examined about half of the available data sources that are available “out of the box”, examined examples of launching these elements, and also examined the example of a simple player based on GStreamer. In the following articles, we will consider the other classes of plugins, consider even more examples of using GStreamer, and of course, write more code.
See you in a week!
UPD 1: Added elements audiotestsrc, videotestsrc.
Previous article | Next article
Previous article: Introducing GStreamer: Introduction .
Introduction
Data Sources is a class of GStreamer plugins that allows you to read media data from various sources, such as the file system or audio inputs of a sound card. Also, they allow you to receive media stream from various streaming servers, such as HTTP (ICECast, ShoutCast), RTSP, RTMP, TCP and UDP. And there is also the opportunity to read data from DVB cards, CDDA-disks (popularly known simply as "CDs"), and much more, with the help of various plug-ins, which are currently about 30.
Note: as mentioned in a previous article, data sources have only one pad called src, since it can be connected to another element, but nothing can be connected to it.
In this article we will analyze some (perhaps the most popular) data sources, write some Python code and learn a lot.
Go
0. gst-launch-1.0
The gst-launch-1.0 utility allows you to start the GStreamer pipeline without writing a single line of code. I will use it to run small examples, and what we need to know today is that the pipeline launch looks something like this:
gst-launch-1.0 описание-pipeline
and the pipeline description, in turn, is divided into a description of elements of the form
element1 property1=value1 property2=value2 ! element2
It seems that everything is clear in this scheme; there is an element of type element1 with properties property1 and property2 which have values value1 and value2 respectively, and there is an element of type element2. The “!” Symbol indicates that the output of element1 must be connected to the input of element2.
The possibilities of gst-launch are not limited to this, they simply will not be useful to us in today's article and will be considered in the future.
1. filesrc
filesrc - in my opinion, this is the most commonly used source of multimedia data. As its name implies, it is an element for reading data from files. In the last article, we looked at the scheme of a primitive player, in which filesrc appeared as a data source.
The filesrc element has several parameters. Let's consider some of them.
location
The location property must contain the path to the file from which to read.
Example: /foo/bar.mp3
blocksize
This parameter sets the size of the read buffer. The value must be described with an unsigned number.
num buffers
The number of blocks after reading which an EOS message will be sent.
The blockize and num-buffers properties can be omitted. In this case, the default values will be used. And the indicated properties are available at all data sources, and will not be considered further.
An example of the use of the filesrc source is taken from the previous article:
gst-launch-1.0 filesrc location=/foo/bar.mp3 ! decodebin ! autoaudiosink
2. souphttpsrc
The souphttpsrc element provides the ability to read data over the HTTP (HTTPS) protocol. Reading can be done either from a regular file accessible via the HTTP protocol, or from an ICECast / ShoutCast server. This element also allows reading data over the ICY and ICYX protocols. According to these protocols, I could not find any information.
souphttpsrc has slightly more parameters than filesrc, let's consider the most important ones.
location
This parameter is similar to the filesrc element parameter of the same name, except that only http, https, icy or icyx URI schemes are supported.
user-agent
The name speaks for itself - this parameter sets which User-Agent to use when sending HTTP requests.
automatic-redirect
Sets the automatic transition in case the server replies that the resource has been moved (statuses 301 and 302).
user-id and user-pw
These properties set the username and password, in case the resource requires authorization (HTTP Basic Authentication).
proxy, proxy-id and proxy-pw
These properties set the proxy server address, username and password for authorization (if necessary). HTTP and HTTPS proxies are supported.
Using the souphttpsrc source is no different than using the filesrc source:
gst-launch-1.0 souphttpsrc location=http://radio.localhost:1234 ! decodebin ! autoaudiosink
3. giosrc
The GStreamer plugin suite has an interesting giosrc element. This is almost the same as filesrc, it only works through the GIO library , and allows you to receive data from various sources with which GIO can work. There are 22 such sources, from file to mtp. Also, this element supports ftp, sftp, smb and webdav protocols.
This element has parameters similar to the filesrc element. An example of using this source is similar to using the previous ones:
gst-launch-1.0 giosrc location=ftp://ftp.localhost/foo/bar.mp3 ! decodebin ! autoaudiosink
4.rtspsrc
As the name implies, this element allows you to read data from the RTSP server. We will not consider in detail the properties of this element, I will only say that it has a location property for specifying the address of the data source, proxy, proxy-id and proxy-pw for specifying proxy settings, and with dozens of properties specific to the RTSP protocol, the consideration of which is not included into our plans.
But, rtspsrc has one feature, it has several pad-s, and they have names suitable for the source_% u pattern. This is due to the fact that the server can broadcast several streams simultaneously, for example, audio and video can be scattered across two streams.
Note:
I am not familiar with the RTSP protocol, so I could be mistaken in the assumption above.
An example of using rtspsrc is similar to the previous elements:
gst-launch-1.0 rtspsrc location=rtsp://rtsp.localhost/movie.mp4 ! decodebin ! autoaudiosink
5. multifilesrc
The essence of the multifilesrc element is the same as the filerc element - to read files from disk, but there is one difference - multifilesrc reads files sequentially (1.mp3, 2.mp3 ...), while filesrc can read only one file . This element has one drawback - file names should differ only in number, for example, file-1.mp3, file-2.mp3 and so on. But such a flaw can be attributed to the category of features, because with the help of this “flaw” you can easily create a video clip from images 0000.png-9999.png from photos, or a melody from samples, or a mix from tracks.
The multifilesrc element has few parameters, and oddly enough - they are all useful. Let's consider them in more detail.
location
The file path can (and should) include the % d escape sequence , which will be automatically replaced using the sprintf () function with a number equal to the current index.
Example: /foo/bar-%04d.mp3
index, start-index and stop-index
These properties contain the current index, start index, and final index. These properties contain an integer.
loop
This property may contain a Boolean value, and if it is true, the reading of the files will be looped. Roughly speaking, this is an analogue of the “Repeat All” function.
An example of using the multifilesrc element that reads files with names starting from bar-1.mp3 with the loop option enabled:
gst-launch-1.0 multifilesrc location="/foo/bar-%d.mp3" loop=true start-index=1 ! decodebin ! autoaudiosink
6. fdsrc
In my opinion, the most useless element. Reads data from a file descriptor. It can be used when too lazy to describe filesrc for gst-launch.
This element has one parameter:
fd
This parameter must contain a number equal to the open file descriptor. The default is zero.
Usage example:
cat /foo/bar.mp3 | gst-launch-1.0 fdsrc ! decodebin ! autoaudiosink
7. fakesrc
As the name implies, this element does nothing - it is simply a dummy source of empty buffers.
8. alsasrc, pulsesrc, osssrc, jackaudiosrc, autoaudiosrc
These elements are designed to receive the stream from the microphone input of the sound card. From the names it follows that they are used to work with various sound subsystems, such as Alsa, PulseAudio, OSS, Jack. The autoaudiosrc element is a “wrapper” over the rest of the elements, and simplifies development by taking all the work of choosing a sound source onto your shoulders.
The properties of these elements are not very many. The main property is device , it is present in the first three elements and should contain the name of the device (hw: 0, hw: 1 ...). The remaining properties are used specific for each sound subsystem, and we will not consider them.
9. v4l2src
This element is intended for opening a video stream through the Video4Linux2 interface . The V4L2 interface supports most webcams, TV tuners, DVB cards and video capture cards.
Consider the most important parameters:
device
Indicates the name of the device from which to capture. Find out the list of available devices using
ls -l /dev/v4l/by-id
The value of this parameter must contain the full path to the device.
Example: device = / dev / video0
hue, saturation, contrast. brightness
These parameters are responsible for color balance, saturation, contrast and brightness, respectively. Of course, if the video capture device does not support these parameters, then the image will be what the device gives it. The values of these properties must be in the range from -2147483648 to 2147483647.
An example of the use of an element:
gst-launch-1.0 v4l2src ! xvimagesink
10. audiotestsrc
This element is similar to the fakesrc element, but differs in that it generates a real audio signal. It can be used both for testing complex pipelines launched through gst-launch, for testing the pipeline in a written application, and it can also be used as a reference generator for measuring the frequency response and distortions introduced by various filters.
Consider the most interesting parameters of this element.
wave
This parameter sets the waveform. 13 values are available: sine, square, saw, triangle, silence, white-noise, pink-noise, sine-table, ticks, gaussian-noise, red-noise, blue-noise, violet-noise .
volume
It is not difficult to guess that this parameter sets the signal level (volume). Valid values range from 0 to 1.
freq
This property sets the frequency of the signal, and the value can range from 0 to 20,000.
An example of using the audiotestsrc element:
gst-launch-1.0 audiotestsrc wave=saw freq=205 ! autoaudiosink
11. videotestsrc
This element, like audiotestsrc, is used to generate test data, namely, a video stream. It can be used either to check the operability of the entire pipeline, or to check for distortions that are introduced by filters.
This element has a lot of parameters, we will consider only one, the most interesting in my opinion.
pattern
This parameter sets the video signal pattern. The valid value can be one of 21. Full list: smpte, snow, black, white, red, green, blue, checkers-1, checkers-2, checkers-4, checkers-8, circular, blink, smpte75, zone-plate , gamut, chroma-zone-plate, solid-color, ball, smpte100, bar .
I will not dwell on all the values, I will only say that the values smpte, smpte75 and smpte100 are SMPTE settings tables that are used in the settings of the TV / video equipment. Patterns checkers- * is a grid of NxN pixels, where N equals the number in the pattern name. The remaining values set a template with a monotonous image, for example, a color fill.
Usage example:
gst-launch-1.0 videotestsrc pattern=smpte ! xvimagesink
Explanations
Above we examined 15 main data sources, this is about half of all available plug-ins from the source category. The remaining plugins were not considered for two reasons:
- They are highly specialized;
- I don’t have the necessary hardware to test this or that plugin.
When considering plugins, many parameters were omitted, since basically they are all used for "fine tuning", and as you remember, we are getting to know GStreamer. Or, these options are read-only. To get a list of all available properties, as well as values that are valid for them, it is enough to perform
gst-inspect-1.0 <имя_элемента>, for example:gst-inspect-1.0 filesrc
and in the "Element Properties" section you can observe all properties (including those inherited).
Examples
Today we will look at one simple example of using GStreamer. In the example, I will use Python 2.7, GStreamer 1.0, and GObject Introspection.
I’ll make a reservation right away, the code may not be ideal, and it may even be the so-called “shit code,” but its only purpose is to show how to use GStreamer, no more.
Simple player
This example shows how elements are linked, how messages are processed, and the properties of elements are changed.
Player code
#env python2
# coding=utf-8
import gi
gi.require_version("Gst", "1.0")
gi.require_version("Gtk", "3.0")
from gi.repository import Gst
from gi.repository import Gtk
from gi.repository import GObject
import os
import signal
import argparse
Gst.init("")
signal.signal(signal.SIGINT, signal.SIG_DFL)
GObject.threads_init()
def parse_args():
parser = argparse.ArgumentParser(prog='example1.py')
parser.add_argument('--volume', help='Указать громкость (0-100)', type=int, default=100)
parser.add_argument('location')
args = parser.parse_args()
return args
class Player():
def __init__(self, args):
self.pipeline = self.create_pipeline(args)
## получаем шину по которой рассылаются сообщения
## и вешаем на нее обработчик
message_bus = self.pipeline.get_bus()
message_bus.add_signal_watch()
message_bus.connect('message', self.message_handler)
## устанавливаем громкость
self.pipeline.get_by_name('volume').set_property('volume', args.volume / 100.)
def create_source(self, location):
"""create_source(str) -> Gst.Element"""
if not args.location.startswith('http') and not os.path.exists(args.location):
raise IOError("File %s doesn't exists" % args.location)
if location.startswith('http'):
source = Gst.ElementFactory.make('souphttpsrc', 'source')
else:
source = Gst.ElementFactory.make('filesrc', 'source')
source.set_property('location', location)
return source
def create_pipeline(self, args):
"""create_pipeline() -> Gst.Pipeline"""
pipeline = Gst.Pipeline()
## Создаем нужные элементы для плеера
source = self.create_source(args.location)
decodebin = Gst.ElementFactory.make('decodebin', 'decodebin')
audioconvert = Gst.ElementFactory.make('audioconvert', 'audioconvert')
volume = Gst.ElementFactory.make('volume', 'volume')
audiosink = Gst.ElementFactory.make('autoaudiosink', 'autoaudiosink')
## decodebin имеет динамические pad'ы, которые так же динамически
## необходимо линковать
def on_pad_added(decodebin, pad):
pad.link(audioconvert.get_static_pad('sink'))
decodebin.connect('pad-added', on_pad_added)
## добавляем все созданные элементы в pipeline
[pipeline.add(k) for k in [source, decodebin, audioconvert, volume, audiosink]]
## линкуем элементы между собой по схеме:
## *src* -> (decodebin + audioconvert) -> volume -> autoaudiosink
source.link(decodebin)
audioconvert.link(volume)
volume.link(audiosink)
return pipeline
def play(self):
self.pipeline.set_state(Gst.State.PLAYING)
def message_handler(self, bus, message):
"""Обработчик сообщений"""
struct = message.get_structure()
if message.type == Gst.MessageType.EOS:
print('Воспроизведение окончено.')
Gtk.main_quit()
elif message.type == Gst.MessageType.TAG and message.parse_tag() and struct.has_field('taglist'):
print('GStreamer обнаружил в потоке мета-теги')
taglist = struct.get_value('taglist')
for x in range(taglist.n_tags()):
name = taglist.nth_tag_name(x)
print(' %s: %s' % (name, taglist.get_string(name)[1]))
else:
pass
if __name__ == "__main__":
args = parse_args()
player = Player(args)
player.play()
Gtk.main()
Examples of running this script:
python2 ./example1.py /foo/bar.mp3
python2 ./example1.py --volume 50 /foo/bar.mp3
python2 ./example1.py http://myradio.localhost:5678/foo.mp3
In case everything goes as it should - you will hear the track you need and see a list of ID3 tags, if any.
An example is available on GitHub .
That's it
Today we examined about half of the available data sources that are available “out of the box”, examined examples of launching these elements, and also examined the example of a simple player based on GStreamer. In the following articles, we will consider the other classes of plugins, consider even more examples of using GStreamer, and of course, write more code.
See you in a week!
Literature
- GStreamer Application Development Manual
- GStreamer Core Plugins Reference Manual
- GStreamer Base Plugins Reference Manual
- GStreamer Good Plugins Reference Manual
- Video4Linux
- GIO Reference Manual
- The ID3v2 / Shoutcast standart
- SHOUTCast
UPD 1: Added elements audiotestsrc, videotestsrc.
Previous article | Next article