Interview with Edward Hervey about PiTiVi Video Editor

Original author: Christian FK Schaller
  • Transfer
This is the fourth opensource multimedia interview. Previous conversations concerned Jokosher , Totem, and Empathy projects ( translation ). We will talk with Edward Hervey accompanying the PiTiVI project . Edward will talk about the current state of the video editor and plans for the future.

So, first of all, tell us a little about yourself and open projects in which you take part?

My name is Edward Hervey, I am half French, half Scot living in Barcelona, ​​and I am one of the founders of Collabora Multimedia . I have been using Linux and free software since about 1995. But I began to make my contribution from 2003, when I finally got enough programming knowledge to start working on PiTiVi. For several years now, I have been working on the GStreamer multimedia framework to take video editing to a decent level. I also accompany python binders for GStreamer, GNonLin (the main PiTiVi dependencies, are responsible for non-linear editing), as well as FFMpeg plugins for GStreamer over the years .

Describe the goals of PiTiVI against the backdrop of dominant video editing apps like iMovie, Avid or Final Cut Pro?

PiTiVi has different goals, but I would note the fundamental one - the desire to create a video editor without any restrictions (unlike analogues, which have very specific limitations in use or support). To get rid of the limitations of formats, devices, filters, ... we can support them through the GStreamer framework. All other editors are tightly bound to their code, but we are proud to bear the name of a free editor without bindings to any patented library! This means that if a company makes its own GStreamer plugin for a specific format / codec / device ... you are free to use it in PiTiVi. This barrier is overcome by choosing our development model.

We do not want to get a monolithic video editor, so it was decided to rewrite PiTiVi in Python, and make the project modular. The user interface is not tied to the engine, in fact, you can use the engine separately. It is designed so that you can implement the chronological tape in different ways, it depends more on your knowledge and how you place accents, which in turn depend on the application in the production process.

The ultimate goal of PiTiVi is to provide basic editing tools without any restrictions. This, unfortunately, is worth a lot of effort: you need to make sure that all errors, problems or restrictions are fixed at the proper level, instead of adding a hack after a hack. And it takes time ...

PiTiVI is written in Python. The first question many people ask when they hear this is: “How does the use of Python affect the performance of multimedia applications, especially resource-intensive ones like a video editor?”

Everything is simple :) What puts the greatest load on the CPU when editing video? That's right, multimedia processing (whether it's viewing, applying effects, rendering, ...). All this is done exclusively with the help of GStreamer and its plugins (written in C, work in separate streams and are used in PiTiVi via Python binders). And what is loading the rest of the processing power of the processor? User interface. GTK + / X / Cairo are doing this (written in C and used in PiTiVi via Python binders).

So all that remains is the logic of connecting everything together and engaging concepts specific to non-linear video editing systems (NLE). We want to create code as fast and flexible as possible, so a language like Python is very convenient. In addition, writing in Python is a pleasure :)

For the curious, I’ll give you an interesting fact: when playing in PiTiVi, the only part implemented in Python is the code responsible for ... displaying the current position. The same thing when rendering. Everything else is fully handled by GStreamer, its plugins and GTK +.

A lot of work has been done in GStreamer to support the MXF container format . Most people have probably never heard of such a format, what value does it have?

Since the advent of computer editing (also called non-linear editing), each program has used different formats / codecs to work. What professionals and amateurs used was also different, including for cameras, VCRs, capture devices. Add to this the fact that most formats have not been documented. We get various formats in software, in the camera, at the output, ... a complete mess.

Some of the key players in the professional market, SMPTE representatives(Society of Film and Television Engineers) decided to work together and wrote specifications for a new container format that will be fully documented, will allow metadata to be stored, recorded directly to the camera, enable streaming and quick search ... It uses the same structural elements as AAF ( advanced development format), which is open and used by Avid, Final Cut Pro and many other non-linear editing systems.

With the addition of MXF support in GStreamer, we can use the same formats in PiTiVi as professional NLE! The ability to directly work with common formats is the main factor that allows you to use PiTiVi in existing production processes.

What multimedia formats does PiTiVI support?

Everything that is available through GStreamer plugins installed on your system, as well as some formats for quick video editing, precise search, I-frame codecs, etc.

At what stage is PiTiVI now and what are your plans for the future?

Over the past 6 months, we have done a deep refactoring of the PiTiVi code, based on the analysis that I performed last summer, summarizing the responses over the past 5 years. In addition to adding the most easily implemented functions, we already managed with multi-layer editing, sketching, cropping, ripple / roll modes, support for the general file format appeared, and much more.

We are going to do the first pre-release of the new pitivi in ​​May. Gradually, new functions will be added to the interface, for example, transitions, effects, overlaying, slip / slide editing and support for a larger number of formats, which will allow, for example, importing projects from other editors.

For those unfamiliar with terminology, what do you mean by layered editing, sketching, cropping, and ripple / roll modes?

Multilayer editing: the ability to sort video not only by time, but also into several parallel layers.
Sketching: the resulting image is displayed for each source used in the Timeline scale, which will make it easier for you to navigate in the chronological feed without the need for a search.
Crop: Change the position of the beginning or end of the source.
Ripple editing: reducing or enlarging one of the parts, while the rest of the data adjusts to the new size of the changed clip.
Roll editing: the same as ripple, except that the total time in a group of clips remains unchanged.

Who is participating in the PiTiVI project besides you?

The two main developers besides me are Brandon Lewis(Brandon Lewis), who started working with GSoC 2007 and is busy with the user interface. Alessandro Decina (Alessandro Decina) joined us in October, he helps me in the low-level part of PiTiVi and improves GStreamer. Both have been working at Collabora Multimedia since last fall and are working on PiTiVi 100% of the time. Explanation for those who think about the business reasons for attracting so many resources in Collabora Multimedia PiTiVi: firstly, we provide consulting services for customers in the content creation industry, they want to use PiTiVI to perform specific tasks in their production processes, secondly , we have clients who want to use the PiTiVi infrastructure we created to set up automatic video manipulations. For example, automatic cropping, scaling and transcoding.

How will future changes to GStreamer affect PiTiVI?

Despite the fact that there are quite working functions in the current GStreamer core and plugins, we are working on improving a number of features to improve the editing interface.

One of the much needed features is the implementation of GstIndex support with a wider range of plugins. This will allow you to scan inconvenient files for editing during import and create an index that allows you to quickly move inside these files. This will allow applications like PiTiVi to get more information about the file format, the location of key frames, thanks to this you will find out which section is more profitable to skip and where to do lossless clipping.

Endless work remains to be done in GStreamer to port more and more libraries as plugins. People often mistakenly assume that GStreamer is a codec library, which is completely wrong, GStreamer is a framework that allows application developers not to worry about processing, and library developers not to worry about applications that will use them. We plan to port plugins such as gavl and the vast majority of great AviSynth filters (which will be completed this summer as part of the Google Summer of Code) to GStreamer. All this, of course, improves the user experience of PiTiVi.

In addition, we are going to reduce memory consumption and CPU time for the GStreamer core and various plugins. We are looking at more efficient use of hardware devices, such as video cards with VDPAU technology, VA-API and gst-plugins-gl plugins. This will allow GStreamer applications to direct processing to hardware devices, leaving more processor time for universal processing.

Jokosher and PiTiVI are multimedia applications that use GStreamer and are written in Python. The natural question is, are these projects collaborating?

Unfortunately, not as active as we would like. We help each other on all fronts (GStreamer, Python binders, GNonLin plugins), but when it comes to the user interface and the main logic, we have different options for using code and different code bases. It would be great to have closer integration, this would allow you to use Jokosher to edit the audio part of the video clip, and for PiTiVi to leave the video.

What are the main technical problems you encountered while creating a video editor for Linux / Unix systems? Is the Linux infrastructure ready to host such applications? And what do you think of the state of multimedia in Linux / Unix in general?

Open-source multimedia has gone * LONG * way over the past 10 years. For those who don’t remember ... we were forced to use different players depending on the format, the only way to capture video from digital cameras. But the problem is that everyone makes their own software in their corner. The complete lack of consistency between the various multimedia applications, libraries and device support in Linux, which is, in my opinion, the reason why we did not become a suitable multimedia platform. Unlike other platforms (Win / Mac), we do not have to rewrite our own libraries and we can reuse existing ones ... Why does this not happen? GStreamer is trying to get everyone together and solve most of these problems, however, it still gets kicks from some developers for unknown reasons. On the day we solve this issue ... we will become dominant on the desktop platforms, as we did for embedded devices and servers.

What other multimedia applications do you use?

Totem , Rhythmbox , Audacity (Oh, he doesn't use GStreamer). Sometimes AviSynth / VirtualDub (Win32 applications) through WINE, but this can change, thanks to the GSoC project, designed to transfer AviSynth filters to GStreamer.

Well, thank you Edward for taking the time for this interview, good luck in the future!

To learn more about the PiTiVi project, visit pitivi.org or the #pitivi channel on irc.freenode.net.

Interview was prepared by Christian Schaller (Christian FK Schaller).
_____
Thank ShutteR77 andilembitov for help in preparing the translation.

Also popular now: