August 4, 2015 at 14:06Application Development Using the NexStreaming NexPlayer SDK
- Transfer
Developing a mobile player application is not easy. It is necessary to provide support for many formats of video, sound, subtitles. You will have to implement the playback of files that are stored locally and on the Internet. If you remember that the result of all these works will be executed in an environment whose resources should be treated with care, you will have to do optimization. How to start creating a high-quality, fast and economical player for Android? The NexPlayer SDK will help answer this question.
NexStreaming is a global company that develops software for mobile devices. NexStreaming is headquartered in Seoul, Korea, with branches in Spain, the USA, Japan, and China. NexPlayer SDK is its most popular product. This is a software development kit for creating players. It is used in their mobile applications by some very well-known companies that are involved in video streaming. The player is compatible with all modern DRM technologies. In addition, its main functionality can be expanded. For example, add tools to improve the sound, support for advertising inserts, the function of collecting statistical information about the audience.
The NexPlayer SDK from NexStreaming provides tools for decoding and playing back video and audio data. Application developers can use the SDK to quickly create their own products. NexPlayer SDK is reliable, stable, compatible with international standards. In this article, you will learn how to use the NexPlayer SDK to create an x86-based player.
NexPlayer SDK is optimized for the x86 platform, that is, it is fully supported by devices built on this platform. Here are the main features of the SDK:
First you need to download the SDK from the NexStreaming website by filling out a short form and waiting for an email with the data to enter the closed section of the site, where the materials are located. In particular, among the materials available for download, you can find a demo application in an .apk file that can be installed on an Android device right away, an example project that can be imported into an IDE, the SDK itself as a set of libraries, and its documentation . In order to decide which libraries you need, you should familiarize yourself with the documentation and look at examples. Application development using the NexPlayer SDK is quite simple and straightforward, and the presence of examples contributes to its rapid development. It takes about an hour to create a working application that uses the NexPlayer SDK.
Integrating the NexPlayer SDK into an x86-based Android application is to copy the libraries and source code from the SDK to your project. After following a few simple steps, described below, you will equip the application with the basic components that will allow it to work perfectly on devices built on the basis of components from Intel. So, the preparatory stage is as follows:
The set of libraries to copy depends on the needs of your application. In order to understand which libraries you will need, it is worth taking a look at a demo and using the recommendations given below.
After copying, the NexPlayer SDK will detect the changes and will be able to use libraries to organize efficient work with Intel resources. Once all the libraries are in the specified directories, the SDK will take over the switch between the ARM and x86 versions of the application. If you want to update the NexPlayer SDK libraries in an existing project, just overwrite their files.
NexPlayer SDK includes a large number of libraries, including those that are responsible for DRM support. They are located in the app / assets / x86 folder. The required set of libraries includes a player engine (engine), decoders (decoders) and visualization tools (rendering layer). Here is a list of them:
Some of the library names contain the abbreviation ics. It indicates that the library is for Android Ice Cream Sandwich. The abbreviation "jb" means that the library is designed for Android Jelly Bean. If your application is designed to support certain versions of Android, libraries for unsupported versions can be removed.
Here are the libraries responsible for supporting codecs:
Listed below are libraries whose purpose is to support subtitles:
To reduce the size of the application, include only those libraries that you need in the project.
Libraries from the app / libs / x86 folder need to be downloaded. This is done in the initManager () method of the corresponding class. For example, to use “NexHDSample” you need to load the appropriate x86 library in the above method of the NexHDManager class (app / src / NexHDManager.java) using the following code:
System.loadLibrary (“NexHTTPDownloaderSample_jni”);
When working with the NexPlayer SDK, there are two ways to display video. This is the use of output subsystems based on NexVideoRenderer and on OpenGL. It is recommended to use exactly NexVideoRenderer. The fact is that this subsystem provides a level of abstraction over complex surface control operations and video output tasks. She chooses the most suitable way to output video, based on the hardware of the device and the operating system installed on it.
In order to use NexVideoRenderer, you need to do the following:
The code for the NexVideoRenderer class can be found among SDK usage examples at NexPlayerSample / src / com / nexstreaming / nexplayerengine / NexVideoRenderer.java
HTTP Live Streaming technology allows you to work with multiple audio and video streams. The SDK supports this feature, its setMediaStream () API allows you to select streams for playback. You can do this from the user interface while playing content. The following options for working with video and sound:
The following are test results that evaluate the load on the system created by the NexPlayerDemoApp application at startup, simple, playing a local MP4 file and playing streaming video. The application is developed using the NexPlayer SDK. For analysis, we used the VTune analyzer for Android, as well as the Intel SoC Watch. The tests were carried out on a tablet on which Android 4.4.2 is installed. It is equipped with a 1.6 GHz Intel Atom Z3740 processor and Intel HD Graphics (Gen 7). The device is connected to the Internet via Wi-Fi.
Reference Measurement Point for Intel Atom Z3740 Processor
NexPlayerDemoApp inactive
The more time the Intel Atom Z3740 processor spends in C0 state while the application is idle, the higher the power consumption. In the case of NexPlayerDemoApp, the use of the processor is very close to the reference point of measurements, as a result - inaction of the application does not create an unnecessary load on the system.
NexPlayerDemoApp application on startup
These data are also very close to the reference measurement point.
When playing video, the x86 version of NexPlayerDemoApp uses, on average, 33% of the processor power. When analyzing this task, no anomalies were detected; when it is performed, a uniform load on the processor is created.
CPU load created by NexPlayerDemoApp x86 application when playing local video
When playing streaming video, the x86 version of NexPlayerDemoApp uses, on average, 25% of the processor power. When analyzing this task, no anomalies were detected; when it is performed, a uniform load on the processor is created.
CPU load created by NexPlayerDemoApp x86 application when playing streaming video
Power consumption x86-version of the application NexPlayerDemoApp was approximately 3.8 watts. Most of the time during measurements, streaming video was played, there was a small period of inactivity. The rest of the time was spent playing the local video file.
Let's find out how the demo application built using the NexPlayer SDK works on Android 5.0 Lollipop. For testing, we will use the Asus Fonepad 8 tablet based on the Intel Atom Z3530 CPU. The device is connected to the Internet via Wi-Fi.
The main screen of the application
On the main screen of the application you can find several buttons:
During the tests, the application coped with its functions. In particular, video from various sources was played at the correct speed and without delays.
Play local video
Using the NexPlayer SDK, you can quickly and easily develop applications whose main task is video playback. In addition, tests prove the efficiency and low power consumption of such applications on devices built on the x86 platform.
Preliminary information
NexStreaming is a global company that develops software for mobile devices. NexStreaming is headquartered in Seoul, Korea, with branches in Spain, the USA, Japan, and China. NexPlayer SDK is its most popular product. This is a software development kit for creating players. It is used in their mobile applications by some very well-known companies that are involved in video streaming. The player is compatible with all modern DRM technologies. In addition, its main functionality can be expanded. For example, add tools to improve the sound, support for advertising inserts, the function of collecting statistical information about the audience.
The NexPlayer SDK from NexStreaming provides tools for decoding and playing back video and audio data. Application developers can use the SDK to quickly create their own products. NexPlayer SDK is reliable, stable, compatible with international standards. In this article, you will learn how to use the NexPlayer SDK to create an x86-based player.
Key features
NexPlayer SDK is optimized for the x86 platform, that is, it is fully supported by devices built on this platform. Here are the main features of the SDK:
- Support Android 1.6 or higher
- Playing mp4, 3gp, avi, asf, piff file formats
- Support for HTTP Live Streaming 5.0 and 3GPP Progressive Download. Support for AES128 and HTTPS protocols, h.264, AAC, AAC +, eAAC + codecs. SDK allows you to work with both software and hardware codecs.
- Support for subtitles in the formats .smi, .srt, .sub, 3GPP timed text, TTML (PIFF / CFF only), CEA 608 and CEA 708. Support for Web Video Text Tracks (WebVTT) technology.
How to create a player for the x86 platform using the NexPlayer SDK
First you need to download the SDK from the NexStreaming website by filling out a short form and waiting for an email with the data to enter the closed section of the site, where the materials are located. In particular, among the materials available for download, you can find a demo application in an .apk file that can be installed on an Android device right away, an example project that can be imported into an IDE, the SDK itself as a set of libraries, and its documentation . In order to decide which libraries you need, you should familiarize yourself with the documentation and look at examples. Application development using the NexPlayer SDK is quite simple and straightforward, and the presence of examples contributes to its rapid development. It takes about an hour to create a working application that uses the NexPlayer SDK.
Integrating the NexPlayer SDK into an x86-based Android application is to copy the libraries and source code from the SDK to your project. After following a few simple steps, described below, you will equip the application with the basic components that will allow it to work perfectly on devices built on the basis of components from Intel. So, the preparatory stage is as follows:
- Copy the necessary libraries from the SDK / libs folder to the assets / x86 folder of your own project
- Copy the required libraries from the SDK / libs folder to the libs / x86 folder of the project
- Copy the java files from the SDK / src to the src / com / nexstreaming / nexplayerengine folder of the project
The set of libraries to copy depends on the needs of your application. In order to understand which libraries you will need, it is worth taking a look at a demo and using the recommendations given below.
After copying, the NexPlayer SDK will detect the changes and will be able to use libraries to organize efficient work with Intel resources. Once all the libraries are in the specified directories, the SDK will take over the switch between the ARM and x86 versions of the application. If you want to update the NexPlayer SDK libraries in an existing project, just overwrite their files.
NexPlayer SDK includes a large number of libraries, including those that are responsible for DRM support. They are located in the app / assets / x86 folder. The required set of libraries includes a player engine (engine), decoders (decoders) and visualization tools (rendering layer). Here is a list of them:
- libnexplayerengine.so
- libnexalfactory.so
- libnexadaptation_layer_for_dlsdk.so
- libnexralbody_audio.so
- libnexralbody_video_opengl.so
- libnexral_nw_ics.so
- libnexral_nw_jb.so
- libnexcal_oc_ics.so
- libnexcal_oc_jb.so
- libnexcralbody_mc_jb.so
- libnexcal_in_aac_x86.so
- libnexcal_in_mp3_x86.so
- libnexcal_in_amr_x86.so
Some of the library names contain the abbreviation ics. It indicates that the library is for Android Ice Cream Sandwich. The abbreviation "jb" means that the library is designed for Android Jelly Bean. If your application is designed to support certain versions of Android, libraries for unsupported versions can be removed.
Here are the libraries responsible for supporting codecs:
- libnexcal_h364_x86.so - H.264 video support library
- libnexcal_aac_x86.so - sound support library for AAC, AAC-Plus and HE-AAC formats
- libnexcal_mp3_x86.so - sound support library for MP2 and MP3 formats
Listed below are libraries whose purpose is to support subtitles:
- libnexcal_3gpp_x86.so - support for 3GPP timed text format
- libnexcal_closedcaption_x86.so - support for CEA 608 and CEA 708 formats
- libnexcal_ttml_x86.so - support for the TTML format (CFF)
- libnexcal_webvtt_x86.so - support for the WebVTT format
To reduce the size of the application, include only those libraries that you need in the project.
Libraries from the app / libs / x86 folder need to be downloaded. This is done in the initManager () method of the corresponding class. For example, to use “NexHDSample” you need to load the appropriate x86 library in the above method of the NexHDManager class (app / src / NexHDManager.java) using the following code:
System.loadLibrary (“NexHTTPDownloaderSample_jni”);
How to display video on an x86-based application screen
When working with the NexPlayer SDK, there are two ways to display video. This is the use of output subsystems based on NexVideoRenderer and on OpenGL. It is recommended to use exactly NexVideoRenderer. The fact is that this subsystem provides a level of abstraction over complex surface control operations and video output tasks. She chooses the most suitable way to output video, based on the hardware of the device and the operating system installed on it.
In order to use NexVideoRenderer, you need to do the following:
- Pass context (android.context.Context) to the constructor.
- Install event listeners (NexPlayer.IListener and NexPlayer.IVideoRendererListener).
- Create an instance of an object of the NexPlayer class.
- Perform the necessary preparation of NexPlayer (use the methods NexPlayer.setNexAlFactory and NexPlayer.init).
- Call the init method of an object of the class NexVideoRenderer (NexVideoRenderer.init).
- Add an instance of NexVideoRenderer as an interface element to the layout.
The code for the NexVideoRenderer class can be found among SDK usage examples at NexPlayerSample / src / com / nexstreaming / nexplayerengine / NexVideoRenderer.java
Streaming
HTTP Live Streaming technology allows you to work with multiple audio and video streams. The SDK supports this feature, its setMediaStream () API allows you to select streams for playback. You can do this from the user interface while playing content. The following options for working with video and sound:
- Using a playlist with various soundtrack options. Video and sound with this approach can be selected independently.
- Using a playlist with various video options. Each video clip that is played contains both sound and video, but the user can select alternative video options. For example, it may be material shot by the camera at different angles or from different points.
- A combination of lists of available audio tracks and videos. This is a mixture of the two options described above. The main video stream provides video tracks in different bitrates, but reproduced with the same sound. At the same time, additional audio tracks are available, for example, in different languages.
NexPlayerDemoApp x86 application performance and power analysis
The following are test results that evaluate the load on the system created by the NexPlayerDemoApp application at startup, simple, playing a local MP4 file and playing streaming video. The application is developed using the NexPlayer SDK. For analysis, we used the VTune analyzer for Android, as well as the Intel SoC Watch. The tests were carried out on a tablet on which Android 4.4.2 is installed. It is equipped with a 1.6 GHz Intel Atom Z3740 processor and Intel HD Graphics (Gen 7). The device is connected to the Internet via Wi-Fi.
Reference Measurement Point for Intel Atom Z3740 Processor
| CPU batch state | % of the time spent in this state, core number 0 | % of the time spent in this state, core number 1 | % of the time spent in this state, core number 2 | % of the time spent in this state, core number 3 | % GPU runtime |
| C0 | 4.00 | 2.50 | 0.40 | 0.40 | 0.30 |
| C6 | 95.00 | 95.10 | 98.30 | 98.10 | 99.50 |
NexPlayerDemoApp inactive
| CPU batch state | % of the time spent in this state, core number 0 | % of the time spent in this state, core number 1 | % of the time spent in this state, core number 2 | % of the time spent in this state, core number 3 | % GPU runtime |
| C0 | 4.80 | 3.70 | 1.2 | 0.90 | 0.00 |
| C6 | 93.50 | 94.10 | 93.40 | 93.90 | 100.00 |
The more time the Intel Atom Z3740 processor spends in C0 state while the application is idle, the higher the power consumption. In the case of NexPlayerDemoApp, the use of the processor is very close to the reference point of measurements, as a result - inaction of the application does not create an unnecessary load on the system.
NexPlayerDemoApp application on startup
| CPU batch state | % of the time spent in this state, core number 0 | % of the time spent in this state, core number 1 | % of the time spent in this state, core number 2 | % of the time spent in this state, core number 3 | % GPU runtime |
| C0 | 4.80 | 3.70 | 1.2 | 0.90 | 0.90 |
| C6 | 93.50 | 94.10 | 93.40 | 93.90 | 99.00 |
These data are also very close to the reference measurement point.
Playing local video
When playing video, the x86 version of NexPlayerDemoApp uses, on average, 33% of the processor power. When analyzing this task, no anomalies were detected; when it is performed, a uniform load on the processor is created.
CPU load created by NexPlayerDemoApp x86 application when playing local video
Streaming video
When playing streaming video, the x86 version of NexPlayerDemoApp uses, on average, 25% of the processor power. When analyzing this task, no anomalies were detected; when it is performed, a uniform load on the processor is created.
CPU load created by NexPlayerDemoApp x86 application when playing streaming video
power usage
Power consumption x86-version of the application NexPlayerDemoApp was approximately 3.8 watts. Most of the time during measurements, streaming video was played, there was a small period of inactivity. The rest of the time was spent playing the local video file.
NexPlayerDemoApp app on Android 5.0 Lollipop
Let's find out how the demo application built using the NexPlayer SDK works on Android 5.0 Lollipop. For testing, we will use the Asus Fonepad 8 tablet based on the Intel Atom Z3530 CPU. The device is connected to the Internet via Wi-Fi.
The main screen of the application
On the main screen of the application you can find several buttons:
- History of views (History).
- Open link (Open URL) - when you click on this button, a pop-up window appears for entering a link to streaming video.
- Watch demo streaming video clips (Watch sample streams) - clicking on this button opens a screen where you can select one of the suggested videos for viewing.
- Play local files - after clicking on this button, the program displays a list of device folders in which the video files are located.
During the tests, the application coped with its functions. In particular, video from various sources was played at the correct speed and without delays.
Play local video
Summary
Using the NexPlayer SDK, you can quickly and easily develop applications whose main task is video playback. In addition, tests prove the efficiency and low power consumption of such applications on devices built on the x86 platform.