Drawable video sequence

    After a post on Apple’s approach to encoding video in JPEG, I decided to talk about my similar “bike” for Android.

    In our mobile project, we decided to make the weapon thumbnails not a static picture, but a video. It was understood that the artists would draw beautiful animations, maybe even in 3D, but something didn’t work out and we were given the simplest looped 1-1.5 second clips in 256x256 resolution. In the iOS version, they built in fine, but on Android I had to fight with MediaPlayer and SurfaceView, but still I got some “clumsiness” - the contents of SurfaceView did not move after the parent View, there was a noticeable pause during playback, etc.

    A reasonable solution would be to split the animations into frames and place them in xml for AnimationDrawable, but for 15 types of weapons this would mean a garbage bin of 5000+ frames of 10-15 kb each. Therefore, its own implementation of AnimationDrawable was made, working with a sprite sheet and a relatively quick method of converting video to such a format.

    Sprite sheet


    Without any further ado, they decided to make a map of sprites horizontal, without a description file. This is not ideally practical, but for 1-2 second animations it is not critical.
    The original video is split into sprites via ffmpeg:

        ffmpeg -i gun.avi -f image2 gun-%02d.png

    If more than 32 frames are obtained, the -r 25 or -r 20 option is added to reduce fps. The limit of 32 frames is taken from the maximum reasonable horizontal image size of 8192 pixels. This can be circumvented by a more complex arrangement of sprites, but for a sequence of 1-1.5 seconds this is enough.

    It turns out such a scatter of files:



    To build a sprite sheet, I use Zwoptex , but any similar tool, or even a self-written script, will do.



    In the example with a pistol, a PNG file of 257kb in size and a resolution of 8192x256 was obtained. After trimming to 7424x256 and processing through the TinyPNG website , it decreased to 101kb without loss of quality. If you wish, you can also save it in JPG with a slight loss of quality and reduce it to 50-70kb. For comparison, the original video in .MP4 with high quality takes the same 100kb. For more complex PNG animations, you can get 2-3 times more than the original video, which is actually not critical either.



    Native AnimationDrawable


    In the original version, the bet was that Bitmap.createBitmap does not create a new picture, but a subset of the existing one as described:

        Returns an immutable bitmap from the specified subset of the source bitmap.

    The designer loads the picture, splits it into frames and adds them to AnimationDrawable. Animations in our case are stored in assets for access by name, but the code is very easy to adapt to work with R.drawable. *

    public class AssetAnimationDrawable extends AnimationDrawable {
    	public AssetAnimationDrawable(Context context, String asset, int frames,
    			int fps) throws IOException {
    		BitmapFactory.Options options = new BitmapFactory.Options();
    		options.inPreferredConfig = Config.RGB_565; // A.G.: use 16-bit mode without alpha for animations
    		this.bitmap = BitmapFactory.decodeStream(context.getResources()
    				.getAssets().open(asset), null, options);
    		int width = bitmap.getWidth() / frames;
    		int height = bitmap.getHeight();
    		int duration = 1000 / fps;		// A.G.: note the little gap cause of integer division.
    								// i.e. duration would be 33 for 30fps, meaning 990ms for 30 frames. 
    		for (int i = 0; i < frames; i++) {
    			Bitmap frame = Bitmap.createBitmap(bitmap, i * width, 0, width, height);
    			addFrame(new BitmapDrawable(frame), duration);
    		}
    	}
    }
    


    A class is used, like a regular AnimationDrawable:

    	AnimationDrawable animation = new AssetAnimationDrawable(getContext(), "movies/gun.jpg", 28, 25);
    	animation.setOneShot(false);
    	previewImage.setImageDrawable(animation);
    	animation.start();
    


    Unfortunately, the experiments showed that not an immutable link to the original is created, but a new image for each frame, because this solution turned out to be quite resource-intensive, although it works fine for many situations.

    The next option is already noticeably more complicated and inherits directly from Drawable. In the constructor, the sprite sheet is loaded into the class member, and the current frame is drawn in the draw method. The class also implements the Runnable interface by analogy with the original AnimationDrawable for animation.

    	@Override
    	public void draw(Canvas canvas) {
    		canvas.drawBitmap(m_bitmap, m_frameRect, copyBounds(), m_bitmapPaint);
    	}
    	@Override
    	public void run() {
    		long tick = SystemClock.uptimeMillis();
    		if (tick - m_lastUpdate >= m_duration) {
    			m_frame = (int) (m_frame + (tick - m_lastUpdate) / m_duration)
    					% m_frames;
    			m_lastUpdate = tick; // TODO: time shift for incomplete frames
    			m_frameRect = new Rect(m_frame * m_width, 0, (m_frame + 1)
    					* m_width, m_height);
    			invalidateSelf();
    		}
    		scheduleSelf(this, tick + m_duration);
    	}
    	public void start() {
    		run();
    	}
    	public void stop() {
    		unscheduleSelf(this);
    	}
    	public void recycle() {
    		stop();
    		if (m_bitmap != null && !m_bitmap.isRecycled())
    			m_bitmap.recycle();
    	}
    


    The run () method calculates the current frame and queues the task. The accuracy of the given code will not be ideal, because the fractional time of the frame is not taken into account (for example, when tick - m_lastUpdate is 1ms less than duration), but this was not relevant in our task, and those who wish can modify the class on their own.
    The full code for paste2: paste2.org/p/2240487

    I want to pay attention to the recycle () method, which clears m_bitmap. In most cases, it is not needed, but we can quickly click on purchases in the store, because of which several AssetAnimationDrawable are created and memory may end, therefore, when creating a new animation, we clear the resources of the old one.

    Pros and cons


    Of course, the approach is far from ideal and is not suitable for large animations or significantly different frames, but for our task it came up perfectly, without a noticeable increase in the project and visual bugs.

    Minuses:
    • inheriting from Drawable we lose some AnimationDrawable features like setOneShot
    • 8192x256x32bpp image occupies 8MB of memory
    • you need to store the number of frames and fps for each animation somewhere
    • native code for standard solutions degrades the readability of the program and complicates its support
    • compressing sprites with jpg we get the worst quality, with the same size as the original video. Compressing in png we get the same quality, with 1-3 times the size


    Pros:
    • no bugs with SurfaceView, MediaPlayer, load braking
    • in RGB_565 mode, picture 8192x256 takes 4 MB of memory, and if necessary, you can put options.inSampleSize = 2 or more in the constructor to reduce the size and occupied memory (with a value of 2, 0.5 MB of memory and a resolution of 4096x128 are obtained)
    • You can scale the sprite sheet in your favorite editor to any size. the main rule is that the width remains a multiple of the number of frames
    • you can easily control the playback speed via fps without any problems without changing the finished files
    • it is quite possible to play animation with transparency in ARGB_8888 or ARGB_4444 modes
    • at any time you can stop the animation and free up resources


    PS


    If this will be interesting to someone, I can separately tell you about the experience of integrating small videos into the GUI in MonoTouch for an iOS project. The Mono documentation is relatively small, and there are plenty of pitfalls there.

    Also popular now: