webp lossy hardware decoding using Android MediaCodec

[James Zern]

Hi,

I posted an archive [1] containing an eclipse (4.4.1) project which produces a MediaCodec wrapper library allowing use of a VP8 hardware decoder and optionally the software one (which uses libvpx). It's far from perfect, but maybe it can serve as a reference. There's some light javadoc in the archive describing the usage. I probably won't touch this again, so if you find it useful please take it.

Here's a quick run-down on implementation notes / open areas for research:

- most testing was done on a Nexus 10 (4.4.2); the Nexus 5 is black-listed as it failed on most images I had -- I didn't try the reference test vectors [2], though

- this implementation uses the pre-API level 21 changes for compatibility, I didn't touch anything L related beyond running a test app on an N5

- there's currently a dependency on libwebp for image validation / image dimension extraction -- this isn't strictly necessary

- with the N10 (at least) there's a 1 frame lag, so the decoder is flushed after each frame

- startup time for both the h/w and s/w decoders can be large (10s of ms)

- the h/w decoder requires a reconfigure on resolution change, which is a bit costly

- when rendering to a Surface (both SurfaceView/TextureView) the N10+h/w would periodically miss render updates, using the flush approach or not; maybe someone with more MediaCodec knowledge might have some insight here

- when decoding to a byte buffer you lose the color conversion benefit so would need to handle that separately -- probably not worth it given the timings

The best use scenario would seem to be if there were a large amount of fairly sizable (>=512x512) images to be decoded. Additionally, using the software decoder would help performance on older Android versions as until recently webp was slower than libvpx on arm.

For timing I used a set of 20 images I had around, the average decode time is in milliseconds.

resolution | Bitmap.decodeStream | S/W | H/W

1 decode / file:

180x180 | 3.40 | 4.75 | 8.55

256x256 | 5.12 | 5.06 | 6.96

666x888 | 71.14 | 71.18 | 27.55

20 decodes / file:

180x180 | 3.35 | 4.96 | 5.44

256x256 | 5.12 | 5.80 | 5.68

666x888 | 64.97 | 51.71 | 20.44

H/W startup time ~5-10ms

S/W startup time ~12ms

[1] http://downloads.webmproject.org/webp/2014-11-26-MediaCodecWebpDecoder.tar.gz

[2] http://www.webmproject.org/code/#libwebp-test-data-dwebp-validation-tool