HardwareVideoDecoder Issue on Android

[Nithyesh Sankar]

I am trying to run AppRTC android application.

I am able to establish a call with an Android phone.But the video received on my device is not decoded properly. I am seeing a stuttering effect on the decoded video. I am using H264 High codec with hardware acceleration enabled for both encoding and decoding. I enabled call statistics on the application and got the following sets of data:

FrameHeightReceived = 720
FrameRateReceived = 29
FrameRateDecoded = 2
FrameRateOutput = 2
FrameWidthReceived = 1280
------------------------------------------------------------------
FrameHeightReceived = 720
FrameRateReceived = 30
FrameRateDecoded = 4
FrameRateOutput = 4
FrameWidthReceived = 1280
------------------------------------------------------------------
FrameHeightReceived = 720
FrameRateReceived = 26
FrameRateDecoded = 1
FrameRateOutput = 1
FrameWidthReceived = 1280

Q1: Am I correct in assuming that the receiver has received 30 frames but is not able to decode all of them? If i am wrong, what do the above values mean?

I tried to find out what was going on and was able to get the following logs from logcat.

adb logcat | grep -e HardwareVideoDecoder

HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal
SurfaceTextureHelper: SurfaceTextureHelper: Setting listener to org.webrtc.HardwareVideoDecoder@ea709c2
HardwareVideoDecoder: HardwareVideoDecoder: Creating video of format video/avc width = + 1280 height = 720 color-format 19
HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal done
HardwareVideoDecoder: HardwareVideoDecoder:  reformatted result -2
HardwareVideoDecoder: HardwareVideoDecoder: Decoder format changed: {mime=video/raw, crop-top=0, crop-right=1279, slice-height=736, color-format=19, height=720, width=1280, what=1869968451, crop-bottom=719, crop-left=0, stride=1280}
HardwareVideoDecoder: HardwareVideoDecoder: Frame stride and slice height: 1280 x 736
HardwareVideoDecoder: HardwareVideoDecoder: decode() - no HW buffers available; decoder falling behind
HardwareVideoDecoder: HardwareVideoDecoder: release
HardwareVideoDecoder: HardwareVideoDecoder: Releasing MediaCodec on output thread
HardwareVideoDecoder: HardwareVideoDecoder: Release on output thread done

HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal
SurfaceTextureHelper: SurfaceTextureHelper: Setting listener to org.webrtc.HardwareVideoDecoder@ea709c2
HardwareVideoDecoder: HardwareVideoDecoder: Creating video of format video/avc width = + 1280 height = 720 color-format 19
HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal done
HardwareVideoDecoder: HardwareVideoDecoder: decode() - key frame required first
HardwareVideoDecoder: HardwareVideoDecoder: release
HardwareVideoDecoder: HardwareVideoDecoder: Releasing MediaCodec on output thread
HardwareVideoDecoder: HardwareVideoDecoder: Release on output thread done

HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal
SurfaceTextureHelper: SurfaceTextureHelper: Setting listener to org.webrtc.HardwareVideoDecoder@ea709c2
HardwareVideoDecoder: HardwareVideoDecoder: Creating video of format video/avc width = + 1280 height = 720 color-format 19
HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal done
HardwareVideoDecoder: HardwareVideoDecoder: decode() - no HW buffers available; decoder falling behind
HardwareVideoDecoder: HardwareVideoDecoder: release
HardwareVideoDecoder: HardwareVideoDecoder: Releasing MediaCodec on output thread
HardwareVideoDecoder: HardwareVideoDecoder: Release on output thread done

HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal
SurfaceTextureHelper: SurfaceTextureHelper: Setting listener to org.webrtc.HardwareVideoDecoder@ea709c2
HardwareVideoDecoder: HardwareVideoDecoder: Creating video of format video/avc width = + 1280 height = 720 color-format 19
HardwareVideoDecoder: HardwareVideoDecoder: initDecodeInternal done
HardwareVideoDecoder: HardwareVideoDecoder: decode() - key frame required first
HardwareVideoDecoder: HardwareVideoDecoder: release
HardwareVideoDecoder: HardwareVideoDecoder: Releasing MediaCodec on output thread
HardwareVideoDecoder: HardwareVideoDecoder: Release on output thread done

Q2: What do "no HW buffers available; decoder falling behind" and "key frame required first" mean? In what case would these happen?


Any help regarding the above issues would be useful. Thank you