I want to decode single frame of a H.264 video stream which is sent by server but when I do, the result picture has large padding. Code & Result: Code: now the result has large adding like this: What am I doing wrong? Rescaling the YUV image did not help and causes the picture to have 0 dimensions. (I put

Android Mediacodec decodes h264 video stream single frames with large green padding

I want to decode single frame of a H.264 video stream which is sent by server but when I do, the result picture has large padding.

Code & Result:

Code:

val singleFrameMediaCodec= MediaCodec.createDecoderByType("video/hevc")
mediaFormat.setInteger(
            MediaFormat.KEY_COLOR_FORMAT,
            MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible
        )

singleFrameMediaCodec.configure(mediaFormat, null, null, 0)


singleFrameMediaCodec.setCallback(object : MediaCodec.Callback() {
                        override fun onInputBufferAvailable(
                            _codec: MediaCodec,
                            index: Int
                        ) {
                            val buffer = _codec.getInputBuffer(index)
                            singleFrameMediaCodec.queueInputBuffer(
                                index,
                                0,
                                data.size,
                                0,
                                /*BUFFER_FLAG_END_OF_STREAM*/0
                            )
                        }

                        override fun onOutputBufferAvailable(
                            _codec: MediaCodec,
                            index: Int,
                            info: MediaCodec.BufferInfo
                        ) {
                            try {    
                                val info = MediaCodec.BufferInfo()
                                val outputIndex = index
                                val image: Image? = _codec.getOutputImage(outputIndex)
                                val rect = image.cropRect
                                val yuvImage = YuvImage(
                                    YUV_420_888toNV21(image),
                                    NV21,
                                    rect.width(),
                                    rect.height(),
                                    null
                                )

                                val stream = ByteArrayOutputStream()
                                yuvImage.compressToJpeg(
                                    Rect(0, 0, rect.width(), rect.height()),
                                    80,
                                    stream
                                )
                                val frameBitmap: Bitmap =
                                    BitmapFactory.decodeByteArray(
                                        stream.toByteArray(),
                                        0,
                                        stream.size()
                                    )    
                                imageView.setImageBitmap(frameBitmap)

                                _codec.stop()
                                stream.close()
                                image.close()
                                if (outputIndex >= 0) {
                                    _codec.releaseOutputBuffer(outputIndex, false)
                                }

                            } catch (e: Exception) {
                                Log.d(SLIDER_PRECISION, "errors here: " + e.toString())
                            }
                        }

                        override fun onError(
                            _codec: MediaCodec,
                            e: MediaCodec.CodecException
                        ) {
                        }

                        override fun onOutputFormatChanged(
                            _codec: MediaCodec,
                            format: MediaFormat
                        ) {
                        }
                    })
                    singleFrameMediaCodec.start();

now the result has large adding like this:

What am I doing wrong? Rescaling the YUV image did not help and causes the picture to have 0 dimensions. (I put 200 & 600)

My YUV conversion code:

public static byte[] YUV_420_888toNV21(Image image) {
    byte[] nv21;
    ByteBuffer yBuffer = image.getPlanes()[0].getBuffer();
    ByteBuffer uBuffer = image.getPlanes()[1].getBuffer();
    ByteBuffer vBuffer = image.getPlanes()[2].getBuffer();
    int ySize = yBuffer.remaining();
    int uSize = uBuffer.remaining();
    int vSize = vBuffer.remaining();
    nv21 = new byte[ySize + uSize + vSize];
    //U and V are swapped
    yBuffer.get(nv21, 0, ySize);
    vBuffer.get(nv21, ySize, vSize);
    uBuffer.get(nv21, ySize + vSize, uSize);
    return nv21;
}

I also have this exception thrown:

android mediacodec Allocating component ‘OMX.qcom.video.decoder.hevc’ failed, try next one.

Answer

I want to explain where the problem was so anyone with same issue can learn. I had set the right padding as per above code. So what was causing this false output? The problem was that I was producing TWO instances of MediaCodec in the activity, hence OS was encountered with low resources and gave this erroneous result. Having just ONE instance of MediaCodec solved the problem. (Note that this problem occurs, yet more probable in phones with older Android Versions (a.k.a =<8).)

Advertisement

Answer