Camera2 API替换Camera API之后的问题
camera和camera2的最主要区别之一就是camera2不再支持nv21的输出，通常我们为了使视频预览更加的流畅，会采用YUV_420_888的输出格式，以下是一段camera2的设置代码片段
...mImageReader = ImageReader.newInstance(width, height, ImageFormat.YUV_420_888, 2)mImageReader?.setOnImageAvailableListener(imageAvailableListener, null)mPreviewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)mCameraDevice.createCaptureSession(listOf(mPreviewSurface, mImageReader.surface),object : CameraCaptureSession.StateCallback() {override fun onConfigured(session: CameraCaptureSession) {mCaptureSession = session// 设置完后自动开始预览// 设置预览输出的 SurfacemPreviewRequestBuilder.addTarget(mPreviewSurface) mPreviewRequestBuilder.addTarget(mImageReader.surface)mPreviewRequest = mPreviewRequestBuilder.build()mCaptureSession.setRepeatingRequest(mPreviewRequest, null, mBackgroundHandler)}override fun onConfigureFailed(session: CameraCaptureSession) {Log.e(TAG, "ConfigureFailed. session: mCaptureSession")}}, mBackgroundHandler) // handle 传入 null 表示使用当前线程的 Looper......private val imageAvailableListener = ImageReader.OnImageAvailableListener {val image = it.acquireLatestImage()val data = image.getNV21DataFromYUV420888Image()val imageWidth = image.widthval imageHeight = image.heightimage.close()// 摄像头预览数据进行人脸检测，使用rgb活体检测FaceSDKManager.getInstance().onDetectCheck(data, null, null,imageHeight, imageWidth, 1, mFaceDetectCallBack)}...
百度人脸识别SDK接受的图片格式类型
百度人脸识别SDK则使用的仍是NV21格式的图片数据输出，因此我们需要在mImageAvailableListener获得image数据的时候，将YUV_420_888的图片数据流转换为NV_21
YUV_420_888转换NV_21
先来看一下android中对于YUV_420_888是如何定义的
/*** <p>Multi-plane Android YUV 420 format</p>** <p>This format is a generic YCbCr format, capable of describing any 4:2:0* chroma-subsampled planar or semiplanar buffer (but not fully interleaved),* with 8 bits per color sample.</p>** <p>Images in this format are always represented by three separate buffers* of data, one for each color plane. Additional information always* accompanies the buffers, describing the row stride and the pixel stride* for each plane.</p>** <p>The order of planes in the array returned by* {@link android.media.Image#getPlanes() Image#getPlanes()} is guaranteed such that* plane #0 is always Y, plane #1 is always U (Cb), and plane #2 is always V (Cr).</p>** <p>The Y-plane is guaranteed not to be interleaved with the U/V planes* (in particular, pixel stride is always 1 in* {@link android.media.Image.Plane#getPixelStride() yPlane.getPixelStride()}).</p>** <p>The U/V planes are guaranteed to have the same row stride and pixel stride* (in particular,* {@link android.media.Image.Plane#getRowStride() uPlane.getRowStride()}* == {@link android.media.Image.Plane#getRowStride() vPlane.getRowStride()} and* {@link android.media.Image.Plane#getPixelStride() uPlane.getPixelStride()}* == {@link android.media.Image.Plane#getPixelStride() vPlane.getPixelStride()};* ).</p>** <p>For example, the {@link android.media.Image} object can provide data* in this format from a {@link android.hardware.camera2.CameraDevice}* through a {@link android.media.ImageReader} object.</p>** @see android.media.Image* @see android.media.ImageReader* @see android.hardware.camera2.CameraDevice*/public static final int YUV_420_888 = 0x23;
由此可见：
对于YUV_420_888的图片数据，image.planes.size==3image.planes[0]的数据为Y（YUV编码格式中的Y值）image.planes[1]的数据为U（YUV编码格式中的U值）image.planes[2]的数据为V（YUV编码格式中的V值）
参考这篇文章对于YUV_420_888数据内容的分析，image.planes[1]和image.planes[2]都为UV数据的交错存储，只是顺序错开一位
image.planes[1]为UVUVUVUVUVUVUVUV，而image.planes[2]为VUVUVUVUVUVUVUVU
而NV21格式的编码结构如下图所示，是Y plane + VU plane：
可见NV21格式即为image.planes[0]+image.planes[2]
(同理NV12格式为image.planes[0]+image.planes[1])
因此我们可以得到从YUV_420_888中提取出NV21编码格式图片数据的方法为：
fun Image.getNV21DataFromYUV420888Image(): ByteArray? {return if (format == ImageFormat.YUV_420_888) {//plane[0] + plane[2] =NV21//plane[0] + plane[1] =NV12val data = ByteArray(planes[0].buffer.capacity() * 3 / 2)val buff0Offset: Int = planes[0].buffer.capacity()planes[0].buffer.get(data, 0, buff0Offset)planes[2].buffer.get(data, buff0Offset, planes[2].buffer.capacity())data} else {null}}
由于这里的数据提取并没有对数据进行转换操作和计算，仅仅是把两个plane的数据直接提取到byteArray中，因此并不需要使用cpp代码来提升转换效率，java/kotlin代码也足够了。
参考文章：
Camera2 YUV420_888
NV21与I420
