Android Camera HAL3中拍照Capture模式下多模块间的交互与帧Result与帧数据回调.doc

Android Camera HAL3中拍照Capture模式下多模块间的交互与帧Result与帧数据回调本文均属自己阅读源码的点滴总结，转账请注明出处谢谢。欢迎和大家交流。qq:1037701636 email:Software：系统源码Android5.1前沿： 之前的两篇博文算是比较详细的记录了整个Camera3 HAL3架构下完全不同于HAL1的preview预览处理过程，包括主要涉及到的控制流和视频流等。比较详细的阐述了Camera2Client下streamProcessor、CallbackProcessor、CaptureSequencer等模块在Camera3架构下的功能。分析得出每个模块下均会在Camera3Device下以一个Stream的形式存在，而每个stream又是由多个buffer来构成主体的。与HAL3进行数据的交互时，以Request和result来作为数据传输的载体。在这些基础上本文将描述具体拍照Capture模式下的数据流和控制流，主要会涉及到jpegprocessor、CaptureSequencer这几个模块的工作原理。鉴于Capture模式下的数据流更复杂，在这里重点会分析数据流result回传时，每个模块的响应以及处理过程，填补前一博文的空白。1. HAL3中Camera2Client下的take picture的入口函数作为标准的capture picture功能的入口，主要完成了以下两件事情：updateProcessorStream(mJpegProcessor, l.mParameters);mCaptureSequencer-startCapture(msgType) 对于JpegProcessor模块而言，他的stream流第一次是在preview阶段进行了create与初始化，这里之所以再次调用JpegProcessor:updateStream目的是参考原先JpegProcessorstream的width与height是否变化即是否照片要求的分辨率发生了变化，如果是的话就需要delete原先的stream，重新建立一个stream。 在JpegProcessor中重点关注CpuConsumer与Surface的生产者与消费者处理模式，官方称之为Create CPU buffer queue endpoint。2.CaptureSequencer模块 CaptureSequencer模块是take picture下操作的重点，在Camera2Client中进行了创建，首先来看CaptureSequencer线程的threadLoop函数：cppview plaincopy1. boolCaptureSequencer:threadLoop()2. 3. spclient=mCmote();4. if(client=0)returnfalse;5. 6. CaptureStatecurrentState;7. 8. Mutex:Autolockl(mStateMutex);9. currentState=mCaptureState;10. 11. 12. currentState=(this-*kStateManagerscurrentState)(client);13. 14. Mutex:Autolockl(mStateMutex);15. if(currentState!=mCaptureState)16. if(mCaptureState!=IDLE)17. ATRACE_ASYNC_END(kStateNamesmCaptureState,mStateTransitionCount);18. 19. mCaptureState=currentState;/保留新的状态20. mStateTransitionCount+;21. if(mCaptureState!=IDLE)22. ATRACE_ASYNC_BEGIN(kStateNamesmCaptureState,mStateTransitionCount);23. 24. ALOGV(Camera%d:Newcapturestate%s,25. client-getCameraId(),kStateNamesmCaptureState);26. mStateChanged.signal();27. 28. 29. if(mCaptureState=ERROR)30. ALOGE(Camera%d:Stoppingcapturesequencerduetoerror,31. client-getCameraId();32. returnfalse;33. 34. 35. returntrue;36. CaptureSequencer是一个以不同的state状态机来循环工作的模块，currentState = (this-*kStateManagerscurrentState)(client)函数是执行对应状态机下的执行函数，其中的state值如下：cppview plaincopy1. constCaptureSequencer:StateManager2. CaptureSequencer:kStateManagersCaptureSequencer:NUM_CAPTURE_STATES-1=3. &CaptureSequencer:manageIdle,4. &CaptureSequencer:manageStart,5. &CaptureSequencer:manageZslStart,6. &CaptureSequencer:manageZslWaiting,7. &CaptureSequencer:manageZslReprocessing,8. &CaptureSequencer:manageStandardStart,9. &CaptureSequencer:manageStandardPrecaptureWait,10. &CaptureSequencer:manageStandardCapture,11. &CaptureSequencer:manageStandardCaptureWait,12. &CaptureSequencer:manageBurstCaptureStart,13. &CaptureSequencer:manageBurstCaptureWait,14. &CaptureSequencer:manageDone,15. ;我们以一个standard capture的操作模式，来分析一次完成的take picture的过程。初始化的mCaptureState(IDLE)，进入的函数入口为manageIdle：cppview plaincopy1. CaptureSequencer:CaptureStateCaptureSequencer:manageIdle(2. sp&/*client*/)3. status_tres;4. Mutex:Autolockl(mInputMutex);5. while(!mStartCapture)6. res=mStartCaptureSignal.waitRelative(mInputMutex,7. kWaitDuration);8. if(res=TIMED_OUT)break;9. 10. if(mStartCapture)11. mStartCapture=false;12. mBusy=true;13. returnSTART;14. 15. returnIDLE;16. 函数主要在轮训mStartCapture的值，这个值是由CameraService端的拍照触发线程来启动的，代码如下：cppview plaincopy1. status_tCaptureSequencer:startCapture(intmsgType)2. ALOGV(%s,_FUNCTION_);3. ATRACE_CALL();4. Mutex:Autolockl(mInputMutex);5. if(mBusy)6. ALOGE(%s:Alreadybusycapturing!,_FUNCTION_);7. returnINVALID_OPERATION;8. 9. if(!mStartCapture)10. mMsgType=msgType;11. mStartCapture=true;12. mStartCaptureSignal.signal();/启动CaptureSequencer13. 14. returnOK;15. 对比CaptureSequencerThreadloop线程中，在阻塞式的等待mStartCapture = true，并在修改完mStartCapture 后向Threadloop发出signal。Threadloop线程被唤醒后，执行返回一个新的状态机mCaptureState =START：2.1 START状态机主要调用了updateCaptureRequest(l.mParameters, client)函数：cppview plaincopy1. status_tCaptureSequencer:updateCaptureRequest(constParametersms,2. sp&client)3. ATRACE_CALL();4. status_tres;5. if(mCaptureRequest.entryCount()=0)6. res=client-getCameraDevice()-createDefaultRequest(7. CAMERA2_TEMPLATE_STILL_CAPTURE,8. &mCaptureRequest);9. if(res!=OK)10. ALOGE(%s:Camera%d:Unabletocreatedefaultstillimagerequest:11. %s(%d),_FUNCTION_,client-getCameraId(),12. strerror(-res),res);13. returnres;14. 15. 16. 17. res=params.updateRequest(&mCaptureRequest);18. if(res!=OK)19. ALOGE(%s:Camera%d:Unabletoupdatecommonentriesofcapture20. request:%s(%d),_FUNCTION_,client-getCameraId(),21. strerror(-res),res);22. returnres;23. 24. 25. res=params.updateRequestJpeg(&mCaptureRequest);/更新JPEG需要的参数26. if(res!=OK)27. ALOGE(%s:Camera%d:UnabletoupdateJPEGentriesofcapture28. request:%s(%d),_FUNCTION_,client-getCameraId(),29. strerror(-res),res);30. returnres;31. 32. 33. returnOK;34. 该函数和preview模式下的updatePreviewRequest很类似，这里首先检查mCaptureRequest是否是一个空的CameraMetadata，如果为空则由createDefaultRequest来请求HAL3来创建一个Request，其中相应的类型为CAMERA2_TEMPLATE_STILL_CAPTURE。随后分别是使用当前模式下的配置参数来更新CameraMetadatamCaptureRequest中不同tag的参数值，便于传递给HAL3，这个过程是类似与以前Camera1中直接的setParamters操作string的过程。2.2STANDARD_START状态manageStandardCapture该状态是启动整个take picture的重点所在：cppview plaincopy1. CaptureSequencer:CaptureStateCaptureSequencer:manageStandardCapture(2. sp&client)3. status_tres;4. ATRACE_CALL();5. SharedParameters:Lockl(client-getParameters();6. VectoroutputStreams;7. uint8_tcaptureIntent=static_cast(ANDROID_CONTROL_CAPTURE_INTENT_STILL_CAPTURE);8. 9. /*10. *Setupoutputstreamsintherequest11. *-preview12. *-capture/jpeg13. *-callback(ifpreviewcallbacksenabled)14. *-recording(ifrecordingenabled)15. */16. outputStreams.push(client-getPreviewStreamId();/previewStream17. outputStreams.push(client-getCaptureStreamId();/captureStream18. 19. if(l.mParameters.previewCallbackFlags&20. CAMERA_FRAME_CALLBACK_FLAG_ENABLE_MASK)21. outputStreams.push(client-getCallbackStreamId();/capturecallback22. 23. 24. if(l.mParameters.state=Parameters:VIDEO_SNAPSHOT)25. outputStreams.push(client-getRecordingStreamId();26. captureIntent=static_cast(ANDROID_CONTROL_CAPTURE_INTENT_VIDEO_SNAPSHOT);27. 28. 29. res=mCaptureRequest.update(ANDROID_REQUEST_OUTPUT_STREAMS,30. outputStreams);31. if(res=OK)32. res=mCaptureRequest.update(ANDROID_REQUEST_ID,33. &mCaptureId,1);/当前request对应的ID34. 35. if(res=OK)36. res=mCaptureRequest.update(ANDROID_CONTROL_CAPTURE_INTENT,37. &captureIntent,1);38. 39. if(res=OK)40. res=mCaptureRequest.sort();41. 42. 43. if(res!=OK)44. ALOGE(%s:Camera%d:Unabletosetupstillcapturerequest:%s(%d),45. _FUNCTION_,client-getCameraId(),strerror(-res),res);46. returnDONE;47. 48. 49. /CreateacapturecopysinceCameraDeviceBase#capturetakesownership50. CameraMetadatacaptureCopy=mCaptureRequest;51. if(captureCopy.entryCount()=0)52. ALOGE(%s:Camera%d:UnabletocopycapturerequestforHALdevice,53. _FUNCTION_,client-getCameraId();54. returnDONE;55. 56. 57. /*58. *Clearthestreamingrequestforstill-capturepictures59. *(asopposedtoi.e.videosnapshots)60. */61. if(l.mParameters.state=Parameters:STILL_CAPTURE)62. /APIdefinitionoftakePicture()-stoppreviewbeforetakingpic63. res=client-stopStream();64. if(res!=OK)65. ALOGE(%s:Camera%d:Unabletostoppreviewforstillcapture:66. %s(%d),67. _FUNCTION_,client-getCameraId(),strerror(-res),res);68. returnDONE;69. 70. 71. /TODO:CaptureshouldbeatomicwithsetStreamingRequesthere72. res=client-getCameraDevice()-capture(captureCopy);/启动camera3device的capture，提交capturerequest73. if(res!=OK)74. ALOGE(%s:Camera%d:Unabletosubmitstillimagecapturerequest:75. %s(%d),76. _FUNCTION_,client-getCameraId(),strerror(-res),res);77. returnDONE;78. 79. 80. mTimeoutCount=kMaxTimeoutsForCaptureEnd;81. returnSTANDARD_CAPTURE_WAIT;82. 83. 84. CaptureSequencer:CaptureStateCaptureSequencer:manageStandardCaptureWait(85. sp&client)86. status_tres;87. ATRACE_CALL();88. Mutex:Autolockl(mInputMutex);89. 90. /Waitfornewmetadataresult(mNewFrame)91. while(!mNewFrameReceived)92. res=mNewFrameSignal.waitRelative(mInputMutex,kWaitDuration);/waitnew一帧metadata93. if(res=TIMED_OUT)94. mTimeoutCount-;95. break;96. 97. 98. 99. /Approximationoftheshutterbeingclosed100. /-TODO:usethehal3exposurecallbackinCamera3Deviceinstead101. if(mNewFrameReceived&!mShutterNotified)102. SharedParameters:Lockl(client-getParameters();103. /*warning:thisalsolocksaSharedCameraCallbacks*/104. shutterNotifyLocked(l.mParameters,client,mMsgType);105. mShutterNotified=true;106. 107. 108. /WaituntiljpegwascapturedbyJpegProcessor109. while(mNewFrameReceived&!mNewCaptureReceived)110. res=mNewCaptureSignal.waitRelative(mInputMutex,kWaitDuration);/等待JPEG数据111. if(res=TIMED_OUT)112. mTimeoutCount-;113. break;114. 115. 116. if(mTimeoutCountremoveFrameListener(mCaptureId,mCaptureId+1,this);140. 141. mNewFrameReceived=false;142. mNewCaptureReceived=false;143. returnDONE;144. 145. returnSTANDARD_CAPTURE_WAIT;146. 整个函数的处理可以分为以下几个小点：a:Vector outputStreams;outputStreams.push(client-getPreviewStreamId();/preview StreamoutputStreams.push(client-getCaptureStreamId();/capture jpeg StreamoutputStreams.push(client-getCallbackStreamId();/capture callback通过以上的操作，可以很清楚是看到，这里集合了take picture所需要使用到的stream流，对应的模块分别是：streamProcessor、jpegProcessor、CallbackProcessor。这个过程和Preview模式下是类似的，收集当前Camera2Client下的所有stream，并以stream的ID号作为区别。b: 将当前操作所有的stream信息全部加入到CameraMetadatamCaptureRequest res = mCaptureRequest.update(ANDROID_REQUEST_OUTPUT_STREAMS, outputStreams); if (res = OK) res = mCaptureRequest.update(ANDROID_REQUEST_ID, &mCaptureId, 1);/当前request对应的ID ANDROID_REQUEST_ID这项值表明，当前只存在3种Request类型：cppview plaincopy1. 预览RequestmPreviewRequest：mPreviewRequestId(Camera2Client:kPreviewRequestIdStart),2. 拍照RequestmCaptureRequest：mCaptureId(Camera2Client:kCaptureRequestIdStart),3. 录像RequestmRecordingRequest：mRecordingRequestId(Camera2Client:kRecordingRequestIdStart),c. 对于STILL_CAPTURE类型的pictureclient-stopStream()，实现的本质是res = device-clearStreamingRequest()，mRequestThread-clearRepeatingRequests(lastFrameNumber);该函数是将之前Preview模式下的建立的captureRequest作delete处理，之前在预览模式下是将最终生产的capturelist加入到了一个mRepeatingRequests当中，这里通过clear使之为empty，即不会再发送Request和HAL3进行数据的交互。d.Camera3Device capture函数首先关注capture函数传入的参数为captureCopy，即CameraMetadatamCaptureRequest的一个copy值。cppview plaincopy1. status_tCamera3Device:capture(CameraMetadata&request,int64_t*/*lastFrameNumber*/)2. ATRACE_CALL();3. 4. Listrequests;5. requests.push_back(request);/对于一个CameraMetadata转为list6. returncaptureList(requests,/*lastFrameNumber*/NULL);7. cppview plaincopy1. status_tCamera3Device:captureList(constList&requests,2. int64_t*lastFrameNumber)3. ATRACE_CALL();4. 5. returnsubmitRequestsHelper(requests,/*repeating*/false,lastFrameNumber);/非重复的，制定于拍照6. capture函数由Camera3Device来响应处理，其传入的mCaptureRequest转变为一个list，再交由submitRequestsHelper来处理，对比之前Preview下的处理方式，其startstream入口为setStreamingRequest-setStreamingRequestList-submitRequestsHelper。这也表明了最终CameraMetadata类型的Request都是由submitRequestsHelper来完成的，所以convertMetadataListToRequestListLocked这个将CameraMetadata转换为Listsp RequestList的处理过程对两者来说都是一致的。但在后续处理时，对picture模式下的Request，其不再是repeating的处理，mRequestThread-queueRequestList()：cppview plaincopy1. status_tCamera3Device:RequestThread:queueRequestList(2. Listsp&requests,3. /*out*/4. int64_t*lastFrameNumber)5. Mutex:Autolockl(mRequestLock);6. for(Listsp:iteratorit=requests.begin();it!=requests.end();7. +it)8. mRequestQueue.push_back(*it);9. .10. unpauseForNewRequests();11. 12. returnOK;13. 这里直接是将CaptureRequest加入到RequestQueue这个队列之中，区别于Preview模式是将captureRequest加入到一个mRepeatingRequests，重复的将其中的captureRequest加入到RequestQueue。最简单的理解是picture模式下是拍去几帧的数据流即可，Preview模式下是实时的获取帧，前者是几次one snop，后者是连续continuous。到这里为止，可以说CaptureSequence已经完成了START状态机的处理。e. 从START到STANDARD_CAPTURE_WAIT该状态下对应的状态机处理函数为manageStandardCaptureWait：cppview plaincopy1. CaptureSequencer:CaptureStateCaptureSequencer:manageStandardCaptureWait(2. sp&client)3. status_tres;4. ATRACE_CALL();5. Mutex:Autolockl(mInputMutex);6. 7. /Waitfornewmetadataresult(mNewFrame)8. while(!mNewFrameReceived)9. res=mNewFrameSignal.waitRelative(mInputMutex,kWaitDuration);/waitnew一帧metadata10. if(res=TIMED_OUT)11.