Android N VR代码简析.doc

Android N VR代码简析虽然Google早在2016的IO大会上就说DayDream会随着Android N发布，但是从Android N的前几个版本的代码来看，都是没有集成的，直到Android-7.0.0_r14这个版本，才可以看到DayDream的一些简单代码，跟之前预见的一样，DayDream跟三星的GearVR采用的是同样的架构，都分离出来一个单独的VR SDK，比如三星的Oculus SDK，Google的Google VR SDK，应用基于他们的SDK开发应用程序，然后运行在支持这些SDK的Android系统上面。单独分离出一个SDK，除了方便应用开发者之外，主要的目的是用来把VR的一些核心算法隐藏起来，比如用来减少延迟的ATW异步时间扭曲算法就是被封装在这里，是闭源不公开的，众所周知，在手机上面运行VR的时候一个最大的挑战就是延迟，当一副画面的延迟超过20MS，人就会感觉不舒服恶心，严重影响用户体验，一个好的VR产品都会尽量避免延迟。目前在Mobile VR上主要有以下几种方式来减少延迟：1、 硬件层面的优化 提升传感器的采样频率,减少刷新率与传感器频率的同步等待时间消耗。 提升传感器的精度, 减少对采样数据进行稳定性过滤产生的延迟。 采用有线传输也有一部分原因是出于延迟的考虑。 屏幕使用OLED替代LCD, 减少像素颜色切换的时间。 提升屏幕刷新率, 主流的屏幕是60Hz, 那每帧就是16.67ms; 如果提升到90Hz, 那每帧就是11.11ms. 双GPU渲染，一个GPU负责左眼的画面，一个GPU负责右眼的渲染。2、 软件层面的优化 多分辨率渲染，VR画面的中心部分可以高分辨率渲染，周边部分低分辨率渲染 上面提过的ATW算法，一个具有高优先级的线程，以一定的帧率，不断的根据Android渲染出来的画面生成一个新的画面，然后投递到屏幕。 Front Buffer Render，即只有一个render buffer，应用赶在下一个屏幕刷新周期到来之前把画面更新到这个render buffer，然后屏幕使用这个render buffer更新显示内容，这个需要修改GPU和Display的Driver部分代码。Android N的DayDream对硬件和软件的一些需求，其实都是根据上面这些优化手段提出来的，具体到Android N的代码来说，它提供了一个SDK，应该是解决了ATW算法和多分辨率渲染等优化问题，然后新增了一个系统服务VrManagerService和 vr HAL层， 应用通过VrManagerService来设置系统进入VR模式，VrManagerService又是通过vr HAL层去实现具体模式的切换的。Implement this HAL to receive callbacks when a virtual reality (VR)application is being used. VR applications characteristically have a number of special display and performance requirements, including: Low sensor latencyTotal end-to-end latency from the IMU, accelerometer,and gyro to an application-visible callback must be extremely low (5ms typically). This is required for HIFI sensor support. Low display latencyTotal end-to-end latency from the GPU draw calls to the actual display update must be as low as possible. This is achieved by using SurfaceFlinger in a single-buffered mode, and assuring that draw calls are synchronized with the display scanout correctly. This behavior is exposed via an EGL extension to applications. See below for the EGL extensions needed for this. Low-persistence displayDisplay persistence settings must be set as low as possible while still maintaining a reasonable brightness. For a typical display running at 60Hz, pixels should be illuminated for =3.5ms to be considered low-persistence. This avoids ghosting during movements in a VR setting, and should be enabled from the lights.h HAL when BRIGHTNESS_MODE_LOW_PERSISTENCE is set. Consistent performance of the GPU and CPUWhen given a mixed GPU/CPU workload for a VR application with bursts of work at regular intervals several times a frame, the CPU scheduling should ensure that the application render thread work is run consistently within 1ms of when scheduled, and completed before the end of the draw window. To this end, a single CPU core must be reserved for solely for the currently running VR applications render thread while in VR mode, and made available in the top-app cpuset. Likewise, an appropriate CPU, GPU, and bus clockrate must be maintained to ensure that the rendering workload finishes within the time allotted to render each frame when the POWER_HINT_SUSTAINED_PERFORMANCE flag has been set in the power.h HAL while in VR mode when the device is not being thermally throttled. Required EGL extensions must be presentAny GPU settings required to allow the above capabilities are required, including the EGL extensions:EGL_ANDROID_create_native_client_buffer, EGL_ANDROID_front_buffer_auto_refresh,EGL_EXT_protected_content, EGL_KHR_mutable_render_buffer,EGL_KHR_reusable_sync, and EGL_KHR_wait_sync. Accurate thermal reportingAccurate thermal temperatures and limits must be reported in the thermal.h HAL. Specifically, the current skin temperature must accurately be reported for DEVICE_TEMPERATURE_SKIN and the vr_throttling_threshold reported for this device must accurately report the temperature limit above which the devices thermal governor throttles the CPU, GPU, and/or bus clockrates below the minimum necessary for consistent performance (see previous bullet point).In general, vendors implementing this HAL are expected to use set_vr_mode as a hint to enable VR-specific performance tuning needed for any of the above requirements, and to turn on any device features optimal for VR display modes. The set_vr_mode call may simply do nothing if no optimizations are available or necessary to meet the above requirements.No methods in this HAL will be called concurrently from the Android framework. typedef struct vr_module /* * Common methods of the module. This *must* be the first member of * vr_module as users of this structure may cast a hw_module_t to a * vr_module pointer in contexts where its known that the hw_module_t * references a vr_module. */ struct hw_module_t common; /* * Convenience method for the HAL implementation to set up any state * needed * at runtime startup. This is called once from the VrManagerService * during * its boot phase. No methods from this HAL will be called before init. */ void (*init)(struct vr_module *module); /* * Set the VR mode state. Possible states of the enabled parameter * are: * false - VR mode is disabled, turn off all VR-specific settings. * true - VR mode is enabled, turn on all VR-specific settings. * This is called whenever the the Android system enters or leaves VR * mode. * This will typically occur when the user switches to or from a VR * application * that is doing stereoscopic rendering. */ void (*set_vr_mode)(struct vr_module *module, bool enabled); /* Reserved for future use. Must be NULL. */ void* reserved8 - 2; vr_module_t;这个HAL层的实现是可选择的，毕竟不是每个Android N的手机都必须要支持DayDream平台。VrManagerService是一个系统级别的服务，它的主要作用描述如下: Service tracking whether VR mode is active, and notifying listening services of state changes. Services running in system server may modify the state of VrManagerService via the interface in VrManagerInternal, and may register to receive callbacks when the system VR mode changes via the interface given in VrStateListener. Device vendors may choose to receive VR state changes by implementing the VR mode HAL, e.g.: hardware/libhardware/modules/vr In general applications may enable or disable VR mode by calling android.app.Activity.setVrModeEnabled An application may also implement a service to be run while in VR mode by implementing android.service.vr.VrListenerService.从描述来看，它是一个vr模式的控制中心，system_server里面的其他服务除了可以通过VrManagerInternal这个接口去控制vr模式，还可以通过VrStateListener接口注册一些vr状态变化的通知，它也是通过底层的vr HAL层去实际打开VR模式的。在system_server进程里面可以通过如下两种方式来访问到这个服务： 通过binder查找服务的方式: IVrManager vrManager = IVrManager.Stub.asInterface( ServiceManager.getService(VrManagerService.VR_MANAGER_BINDER_SERVICE); if (vrManager != null) try vrManager.registerListener(mVrStateCallbacks); mVrModeEnabled = vrManager.getVrModeState(); catch (RemoteException re) System_server进程内部通过localservice的方式: mVrManagerInternal = getLocalService(VrManagerInternal.class);在Android N上面Activity提供了一个新的接口setVrModeEnabled,这个函数会通过binder调用AMS的setVrmode接口，而在AMS的setVrmode最终调用的是VrManagerService的setVrmode函数，VrManagerService又是通过vr HAL层去打开这个VR模式，通过这样的一个调用链，使手机进入VR模式。/enabled:true进入VR模式，false退出VR模式./requestedComponent 指定一个继承android.service.vr.VrListenerService 的服务,这个服务由VrManagerService来调用，如果没有指定，不会进入VR模式.public void setVrModeEnabled(boolean enabled, NonNull ComponentName requestedComponent)throws PackageManager.NameNotFoundException try if (ActivityManagerNative.getDefault().setVrMode(mToken, enabled, requestedComponent)!= 0) throw new PackageManager.NameNotFoundException( requestedComponent.flattenToString(); catch (RemoteException e) / pass 上面说过system_server进程里面的其他服务和其他进程可以通过VrStateListener的接口获知当前的VR模式变化信息，搜索了一下目前系统里面实现了VrStateListener接口的有以下几处代码:Android N 实现了VrStateListener的地方VR之所以被人喜欢，是因为能获得一种沉浸式体验，所以在VR模式下，人们是不喜欢被其他事情打扰的，Powe