Implementing graphics

To implement the Android graphics HAL, review the following requirements, implementation details, and testing advice.

Requirements

Android graphics support requires the following components:

  • EGL driver
  • OpenGL ES 1.x driver
  • OpenGL ES 2.0 driver
  • OpenGL ES 3.x driver (optional)
  • Vulkan (optional)
  • Gralloc HAL implementation
  • Hardware Composer HAL implementation

Implementation

OpenGL and EGL drivers

You must provide drivers for EGL, OpenGL ES 1.x, and OpenGL ES 2.0 (support for OpenGL 3.x is optional). Key considerations include:

  • GL driver must be robust and conformant to OpenGL ES standards.
  • Do not limit the number of GL contexts. Because Android allows apps in the background and tries to keep GL contexts alive, you should not limit the number of contexts in your driver.
  • It is common to have 20-30 active GL contexts at once, so be mindful of the amount of memory allocated for each context.
  • Support the YV12 image format and other YUV image formats that come from other components in the system, such as media codecs or the camera.
  • Support the mandatory extensions: GL_OES_texture_external, EGL_ANDROID_image_native_buffer, and EGL_ANDROID_recordable. In addition, the EGL_ANDROID_framebuffer_target extension is required for Hardware Composer v1.1 and higher.

We highly recommend also supporting EGL_ANDROID_blob_cache, EGL_KHR_fence_sync, EGL_KHR_wait_sync, and EGL_ANDROID_native_fence_sync.

Note: The OpenGL API exposed to app developers differs from the OpenGL implemented on the device. Apps cannot directly access the GL driver layer and must go through the interface provided by the APIs.

Pre-rotation

Many hardware overlays do not support rotation (and even if they do it costs processing power); the solution is to pre-transform the buffer before it reaches SurfaceFlinger. Android supports a query hint (NATIVE_WINDOW_TRANSFORM_HINT) in ANativeWindow to represent the most likely transform to be applied to the buffer by SurfaceFlinger. GL drivers can use this hint to pre-transform the buffer before it reaches SurfaceFlinger so when the buffer arrives, it is correctly transformed.

For example, when receiving a hint to rotate 90 degrees, generate and apply a matrix to the buffer to prevent it from running off the end of the page. To save power, do this pre-rotation. For details, see the ANativeWindow interface defined in system/core/include/system/window.h.

Gralloc HAL

The graphics memory allocator allocates memory requested by image producers. You can find the interface definition of the HAL at hardware/libhardware/include/hardware/gralloc.h.

Protected buffers

The gralloc usage flag GRALLOC_USAGE_PROTECTED allows the graphics buffer to be displayed only through a hardware-protected path. These overlay planes are the only way to display DRM content (DRM-protected buffers cannot be accessed by SurfaceFlinger or the OpenGL ES driver).

DRM-protected video can be presented only on an overlay plane. Video players that support protected content must be implemented with SurfaceView. Software running on unprotected hardware cannot read or write the buffer; hardware-protected paths must appear on the Hardware Composer overlay (i.e., protected videos will disappear from the display if Hardware Composer switches to OpenGL ES composition).

For details on protected content, see DRM.

Hardware Composer HAL

The Hardware Composer HAL (HWC) is used by SurfaceFlinger to composite surfaces to the screen. It abstracts objects such as overlays and 2D blitters and helps offload some work that would normally be done with OpenGL. For details on the HWC, see Hardware Composer HAL.

VSYNC

VSYNC synchronizes certain events to the refresh cycle of the display. Applications always start drawing on a VSYNC boundary, and SurfaceFlinger always composites on a VSYNC boundary. This eliminates stutters and improves visual performance of graphics. For details on VSYNC, see Implementing VSYNC.

Vulkan

Vulkan is a low-overhead, cross-platform API for high-performance 3D graphics. Like OpenGL ES, Vulkan provides tools for creating high-quality, real-time graphics in applications. Vulkan advantages include reductions in CPU overhead and support for the SPIR-V Binary Intermediate language. For details on Vulkan, see Implementing Vulkan.

Virtual displays

Android added platform support for virtual displays in Hardware Composer v1.3. The virtual display composition is similar to the physical display: Input layers are described in prepare(), SurfaceFlinger conducts GPU composition, and layers and GPU framebuffer are provided to Hardware Composer in set(). For details on virtual displays, see Implementing Virtual Displays.

Testing

For benchmarking, use the following flow by phase:

  • Specification. When initially specifying the device (such as when using immature drivers), use predefined (fixed) clocks and workloads to measure frames per second (fps) rendered. This gives a clear view of hardware capabilities.
  • Development. As drivers mature, use a fixed set of user actions to measure the number of visible stutters (janks) in animations.
  • Production. When a device is ready for comparison against competitors, increase the workload until stutters increase. Determine if the current clock settings can keep up with the load. This can help you identify where to slow the clocks and reduce power use.

For help deriving device capabilities during the specification phase, use the Flatland tool at platform/frameworks/native/cmds/flatland/. Flatland relies upon fixed clocks and shows the throughput achievable with composition-based workloads. It uses gralloc buffers to simulate multiple window scenarios, filling in the window with GL then measuring the compositing.

Note: Flatland uses the synchronization framework to measure time, so your implementation must support the synchronization framework.