Implement Hardware Composer HAL

The Hardware Composer (HWC) HAL composites layers received from SurfaceFlinger, reducing the amount of composition OpenGL ES (GLES) and the GPU perform.

The HWC abstracts objects, such as overlays and 2D blitters, to composite surfaces and communicates with specialized window composition hardware to composite windows. Use the HWC to composite windows instead of having SurfaceFlinger composite with the GPU. Most GPUs aren't optimized for composition, and when the GPU composes layers from SurfaceFlinger, apps can't use the GPU for their own rendering.

HWC implementations should support:

At least four overlays:
- Status bar
- System bar
- App
- Wallpaper/background
Layers that are larger than the display (for example, a wallpaper)
Simultaneous premultiplied per-pixel alpha blending and per-plane alpha blending
Hardware path for protected video playback
RGBA packing order, YUV formats, and tiling, swizzling, and stride properties

To implement the HWC:

Implement a nonoperational HWC and send all composition work to GLES.
Implement an algorithm to delegate composition to the HWC incrementally. For example, delegate only the first three or four surfaces to the overlay hardware of the HWC.
Optimize the HWC. This may include:
- Selecting surfaces that maximize the load taken off the GPU and sending them to the HWC.
- Detecting whether the screen is updating. If it isn't, delegate composition to GLES instead of the HWC to save power. When the screen updates again, continue to offload composition to the HWC.
- Preparing for common use cases such as:
  - The home screen, which includes the status bar, system bar, app window, and live wallpapers
  - Full-screen games in portrait and landscape mode
  - Full-screen video with closed captioning and playback control
  - Protected video playback
  - Split-screen multiwindow

HWC primitives

The HWC provides two primitives, layers and displays, to represent composition work and its interaction with the display hardware. The HWC also provides control over VSYNC and a callback to SurfaceFlinger to notify it when a VSYNC event occurs.

HIDL interface

Android 8.0 and higher uses a HIDL interface called Composer HAL for binderized IPC between the HWC and SurfaceFlinger. The Composer HAL replaces the legacy hwcomposer2.h interface. If vendors provide a Composer HAL implementation of the HWC, Composer HAL directly accepts HIDL calls from SurfaceFlinger. If vendors provide a legacy implementation of the HWC, Composer HAL loads function pointers from hwcomposer2.h, forwarding HIDL calls into function pointer calls.

The HWC provides functions to determine the properties of a given display; to switch between different display configurations (such as 4k or 1080p resolution) and color modes (such as native color or true sRGB); and to turn the display on, off, or into a low-power mode if supported.

Function pointers

If vendors implement Composer HAL directly, SurfaceFlinger calls its functions through HIDL IPC. For example, to create a layer, SurfaceFlinger calls createLayer() on the Composer HAL.

If vendors implement the hwcomposer2.h interface, Composer HAL calls into hwcomposer2.h function pointers. In hwcomposer2.h comments, HWC interface functions are referred to by lowerCamelCase names that don't exist in the interface as named fields. Almost every function is loaded by requesting a function pointer using getFunction provided by hwc2_device_t. For example, the function createLayer is a function pointer of type HWC2_PFN_CREATE_LAYER, which is returned when the enumerated value HWC2_FUNCTION_CREATE_LAYER is passed into getFunction.

For detailed documentation on Composer HAL functions and HWC function passthrough functions, see composer. For detailed documentation on HWC function pointers, see the hwcomposer2.h.

Layer and display handles

Layers and displays are manipulated by handles generated by the HWC. The handles are opaque to SurfaceFlinger.

When SurfaceFlinger creates a new layer, it calls createLayer, which returns of type Layer for direct implementations or hwc2_layer_t for passthrough implementations. When SurfaceFlinger modifies a property of that layer, SurfaceFlinger passes the hwc2_layer_t value into the appropriate modification function along with any other information needed to make the modification. The hwc2_layer_t type is large enough to hold either a pointer or an index.

Physical displays are created by being hotplugged. When a physical display is hotplugged, the HWC creates a handle and passes the handle to SurfaceFlinger through the hotplug callback. Virtual displays are created by SurfaceFlinger calling createVirtualDisplay() to request a display. If the HWC supports virtual display composition, it returns a handle. Then, SurfaceFlinger delegates the displays's composition to the HWC. If the HWC doesn't support virtual display composition, SurfaceFlinger creates the handle and composites the display.

Display composition operations

Once per VSYNC, SurfaceFlinger wakes if it has new content to composite. This new content can be new image buffers from apps or a change in the properties of one or more layers. When SurfaceFlinger wakes it:

Handles transactions, if present.
Latches new graphic buffers, if present.
Performs a new composition, if step 1 or 2 resulted in a change to the display contents.

To perform a new composition, SurfaceFlinger creates and destroys layers or modifies layer states, as applicable. It also updates layers with their current contents, using calls such as setLayerBuffer or setLayerColor. After all layers are updated, SurfaceFlinger calls validateDisplay, which tells the HWC to examine the state of the layers and determine how composition will proceed. By default, SurfaceFlinger attempts to configure every layer such that the layer is composited by the HWC; though in some circumstances, SurfaceFlinger composites layers through the GPU fallback.

After the call to validateDisplay, SurfaceFlinger calls getChangedCompositionTypes to see if the HWC wants any of the layer composition types changed before performing the composition. To accept the changes, SurfaceFlinger calls acceptDisplayChanges.

If any layers are marked for SurfaceFlinger composition, SurfaceFlinger composites them into the target buffer. SurfaceFlinger then calls setClientTarget to give the buffer to the display so that the buffer can be displayed on the screen or further composited with layers that haven't been marked for SurfaceFlinger composition. If no layers are marked for SurfaceFlinger composition, SurfaceFlinger bypasses the composition step.

Finally, SurfaceFlinger calls presentDisplay to tell the HWC to complete the composition process and display the final result.

Multiple displays

Android 10 supports multiple physical displays. When designing an HWC implementation intended for use on Android 7.0 and higher, there are some restrictions not present in the HWC definition:

It's assumed that there's exactly one internal display. The internal display is the display that the initial hotplug reports during boot. After the internal display is hotplugged, it can't be disconnected.
In addition to the internal display, any number of external displays may be hotplugged during normal operation of the device. The framework assumes that all hotplugs after the first internal display are external displays, so if any more internal displays are added, they're categorized incorrectly as Display.TYPE_HDMI instead of Display.TYPE_BUILT_IN.

While the SurfaceFlinger operations described above are performed per-display, they're performed sequentially for all active displays, even if the contents of only one display are updated.

For example, if the external display is updated, the sequence is:

// In Android 9 and lower:

// Update state for internal display
// Update state for external display
validateDisplay(<internal display>)
validateDisplay(<external display>)
presentDisplay(<internal display>)
presentDisplay(<external display>)

// In Android 10 and higher:

// Update state for internal display
// Update state for external display
validateInternal(<internal display>)
presentInternal(<internal display>)
validateExternal(<external display>)
presentExternal(<external display>)

Virtual display composition

Virtual display composition is similar to external display composition. The difference between virtual display composition and physical display composition is that virtual displays send output to a Gralloc buffer instead of to the screen. Hardware Composer (HWC) writes the output to a buffer, provides the completion fence, and sends the buffer to a consumer (such as the video encoder, GPU, CPU, and so on). Virtual displays can use 2D/blitter or overlays if the display pipeline writes to memory.

Modes

Each frame is in one of three modes after SurfaceFlinger calls the validateDisplay() HWC method:

GLES — The GPU composites all layers, writing directly to the output buffer. The HWC isn't involved in composition.
MIXED — The GPU composites some layers to the framebuffer and HWC composites the framebuffer and the remaining layers, writing directly to the output buffer.
HWC — HWC composites all layers and writes directly to the output buffer.

Output format

Virtual display buffer output formats depend on their mode:

GLES mode — The EGL driver sets the output buffer format in dequeueBuffer(), typically RGBA_8888. The consumer must be able to accept the output format the driver sets or the buffer can't be read.
MIXED and HWC modes — If the consumer needs CPU access, the consumer sets the format. Otherwise, the format is IMPLEMENTATION_DEFINED, and Gralloc sets the best format based on the usage flags. For example, Gralloc sets a YCbCr format if the consumer is video encoder and HWC can write the format efficiently.

Note: Android 10 removes the requirement that eglSwapBuffers() dequeues buffers after rendering begins. Buffers may be dequeued immediately.

Synchronization fences

Synchronization (sync) fences are a crucial aspect of the Android graphics system. Fences let CPU work proceed independently from concurrent GPU work, blocking only when there's a true dependency.

For example, when an app submits a buffer that's being produced on the GPU, it also submits a sync fence object. This fence signals when the GPU has finished writing into the buffer.

The HWC requires that the GPU finish writing buffers before buffers are displayed. Sync fences are passed through the graphics pipeline with buffers and signal when buffers are written. Before a buffer is displayed, the HWC checks if the sync fence has signaled, and if it has, it displays the buffer.

For more information about sync fences see Hardware Composer Integration.