The Hardware Composer HAL (HWC) is used by SurfaceFlinger to composite surfaces to the screen. The HWC abstracts objects such as overlays and 2D blitters and helps offload some work that would normally be done with OpenGL.
Android 7.0 includes a new version of HWC (HWC2) used by SurfaceFlinger to talk to specialized window composition hardware. SurfaceFlinger contains a fallback path that uses the 3D graphics processor (GPU) to perform the task of window composition, but this path is not ideal for a couple of reasons:
- Typically, GPUs are not optimized for this use case and may use more power than necessary to perform composition.
- Any time SurfaceFlinger is using the GPU for composition is time that applications cannot use the processor for their own rendering, so it is preferable to use specialized hardware for composition instead of the GPU whenever possible.
As the physical display hardware behind the Hardware Composer abstraction layer can vary from device to device, it's difficult to give recommendations on specific features. In general, use the following guidance:
- The HWC should support at least four overlays (status bar, system bar, application, and wallpaper/background).
- Layers can be bigger than the screen, so the HWC should be able to handle layers that are larger than the display (for example, a wallpaper).
- Pre-multiplied per-pixel alpha blending and per-plane alpha blending should be supported at the same time.
- The HWC should be able to consume the same buffers the GPU, camera, and
video decoder are producing, so supporting some of the following
properties is helpful:
- RGBA packing order
- YUV formats
- Tiling, swizzling, and stride properties
- To support protected content, a hardware path for protected video playback must be present.
The general recommendation is to implement a non-operational HWC first; after the structure is complete, implement a simple algorithm to delegate composition to the HWC (for example, delegate only the first three or four surfaces to the overlay hardware of the HWC).
Focus on optimization, such as intelligently selecting the surfaces to send to the overlay hardware that maximizes the load taken off of the GPU. Another optimization is to detect whether the screen is updating; if it isn't, delegate composition to OpenGL instead of the HWC to save power. When the screen updates again, continue to offload composition to the HWC.
Prepare for common use cases, such as:
- Full-screen games in portrait and landscape mode
- Full-screen video with closed captioning and playback control
- The home screen (compositing the status bar, system bar, application window, and live wallpapers)
- Protected video playback
- Multiple display support
These use cases should address regular, predictable uses rather than edge cases that are rarely encountered (otherwise, optimizations will have little benefit). Implementations must balance two competing goals: animation smoothness and interaction latency.
HWC2 interface activities
HWC2 provides a few primitives (layer, display) to represent composition work and its interaction with the display hardware.
A layer is the most important unit of composition; every layer has a set of properties that define how it interacts with other layers. Property categories include the following:
- Positional. Defines where the layer appears on its display. Includes information such as the positions of a layer's edges and its Z order relative to other layers (whether it should be in front of or behind other layers).
- Content. Defines how content displayed on the layer should be presented within the bounds defined by the positional properties. Includes information such as crop (to expand a portion of the content to fill the bounds of the layer) and transform (to show rotated or flipped content).
- Composition. Defines how the layer should be composited with other layers. Includes information such as blending mode and a layer-wide alpha value for alpha compositing.
- Optimization. Provides information not strictly necessary to correctly composite the layer, but which can be used by the HWC device to optimize how it performs composition. Includes information such as the visible region of the layer and which portion of the layer has been updated since the previous frame.
A display is another important unit of composition. Every layer can be present only on one display. A system can have multiple displays, and displays can be added or removed during normal system operations. This addition/removal can come at the request of the HWC device (typically in response to an external display being plugged into or removed from the device, called hotplugging), or at the request of the client, which permits the creation of virtual displays whose contents are rendered into an off-screen buffer instead of to a physical display.
HWC2 provides functions to determine the properties of a given display, to switch between different configurations (e.g., 4k or 1080p resolution) and color modes (e.g., native color or true sRGB), and to turn the display on, off, or into a low-power mode if supported.
In addition to layers and displays, HWC2 also provides control over the hardware vertical sync (VSYNC) signal along with a callback into the client to notify it of when a vsync event has occurred.
In this section and in HWC2 header comments, HWC interface functions are
referred to by lowerCamelCase names that do not actually exist in the interface
as named fields. Instead, almost every function is loaded by requesting a
function pointer using
getFunction provided by
hwc2_device_t. For example, the function
is a function pointer of type
HWC2_PFN_CREATE_LAYER, which is
returned when the enumerated value
For detailed documentation on functions (including functions required for every HWC2 implementation), refer to the HWC2 header.
Layer and display handles
Layers and displays are manipulated by opaque handles.
When SurfaceFlinger wants to create a new layer, it calls the
createLayer function, which then returns an opaque handle of type
hwc2_layer_t. From that point on, any time SurfaceFlinger wants to
modify a property of that layer, it passes that
into the appropriate modification function, along with any other information
needed to make the modification. The
hwc2_layer_t type was made
large enough to be able to hold either a pointer or an index, and it will be
treated as opaque by SurfaceFlinger to provide HWC implementers maximum
Most of the above also applies to display handles, though handles are created
differently depending on whether they are hotplugged (where the handle is passed
through the hotplug callback) or requested by the client as a virtual display
(where the handle is returned from
Display composition operations
Once per hardware vsync, SurfaceFlinger wakes if it has new content to composite. This new content could be new image buffers from applications or just a change in the properties of one or more layers. When it wakes, it performs the following steps:
- Apply transactions, if present. Includes changes in the properties of layers specified by the window manager but not changes in the contents of layers (i.e., graphic buffers from applications).
- Latch new graphic buffers (acquire their handles from their respective applications), if present.
- If step 1 or 2 resulted in a change to the display contents, perform a new composition (described below).
Steps 1 and 2 have some nuances (such as deferred transactions and presentation timestamps) that are outside the scope of this section. However, step 3 involves the HWC interface and is detailed below.
At the beginning of the composition process, SurfaceFlinger will create and
destroy layers or modify layer state as applicable. It will also update the
layers with their current contents, using calls such as
setLayerColor. After all layers have
been updated, it will call
validateDisplay, which tells the device
to examine the state of the various layers and determine how composition will
proceed. By default, SurfaceFlinger usually attempts to configure every layer
such that it will be composited by the device, though there may be some
circumstances where it will mandate that it be composited by the client.
After the call to
validateDisplay, SurfaceFlinger will follow up
with a call to
getChangedCompositionTypes to see if the device
wants any of the layers' composition types changed before performing the actual
composition. SurfaceFlinger may choose to:
- Change some of the layer composition types and re-validate the display.
acceptDisplayChanges, which has the same effect as changing the composition types as requested by the device and re-validating without actually having to call
In practice, SurfaceFlinger always takes the latter path (calling
acceptDisplayChanges) though this behavior may change in the
At this point, the behavior differs depending on whether any of the layers
have been marked for client composition. If any (or all) layers have been marked
for client composition, SurfaceFlinger will now composite all of those layers
into the client target buffer. This buffer will be provided to the device using
setClientTarget call so that it may be either displayed
directly on the screen or further composited with layers that have not been
marked for client composition. If no layers have been marked for client
composition, then the client composition step is bypassed.
Finally, after all of the state has been validated and client composition has
been performed if needed, SurfaceFlinger will call
This is the HWC device's cue to complete the composition process and display the
Multiple displays in Android 7.0
While the HWC2 interface is quite flexible when it comes to the number of displays in the system, the rest of the Android framework is not yet as flexible. When designing a HWC2 implementation intended for use on Android 7.0, there are some additional restrictions not present in the HWC definition itself:
- It is assumed that there is exactly one primary display; that is, that there is one physical display that will be hotplugged immediately during the initialization of the device (specifically after the hotplug callback is registered).
- In addition to the primary display, exactly one external display may be hotplugged during normal operation of the device.
While the SurfaceFlinger operations described above are performed per-display (eventual goal is to be able to composite displays independently of each other), they are currently performed sequentially for all active displays, even if only the contents of one display are updated.
For example, if only the external display is updated, the sequence is:
// Update state for internal display // Update state for external display validateDisplay(<internal display>) validateDisplay(<external display>) presentDisplay(<internal display>) presentDisplay(<external display>)
Synchronization (sync) fences are a crucial aspect of the Android graphics system. Fences allow CPU work to proceed independently from concurrent GPU work, blocking only when there is a true dependency.
For example, when an application submits a buffer that is being produced on the GPU, it will also submit a fence object; this fence signals only when the GPU has finished writing into the buffer. Since the only part of the system that truly needs the GPU write to have finished is the display hardware (the hardware abstracted by the HWC HAL), the graphics pipeline is able to pass this fence along with the buffer through SurfaceFlinger to the HWC device. Only immediately before that buffer would be displayed does the device need to actually check that the fence has signaled.
Sync fences are integrated tightly into HWC2 and organized in the following categories:
- Acquire fences are passed along with input buffers to the
setClientTargetcalls. These represent a pending write into the buffer and must signal before the HWC client or device attempts to read from the associated buffer to perform composition.
- Release fences are retrieved after the call to
getReleaseFencescall and are passed back to the application along with buffers that will be replaced during the next composition. These represent a pending read from the buffer, and must signal before the application attempts to write new contents into the buffer.
- Retire fences are returned, one per frame, as part of the call to
presentDisplayand represent when the composition of this frame has completed, or alternately, when the composition result of the prior frame is no longer needed. For physical displays, this is when the current frame appears on the screen and can also be interpreted as the time after which it is safe to write to the client target buffer again (if applicable). For virtual displays, this is the time when it is safe to read from the output buffer.
Changes in HWC2
The meaning of sync fences in HWC 2.0 has changed significantly relative to previous versions of the HAL.
In HWC v1.x, the release and retire fences were speculative. A release fence for a buffer or a retire fence for the display retrieved in frame N would not signal any sooner than frame N + 1. In other words, the meaning of the fence was "the content of the buffer you provided for frame N is no longer needed." This is speculative because in theory SurfaceFlinger may not run again after frame N for an indeterminate period of time, which would leave those fences unsignaled for the same period.
In HWC 2.0, release and retire fences are non-speculative. A release or
retire fence retrieved in frame N will signal as soon as the content of the
associated buffers replaces the contents of the buffers from frame N - 1, or in
other words, the meaning of the fence is "the content of the buffer you provided
for frame N has now replaced the previous content." This is non-speculative,
since this fence should signal shortly after
called as soon as the hardware presents this frame's content.
For implementation details, refer to the HWC2 header.