The BufferQueue class connects components that generate buffers of graphical data (producers) to components that accept the data for display or further processing (consumers). Nearly everything that moves buffers of graphical data through the system relies on BufferQueue.
The Gralloc memory allocator performs buffer allocations and is
implemented through two vendor-specific HIDL interfaces (see
hardware/interfaces/graphics/allocator/ and
hardware/interfaces/graphics/mapper/). The
allocate() function takes expected arguments (width, height, pixel
format) as well as a set of usage flags.
BufferQueue producers and consumers
Consumers create and own the BufferQueue data structure and can exist in
different processes than their producers. When a producer needs a buffer, it
requests a free buffer from BufferQueue by calling
dequeueBuffer(), specifying the buffers's width, height,
pixel format, and usage flags. The producer then populates the buffer and
returns the buffer to the queue by calling queueBuffer(). Next,
the consumer acquires the buffer with acquireBuffer() and makes
use of the buffer contents. When the consumer is done, it returns the buffer to
the queue by calling releaseBuffer(). The sync framework controls
how buffers move through the Android graphics pipeline.
Some characteristics of the BufferQueue, such as the maximum number of buffers it can hold, are determined jointly by the producer and the consumer. However, the BufferQueue allocates buffers as it needs them. Buffers are retained unless the characteristics change; for example, if a producer requests buffers with a different size, old buffers are freed and new buffers are allocated on demand.
Buffer contents are never copied by BufferQueue, as moving that much data around is inefficient. Instead, buffers are always passed by a handle.
Track BufferQueue with Systrace
To understand how graphics buffers move around, use Systrace, a tool that records device activity over a short period of time. The system-level graphics code is well instrumented, as is much of the relevant app framework code.
To use Systrace, enable the gfx, view, and
sched tags. BufferQueue objects are displayed in the trace.
As an example, if you take a trace while
Grafika's Play
video (SurfaceView) is running, the row labeled SurfaceView tells you
how many buffers were queued up at any given time.
The value increments while the app is active, which triggers the rendering of frames by the MediaCodec decoder. The value decrements while SurfaceFlinger is working and consuming buffers. When showing video at 30 fps, the queue's value varies from 0 to 1 because the ~60 fps display can keep up with the source. SurfaceFlinger wakes only when there's work to be done, not 60 times per second. The system tries to avoid work and disables VSYNC if nothing is updating the screen.
If you switch to Grafika's Play video (TextureView) and grab a new trace,
you see a row labeled
com.android.grafika / com.android.grafika.PlayMovieActivity.
This is the main UI layer, which is another BufferQueue. Because TextureView
renders into the UI layer rather than a separate layer, all of the
video-driven updates are displayed here.
Gralloc
The Gralloc allocator HAL
hardware/libhardware/include/hardware/gralloc.h
performs buffer allocations through usage flags. Usage flags include attributes
such as:
- How often the memory will be accessed from software (CPU)
- How often the memory will be accessed from hardware (GPU)
- Whether the memory will be used as an OpenGL ES (GLES) texture
- Whether the memory will be used by a video encoder
For example, if a producer's buffer format specifies RGBA_8888 pixels, and
the producer indicates that the buffer will be accessed from software (meaning an app
will touch pixels on the CPU), Gralloc creates a buffer with 4 bytes per pixel
in R-G-B-A order. If instead, a producer specifies its buffer will be only
accessed from hardware and as a GLES texture, Gralloc can do anything the GLES
driver wants, such as BGRA ordering, nonlinear swizzled layouts, and alternative
color formats. Allowing the hardware to use its preferred format can
improve performance.
Some values can't be combined on certain platforms. For example, the video
encoder flag may require YUV pixels, so adding software access and specifying
RGBA_8888 fails.
The handle returned by Gralloc can be passed between processes through Binder.
Protected buffers
The Gralloc usage flag GRALLOC_USAGE_PROTECTED allows the
graphics buffer to be displayed only through a hardware-protected path. These
overlay planes are the only way to display DRM content (DRM-protected buffers
can't be accessed by SurfaceFlinger or the OpenGL ES driver).
DRM-protected video can be presented only on an overlay plane. Video players that support protected content must be implemented with SurfaceView. Software running on unprotected hardware can't read or write the buffer; hardware-protected paths must appear on the Hardware Composer overlay (that is, protected videos disappear from the display if Hardware Composer switches to OpenGL ES composition).
For details on protected content, see DRM.