Android 13 introduces a vendor-configurable static library called libtonemap,
which defines tone mapping operations and is shared with the SurfaceFlinger
process and Hardware Composer (HWC) implementations. This feature enables OEMs
to define and share their display tone mapping algorithms between the framework
and vendors, lessening a mismatch in tone mapping.
In Android 12 and lower, display-specific tone mapping operations weren't shared between the HWC, SurfaceFlinger, and apps. Depending on the rendering path, for HDR content, this led to mismatches in image quality, where the HDR content was tone mapped to an output space in different ways. This was perceptible in scenarios such as screen rotation, where the composition strategy changes between the GPU and the DPU, and in differences in rendering behavior between TextureView and SurfaceView.
This page describes the interface, customization, and validation details of the
libtonemap library.
Interface to the tone mapping library
The libtonemap
library contains CPU-backed implementations and SkSL shaders, which can be
plugged in by SurfaceFlinger for GPU-backend composition and by the HWC for
generating a tone mapping look-up table (LUT). The entry point to libtonemap
is
android::tonemap::getToneMapper,
which returns an object that implements the
ToneMapper
interface.
The ToneMapper interface supports the following capabilities:
Generate a tone-mapping LUT
The interface
ToneMapper::lookupTonemapGainis a CPU implementation of the shader defined inlibtonemap_LookupTonemapGain. This is used by unit tests in the framework, and can be used by partners for assistance with generating a tone-mapping LUT inside their color pipeline.libtonemap_LookupTonemapGaintakes in color values in absolute, unnormalized linear space, both in linear RGB and in XYZ, and returns a float describing how much to multiply the input colors in linear space.Generate an SkSL shader
The interface
ToneMapper::generateTonemapGainShaderSkSLreturns an SkSL shader string, given a source and destination dataspace. The SkSL shader is plugged into the Skia implementation forRenderEngine, the GPU-accelerated compositing component for SurfaceFlinger. The shader is also plugged intolibhwui, so that HDR-to-SDR tone mapping can be performed efficiently forTextureView. Because the generated string is in-lined into other SkSL shaders used by Skia, the shader must adhere to the following rules:- The shader string must have an entry point with the
float libtonemap_LookupTonemapGain(vec3 linearRGB, vec3 xyz)signature, wherelinearRGBis the value of the absolute nits of the RGB pixels in linear space andxyzislinearRGBconverted into XYZ. - Any helper methods used by the shader string must be prefixed with the
string
libtonemap_so that framework shader definitions don't conflict. Similarly, input uniforms must be prefixed within_libtonemap_.
- The shader string must have an entry point with the
Generate SkSL uniforms
The interface
ToneMapper::generateShaderSkSLUniformsreturns the following, given a metadatastructdescribing metadata from different HDR standards and display conditions:A list of uniforms that are bound by an SkSL shader.
The uniform values
in_libtonemap_displayMaxLuminanceandin_libtonemap_inputMaxLuminance. These values are used by framework shaders when scaling the input intolibtonemap, and normalizing the output as applicable.The process of generating uniforms is agnostic to the input and output dataspace.
Customization
The reference implementation of the
libtonemap
library produces acceptable results. However, because the tone mapping algorithm
used by GPU composition can differ from that used by the DPU composition, using
the reference implementation can cause flicker in some scenarios such as the
rotation animation. Customization can resolve such vendor-specific image quality
issues.
OEMs are strongly encouraged to override the implementation of libtonemap to
define their own ToneMapper subclass, which is returned by getToneMapper.
When customizing the implementation, partners are expected to do one of the
following:
- Modify the implementation of
libtonemapdirectly. - Define their own static library, compile the library as a standalone, and
replace
libtonemaplibrary's.afile with the one generated from their custom library.
Vendors don't need to modify any kernel code, but multiple vendors must communicate details about the DPU tone-mapping algorithms for proper implementation.
Validation
Follow these steps to validate your implementation:
Play HDR videos on screen of any HDR standards that your display system supports, such as HLG, HDR10, HDR10+, or DolbyVision.
Toggle GPU composition to ensure that there's no user perceptible flicker.
Use the following
adbcommand to toggle the GPU composition:adb shell service call SurfaceFlinger 1008 i32 <0 to enable HWC composition, 1 to force GPU composition>
Common issues
The following issues can occur with this implementation:
Banding is caused when the render target used by GPU composition is of lower precision than the typical value for HDR content. For example, banding can occur when an HWC implementation supports opaque 10-bit formats for HDR such as RGBA1010102 or P010, but requires that GPU composition writes to an 8-bit format like RGBA8888 to support alpha.
A subtle color shift is caused by quantization differences if the DPU operates at a different precision than the GPU.
Each of these issues is related to the relative precision differences of the underlying hardware. A typical workaround is to ensure that there's a dithering step in the lower precision paths, making any precision differences less human perceptible.