High-Resolution Audio Effects

The Android 9 release includes the following improvements for high-resolution audio:

  • Effect processing is converted from int16 to float format, one of the few remaining int16 pipelines in AudioFlinger. This effect processing change requires implementation work from vendors providing custom effects. That work is described in the following sections.
  • The following improvements do not require any partner implementation:
    • The number of simultaneous client output tracks increases from 14 to 40, as limited client AudioTracks have been an issue for applications in Android 8.x.
    • Maximum client/server memory increases from 4MB to 32MB (depending on total device memory) to allow more simultaneous high-resolution audio tracks.
    • Total mixed tracks increases from 32 to 256 to prevent resource contention between applications and the System UI.

Output effect changes

Prior to the Android 9 release, effect chain processing was implemented in stereo int16 sample format. This had several limitations:

  • All output effects forced conversion from floating point audio data to int16, causing loss of precision.
  • Output effects were rejected from output sinks with a channel count greater than 2.

In the Android 9 release, the effect chain processing pipeline is upgraded to support the multichannel float format. Key points:

  • Android software effects are already migrated to stereo float.
  • Legacy effects are supported with format adapters, which convert float to int16 as needed.

Implementing output effects

A reference implementation for output effects is available under frameworks/av/media/libeffects.

Partners implementing their own custom output effects should do the following for the Android 9 release:

  • Update output effects to support the multichannel float format:
    • Int16 processing support is no longer required.
    • Support output channel counts from 2 - 8 (for future compatibility consider counts from 1 - 30).
    • Support input channel counts matching output channel count for insert effects. Auxiliary effects will continue to see an input channel count of 1 (mono).
    • Support both channel position masks (canonical) and channel index masks of (1 << n) - 1.
  • If you must continue to support legacy vendor output effects and cannot update them, then verify legacy code as follows:
    • Legacy output (insert) effects must reject unsupported configurations in EFFECT_CMD_SET_CONFIG.
      • Check that format is int16.
      • Check that the input and output channel masks are stereo.
      • If not, return -EINVAL.
    • Legacy output (auxiliary) effects are configured by AudioFlinger with a mono input channel mask and potentially multichannel output channel masks, depending on whether the output sink is multichannel. They must reject unsupported configurations in EFFECT_CMD_SET_CONFIG.
      • Check that the format is int16.
      • Check that the input channel mask is mono and the output channel mask is stereo.
      • If not, return -EINVAL.
    • Verify legacy code. Please do not assume it will work!