to top

Audio Latency

In this document

Audio latency is the time delay as an audio signal passes through a system. For a complete description of audio latency for the purposes of Android compatibility, see Section 5.5 Audio Latency in the Android CDD. See Design For Reduced Latency for an understanding of Android's audio latency-reduction efforts.

This page focuses on the contributors to output latency, but a similar discussion applies to input latency.

Assuming the analog circuitry does not contribute significantly, then the major surface-level contributors to audio latency are the following:

  • Application
  • Total number of buffers in pipeline
  • Size of each buffer, in frames
  • Additional latency after the app processor, such as from a DSP

As accurate as the above list of contributors may be, it is also misleading. The reason is that buffer count and buffer size are more of an effect than a cause. What usually happens is that a given buffer scheme is implemented and tested, but during testing, an audio underrun is heard as a "click" or "pop." To compensate, the system designer then increases buffer sizes or buffer counts. This has the desired result of eliminating the underruns, but it also has the undesired side effect of increasing latency.

A better approach is to understand the causes of the underruns and then correct those. This eliminates the audible artifacts and may permit even smaller or fewer buffers and thus reduce latency.

In our experience, the most common causes of underruns include:

  • Linux CFS (Completely Fair Scheduler)
  • high-priority threads with SCHED_FIFO scheduling
  • long scheduling latency
  • long-running interrupt handlers
  • long interrupt disable time

Linux CFS and SCHED_FIFO scheduling

The Linux CFS is designed to be fair to competing workloads sharing a common CPU resource. This fairness is represented by a per-thread nice parameter. The nice value ranges from -19 (least nice, or most CPU time allocated) to 20 (nicest, or least CPU time allocated). In general, all threads with a given nice value receive approximately equal CPU time and threads with a numerically lower nice value should expect to receive more CPU time. However, CFS is "fair" only over relatively long periods of observation. Over short-term observation windows, CFS may allocate the CPU resource in unexpected ways. For example, it may take the CPU away from a thread with numerically low niceness onto a thread with a numerically high niceness. In the case of audio, this can result in an underrun.

The obvious solution is to avoid CFS for high-performance audio threads. Beginning with Android 4.1, such threads now use the SCHED_FIFO scheduling policy rather than the SCHED_NORMAL (also called SCHED_OTHER) scheduling policy implemented by CFS.

Though the high-performance audio threads now use SCHED_FIFO, they are still susceptible to other higher priority SCHED_FIFO threads. These are typically kernel worker threads, but there may also be a few non-audio user threads with policy SCHED_FIFO. The available SCHED_FIFO priorities range from 1 to 99. The audio threads run at priority 2 or 3. This leaves priority 1 available for lower priority threads, and priorities 4 to 99 for higher priority threads. We recommend you use priority 1 whenever possible, and reserve priorities 4 to 99 for those threads that are guaranteed to complete within a bounded amount of time and are known to not interfere with scheduling of audio threads.

Scheduling latency

Scheduling latency is the time between when a thread becomes ready to run, and when the resulting context switch completes so that the thread actually runs on a CPU. The shorter the latency the better, and anything over two milliseconds causes problems for audio. Long scheduling latency is most likely to occur during mode transitions, such as bringing up or shutting down a CPU, switching between a security kernel and the normal kernel, switching from full power to low-power mode, or adjusting the CPU clock frequency and voltage.

Interrupts

In many designs, CPU 0 services all external interrupts. So a long-running interrupt handler may delay other interrupts, in particular audio direct memory access (DMA) completion interrupts. Design interrupt handlers to finish quickly and defer any lengthy work to a thread (preferably a CFS thread or SCHED_FIFO thread of priority 1).

Equivalently, disabling interrupts on CPU 0 for a long period has the same result of delaying the servicing of audio interrupts. Long interrupt disable times typically happen while waiting for a kernel spin lock. Review these spin locks to ensure that they are bounded.