AddressSanitizer

AddressSanitizer (ASan) is a fast compiler-based tool for detecting memory bugs in native code. Android supports both regular ASan and hardware-accelerated ASan (HWASan). HWAsan is based on memory tagging and is only available on AArch64 because it relies on the Top-Byte-Ignore feature.

These tools detect:

  • Stack and heap buffer overflow/underflow.
  • Heap use after free.
  • Stack use outside scope.
  • Stack use after return (HWAsan only on Android).
  • Double free/wild free.

ASan runs on both 32-bit and 64-bit ARM, plus x86 and x86-64. ASan's CPU overhead is roughly 2x, code size overhead is between 50% and 2x, and a large memory overhead (dependent on your allocation patterns, but on the order of 2x).

HWASan has similar CPU and code size overheads, but a much smaller RAM overhead (15%). HWASan is non-deterministic. There are only 256 possible tag values, so there is a flat 0.4% probability of missing any bug. HWAsan does not have ASan's limited-size redzones for detecting overflows and limited-capacity quarantine for detecting use-after-free, so it does not matter to HWAsan how large the overflow is or how long ago the memory was deallocated. This makes HWASan better than ASan. You can read more about the design of HWAsan.

Valgrind's Memcheck tool is similar, but ASan also detects stack/global overflows in addition to heap overflows, and is much faster with less memory overhead. Conversely, Valgrind detects uninitialized reads and memory leaks that ASan does not. Valgrind may be useful for debugging apps but is not practical for the entire OS, which is why the Android team uses ASan instead.

This document describes how to build and run parts/all of the Android OS itself with AddressSanitizer. If you are building an SDK/NDK application with AddressSanitizer, see AddressSanitizerOnAndroid instead.

Using HWAsan

As of February 2019 only Pixel 2 and Pixel 2 XL support HWAsan. The Android team is working on getting the necessary patches into the common kernel, but for now supporting another device requires backporting these kernel patches:

You may also need to remove some optional extras to make room on your system partition for the larger libraries. See the walleye_hwasan target for an example.

Use the following commands to build the entire platform using HWASan:

lunch walleye_hwasan-userdebug
make SANITIZE_TARGET=hwaddress

Unlike ASan, with HWASan there's no need to build twice, incremental builds just work, there are no special flashing instructions or wiping, static executables are supported, and it's okay to skip sanitization of any library other than libc. There's also no requirement that if a library is sanitized, any executable that links to it must also be sanitized.

To skip sanitization of a module, use LOCAL_NOSANITIZE := hwaddress or sanitize: { hwaddress: false }.

Sanitizing individual executables with ASan

Add LOCAL_SANITIZE:=address or sanitize: { address: true } } to the build rule for the executable. You can search the code for existing examples or to find the other available sanitizers.

When a bug is detected, ASan prints a verbose report both to the standard output and to logcat and then crashes the process.

Sanitizing shared libraries with ASan

Due to the way ASan works, a library built with ASan cannot be used by an executable that's built without ASan.

Note: In runtime situations where an ASan library is loaded into an incorrect process, you will see unresolved symbol messages starting with _asan or _sanitizer.

To sanitize a shared library that is used in multiple executables, not all of which are built with ASan, you'll need two copies of the library. The recommended way to do this is to add the following to Android.mk for the module in question:

LOCAL_SANITIZE:=address
LOCAL_MODULE_RELATIVE_PATH := asan

This puts the library in /system/lib/asan instead of /system/lib. Then, run your executable with: LD_LIBRARY_PATH=/system/lib/asan

For system daemons, add the following to the appropriate section of /init.rc or /init.$device$.rc.

setenv LD_LIBRARY_PATH /system/lib/asan

Warning: The LOCAL_MODULE_RELATIVE_PATH setting moves your library to /system/lib/asan, meaning that clobbering and rebuilding from scratch will result in the library missing from /system/lib, and probably an unbootable image. That's an unfortunate limitation of the current build system. Don't clobber; do make -j $N and adb sync.

Verify the process is using libraries from /system/lib/asan when present by reading /proc/$PID/maps. If it's not, you may need to disable SELinux, like so:

adb root
adb shell setenforce 0
# restart the process with adb shell kill $PID
# if it is a system service, or may be adb shell stop; adb shell start.

Better stack traces

AddressSanitizer uses a fast, frame-pointer-based unwinder to record a stack trace for every memory allocation and deallocation event in the program. Most of Android is built without frame pointers. As a result, you will often get only one or two meaningful frames. To fix this, either rebuild the library with ASan (recommended!), or with:

LOCAL_CFLAGS:=-fno-omit-frame-pointer
LOCAL_ARM_MODE:=arm

Or set ASAN_OPTIONS=fast_unwind_on_malloc=0 in the process environment. The latter can be very CPU-intensive, depending on the load.

Symbolization

Initially, ASan reports contain references to offsets in binaries and shared libraries. There are two ways to obtain source file and line information:

  • Ensure llvm-symbolizer binary is present in /system/bin. Llvm-symbolizer is built from sources in: third_party/llvm/tools/llvm-symbolizer
  • Filter the report through the external/compiler-rt/lib/asan/scripts/symbolize.py script.

The second approach can provide more data (i.e. file:line locations) because of the availability of symbolized libraries on the host.

AddressSanitizer in apps

AddressSanitizer cannot see into Java code, but it can detect bugs in the JNI libraries. For that, you'll need to build the executable with ASan, which in this case is /system/bin/app_process(32|64). This will enable ASan in all apps on the device at the same time, which is a bit stressful, but nothing that a 2GB RAM device cannot handle.

Add the usual LOCAL_SANITIZE:=address to the app_process build rule in frameworks/base/cmds/app_process. Ignore the app_process__asan target in the same file for now (if it is still there at the time you read this). Edit the Zygote record in system/core/rootdir/init.zygote(32|64).rc to add the following lines:

setenv LD_LIBRARY_PATH /system/lib/asan:/system/lib
setenv ASAN_OPTIONS
allow_user_segv_handler=true

Build, adb sync, fastboot flash boot, reboot.

Using the wrap property

The approach in the previous section puts AddressSanitizer into every application in the system (actually, into every descendant of the Zygote process). It is possible to run only one (or several) applications with ASan, trading some memory overhead for slower application startup.

This can be done by starting your app with the “wrap.” property, the same one that’s used to run apps under Valgrind. The following example runs the Gmail app under ASan:

adb root
adb shell setenforce 0  # disable SELinux
adb shell setprop wrap.com.google.android.gm "asanwrapper"

In this context, asanwrapper rewrites /system/bin/app_process to /system/bin/asan/app_process, which is built with AddressSanitizer. It also adds /system/lib/asan at the start of the dynamic library search path. This way ASan-instrumented libraries from /system/lib/asan are preferred to normal libraries in /system/lib when running with asanwrapper.

Again, if a bug is found, the app will crash, and the report will be printed to the log.

SANITIZE_TARGET

Since Android 7.0 Nougat, there is support for building the entire Android platform with ASan at once. (If you're building a release newer than Android 9.0 Pie, HWASan is a better choice.)

Run the following commands in the same build tree.

make -j42
SANITIZE_TARGET=address make -j42

In this mode, userdata.img contains extra libraries and must be flashed to the device as well. Use the following command line:

fastboot flash userdata && fastboot flashall

This works by building two sets of shared libraries: normal in /system/lib (the first make invocation), ASan-instrumented in /data/asan/lib (the second make invocation). Executables from the second build overwrite the ones from the first build. ASan-instrumented executables get a different library search path that includes /data/asan/lib before /system/lib through the use of "/system/bin/linker_asan" in PT_INTERP.

The build system clobbers intermediate object directories when the $SANITIZE_TARGET value has changed. This forces a rebuild of all targets while preserving installed binaries under /system/lib.

Some targets cannot be built with ASan:

  • Statically linked executables.
  • LOCAL_CLANG:=false targets
  • LOCAL_SANITIZE:=false will not be ASan'd for SANITIZE_TARGET=address

Executables like these are skipped in the SANITIZE_TARGET build, and the version from the first make invocation is left in /system/bin.

Libraries like this are simply built without ASan. They can contain some ASan code anyway from the static libraries they depend upon.

Supporting documentation

AddressSanitizerOnAndroid public project site

AddressSanitizer and Chromium

Other Google Sanitizers