Hardware-assisted AddressSanitizer

See Understanding HWASan reports for information on how to read HWASan crashes!

Hardware-assisted AddressSanitizer (HWASan) is a memory error detection tool similar to AddressSanitizer. HWASan uses a lot less RAM compared to ASan, which makes it suitable for whole system sanitization. HWASan is only available on Android 10 and higher, and only on AArch64 hardware.

Although primarily useful for C/C++ code, HWASan can also help debug Java code that causes crashes in C/C++ used to implement Java interfaces. It's helpful because it catches memory errors when they happen, pointing you directly at the responsible code.

You can flash prebuilt HWASan images to supported Pixel devices from ci.android.com (detailed setup instructions).

Compared to classic ASan, HWASan has:

  • Similar CPU overhead (~2x)
  • Similar code size overhead (40 – 50%)
  • Much smaller RAM overhead (10% – 35%)

HWASan detects the same set of bugs as ASan:

  • Stack and heap buffer overflow/underflow
  • Heap use after free
  • Stack use outside scope
  • Double free/wild free

Additionally, HWASan detects stack use after return.

HWASan (same as ASan) is compatible with UBSan, both can be enabled on a target at the same time.

Implementation details and limitations

HWASan is based on the memory tagging approach, where a small random tag value is associated both with pointers and with ranges of memory addresses. For a memory access to be valid, the pointer and memory tags have to match. HWASan relies on ARMv8 feature top byte ignore (TBI), also called virtual address tagging, to store the pointer tag in the highest bits of the address.

You can read more about the design of HWASan on the Clang documentation site.

By design, HWASan doesn't have ASan's limited-size redzones for detecting overflows or ASan's limited-capacity quarantine for detecting use after free. For this reason, HWASan can detect a bug no matter how large the overflow is or how long ago the memory was deallocated. This gives HWASan a big advantage over ASan.

However, HWASan has a limited number of possible tag values (256), which means that there is a 0.4% probability of missing any bug during one execution of the program.

Requirements

Recent versions (4.14+) of the common Android kernel support HWASan out-of-the-box. The Android 10 specific branches do not have support for HWASan.

Userspace support for HWASan is available starting with Android 11.

If you are working with a different kernel, HWASan requires the Linux kernel to accept tagged pointers in system call arguments. Support for this was implemented in the following upstream patchsets:

If you are building with a custom toolchain, make sure that it includes everything up to LLVM commit c336557f.

Use HWASan

Use the following commands to build the entire platform using HWASan: 

lunch aosp_walleye-userdebug # (or any other product)
export SANITIZE_TARGET=hwaddress
m -j

For convenience, you can add the SANITIZE_TARGET setting to a product definition, similar to aosp_coral_hwasan.

For users familiar with AddressSanitizer, a lot of build complexity is gone:

  • No need to run make twice.
  • Incremental builds work out of the box.
  • No need to flash userdata.

Some AddressSanitizer restrictions are also gone:

  • Static executables are supported.
  • It’s OK to skip sanitization of any target other than libc. Unlike with ASan there is no requirement that if a library is sanitized, then any executable that links it must be, too.

Switching between HWASan and regular images at the same (or higher) build number can be done freely. Wiping the device is not required.

To skip sanitization of a module, use LOCAL_NOSANITIZE := hwaddress (Android.mk) or sanitize: { hwaddress: false } (Android.bp).

Sanitize individual targets

HWASan can be enabled per-target in a regular (unsanitized) build, as long as libc.so is also sanitized. Add hwaddress: true to the sanitize block in "libc_defaults" in bionic/libc/Android.bp. Then do the same in the target you are working on.

Note that sanitizing libc enables tagging of heap memory allocations system-wide, as well as the checking of the tags for memory operations inside libc.so. This may catch bugs even in binaries that HWASan was not enabled on if the bad memory access is in libc.so (ex. pthread_mutex_unlock() on a delete()ed mutex).

It is not necessary to change any build files if the whole platform is built using HWASan.

Flashstation

For development purposes, you can flash a HWASan-enabled build of AOSP onto a Pixel device with unlocked bootloader using Flashstation. Select the _hwasan target, e.g. aosp_flame_hwasan-userdebug. See the NDK documentation for HWASan for app developers for more details.

Better stack traces

HWASan uses a fast, frame-pointer-based unwinder to record a stack trace for every memory allocation and deallocation event in the program. Android enables frame pointers in AArch64 code by default, so this works great in practice. If you need to unwind through managed code, set HWASAN_OPTIONS=fast_unwind_on_malloc=0 in the process environment. Note that bad memory access stack traces use the "slow" unwinder by default; this setting only affects allocation and deallocation traces. This option can be very CPU-intensive, depending on the load.

Symbolization

See Symbolization in "Understanding HWASan reports".

HWASan in apps

Similar to AddressSanitizer, HWASan can't see into Java code, but it can detect bugs in the JNI libraries. Until Android 14, running HWASan apps on a non-HWASan device was not supported.

On a HWASan device, apps can be checked with HWASan by building their code with SANITIZE_TARGET:=hwaddress in Make, or -fsanitize=hwaddress in compiler flags. On a non-HWASan device (running Android 14 or newer), a wrap.sh file setting LD_HWASAN=1 has to be added. See the app developer documentation for more details.