Debugging Native Android Platform Code

This section summarizes useful tools and related commands for debugging, tracing, and profiling native Android platform code when developing platform-level features.

Note: The pages in this section and elsewhere within this site recommend use of adb in conjunction with the setprop argument to debug certain aspects of Android. Please note, on pre-O versions of the Android OS, property names had a length limit of 32 characters. This meant that to create a wrap property with the name of the app, it was necessary to truncate the name to fit. In Android O and later, this limit is much greater and should require no truncation.

This page covers use of debuggerd, a daemon process for collecting error information after applications crash. Other pages explore system services with dumpsys, viewing native memory, network, and RAM usage, using AddressSanitizer to detect memory bugs in native code, evaluating performance issues (includes systrace), and using GNU Project debugger (GDB) and other debugging tools.

Using debuggerd

The debuggerd process dumps registers and unwinds the stack. When a dynamically linked executable starts, several signal handlers are registered that connect to debuggerd (or debuggerd64) in the event that signals (such as SIGSEGV or SIGABRT) are sent to the process.

It's possible for debuggerd to attach only if nothing else is already attached, which means using tools such as strace or gdb will prevent debuggerd from working. You can also explicitly prevent debuggerd from attaching by calling prctl(PR_SET_DUMPABLE, 0), which can be useful when you need to opt out of crash reporting.

Example debuggerd output (with timestamps and extraneous information removed):

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'Android/aosp_angler/angler:7.1.1/NYC/enh12211018:eng/test-keys'
Revision: '0'
ABI: 'arm'
pid: 17946, tid: 17949, name: crasher  >>> crasher <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
    r0 0000000c  r1 00000000  r2 00000000  r3 00000000
    r4 00000000  r5 0000000c  r6 eccdd920  r7 00000078
    r8 0000461a  r9 ffc78c19  sl ab209441  fp fffff924
    ip ed01b834  sp eccdd800  lr ecfa9a1f  pc ecfd693e  cpsr 600e0030

backtrace:
    #00 pc 0004793e  /system/lib/libc.so (pthread_mutex_lock+1)
    #01 pc 0001aa1b  /system/lib/libc.so (readdir+10)
    #02 pc 00001b91  /system/xbin/crasher (readdir_null+20)
    #03 pc 0000184b  /system/xbin/crasher (do_action+978)
    #04 pc 00001459  /system/xbin/crasher (thread_callback+24)
    #05 pc 00047317  /system/lib/libc.so (_ZL15__pthread_startPv+22)
    #06 pc 0001a7e5  /system/lib/libc.so (__start_thread+34)
Tombstone written to: /data/tombstones/tombstone_06

The last line of debuggerd output dumps a summary to the log and writes a full tombstone to disk. The tombstone is simply a file with extra data about the crashed process; it contains information that can be helpful in debugging a crash, in particular the stack traces for all threads in the crashing process (not just the thread that caught the signal) and a full memory map.

Assuming the unstripped binaries can be found, you can get a more detailed unwind with line number information by pasting the above example into development/scripts/stack:

Tip: For convenience, if you've lunched stack will be on your $PATH already so you don't need to give the full path.

development/tools/stack

Example output:

Reading native crash info from stdin
03-02 23:53:49.477 17951 17951 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
03-02 23:53:49.477 17951 17951 F DEBUG   : Build fingerprint: 'Android/aosp_angler/angler:7.1.1/NYC/enh12211018:eng/test-keys'
03-02 23:53:49.477 17951 17951 F DEBUG   : Revision: '0'
03-02 23:53:49.477 17951 17951 F DEBUG   : ABI: 'arm'
03-02 23:53:49.478 17951 17951 F DEBUG   : pid: 17946, tid: 17949, name: crasher  >>> crasher <<<
03-02 23:53:49.478 17951 17951 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
03-02 23:53:49.478 17951 17951 F DEBUG   :     r0 0000000c  r1 00000000  r2 00000000  r3 00000000
03-02 23:53:49.478 17951 17951 F DEBUG   :     r4 00000000  r5 0000000c  r6 eccdd920  r7 00000078
03-02 23:53:49.478 17951 17951 F DEBUG   :     r8 0000461a  r9 ffc78c19  sl ab209441  fp fffff924
03-02 23:53:49.478 17951 17951 F DEBUG   :     ip ed01b834  sp eccdd800  lr ecfa9a1f  pc ecfd693e  cpsr 600e0030
03-02 23:53:49.491 17951 17951 F DEBUG   :
03-02 23:53:49.491 17951 17951 F DEBUG   : backtrace:
03-02 23:53:49.492 17951 17951 F DEBUG   :     #00 pc 0004793e  /system/lib/libc.so (pthread_mutex_lock+1)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #01 pc 0001aa1b  /system/lib/libc.so (readdir+10)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #02 pc 00001b91  /system/xbin/crasher (readdir_null+20)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #03 pc 0000184b  /system/xbin/crasher (do_action+978)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #04 pc 00001459  /system/xbin/crasher (thread_callback+24)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #05 pc 00047317  /system/lib/libc.so (_ZL15__pthread_startPv+22)
03-02 23:53:49.492 17951 17951 F DEBUG   :     #06 pc 0001a7e5  /system/lib/libc.so (__start_thread+34)
03-02 23:53:49.492 17951 17951 F DEBUG   :     Tombstone written to: /data/tombstones/tombstone_06
Reading symbols from /huge-ssd/aosp-arm64/out/target/product/angler/symbols
Revision: '0'
pid: 17946, tid: 17949, name: crasher  >>> crasher <<<
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xc
     r0 0000000c  r1 00000000  r2 00000000  r3 00000000
     r4 00000000  r5 0000000c  r6 eccdd920  r7 00000078
     r8 0000461a  r9 ffc78c19  sl ab209441  fp fffff924
     ip ed01b834  sp eccdd800  lr ecfa9a1f  pc ecfd693e  cpsr 600e0030
Using arm toolchain from: /huge-ssd/aosp-arm64/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/

Stack Trace:
  RELADDR   FUNCTION                   FILE:LINE
  0004793e  pthread_mutex_lock+2       bionic/libc/bionic/pthread_mutex.cpp:515
  v------>  ScopedPthreadMutexLocker   bionic/libc/private/ScopedPthreadMutexLocker.h:27
  0001aa1b  readdir+10                 bionic/libc/bionic/dirent.cpp:120
  00001b91  readdir_null+20            system/core/debuggerd/crasher.cpp:131
  0000184b  do_action+978              system/core/debuggerd/crasher.cpp:228
  00001459  thread_callback+24         system/core/debuggerd/crasher.cpp:90
  00047317  __pthread_start(void*)+22  bionic/libc/bionic/pthread_create.cpp:202 (discriminator 1)
  0001a7e5  __start_thread+34          bionic/libc/bionic/clone.cpp:46 (discriminator 1)

Note: Some system libraries are built with LOCAL_STRIP_MODULE := keep_symbols to provide usable backtraces directly from debuggerd without taking up anywhere near as much space as an unstripped version.

You can also stack an entire tombstone. Example:

stack < FS/data/tombstones/tombstone_05

This is useful if you've just unzipped a bugreport in the current directory. For more information about diagnosing native crashes and tombstones, see Diagnosing Native Crashes.

Getting a stack trace/tombstone from a running process

You can also use debuggerd on a running process. From the command line, invoke debuggerd using a process ID (PID) to dump the full tombstone to stdout. To get just the stack for every thread in the process, include the -b or --backtrace flag.