Detecting Memory Corruption Bugs With HWASan

Posted by Evgenii Stepanov, Staff Software Engineer, Dynamic Tools

Native code in memory-unsafe languages like C and C++ is often vulnerable to memory corruption bugs. Our data shows that issues like use-after-free, double-free, and heap buffer overflows generally constitute more than 65% of High & Critical security bugs in Chrome and Android.

In previous years our memory bug detection efforts were focused on Address Sanitizer (ASan). ASan catches these errors but causes your app to use 2x-3x extra memory and to run slower.

To better tackle these problems we’ve developed Hardware-Assisted Address Sanitizer (HWASan). HWASan typically only requires 15% more memory. It’s also a lot faster than ASan. HWASan’s performance makes it usable not only for unit testing, but also for interactive human-driven testing. We use this to find memory issues in the Android OS itself, and now we've made it easy for app developers to use it too. HWASan is fast enough that some Android developers use it on their development devices for everyday tasks.

Under the hood

HWASan is based on memory tagging and depends on the Top Byte Ignore feature present in all 64-bit ARM CPUs and the associated kernel support. Every memory allocation is assigned a random 8-bit tag that is stored in the most significant byte (MSB) of the address, but ignored by the CPU. As a result, this tagged pointer can be used in place of a regular pointer without any code changes.

Under the hood, HWASan uses shadow memory - a sparse map that assigns a tag value to each 16 byte block of program memory. Compile time code instrumentation is used to insert checks that compare pointer and memory tags for every memory access, and raise an error if they do not match.

This approach allows us to detect both use-after-free and buffer-overflow types of bugs. The memory tag in the shadow is changed to a random value during allocation and deallocation. As a result, attempting to access deallocated memory with a dangling pointer will almost certainly fail due to a tag mismatch. The same is true for an attempt to access memory outside of the allocated region, which is very likely to have a different tag. Stack and global variables are similarly protected.

Use-after-free bug detection with memory tagging.

This approach is non deterministic: because of the limited number of possible tags, an invalid memory access has 1 chance out of 256 (approximately 0.4%) to pass undetected. We have not observed this as a problem in practice, but, due to the tag randomness, running the program the second time is very likely to find any bugs that the first run has missed.

An advantage of HWASan over ASan is its ability to find bugs that happen far from their origination point - for example, a use-after-free where the memory is accessed long after it has been deallocated, or a buffer overflow with a large offset. This is not the case with ASan, which uses red zones around memory allocations, and a quarantine for the temporary storage of recently deallocated memory blocks. Both redzones and the quarantine are of limited size, and error detection is unlikely beyond that. HWASan uses a different approach that does not have these limitations.

Usage

When a bug is discovered the process is terminated and a crash dump is printed to logcat. The “Abort message” field contains a HWASan report, which shows the access type (read or write), access address, thread id and the stack trace of the bad memory access. This is followed by a stack trace for the original allocation, and, for use-after-free bugs, a stack trace showing where the deallocation took place. Advanced users can find extra debugging information below this, including a map of memory tags for nearby locations.

signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: '==21586==ERROR: HWAddressSanitizer: tag-mismatch on address 0x0042a0807af0 at pc 0x007b23b8786c
WRITE of size 1 at 0x0042a0807af0 tags: db/19 (ptr/mem) in thread T0
    #0 0x7b23b87868 (/data/app/com.example.myapp/lib/arm64/native.so+0x2868)
    #1 0x7b8f1e4ccc (/apex/com.android.art/lib64/libart.so+0x198ccc)
[...]

0x0042a0807af0 is located 0 bytes to the right of 16-byte region [0x0042a0807ae0,0x0042a0807af0)
allocated here:
    #0 0x7b92a322bc (/path/to/libclang_rt.hwasan-aarch64-android.so+0x212bc)
    #1 0x7b23b87840 (/data/app/com.example.myapp/lib/arm64/native.so+0x2840)
[...]

An example snippet from a HWASan crash report.

Google uses HWASan extensively in Android development, and now you can too. Find out more -- including the details of how to rebuild your app for use with HWASan -- at https://developer.android.com/ndk/guides/hwasan. Prebuilt HWASan system images are available on the AOSP build server (or you can build your own). They can be easily flashed onto a compatible device using the recently announced web flash tool.