Posted by Paul Lawrence, Android Security Engineer
In Android-powered devices, the kernel does the heavy lifting to enforce the Android security model. As the security team has worked to harden Android's userspace and isolate and deprivilege processes, the kernel has become the focus of more security attacks. System calls are a common way for attackers to target the kernel.
All Android software communicates with the Linux kernel using system calls, or syscalls for short. The kernel provides many device- and SOC-specific syscalls that allow userspace processes, including apps, to directly interact with the kernel. All apps rely on this mechanism to access collections of behavior indexed by unique system calls, such as opening a file or sending a Binder message. However, many of these syscalls are not used or officially supported by Android.
Android O takes advantage of a Linux feature called seccomp that makes unused system calls inaccessible to application software. Because these syscalls cannot be accessed by apps, they can't be exploited by potentially harmful apps.

seccomp filter

Android O includes a single seccomp filter installed into zygote, the process from which all the Android applications are derived. Because the filter is installed into zygote—and therefore all apps—the Android security team took extra caution to not break existing apps. The seccomp filter allows:

• all the syscalls exposed via bionic (the C runtime for Android). These are defined in bionic/libc/SYSCALLS.TXT.
• syscalls to allow Android to boot
• syscalls used by popular Android applications, as determined by running Google's full app compatibility suite

Android O's seccomp filter blocks certain syscalls, such as swapon/swapoff, which have been implicated in some security attacks, and the key control syscalls, which are not useful to apps. In total, the filter blocks 17 of 271 syscalls in arm64 and 70 of 364 in arm.

Developers

Test your app for illegal syscalls on a device running Android O.

Detecting an illegal syscall

In Android O, the system crashes an app that uses an illegal syscall. The log printout shows the illegal syscall, for example:

03-09 16:39:32.122 15107 15107 I crash_dump32: performing dump of process 14942 (target tid = 14971)
03-09 16:39:32.127 15107 15107 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
03-09 16:39:32.127 15107 15107 F DEBUG   : Build fingerprint: 'google/sailfish/sailfish:O/OPP1.170223.013/3795621:userdebug/dev-keys'
03-09 16:39:32.127 15107 15107 F DEBUG   : Revision: '0'
03-09 16:39:32.127 15107 15107 F DEBUG   : ABI: 'arm'
03-09 16:39:32.127 15107 15107 F DEBUG   : pid: 14942, tid: 14971, name: WorkHandler  >>> com.redacted <<<
03-09 16:39:32.127 15107 15107 F DEBUG   : signal 31 (SIGSYS), code 1 (SYS_SECCOMP), fault addr --------
03-09 16:39:32.127 15107 15107 F DEBUG   : Cause: seccomp prevented call to disallowed system call 55
03-09 16:39:32.127 15107 15107 F DEBUG   :     r0 00000091  r1 00000007  r2 ccd8c008  r3 00000001
03-09 16:39:32.127 15107 15107 F DEBUG   :     r4 00000000  r5 00000000  r6 00000000  r7 00000037

Affected developers should rework their apps to not call the illegal syscall.

Toggling seccomp filters during testing

In addition to logging errors, the seccomp installer respects setenforce on devices running userdebug and eng builds, which allows you to test whether seccomp is responsible for an issue. If you type:

adb shell setenforce 0 && adb stop && adb start

then no seccomp policy will be installed into zygote. Because you cannot remove a seccomp policy from a running process, you have to restart the shell for this option to take effect.

Device manufacturers

Because Android O includes the relevant seccomp filters at //bionic/libc/seccomp, device manufacturers don't need to do any additional implementation. However, there is a CTS test that checks for seccomp at //cts/tests/tests/security/jni/android_security_cts_SeccompTest.cpp. The test checks that add_key and keyctl syscalls are blocked and openat is allowed, along with some app-specific syscalls that must be present for compatibility.