The latest Android and Google Play news for app and game developers.
In less than a month, we'll be kicking off Android Dev Summit 2018, broadcasting live from the Computer History Museum in Mountain View, CA on November 7 and 8. We'll have two days of deep technical sessions from the Android engineering team, with over 30 sessions livestreamed. The first wave of sessions were just posted to the website: check them out and start planning.
The summit kicks off on November 7 at 10AM PST with the keynote, where you'll hear directly from Dave Burke and others on the present and future of Android development. From there, we'll dive into two tracks (and two days!) of deep technical content from the Google engineering team, on topics such as Android Pie, Android Studio, Kotlin, Android Jetpack, Google Play and more. We'll also have demos and office hours for those attending in person; more on that in the coming weeks!
We received a ton of interest from developers looking to attend in person; if you were one of those who expressed interest but didn't receive a ticket, we've already reached out to you and shared this news, but we want to apologize again that we weren't able to find you a spot. Rest assured, though, that we're still doing all that we can to free up more tickets, and we'll be reaching out to folks we're able to accommodate in the lead-up to the show. And if you did receive a ticket but your plans have changed and you're no longer able to attend, please let us know by sending an email to android-dev-summit@google.com, and we'll free up your spot for others on the waitlist.
If you can't join in person, you can always join us online: we'll be livestreaming all of the sessions on the Android Dev Summit website and making them available on YouTube throughout the conference to watch at your own pace. Plus, we'll be sharing updates directly from the Computer History Museum to our social channels, so be sure to follow along!
Posted by Don Turner, Developer Advocate, Android Audio Framework
This week we released the first production-ready version of Oboe - a C++ library for building real-time audio apps. Oboe provides the lowest possible audio latency across the widest range of Android devices, as well as several other benefits.
Oboe takes advantage of the improved performance and features of AAudio on Oreo MR1 (API 27+) whilst maintaining backward compatibility (using OpenSL ES) on API 16+. It's kind of like AndroidX for native audio.
Using Oboe you can create an audio stream in just 3 lines of code (vs 50+ lines in OpenSL ES):
AudioStreamBuilder builder; AudioStream *stream = nullptr; Result result = builder.openStream(&stream);
Take a look at the short video introduction:
Check out the documentation, code samples and API reference. There's even a codelab which shows you how to build a rhythm-based game.
If you have any issues, please file them here, we'd love to hear how you get on.
Posted by Sami Tolvanen, Staff Software Engineer, Android Security
Android's security model is enforced by the Linux kernel, which makes it a tempting target for attackers. We have put a lot of effort into hardening the kernel in previous Android releases and in Android 9, we continued this work by focusing on compiler-based security mitigations against code reuse attacks.
Google's Pixel 3 will be the first Android device to ship with LLVM's forward-edge Control Flow Integrity (CFI) enforcement in the kernel, and we have made CFI support available in Android kernel versions 4.9 and 4.14. This post describes how kernel CFI works and provides solutions to the most common issues developers might run into when enabling the feature.
A common method of exploiting the kernel is using a bug to overwrite a function pointer stored in memory, such as a stored callback pointer or a return address that had been pushed to the stack. This allows an attacker to execute arbitrary parts of the kernel code to complete their exploit, even if they cannot inject executable code of their own. This method of gaining code execution is particularly popular with the kernel because of the huge number of function pointers it uses, and the existing memory protections that make code injection more challenging.
CFI attempts to mitigate these attacks by adding additional checks to confirm that the kernel's control flow stays within a precomputed graph. This doesn't prevent an attacker from changing a function pointer if a bug provides write access to one, but it significantly restricts the valid call targets, which makes exploiting such a bug more difficult in practice.
Figure 1. In an Android device kernel, LLVM's CFI limits 55% of indirect calls to at most 5 possible targets and 80% to at most 20 targets.
In order to determine all valid call targets for each indirect branch, the compiler needs to see all of the kernel code at once. Traditionally, compilers work on a single compilation unit (source file) at a time and leave merging the object files to the linker. LLVM's solution to CFI is to require the use of LTO, where the compiler produces LLVM-specific bitcode for all C compilation units, and an LTO-aware linker uses the LLVM back-end to combine the bitcode and compile it into native code.
Figure 2. A simplified overview of how LTO works in the kernel. All LLVM bitcode is combined, optimized, and generated into native code at link time.
Linux has used the GNU toolchain for assembling, compiling, and linking the kernel for decades. While we continue to use the GNU assembler for stand-alone assembly code, LTO requires us to switch to LLVM's integrated assembler for inline assembly, and either GNU gold or LLVM's own lld as the linker. Switching to a relatively untested toolchain on a huge software project will lead to compatibility issues, which we have addressed in our arm64 LTO patch sets for kernel versions 4.9 and 4.14.
In addition to making CFI possible, LTO also produces faster code due to global optimizations. However, additional optimizations often result in a larger binary size, which may be undesirable on devices with very limited resources. Disabling LTO-specific optimizations, such as global inlining and loop unrolling, can reduce binary size by sacrificing some of the performance gains. When using GNU gold, the aforementioned optimizations can be disabled with the following additions to LDFLAGS:
LDFLAGS += -plugin-opt=-inline-threshold=0 \ -plugin-opt=-unroll-threshold=0
Note that flags to disable individual optimizations are not part of the stable LLVM interface and may change in future compiler versions.
LLVM's CFI implementation adds a check before each indirect branch to confirm that the target address points to a valid function with a correct signature. This prevents an indirect branch from jumping to an arbitrary code location and even limits the functions that can be called. As C compilers do not enforce similar restrictions on indirect branches, there were several CFI violations due to function type declaration mismatches even in the core kernel that we have addressed in our CFI patch sets for kernels 4.9 and 4.14.
Kernel modules add another complication to CFI, as they are loaded at runtime and can be compiled independently from the rest of the kernel. In order to support loadable modules, we have implemented LLVM's cross-DSO CFI support in the kernel, including a CFI shadow that speeds up cross-module look-ups. When compiled with cross-DSO support, each kernel module contains information about valid local branch targets, and the kernel looks up information from the correct module based on the target address and the modules' memory layout.
Figure 3. An example of a cross-DSO CFI check injected into an arm64 kernel. Type information is passed in X0 and the target address to validate in X1.
CFI checks naturally add some overhead to indirect branches, but due to more aggressive optimizations, our tests show that the impact is minimal, and overall system performance even improved 1-2% in many cases.
CFI for arm64 requires clang version >= 5.0 and binutils >= 2.27. The kernel build system also assumes that the LLVMgold.so plug-in is available in LD_LIBRARY_PATH. Pre-built toolchain binaries for clang and binutils are available in AOSP, but upstream binaries can also be used.
The following kernel configuration options are needed to enable kernel CFI:
CONFIG_LTO_CLANG=y CONFIG_CFI_CLANG=y
Using CONFIG_CFI_PERMISSIVE=y may also prove helpful when debugging a CFI violation or during device bring-up. This option turns a violation into a warning instead of a kernel panic.
As mentioned in the previous section, the most common issue we ran into when enabling CFI on Pixel 3 were benign violations caused by function pointer type mismatches. When the kernel runs into such a violation, it prints out a runtime warning that contains the call stack at the time of the failure, and the call target that failed the CFI check. Changing the code to use a correct function pointer type fixes the issue. While we have fixed all known indirect branch type mismatches in the Android kernel, similar problems may be still found in device specific drivers, for example.
CFI failure (target: [<fffffff3e83d4d80>] my_target_function+0x0/0xd80): ------------[ cut here ]------------ kernel BUG at kernel/cfi.c:32! Internal error: Oops - BUG: 0 [#1] PREEMPT SMP … Call trace: … [<ffffff8752d00084>] handle_cfi_failure+0x20/0x28 [<ffffff8752d00268>] my_buggy_function+0x0/0x10 …
Figure 4. An example of a kernel panic caused by a CFI failure.
Another potential pitfall are address space conflicts, but this should be less common in driver code. LLVM's CFI checks only understand kernel virtual addresses and any code that runs at another exception level or makes an indirect call to a physical address will result in a CFI violation. These types of failures can be addressed by disabling CFI for a single function using the __nocfi attribute, or even disabling CFI for entire code files using the $(DISABLE_CFI) compiler flag in the Makefile.
static int __nocfi address_space_conflict() { void (*fn)(void); … /* branching to a physical address trips CFI w/o __nocfi */ fn = (void *)__pa_symbol(function_name); cpu_install_idmap(); fn(); cpu_uninstall_idmap(); … }
Figure 5. An example of fixing a CFI failure caused by an address space conflict.
Finally, like many hardening features, CFI can also be tripped by memory corruption errors that might otherwise result in random kernel crashes at a later time. These may be more difficult to debug, but memory debugging tools such as KASAN can help here.
We have implemented support for LLVM's CFI in Android kernels 4.9 and 4.14. Google's Pixel 3 will be the first Android device to ship with these protections, and we have made the feature available to all device vendors through the Android common kernel. If you are shipping a new arm64 device running Android 9, we strongly recommend enabling kernel CFI to help protect against kernel vulnerabilities.
LLVM's CFI protects indirect branches against attackers who manage to gain access to a function pointer stored in kernel memory. This makes a common method of exploiting the kernel more difficult. Our future work involves also protecting function return addresses from similar attacks using LLVM's Shadow Call Stack, which will be available in an upcoming compiler release.
Posted by Paul Bankhead, Director, Product Management, Google Play
We focus relentlessly on security and privacy on the Google Play Store to ensure Android users have a positive experience discovering and installing apps and games they love. We regularly update our Google Play Developer policies and today have introduced stronger controls and new policies to keep user data safe. Here are a few updates:
As previously announced, as of November 1, 2018, Google Play will require updates to existing apps to target API level 26 (Android 8.0) or higher (this is already required for all new apps). Our goal is to ensure all apps on Google Play are built using the latest APIs that are optimized for security and performance.
Our Google Play Developer policies are designed to provide a safe and secure experience for our users while also giving developers the tools they need to succeed. For example, we have always required developers to limit permission requests to only what is needed for their app to function and to be clear with users about what data they access.
As part of today's Google Play Developer Policy update, we're announcing changes related to SMS and Call Log permissions. Some Android apps ask for permission to access a user's phone (including call logs) and SMS data. Going forward, Google Play will limit which apps are allowed to ask for these permissions. Only an app that has been selected as a user's default app for making calls or text messages will be able to access call logs and SMS, respectively.
Please visit our Google Play Developer Policy Center and this Help Center article for detailed information on product alternatives to SMS and call logs permissions. For example, the SMS Retriever API enables you to perform SMS-based user verification and SMS Intent enables you to initiate an SMS or MMS text message to share content or invitations. We'll be working with our developer partners to give them appropriate time to adjust and update their apps, and will begin enforcement 90 days from this policy update.
In the coming months, we'll be rolling out additional controls and policies across our various products and platforms, and will continue to work with you, our developers, to help with the transition.
The trust of our users is critical and together we'll continue to build a safe and secure Android ecosystem.
Posted by James Lau (@jmslau), Product Manager
Today marks the beginning of KotlinConf 2018 - the largest in-person gathering of the Kotlin community annually. 2018 has been a big year for Kotlin, as the language continues to gain adoption and earn the love of developers. In fact, 27% of the top 1000 Android apps on Google Play already use Kotlin. More importantly, Android developers are loving the language with over 97% satisfaction in our most recent survey. It's no surprise that Kotlin was voted as the #2 most-loved language in the 2018 StackOverflow survey.
Google supports Kotlin as a first-class programming language for Android development. In the past 12 months, we have delivered a number of important improvements to the Kotlin developer experience. This includes the Kotlin-friendly SDK, Android KTX, new Lint checks and various Kotlin support improvements in Android Studio. We have also launched Kotlin support in our official documentation, new flagship samples in Kotlin, a new Kotlin Bootcamp Udacity course, #31DaysOfKotlin and other deep dive content. We are committed to continuing to improve the Kotlin developer experience.
As the language continues to advance, more developers are discovering the benefits of Kotlin across the globe. Recently, we traveled to India and worked with local developers like Zomato to better understand how adopting Kotlin has benefited their Android development. Zomato is a leading restaurant search & discovery service that operates in 24 countries, with over 150 million monthly users. Kotlin helped Zomato reduce the number of lines of code in their app significantly, and it has also helped them find important defects in their app at compile time. You can watch their Kotlin adoption story in the video below.
Android Developer Story: Zomato uses Kotlin to write safer, more concise code.
Going beyond Android, we are happy to announce that the Google Cloud Platform team is launching a dedicated Kotlin portal today. This will help developers more easily find resources related to Kotlin on Google Cloud. We want to make it as easy as possible for you to use Kotlin, whether it's on mobile or in the Cloud.
Google Cloud Platform's Kotlin Homepage
Adopting a new language is a major decision for most companies, and you need to be confident that the language you choose will have a bright future. That's why Google has joined forces with JetBrains and established the Kotlin Foundation. The Foundation will ensure that Kotlin continues to advance rapidly, remain free and stay open. You can learn more about the Kotlin Foundation here.
It's an exciting time to be a Kotlin developer. If you haven't tried Kotlin yet, we encourage you to join this growing global community. You can get started by visiting kotlinlang.org or the Android Developer Kotlin page.
Posted by Jamal Eason, Product Manager, Android
Today, Android Studio 3.2 is available for download. Android Studio 3.2 is the best way for app developers to cut into the latest Android 9 Pie release and build the new Android App bundle. Since announcing this update of Android Studio at Google I/O '18, we have refined and polished 20+ new features and focused our efforts on improving the quality for this stable release of Android Studio 3.2.
Every developer should use Android Studio 3.2 to transition to using an Android App Bundle, the new app publishing format. With very minimal work, you can generate an app bundle with Android Studio. Once you upload your app bundle to Google Play you can distribute smaller, optimized apps to your users. Early adopters have already seen between 11% - 64% in app size savings with app bundles over the legacy APK app size.
Another feature you do not want to miss is the Energy Profiler. This new profiler gives you a set of tools that will help you diagnose and improve the energy impact of your app. Better device battery life is one of the top most user requests, and with the Energy Profiler in Android Studio 3.2, you can do your part in improving device battery life by making sure your app is using the right amount of energy at the right time.
Lastly, you should also check out the new Android Emulator Snapshots feature. By using this feature, you can quickly take a snapshot of the current state of your emulator which includes the current state of the screen, apps, and settings. You can resume or boot into your emulator snapshot in under 2 seconds. For any app developer looking for super- fast boot times, or seeking to run tests in a predictable Android environment, Android Emulator Snapshots is a game changing feature for app development
On top of these major features, there are 20 new features plus many under-the-hood quality refinements in Android Studio 3.2. By using Android Studio 3.2, you can also develop for the latest technologies ranging from Android Jetpack, to the latest in Google Artificial Intelligence (AI) APIs with Android Slices.
Thank you to those who gave your early feedback on both the canary and beta releases. Your feedback helped us improve the quality and features in Android Studio 3.2. If you are ready for the next stable release, and want to use a new set of productivity features, Android Studio 3.2 is ready to download for you to get started.
Below is a full list of new features in Android Studio 3.2, organized by key developer flows.
Slices Provider Template
android.useAndroidX=true
gradle.properties
Build Android App Bundle
Android Emulator Snapshots
Energy Profiler
startMethodTracing(String tracePath)
stopMethodTracing()
To recap, the latest canary of Android Studio 3.2 includes these new major features:
Build
Optimize
Check out the release notes for more details.
Download the latest version of Android Studio 3.2 from the download page. If you are using a previous canary release of Android Studio, make sure you update to Android Studio Canary 14 or higher. If you want to maintain a stable version of Android Studio, you can run the stable release version and canary release versions of Android Studio at the same time. Learn more.
To use the mentioned Android Emulator features make sure you are running at least Android Emulator v28.0.7+ downloaded via the Android Studio SDK Manager.
We appreciate any feedback on things you like, and issues or features you would like to see. Please note, to maintain high product quality, a couple features (e.g. Navigation Editor) you saw in earlier release channels are not enabled by default in the stable release channel. If you find a bug or issue, feel free to file an issue. Connect with us -- the Android Studio development team ‐ on our Google+ page or on Twitter.
Posted by Jason Woloz and Mayank Jain, Android Security & Privacy Team
Our Android and Play security reward programs help us work with top researchers from around the world to improve Android ecosystem security every day. Thank you to all the amazing researchers who submitted vulnerability reports.
In the ASR program's third year, we received over 470 qualifying vulnerability reports from researchers and the average pay per researcher jumped by 23%. To date, the ASR program has rewarded researchers with over $3M, paying out roughly $1M per year.
Here are some of the highlights from the Android Security Rewards program's third year:
As part of our ongoing commitment to security we regularly update our programs and policies based on ecosystem feedback. We also updated our severity guidelines for evaluating the impact of reported security vulnerabilities against the Android platform.
In October 2017, we rolled out the Google Play Security Reward Program to encourage security research into popular Android apps available on Google Play. So far, researchers have reported over 30 vulnerabilities through the program, earning a combined bounty amount of over $100K.
If undetected, these vulnerabilities could have potentially led to elevation of privilege, access to sensitive data and remote code execution on devices.
In addition to rewarding for vulnerabilities, we continue to work with the broad and diverse Android ecosystem to protect users from issues reported through our program. We collaborate with manufacturers to ensure that these issues are fixed on their devices through monthly security updates. Over 250 device models have a majority of their deployed devices running a security update from the last 90 days. This table shows the models with a majority of deployed devices running a security update from the last three months:
Thank you to everyone internally and externally who helped make Android safer and stronger in the past year. Together, we made a huge investment in security research that helps Android users everywhere. If you want to get involved to make next year even better, check out our detailed program rules. For tips on how to submit complete reports, see Bug Hunter University.
Play Console
