Posted by Scott Swarthout, Product Manager
We are excited to announce the stable release of Android Studio 3.6 with a targeted set of features addressing quality in primarily code editing and debugging use cases. This is our first release after the end of Project Marble, which was focused on making the fundamental features and flows of the Integrated Development Environment (IDE) rock-solid. We learned a lot from Project Marble and in Android Studio 3.6 we introduce a small set of features, polished existing features, and spent a notable effort addressing bugs and improving underlying performance to ensure we meet the high quality bar we set in the past year.
Some highlights of Android Studio 3.6 include a new way to quickly design, develop and preview app layouts using XML, with a new Split View in the design editors. Additionally, you no longer have to manually type in GPS coordinates to test location with your app because we now embedded Google Maps right into the Android Emulator extended control panel. Finally, we’ve made it easier to optimize your app and find bugs with automatic memory leak detection for Fragments and Activities. We hope all of these features help you be happier and more productive while developing on Android.
Thank you to those who gave your early feedback in preview releases. Your feedback helped us iterate and improve features in Android Studio 3.6. If you are ready for the next stable release, and want to use a new set of productivity features, Android Studio 3.6 is ready to download for you to get started.
Below is a full list of new features in Android Studio 3.6, organized by key developer flows.
Design editors, such as the Layout Editor and Navigation Editor, now provide a Split view that enables you to see both the Design and Code views of your UI at the same time. Split view replaces and improves upon the earlier Preview window, and can be configured on a file-by-file basis to preserve context information like zoom factor and design view options, so you can choose the view that works best for each use case. To enable split view, click the Split icon in the top-right corner of the editor window. Learn more.
Split view for design editors
In this release we wanted to make it easier to apply colors you have defined as color resources. In Android Studio 3.6, the color picker populates the color resources in your app for you to quickly choose and replace color resources values. The color picker is accessible in the design tools as well as in the XML editor.
Color picker resource tab
View binding is a feature that allows you to more easily write code that interacts with views by providing compile-time safety when referencing views in your code. When enabled, view binding generates a binding class for each XML layout file present in that module. In most cases, view binding replaces findViewById. You can reference all views that have an ID with no risk of null pointer or class cast exceptions.These differences mean that incompatibilities between your layout and your code will result in your build failing at compile time rather than at runtime. To enable view binding in your project, include the following in each module’s build.gradle file:
android { viewBinding.enabled = true }
For more information, check out this blog post by one of our developer experts.
The following Android NDK features in Android Studio, previously supported in Java, are now also supported in Kotlin:
Learn more
Android Studio 3.6 includes the IntelliJ 2019.2 platform release. This IntelliJ release includes many improvements from a new services tool window to much improved startup times. Learn more
You can now add a class and then deploy that code change to your running app by clicking either Apply Code Changes or Apply Changes and Restart Activity.
To learn more about the difference between these two actions, see Apply Changes.
Android Gradle plugin 3.6 and higher includes support for the Maven Publish Gradle plugin, which allows you to publish build artifacts to an Apache Maven repository. The Android Gradle plugin creates a component for each build variant artifact in your app or library module that you can use to customize a publication to a Maven repository. This change will make it easier to manage the release lifecycle for your various targets. Learn more
Additionally, Android Gradle plugin has made significant performance improvement for annotation processing/KAPT for large projects. This is caused by AGP now generating R class bytecode directly, instead of .java files.
The Android build team is continuously working on changes to improve build performance, and in this release we changed the default packaging tool to zipflinger for debug builds. Users should see an improvement in build speed, but you can also revert to using the old packaging tool by setting android.useNewApkCreator=false in your gradle.properties file.
android.useNewApkCreator=false
Edit your gradle.properties file to disable the new packaging tool
Android Emulator 29.2.12 includes a new way for app developers to interface with the emulated device location. We embedded the Google Maps user interface in the extended controls menu to make it easier to specify locations and also to construct routes from pairs of locations. Individual points can be saved and re-sent to the device as the virtual location, while routes can be generated through typing in addresses or clicking two points. These routes can be replayed in real time as locations along the route are sent to the guest OS.
Android Emulator location UI with real-time location streaming
Emulator 29.1.10 includes preliminary support for multiple virtual displays. As more devices are available that have multiple displays, it is important to test your app on a variety of multi-display configurations. Users can configure multiple displays through the settings menu (Extended Controls > Settings).
Multi-display support in Android Emulator
Configure secondary displays in the Android Emulator Extended Controls Panel
When downloading Android SDK components and tools using the Android Studio SDK Manager, Android Studio now allows you to resume downloads that were interrupted (for example, due to a network issue) instead of restarting the download from the beginning. This enhancement is especially helpful for large downloads, such as the Android Emulator or system images, when internet connectivity is unreliable.
Pause and resume SDK downloads
Android Studio allows you to import externally-built APKs to debug and profile them. Previously, when changes to those APKs were made, you would have to manually import them again and reattach symbols and sources. Android Studio 3.6 now automatically detects changes made to the imported APK file and gives you an option to re-import it in-place.
Attach Kotlin sources to imported APKs
We added support for attaching Kotlin source files to imported APKs. To learn more, see Attach Kotlin/Java sources.
Attach Kotlin/Java sources to imported APKs
Based on your feedback, we’ve added in the Memory Profiler the ability to detect Activity and Fragment instances which may have leaked. To get started, capture or import a heap dump file in the Memory Profiler, and check the Activity/Fragment Leaks checkbox to generate the results. For more information on how Android Studio detects leaks, please see our documentation.
Detect leaked Activities and Fragments in the Memory Profiler
When using the APK Analyzer to inspect DEX files, you can now deobfuscate class and method bytecode. While in the DEX file viewer, load the ProGuard mappings file for the APK you’re analyzing. When loaded, you will be able to right-click on the class or method you want to inspect by selecting Show bytecode. Learn more
Deobfuscate class and method bytecode by selecting Show Bytecode in the APK Analyzer
To recap, Android Studio 3.6 includes these new enhancements & features:
Design
Develop
Build
Test
Optimize
Download Android Studio 3.6 from the download page. If you are using a previous release of Android Studio, you can simply update to the latest version of Android Studio. To use the mentioned Android Emulator features make sure you are running at least Android Emulator v29.2.12 downloaded via the Android Studio SDK Manager.
As mentioned above, we appreciate any feedback on things you like, and issues or features you would like to see. If you find a bug or issue, feel free to file an issue. Follow us -- the Android Studio development team ‐ on Twitter and on Medium.
By Dan Albert, Software Engineer
One thing that NDK users struggle with is managing native dependencies:
With version 4.0 of the Android Gradle Plugin, we’ve addressed these issues by adding support for distributing and exposing native libraries through the same mechanism that you do for Java libraries: Android Archives (AARs).
Here’s how you’d use curl and jsoncpp for example (and automatically pull in the implicit OpenSSL dependency that curl has):
// build.gradle dependencies { implementation 'com.android.ndk.thirdparty:curl:7.68.0-alpha-1' implementation 'com.android.ndk.thirdparty:jsoncpp:1.8.4-alpha-1' }
Note: With AGP 4.0 this is still experimental, so to enable this functionality you must set the following properties in your project's gradle.properties file:
gradle.properties
# Enables Prefab android.enablePrefab=true # Work around https://issuetracker.google.com/149575364 android.enableParallelJsonGen=false # 4.0.0 canary 9 defaults to Prefab 1.0.0-alpha3, which is not the latest. android.prefabVersion=1.0.0-alpha5
Declaring the dependencies in your build.gradle will cause Gradle to download those dependencies from Maven, but you must still instruct CMake or ndk-build how those dependencies should be used. Fortunately, the necessary CMake package config or ndk-build module will be automatically generated on your behalf. All you need to do is import and use them.
build.gradle
Here’s an example with CMake:
cmake_minimum_required(VERSION 3.6) project(app VERSION 1.0.0 LANGUAGES CXX) find_package(curl REQUIRED CONFIG) find_package(jsoncpp REQUIRED CONFIG) add_library(app SHARED app.cpp) target_link_libraries(app curl::curl jsoncpp::jsoncpp)
And here’s the same example with ndk-build:
LOCAL_PATH := $(call my-dir) include $(CLEAR_VARS) LOCAL_MODULE := libapp LOCAL_SRC_FILES := app.cpp LOCAL_SHARED_LIBRARIES := jsoncpp curl include $(BUILD_SHARED_LIBRARY) $(call import-module,prefab/curl) $(call import-module,prefab/jsoncpp)
And that’s it. In app.cpp you can now do the following:
app.cpp
#include "curl/curl.h" #include "json/json.h"
A very common issue that people have is building OpenSSL to use with curl. While not explicitly mentioned in the build scripts above, the curl package itself depends on OpenSSL so this support is available automatically.
For the complete example, see the curl-ssl sample.
The tool that facilitates all of this is called Prefab. Each AAR that exposes C++ libraries to its consumers packages their libraries, headers, and a small amount of metadata into the prefab directory in the AAR. If the prefab directory is found in an AAR dependency, the Android Gradle Plugin automatically runs Prefab to generate build system scripts from the contained information.
prefab
Each AAR might contain a large number of prebuilts for different configurations, so Prefab will perform a number of compatibility checks to find a suitable library for your build configuration. The selected library will match your build’s ABI, minSdkVersion, STL choice, and be the best fit for the version of the NDK that you’re using.
minSdkVersion
We’ve already published the following libraries:
For an up to date list, search https://maven.google.com/web/index.html for “com.android.ndk.thirdparty”.
For the libraries we currently distribute, we wrote ndkports. This is a good fit if the library you’re building is a typical Linux or cross-platform project that doesn’t fit naturally into a typical Android build. If that’s a fit for the library you want, feel free to use ndkports for it, and consider sending us the patch!
If you’d like to request that Google maintain and publish an open source library in Prefab, use the “Package request” bug template on https://github.com/google/prefab/issues. Please keep in mind that each package does come with an ongoing cost, and we will be adding support on a limited basis so we will not be able to support everything.
Coming next is support for exposing your libraries in AARs using the existing Android Library publishing workflow.
For more information about using native dependencies with the Android Gradle Plugin, see the documentation. For more examples, see the NDK samples.
If you’d like to learn more about Prefab itself, see its documentation on GitHub.
If you encounter any issues, file bugs in our Issue Tracker.
Posted by Dave Burke, VP of Engineering
Android has led the way towards the future of mobile, with new technologies like 5G to foldable displays to machine learning built into the core. A hallmark of our approach is a strong developer community that provides early and thoughtful feedback, helping us deliver a robust platform for apps and games that delight billions of users around the world. So today, we’re releasing the first Developer Preview of Android 11, and building on a strong feedback cycle last year, we’re making this year’s preview available to you earlier than ever.
With Android 11 we’re keeping our focus on helping users take advantage of the latest innovations, while continuing to keep privacy and security a top priority. We’ve added multiple new features to help users manage access to sensitive data and files, and we’ve hardened critical areas of the platform to keep the OS resilient and secure. For developers, Android 11 has a ton of new capabilities for your apps, like enhancements for foldables and 5G, call-screening APIs, new media and camera capabilities, machine learning, and more.
This is just a first look; like prior years, we’ll continue to share new features and updates over the coming months and into Google I/O as we work through your feedback. The most important thing for you to do right now is this: visit the Android 11 developer site, download a system image for your Pixel 2, 3, 3a, or 4 device, and let us know what you think!
Today’s release is an early baseline build for developers only and not intended for daily or consumer use, so we're making it available by manual download and flash only. Remember, getting early input from you is crucial in helping us evolve the platform to meet your needs. Read on for a taste of what’s new in Android 11, and visit the developer site for details on timeline, how to test, and how to give feedback.
5G brings consistently faster speeds and lower latency to more users around the world. With 5G you can extend your Wi-Fi app experiences -- such as streaming 4K video or loading higher-res game assets -- to mobile users, or you can build new experiences designed specifically for 5G. In Android 11 we’re enhancing and updating the existing connectivity APIs so you can take advantage of 5G’s improved speeds.
Moving beyond the home, 5G can for example let you enhance your “on-the-go” experience by providing seamless interactions with the world around you from friends and family to businesses.
Device makers are continuing to innovate by bringing exciting new form-factors and device screens to market. We’ve extended support for these in the platform, with APIs to let you optimize your apps.
Communicating with your friends and colleagues is the most important thing many people do on their phones. In Android 11, we are introducing changes that help developers create deeper conversational experiences, a few of which you’ll see early versions of in DP1:
Neural Networks API (NNAPI) is designed for running computationally intensive operations for machine learning on Android devices. In Android 11, we’re expanding the operations and controls available to developers. In this release, we’ve added new operations and execution controls to help optimize common use cases:
See the NDK sample code for examples using these new APIs.
Watch for more coming in later preview updates. We’re working with hardware vendors and popular machine learning frameworks such as TensorFlow to optimize and roll out support for NNAPI 1.3.
Privacy has always been at the core of Android, and each year we’ve added more ways to keep users secure and increase transparency and control. These changes have been popular with users - for example in Android 10 we added the “While app is in use” permission option to give users more granular control over their location and limit background location access. So far, when given the “While app is in use” option, about half of users select it.
In Android 11 we’re continuing our focus on user privacy with new permission options, updates to scoped storage, and more. Please give these features a try with your apps right away and let us know what you think.
One-time permission dialog in Android 11.
In addition to these platform changes, users tell us that they want more protection on earlier versions of Android and more transparency around how apps will use this data, so we are updating Google Play Policy to ensure that apps only request location permissions when truly necessary. Read more
We focus on raising the bar for security with each version of Android -- from reaching more devices with monthly security updates to building more protections into the latest platform. In Android 11, we’ve extended Android’s defense-in-depth strategies to more areas of the platform and added new features and APIs for apps.
Since Android 10, we’ve been scaling up our investment in Google Play System Updates (Project Mainline) to improve security, privacy, and consistency across the ecosystem. Thanks to strong collaboration with device makers, we’ve made significant progress towards this goal and have expanded our infrastructure to reach a wider range of devices more safely and quickly.
In Android 11, we’ve added 12 new updatable modules, for a total of 22 modules. Highlights include a permissions module that standardizes user and developer access to critical privacy controls on Android devices, a media provider module that’s integral to our privacy efforts around Scoped Storage, and an NNAPI (Neural Networks API) module that optimizes performance and guarantees consistent APIs across devices. To learn more about Google Play System Updates, check out the Project Mainline blog post.
We’re also working to make updates faster and smoother by prioritizing app compatibility as we roll out new platform versions. In Android 11 we’ve added new processes, developer tools, and release milestones to minimize the impact of platform updates.
App compatibility toggles in Developer Options.
The Developer Preview has everything you need to try the Android 11 features, test your apps, and give us feedback. To get started, download and flash a device system image to a Pixel 2 / 2 XL, Pixel 3 / 3 XL, Pixel 3a / 3a XL, or Pixel 4 / 4 XL device. Additionally, you can set up the Android Emulator through Android Studio. The Android Emulator running Android 11 system images includes experimental support to run ARM 32-bit & 64-bit binary app code directly on 64-bit x86 Android Emulator system images. Lastly, for broader testing, GSI images are also available.
Next, update your Android Studio environment with the Android 11 Preview SDK and tools - you can do this from inside Android Studio. See the setup guide for complete details. To take advantage of the latest Android Studio features, we recommend installing the latest version of Android Studio from the canary channel.
When you’re set up, here are some of the things you can do:
For more information, visit the Android 11 developer site. You’ll find an overview of what’s new in this release, details on behavior changes, setup and migration guides, release notes, feedback channels, and more.
We plan to update the preview system images and SDK regularly throughout the Android 11 release cycle. This initial preview release is for developers only and not intended for daily or consumer use, so we're making it available by manual download and flash only. Downloads are here and instructions are here.
As we get closer to a final product, we'll be inviting consumers to try it out as well, and we'll open up enrollments through Android Beta at that time. Stay tuned for details, but for now please note that Android Beta is not currently available for Android 11.
As always, your feedback is crucial, so please let us know what you think — the sooner we hear from you, the more of your feedback we can integrate, and because of timelines, we’re giving priority to input we receive in the next several weeks. When you find issues, please report them here.
Posted by Krish Vitaldevara, Director of Product Management Trust & Safety, Google Play
Last year, we made several changes to our platform and policies to increase user trust and safety. We’re proud of the work we’ve done to improve family safety, limit use of sensitive permissions, and catch bad actors before they ever reach the Play Store.
We realize that changes can lead to work for developers. Last year, you told us that you wanted more detailed communications about impactful updates, why we’re making them, and how to take action. You also asked for as much time as possible to make any changes required.
With that feedback in mind, today, we’re previewing Android and Google Play policy changes that will impact how developers access location in the background.
Giving users more control over their location data
Users consistently tell us that they want more control over their location data and that we should take every precaution to prevent misuse. Since the beginning of Android, users have needed to grant explicit permission to any app that wants access to their location data.
In Android 10, people were given additional control to only grant access when the app is in use, which makes location access more intentional. Users clearly appreciated this option as over half of users select “While app is in use.”
Now in Android 11, we’re giving users even more control with the ability to grant a temporary “one-time” permission to sensitive data like location. When users select this option, apps can only access the data until the user moves away from the app, and they must then request permission again for the next access. Please visit the Android 11 developer preview to learn more.
Preventing unnecessary access to background location
Users tell us they also want more protection on earlier versions of Android - as well as more transparency around how apps use this data.
As we took a closer look at background location usage, we found that many of the apps that requested background location didn’t actually need it. In fact, many of these apps could provide the same user experience by only accessing location when the app is visible to the user. We want to make it easier for users to choose when to share their location and they shouldn't be asked for a permission that the app doesn't need.
Later this year, we will be updating Google Play policy to require that developers get approval if they want to access location data in the background. Factors that will be looked at include:
All apps will be evaluated against the same factors, including apps made by Google, and all submissions will be reviewed by people on our team. Let’s take a look at three examples:
An app that sends emergency or safety alerts as part of its core functionality - and clearly communicates why access is needed to the user - would have a strong case to request background location.
A social networking app that allows users to elect to continuously share their location with friends would also have a strong case to access location in the background.
An app with a store locator feature would work just fine by only accessing location when the app is visible to the user. In this scenario, the app would not have a strong case to request background location under the new policy.
When we spoke to developers for feedback, the vast majority understood user concerns over their information falling into the wrong hands and were willing to change their location usage to be safer and more transparent.
Getting approval for background access
We know that when we update our policies, you want to get actionable feedback and have ample time to make changes. Before we implement this policy change, you will be able to submit your use case via the Play Console and receive feedback on whether it will be allowed under the new policy.
We anticipate the following timeline for this policy rollout; however, it is subject to change.
Review and evaluate your location access
We encourage all developers to review the following best practices for accessing location data in their apps:
We hope you found this policy preview useful in planning your roadmap for the year and we appreciate your efforts to build privacy-friendly apps. Together, we can keep the Android ecosystem safe and secure for everyone.
By Omar El Sheikh, Android Engineer
Francesco Carucci, Developer Advocate
Vulkan provides developers with the power to specify much more information to devices about rendering state compared to OpenGL. With that power though comes some new responsibilities; developers are expected to explicitly implement things that in OpenGL were handled by the driver. One of these things is device orientation and its relationship to render surface orientation. Currently, there are 3 ways that Android can handle reconciling the render surface of the device with the orientation of the device :
There currently isn't a way for an application to know whether surface rotation handled outside of the application will be free. Even if there is a DPU to take care of this for us, there will still likely be a measurable performance penalty to pay. If your application is CPU bound, this becomes a huge power issue due to the increased GPU usage by the Android Compositor which usually is running at a boosted frequency as well; and if your application is GPU bound, then it becomes a potentially large performance issue on top of that as the Android Compositor will preempt your application's GPU work causing your application to drop its frame rate.
On Pixel 4XL, we have seen on shipping titles that SurfaceFlinger (the higher-priority task that drives the Android Compositor) both regularly preempts the application’s work causing 1-3ms hits to frametimes, as well as puts increased pressure on the GPU’s vertex/texture memory as the Compositor has to read the frame to do its composition work.
Handling orientation properly stops GPU preemption by SurfaceFlinger almost entirely, as well as sees the GPU frequency drop 40% as the boosted frequency used by the Android Compositor is no longer needed.
To ensure surface rotations are handled properly with as little overhead as possible (as seen in the case above) we recommend implementing method 3 - this is known as pre-rotation. The primary mechanism with which this works is by telling the Android OS that we are handling the surface rotation by specifying in the orientation during swapchain creation through the passed in surface transform flags, which stops the Android Compositor from doing the rotation itself.
Knowing how to set the surface transform flag is important for every Vulkan application, since applications tend to either support multiple orientations, or support a single orientation where its render surface is in a different orientation to what the device considers its identity orientation; For example a landscape-only application on a portrait-identity phone, or a portrait-only application on a landscape-identity tablet.
In this post we will describe in detail how to implement pre-rotation and handle device rotation in your Vulkan application.
To handle device rotation in your app, change the application’s AndroidManifest.xml to tell Android that your app will handle orientation and screen size changes. This prevents Android from destroying and recreating the Android activity and calling the onDestroy() function on the existing window surface when an orientation change occurs. This is done by adding the orientation (to support API level <13) and screenSize attributes to the activity’s configChanges section:
onDestroy()
orientation
screenSize
<activity android:name="android.app.NativeActivity" android:configChanges="orientation|screenSize">
If your application fixes its screen orientation using the screenOrientation attribute you do not need to do this. Also if your application uses a fixed orientation then it will only need to setup the swapchain once on application startup/resume.
The next thing that needs to be done is to detect the device’s screen resolution that is associated with the VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR as this resolution is the one that the Swapchain will always need to be set to. The most reliable way to get this is to make a call to vkGetPhysicalDeviceSurfaceCapabilitiesKHR() at application startup and storing the returned extent - swapping the width and height based on the currentTransform that is also returned in order to ensure we are storing the identity screen resolution:
VK_SURFACE_TRANSFORM_IDENTITY_BIT_KHR
vkGetPhysicalDeviceSurfaceCapabilitiesKHR()
VkSurfaceCapabilitiesKHR capabilities; vkGetPhysicalDeviceSurfaceCapabilitiesKHR(physDevice, surface, &capabilities); uint32_t width = capabilities.currentExtent.width; uint32_t height = capabilities.currentExtent.height; if (capabilities.currentTransform & VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR || capabilities.currentTransform & VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR) { // Swap to get identity width and height capabilities.currentExtent.height = width; capabilities.currentExtent.width = height; } displaySizeIdentity = capabilities.currentExtent;
displaySizeIdentity is a VkExtent2D that we use to store said identity resolution of the app's window surface in the display’s natural orientation.
displaySizeIdentity
To know when the application has encountered an orientation change, the most reliable means of tracking this change involves checking the return value of vkQueuePresentKHR() and seeing if it returned VK_SUBOPTIMAL_KHR
vkQueuePresentKHR()
VK_SUBOPTIMAL_KHR
auto res = vkQueuePresentKHR(queue_, &present_info); if (res == VK_SUBOPTIMAL_KHR){ orientationChanged = true; }
One thing to note about this solution is that it only works on devices running Android Q and above as that is when Android started to return VK_SUBOPTIMAL_KHR from vkQueuePresentKHR()
orientationChanged is a boolean stored somewhere accessible from the applications main rendering loop
orientationChanged
For devices running older versions of Android below 10, a different implementation is needed since we do not have access to VK_SUBOPTIMAL_KHR.
VK_SUBOPTIMAL_KHR.
On pre-10 devices we can poll the current device transform every pollingInterval frames, where pollingInterval is a granularity decided on by the programmer. The way we do this is by calling vkGetPhysicalDeviceSurfaceCapabilitiesKHR() and then comparing the returned currentTransform field with that of the currently stored surface transformation (in this code example stored in pretransformFlag)
pollingInterval
currentTransform
pretransformFlag
currFrameCount++; if (currFrameCount >= pollInterval){ VkSurfaceCapabilitiesKHR capabilities; vkGetPhysicalDeviceSurfaceCapabilitiesKHR(physDevice, surface, &capabilities); if (pretransformFlag != capabilities.currentTransform) { window_resized = true; } currFrameCount = 0; }
On a Pixel 4 running Android Q, polling vkGetPhysicalDeviceSurfaceCapabilitiesKHR() took between .120-.250ms and on a Pixel 1XL running Android O polling took .110-.350ms
A second option for devices running below Android 10 is to register an onNativeWindowResized() callback to call a function that sets the orientationChanged flag to signal to the application an orientation change has occurred:
onNativeWindowResized()
void android_main(struct android_app *app) { ... app->activity->callbacks->onNativeWindowResized = ResizeCallback; }
Where ResizeCallback is defined as:
ResizeCallback
void ResizeCallback(ANativeActivity *activity, ANativeWindow *window){ orientationChanged = true; }
The drawback to this solution is that onNativeWindowResized() only ever gets called on 90 degree orientation changes (going from landscape to portrait or vice versa), so for example an orientation change from landscape to reverse landscape will not trigger the swapchain recreation, requiring the Android compositor to do the flip for your application.
To actually handle the orientation change, we first have a check at the top of the main rendering loop for whether the orientationChanged variable has been set to true, and if so we'll go into the orientation change routine:
bool VulkanDrawFrame() { if (orientationChanged) { OnOrientationChange(); }
And within the OnOrientationChange() function we will do all the work necessary to recreate the swapchain. This involves destroying any existing Framebuffers and ImageViews; recreating the swapchain while destroying the old swapchain (which will be discussed next); and then recreating the Framebuffers with the new swapchain’s DisplayImages. Note that attachment images (depth/stencil images for example) usually do not need to be recreated as they are based on the identity resolution of the pre-rotated swapchain images.
OnOrientationChange()
void OnOrientationChange() { vkDeviceWaitIdle(getDevice()); for (int i = 0; i < getSwapchainLength(); ++i) { vkDestroyImageView(getDevice(), displayViews_[i], nullptr); vkDestroyFramebuffer(getDevice(), framebuffers_[i], nullptr); } createSwapChain(getSwapchain()); createFrameBuffers(render_pass, depthBuffer.image_view); orientationChanged = false; }
And at the end of the function we reset the orientationChanged flag to false to show that we have handled the orientation change.
In the previous section we mention having to recreate the swapchain. The first steps to doing so involves getting the new characteristics of the rendering surface:
void createSwapChain(VkSwapchainKHR oldSwapchain) { VkSurfaceCapabilitiesKHR capabilities; vkGetPhysicalDeviceSurfaceCapabilitiesKHR(physDevice, surface, &capabilities); pretransformFlag = capabilities.currentTransform;
With the VkSurfaceCapabilities struct populated with the new information, we can now check to see whether an orientation change has occurred by checking the currentTransform field and store it for later in the pretransformFlag field as we will be needing it for later when we make adjustments to the MVP matrix.
VkSurfaceCapabilities
In order to do so we must make sure that we properly specify some attributes within the VkSwapchainCreateInfo struct:
VkSwapchainCreateInfo
VkSwapchainCreateInfoKHR swapchainCreateInfo{ ... .sType = VK_STRUCTURE_TYPE_SWAPCHAIN_CREATE_INFO_KHR, .imageExtent = displaySizeIdentity, .preTransform = pretransformFlag, .oldSwapchain = oldSwapchain, }; vkCreateSwapchainKHR(device_, &swapchainCreateInfo, nullptr, &swapchain_)); if (oldSwapchain != VK_NULL_HANDLE) { vkDestroySwapchainKHR(device_, oldSwapchain, nullptr); }
The imageExtent field will be populated with the displaySizeIdentity extent that we stored at application startup. The preTransform field will be populated with our pretransformFlag variable (which is set to the currentTransform field of the surfaceCapabilities). We also set the oldSwapchain field to the swapchain that we are about to destroy.
imageExtent
preTransform
surfaceCapabilities
oldSwapchain
It is important that the surfaceCapabilities.currentTransform field and the swapchainCreateInfo.preTransform field match because this lets the Android OS know that we are handling the orientation change ourselves, thus avoiding the Android Compositor.
surfaceCapabilities.currentTransform
swapchainCreateInfo.preTransform
The last thing that needs to be done is actually apply the pre-transformation. This is done by applying a rotation matrix to your MVP matrix. What this essentially does is apply the rotation in clip space so that the resulting image is rotated to the device current orientation. You can then simply pass this updated MVP matrix into your vertex shader and use it as normal without the need to modify your shaders.
glm::mat4 pre_rotate_mat = glm::mat4(1.0f); glm::vec3 rotation_axis = glm::vec3(0.0f, 0.0f, 1.0f); if (pretransformFlag & VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR) { pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(90.0f), rotation_axis); } else if (pretransformFlag & VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR) { pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(270.0f), rotation_axis); } else if (pretransformFlag & VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR) { pre_rotate_mat = glm::rotate(pre_rotate_mat, glm::radians(180.0f), rotation_axis); } MVP = pre_rotate_mat * MVP;
If your application is using a non-full screen viewport/scissor region, they will need to be updated according to the orientation of the device. This requires that we enable the dynamic Viewport and Scissor options during Vulkan’s pipeline creation:
VkDynamicState dynamicStates[2] = { VK_DYNAMIC_STATE_VIEWPORT, VK_DYNAMIC_STATE_SCISSOR, }; VkPipelineDynamicStateCreateInfo dynamicInfo = { .sType = VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO, .pNext = nullptr, .flags = 0, .dynamicStateCount = 2, .pDynamicStates = dynamicStates, }; VkGraphicsPipelineCreateInfo pipelineCreateInfo = { .sType = VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO, ... .pDynamicState = &dynamicInfo, ... }; VkCreateGraphicsPipelines(device, VK_NULL_HANDLE, 1, &pipelineCreateInfo, nullptr, &mPipeline);
The actual computation of the viewport extent during command buffer recording looks like this:
int x = 0, y = 0, w = 500, h = 400; glm::vec4 viewportData; switch (device->GetPretransformFlag()) { case VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR: viewportData = {bufferWidth - h - y, x, h, w}; break; case VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR: viewportData = {bufferWidth - w - x, bufferHeight - h - y, w, h}; break; case VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR: viewportData = {y, bufferHeight - w - x, h, w}; break; default: viewportData = {x, y, w, h}; break; } const VkViewport viewport = { .x = viewportData.x, .y = viewportData.y, .width = viewportData.z, .height = viewportData.w, .minDepth = 0.0F, .maxDepth = 1.0F, }; vkCmdSetViewport(renderer->GetCurrentCommandBuffer(), 0, 1, &viewport);
Where x and y define the coordinates of the top left corner of the viewport, and w and h define the width and height of the viewport respectively.
x
y
w
h
The same computation can be used to also set the scissor test, and is included below for completeness:
int x = 0, y = 0, w = 500, h = 400; glm::vec4 scissorData; switch (device->GetPretransformFlag()) { case VK_SURFACE_TRANSFORM_ROTATE_90_BIT_KHR: scissorData = {bufferWidth - h - y, x, h, w}; break; case VK_SURFACE_TRANSFORM_ROTATE_180_BIT_KHR: scissorData = {bufferWidth - w - x, bufferHeight - h - y, w, h}; break; case VK_SURFACE_TRANSFORM_ROTATE_270_BIT_KHR: scissorData = {y, bufferHeight - w - x, h, w}; break; default: scissorData = {x, y, w, h}; break; } const VkRect2D scissor = { .offset = { .x = (int32_t)viewportData.x, .y = (int32_t)viewportData.y, }, .extent = { .width = (uint32_t)viewportData.z, .height = (uint32_t)viewportData.w, }, }; vkCmdSetScissor(renderer->GetCurrentCommandBuffer(), 0, 1, &scissor);
If your application is using derivative computations such as dFdx and dFdy, additional transformations may be needed to account for the rotated coordinate system as these computations are executed in pixel space. This requires the app to pass some indication of the preTransform into the fragment shader (such as an integer representing the current device orientation) and use that to map the derivative computations properly:
In order for your application to get the most out of Vulkan on Android, implementing pre-rotation is a must. The most important takeaways from this blogpost are:
Here is a link to a minimal example of pre-rotation being implemented for an android application:
https://github.com/google/vulkan-pre-rotation-demo
Native code in memory-unsafe languages like C and C++ is often vulnerable to memory corruption bugs. Our data shows that issues like use-after-free, double-free, and heap buffer overflows generally constitute more than 65% of High & Critical security bugs in Chrome and Android.
In previous years our memory bug detection efforts were focused on Address Sanitizer (ASan). ASan catches these errors but causes your app to use 2x-3x extra memory and to run slower.
To better tackle these problems we’ve developed Hardware-Assisted Address Sanitizer (HWASan). HWASan typically only requires 15% more memory. It’s also a lot faster than ASan. HWASan’s performance makes it usable not only for unit testing, but also for interactive human-driven testing. We use this to find memory issues in the Android OS itself, and now we've made it easy for app developers to use it too. HWASan is fast enough that some Android developers use it on their development devices for everyday tasks.
HWASan is based on memory tagging and depends on the Top Byte Ignore feature present in all 64-bit ARM CPUs and the associated kernel support. Every memory allocation is assigned a random 8-bit tag that is stored in the most significant byte (MSB) of the address, but ignored by the CPU. As a result, this tagged pointer can be used in place of a regular pointer without any code changes.
Under the hood, HWASan uses shadow memory - a sparse map that assigns a tag value to each 16 byte block of program memory. Compile time code instrumentation is used to insert checks that compare pointer and memory tags for every memory access, and raise an error if they do not match.
This approach allows us to detect both use-after-free and buffer-overflow types of bugs. The memory tag in the shadow is changed to a random value during allocation and deallocation. As a result, attempting to access deallocated memory with a dangling pointer will almost certainly fail due to a tag mismatch. The same is true for an attempt to access memory outside of the allocated region, which is very likely to have a different tag. Stack and global variables are similarly protected.
Use-after-free bug detection with memory tagging.
This approach is non deterministic: because of the limited number of possible tags, an invalid memory access has 1 chance out of 256 (approximately 0.4%) to pass undetected. We have not observed this as a problem in practice, but, due to the tag randomness, running the program the second time is very likely to find any bugs that the first run has missed.
An advantage of HWASan over ASan is its ability to find bugs that happen far from their origination point - for example, a use-after-free where the memory is accessed long after it has been deallocated, or a buffer overflow with a large offset. This is not the case with ASan, which uses red zones around memory allocations, and a quarantine for the temporary storage of recently deallocated memory blocks. Both redzones and the quarantine are of limited size, and error detection is unlikely beyond that. HWASan uses a different approach that does not have these limitations.
When a bug is discovered the process is terminated and a crash dump is printed to logcat. The “Abort message” field contains a HWASan report, which shows the access type (read or write), access address, thread id and the stack trace of the bad memory access. This is followed by a stack trace for the original allocation, and, for use-after-free bugs, a stack trace showing where the deallocation took place. Advanced users can find extra debugging information below this, including a map of memory tags for nearby locations.
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr -------- Abort message: '==21586==ERROR: HWAddressSanitizer: tag-mismatch on address 0x0042a0807af0 at pc 0x007b23b8786c WRITE of size 1 at 0x0042a0807af0 tags: db/19 (ptr/mem) in thread T0 #0 0x7b23b87868 (/data/app/com.example.myapp/lib/arm64/native.so+0x2868) #1 0x7b8f1e4ccc (/apex/com.android.art/lib64/libart.so+0x198ccc) [...] 0x0042a0807af0 is located 0 bytes to the right of 16-byte region [0x0042a0807ae0,0x0042a0807af0) allocated here: #0 0x7b92a322bc (/path/to/libclang_rt.hwasan-aarch64-android.so+0x212bc) #1 0x7b23b87840 (/data/app/com.example.myapp/lib/arm64/native.so+0x2840) [...]
An example snippet from a HWASan crash report.
Google uses HWASan extensively in Android development, and now you can too. Find out more -- including the details of how to rebuild your app for use with HWASan -- at https://developer.android.com/ndk/guides/hwasan. Prebuilt HWASan system images are available on the AOSP build server (or you can build your own). They can be easily flashed onto a compatible device using the recently announced web flash tool.
Posted by Andrew Ahn, Product Manager, Google Play + Android App Safety
Google Play connects users with great digital experiences to help them be more productive and entertained, as well as providing app developers with tools to reach billions of users around the globe. Such a thriving ecosystem can only be achieved and sustained when trust and safety is one of its key foundations. Over the last few years we’ve made the trust and safety of Google Play a top priority, and have continued our investments and improvements in our abuse detection systems, policies, and teams to fight against bad apps and malicious actors.
In 2019, we continued to strengthen our policies (especially to better protect kids and families), continued to improve our developer approval process, initiated a deeper collaboration with security industry partners through the App Defense Alliance, enhanced our machine learning detection systems analyzing an app’s code, metadata, and user engagement signals for any suspicious content or behaviors, as well as scaling the number and the depth of manual reviews. The combination of these efforts have resulted in a much cleaner Play Store:
In addition we’ve launched a refreshed Google Play Protect experience, our built-in malware protection for Android devices. Google Play Protect scans over 100B apps everyday, providing users with information about potential security issues and actions they can take to keep their devices safe and secure. Last year, Google Play Protect also prevented more than 1.9B malware installs from non-Google Play sources.
While we are proud of what we were able to achieve in partnership with our developer community, we know there is more work to be done. Adversarial bad actors will continue to devise new ways to evade our detection systems and put users in harm's way for their own gains. Our commitment in building the world's safest and most helpful app platform will continue in 2020, and we will continue to invest in the key app safety areas mentioned in last year’s blog post:
Our teams of passionate product managers, engineers, policy experts, and operations leaders will continue to work with the developer community to accelerate the pace of innovation, and deliver a safer app store to billions of Android users worldwide.
