Continuous testing with new Android emulator tools

Posted by Lingfeng Yang, Android Studio team

Developers often use the Android Emulator during their day-to-day development to quickly test the latest changes before they are being committed. In addition, developers are increasingly using the emulator in their continuous integration (CI) systems to run a larger suite of automated tests. To better support this use-case, we are open sourcing the Android Emulator Container Scripts and improving the developer experiences around two pain points:

1. Deployability - finding and running the desired version of Android Emulator.
2. Debuggability - tracking down bugs from remote instances of Android Emulator.

Deployability

Android supports a wide variety of hardware and software configurations, and the Android Emulator is no different. However, this wide variety can create confusion over environment configurations. How should developers obtain emulators and system images? What drivers are required? How do you run with or without CPU or GPU acceleration? (etc. etc.)

To address this we have launched:

• Android Emulator Download Script - This script provides the current up-to-date lists of emulator images (both AOSP and with Google Play Services) as well as emulators binaries (supporting Linux, Mac OS and Windows). You can integrate this with your existing continuous integration system. Going forward, we aim to enhance this service to enable downloading of deprecated versions in addition to the latest versions to make it easier to reproduce historical test results.
• Android Emulator Docker image generator - Android system images and the emulator is only one part of the story. For environment, drivers, and pre-installed system dependencies, we put together a Docker image generator. This creates the complete environment in which the Android Emulator runs. After you start up the Docker image, 1) port forwarding and ADB, or 2) gRPC and WebRTC, makes interaction with the emulator possible. Currently, the Docker image generator is designed to work in Linux. We are also looking at Mac OS and Windows hosts, so stay tuned!

To increase reproducibility, the underlying Dockerfile template makes the required command line flags and system dependencies more explicit (and reproducible via building Docker images from them). For hardware acceleration, note the --privileged flag that is passed to run.sh; we assume CPU acceleration is available when running the emulator, and --privileged is needed to run the containers with CPU acceleration (KVM) enabled.

For more details on how to create and deploy the Android Emulator image, go to the README.

Debuggability

When the emulator is running and a test or the emulator fails, it can be difficult to dive into the running environment and diagnose the error. Often, diagnosis requires direct interaction with the virtual device. We provide two mechanisms for direct interaction:

1. ADB
2. Remote streaming

In the case of ADB, we allow all commands, such as logcat and shell, by forwarding a particular port from the Docker guest to the host. Because the current port is 5555, we'll need to collect more feedback and do more research on how best to separate ports across different containers.

Remote streaming

Security note: With remote streaming, keep in mind that once the service is started, anyone who can connect to your computer on port 80/443 can interact with the emulator. So be careful with running this on a public server!

With remote streaming, you can run the emulator in a container, which is as interactive as running locally. Running the emulator in a container makes it easier to debug issues that can be hard to discover using ADB commands. You can access the emulator using a browser with WebRTC, which is used to stream the video, and gRPC, which is used to send mouse and keyboard events to the emulator. Remote streaming requires three containers:

1. A container that hosts the latest emulator
2. A container with an Envoy web proxy needed for gRPC
3. A container with nginx to serve the React web app

You can compose the Docker containers together using docker-compose, as described in the README. The containers bind to port 80 and 443, so make sure you do not have a web server running. A self-signed certificate will be offered if you point the browser to the host. If you point your browser to the host you should see something like the image below:

Again, keep in mind that anyone who can connect to your host can interact with the emulator. So be careful with running this on a public server!

Let’s scale testing!

Testing can seem to be a tax on development time. However, as many seasoned developers have seen, proper automated testing can increase development velocity as the code base becomes bigger and more complex. Continuous testing should give you confidence that the change you make won’t break your app.