Posted by Harini Chandrasekharan, Staff Software Engineer, Google Play

The Google Play Store, launched 10 years ago in 2012 sits at the heart of Android, connecting billions of users with an equally staggering and ever-growing collection of apps and games worldwide.

Let's take a peek behind the curtains to learn what it takes to design the serving infrastructure of the worlds largest Android marketplace. In the world of consumer facing software, it's not a surprise that out of box engineering solutions fail to meet the requirements that Google scale demands. Therefore every system at Google is carefully crafted and honed with iterative enhancements to meet the unique availability, quality and latency demands of the Google Play Store.

What is feature engineering?

In the domain of consumer facing features, users’ opinions and choices, developer ecosystem and demand often changes faster than infrastructure can. In such an environment, the biggest challenge engineers face is how to be nimble and design infrastructure that’s not only future-proof but also meets the needs of the consumer space within the constraints of scalability and performance. Let’s take a deeper look at some engineering challenges in such a dynamic space.

What does success look like?

In a data driven organization such as the Play store, metrics are built for measuring anything and everything of importance. Here are some of the dimensions that come in handy when measuring and tracking success:

• Product/business metrics - These are metrics specific to the product or service under consideration. Running A/B experiments to measure changes to these metrics for the new treatment builds confidence, particularly when decision making involves several tradeoffs.
• Performance - Measuring latency, error rates and availability makes the backbone of almost every service and for good reason. Knowing these baseline metrics is essential since this closely tracks user experience and perception of the product.
• System health - These are internal system metrics tracking resource utilization and fleet stability.

Challenges in feature engineering infrastructure

Designing backend systems that scale to the requirements of the Play Store that also meet the performance criteria required to make user interactions feel fluid and responsive is paramount. From an engineering perspective, infrastructure needs to continuously evolve to meet the needs of the business. The Play store is no different—the store infrastructure has evolved several times in the last decade to not only support the needs of new features that are available to users today, but also to modernize, eliminate tech debt and most of all reduce latency.

Feature engineering guiding principles

Let’s see how the Play Store solves these challenges to enable state of the art innovation.

Data driven experiments and launches - understand your success metrics

Optimizing for time to market i.e getting the feature to the user and measuring how it impacts app installs and other store business metrics using A/B experiments is of prime importance. Iterating fast based on data helps tune the final feature to the desired end state. Google has several home grown technologies for running A/B experiments at worldwide scale with seamless integration with metric presentation tools that make running these experiments smooth and easy, so developers can spend more time coding and less in analysis.

Design and experiment with polished MVPs - with a focus on quality

Deciding what to build, whether it meets Google quality standards, understanding engineering costs and the user needs it solves are all important questions that need to be answered before designing anything. Feature Engineering is therefore often done in close collaboration with Product Managers. Aligning on the perfect MVP that can be built in a reasonable amount of engineering time that meets the user journey is the key to a successful product.

Frequently modernize the infrastructure - clean up tech debt

Frequent iterations and a fast MVP development culture often comes with its set of cons, the biggest being tech debt. In optimizing for fast velocity, cutting corners results in obsolete code (due to unlaunchable metrics) or discarded experiment flags. These often make testing, maintaining and impact future development velocity if left unfixed. Additionally, using the latest and greatest frameworks to get to the last milliseconds of latency or making development easier yields great dividends in the long run. Frequently modernizing the infrastructure either via refactoring or full rewrites may traditionally spell signs of poorly designed code, but it's one of the bigger tradeoffs that feature engineers often have to make, because after all what use is all the fancy infrastructure if users don't interact with the feature in the first place!

How useful did you find this blog post?

★ ★ ★ ★ ★