Posted by Donovan McMurray, CameraX Developer Relations Engineer
As part of
Android Jetpack, the CameraX library makes complex camera functionality available in an easy-to-use API, helping you create a best-in-class experience that works consistently across Android versions and devices. As of today,
CameraX version 1.2 is officially in Beta. Update from version 1.1 to take advantage of the latest game-changing features: our new ML Kit integration, which can reduce your boilerplate code when using ML Kit in a CameraX app, and Zero-Shutter Lag, which enables faster action shots than were previously possible.
These two advanced features are simple to implement with CameraX 1.2, so let’s take a look at each of them in depth.
ML Kit Integration
Google’s
ML Kit provides several on-device vision APIs for detecting faces, barcodes, text, objects, and more. We’re making it easier to integrate these APIs with CameraX. Version 1.2 introduces
MlKitAnalyzer, an implementation of
ImageAnalysis.Analyzer that handles much of the ML Kit setup for you.
You can use
MlKitAnalyzer with both
cameraController and
cameraProvider workflows. If you use the
cameraController.setImageAnalysisAnalyzer() method, then CameraX can also handle the coordinates transformation between the ML Kit output and your
PreviewView.Here’s a code snippet using
setImageAnalysisAnalyzer() to set a
BarcodeScanner on a
cameraController to detect QR codes. CameraX automatically handles the coordinate transformations when you pass
COORDINATE_SYSTEM_VIEW_REFERENCED into the
MlKitAnalyzer. (Use
COORDINATE_SYSTEM_ORIGINAL to prevent CameraX from applying any coordinate transformations.)
val options = BarcodeScannerOptions.Builder() .setBarcodeFormats(Barcode.FORMAT_QR_CODE) .build() val barcodeScanner = BarcodeScanning.getClient(options)
cameraController.setImageAnalysisAnalyzer( executor, new MlKitAnalyzer(List.of(barcodeScanner), COORDINATE_SYSTEM_VIEW_REFERENCED, executor, result -> { // The value of result.getResult(barcodeScanner) // can be used directly for drawing UI overlay. } ) ) |
Zero-Shutter Lag
Have you ever lined up the perfect photo, but when you click the shutter button the lag causes you to miss the best moment? CameraX 1.2 offers a solution to this problem by introducing Zero-Shutter Lag.
Prior to CameraX 1.2, you could optimize for quality
(CAPTURE_MODE_MAXIMIZE_QUALITY) or efficiency
(CAPTURE_MODE_MINIMIZE_LATENCY) when calling
ImageCapture.Builder.setCaptureMode(). CameraX 1.2 adds a new value
(CAPTURE_MODE_ZERO_SHOT_LAG) that reduces latency even further than
CAPTURE_MODE_MINIMIZE_LATENCY. Note: for devices that cannot support Zero-Shutter Lag, CameraX will fallback to
CAPTURE_MODE_MINIMIZE_LATENCY.
We accomplish this by using a circular buffer of photos. On image capture, we go back in time in the circular buffer to get the frame closest to the actual press of the shutter button. No DeLorean needed. Great Scott!
Here’s an example of how this works in a CameraX app with Preview and ImageCapture use cases:
- Just like any other app with a Preview use case, CameraX sends images from the camera to the UI for the user to see.
- With Zero-Shutter Lag, CameraX also sends images to a circular buffer which holds multiple recent images.
- When the user presses the shutter button, there is inevitably some lag in sending the current camera image to your app. For this reason, Zero-Shutter Lag goes to the circular buffer to fetch an image.
- CameraX finds the photo in the circular buffer closest to the actual time when the user pressed the shutter button, and returns that photo to your app.
There are a few limitations to keep in mind with Zero-Shutter Lag. First, please be mindful that this is still an experimental feature. Second, since keeping a circular buffer of images is computationally intensive, you cannot use
CAPTURE_MODE_ZERO_SHOT_LAG while using
VideoCapture or
extensions. Third, the circular buffer will increase the memory footprint of your app.
Next steps
Check our
full release notes for CameraX 1.2 for more details on the features described here and more! If you’re ready to try out CameraX 1.2, update your project’s CameraX dependency to 1.2.0-beta01 (or the latest version at the time you’re reading this).
If you would like to provide feedback on any of these features or CameraX in general, please
create a CameraX issue. As always, you can also reach out on our
CameraX Discussion Group.