Today, we want to share some best practices on using the OpenGL Shading Language (GLSL) that can optimize the performance of your game and simplify your workflow. Specifically, Layout qualifiers make your code more deterministic and increase performance by reducing your work.
attribute vec4 vertexPosition; attribute vec2 vertexUV; uniform mat4 matWorldViewProjection; varying vec2 outTexCoord; void main() { outTexCoord = vertexUV; gl_Position = matWorldViewProjection * vertexPosition; }
struct Vertex { Vector4 Position; Vector2 TexCoords; };
attribute vec4 vertexPosition; attribute vec2 vertexUV;
glGetAttribLocation
glVertexAttribPointer
GLint handleVertexPos = glGetAttribLocation( myShaderProgram, "vertexPosition" ); glVertexAttribPointer( handleVertexPos, 4, GL_FLOAT, GL_FALSE, 0, 0 ); GLint handleVertexUV = glGetAttribLocation( myShaderProgram, "vertexUV" ); glVertexAttribPointer( handleVertexUV, 2, GL_FLOAT, GL_FALSE, 0, 0 );
layout(location = 0) in vec4 vertexPosition; layout(location = 1) in vec2 vertexUV;
#version 300 es
#version 300 es layout(location = 0) in vec4 vertexPosition; layout(location = 1) in vec2 vertexUV; uniform mat4 matWorldViewProjection; out vec2 outTexCoord; void main() { outTexCoord = vertexUV; gl_Position = matWorldViewProjection * vertexPosition; }
#version 330
const int ATTRIB_POS = 0; const int ATTRIB_UV = 1; glVertexAttribPointer( ATTRIB_POS, 4, GL_FLOAT, GL_FALSE, 0, 0 ); glVertexAttribPointer( ATTRIB_UV, 2, GL_FLOAT, GL_FALSE, 0, 0 );
[This post is by Romain Guy and Chet Haase, Android engineers who have been known to collaborate on the subject of graphics, UIs, and animation. You can read more from them on their blogs at curious-creature.org and graphics-geek.blogspot.com. — Tim Bray]
Earlier this year, Android 3.0 launched with a new 2D rendering pipeline designed to support hardware acceleration on tablets. With this new pipeline, all drawing operations performed by the UI toolkit are carried out using the GPU.
You’ll be happy to hear that Android 4.0, Ice Cream Sandwich, brings an improved version of the hardware-accelerated 2D rendering pipeline to phones, starting with Galaxy Nexus.
In Android 4.0 (API level 14), hardware acceleration, for the first time, is on by default for all applications. For applications at lower API levels, you can turn it on by adding android:hardwareAccelerated="true" to the <application> tag in your AndroidManifest.xml.
android:hardwareAccelerated="true"
<application>
To learn more about the hardware accelerated 2D rendering pipeline visit the official Android developer guide. This guide explains how to control hardware acceleration at various levels, offers several performance tips and tricks and describes in details the new drawing model.
I also encourage you to watch the Android Hardware Accelerated Rendering talk that we gave at Google I/O 2011.
Applications that need to display OpenGL or video content rely today on a special type of UI element called SurfaceView. This widget works by creating a new window placed behind your application’s window. It then punches a hole through your application’s window to reveal the new window. While this approach is very efficient, since the content of the new window can be refreshed without redrawing the application’s window, it suffers from several important limitations.
Because a SurfaceView’s content does not live in the application’s window, it cannot be transformed (moved, scaled, rotated) efficiently. This makes it difficult to use a SurfaceView inside a ListView or a ScrollView. SurfaceView also cannot interact properly with some features of the UI toolkit such as fading edges or View.setAlpha().
View.setAlpha()
To solve these problems, Android 4.0 introduces a new widget called TextureView that relies on the hardware accelerated 2D rendering pipeline and SurfaceTexture. TextureView offers the same capabilities as SurfaceView but, unlike SurfaceView, behaves as a regular view. You can for instance use a TextureView to display an OpenGL scene or a video stream. The TextureView itself can be animated, scrolled, etc.
The following piece of code creates a TextureView to display the video preview from the default camera. The TextureView itself is rotated 45 degrees and semi-transparent.
public class TextureViewActivity extends Activity implements TextureView.SurfaceTextureListener { private Camera mCamera; private TextureView mTextureView; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); mTextureView = new TextureView(this); mTextureView.setSurfaceTextureListener(this); setContentView(mTextureView); } @Override public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) { mCamera = Camera.open(); Camera.Size previewSize = mCamera.getParameters().getPreviewSize(); mTextureView.setLayoutParams(new FrameLayout.LayoutParams( previewSize.width, previewSize.height, Gravity.CENTER)); try { mCamera.setPreviewTexture(surface); } catch (IOException t) { } mCamera.startPreview(); mTextureView.setAlpha(0.5f); mTextureView.setRotation(45.0f); } @Override public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) { // Ignored, the Camera does all the work for us } @Override public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) { mCamera.stopPreview(); mCamera.release(); return true; } @Override public void onSurfaceTextureUpdated(SurfaceTexture surface) { // Called whenever a new frame is available and displayed in the TextureView } }
A TextureView can just as easily be used to embed an OpenGL scene in your application. As of Android 4.0, eglCreateWindowSurface() can be used to render into a SurfaceTexture object.
There were minor improvements in Android 4.0 to some of the new animation facilities that were added in the 3.x releases.
First, if you're new to the new android.animation package and classes added in 3.0 and 3.1, you might want to read Animation in Honeycomb and Introducing ViewPropertyAnimator. These articles discuss the new APIs added in 3.0 that make animation in Android easier, more powerful, and more flexible. The Android 4.0 improvements discussed below are small additions to these core facilities.
One of the constraints of the Java programming language is the lack of “properties”. There is no way to refer generically to a field or a setter/getter of an object. As long as you know what kind of object you have, this is not a problem, because you then know the set of fields and methods that you can call. But if someone passes you some subclass of Object and you'd like to get or set some value “foo” on it, you're out of luck. You can use reflection or JNI to get access to the foo field/methods, but there is no way to refer directly to a field or a method unless you know the instance type of the object that has that field/method on it.
This is a constraint that the new animation system works within. Its whole job, you might say, is to get and set values on generic objects that it's been told about. If someone wants to animate the alpha property on a View, they tell the system the target object and the name of that field (“alpha”), and the animation system uses JNI to figure out which method(s) act on a field of that name. Basically, it looks for setAlpha() and, sometimes, getAlpha() methods. Then when an animation frame occurs, it calculates the new value and passes it into the setter method that it found.
setAlpha()
getAlpha()
This seems like a lot of work for what it does, but there's really no way around it. Unless the animation system were specific to View objects, there's no way that it could know that the target object has appropriate setter/getter methods. And even if it did, there's still no way for callers that construct the animations to tell the animator about the property named “alpha”; there are no function handles like there are in other languages, and there's no way to reference a public field either. (I'm ignoring Reflection here, which has Method and Field objects, because this approach requires much more overhead and processing than the simple function/field references of other languages).
If only there were a way to refer to these properties and to get/set them on generic objects...
Now there is. There is a new Property object in the android.util package that does exactly this. This class offers a set() and a get() method. So if someone hands you a Property object, you can safely call the set() and get() methods without knowing anything about the target object and it will do the job. Moreover, it will do it much more efficiently than the current JNI or reflection approaches because it can, depending on the implementation of it, set a field or call a method on the target object directly. For example, the new ALPHA property on View calls setAlpha() on the View object.
set()
get()
The Property class is a generic utility that can be used anywhere, not just in animations. But it's animations that benefit enormously from it, in their ability to handle properties in a type-safe and efficient manner.
For example prior to Android 4.0, you might construct a fade-out animation on some object myView like this:
ObjectAnimator anim = ObjectAnimator.ofFloat(myView, "alpha", 0);
In Android 4.0, you can use the new ALPHA object on View to construct a Property-based animator instead:
ObjectAnimator anim = ObjectAnimator.ofFloat(myView, View.ALPHA, 0);
There were minor API additions to the ViewPropertyAnimator class (introduced in Android 3.1) which make this class more flexible and powerful:
cancel(): You can now cancel() a running ViewPropertyAnimator.
cancel()
setStartDelay(): You can now set a startDelay on the ViewPropertyAnimator, just like the startDelay of the other Animator classes.
setStartDelay()
start(): ViewPropertyAnimators start automatically, but they do so on the next animation frame handled. This allows them to collect several requests and bundle them together, which is much more efficient and makes it easier to synchronize multiple animations together. However, if you just want to run a single animation, or want to make sure it starts immediately, at the time of construction, you can call start() to avoid that intervening delay.
start()
LayoutTransition (introduced in Android 3.0) continues to provide functionality that makes some kinds of animations easier, specifically when adding, removing, hiding, and showing views. For example, either this snippet in a layout file:
<LinearLayout android:orientation="vertical" android:layout_width="match_parent" android:layout_height="match_parent" android:animateLayoutChanges="true" android:id="@+id/container">
Or this code added at runtime:
container.setLayoutTransition(new LayoutTransition());
will result in a container that animates the visibility of its children views. So when a view is added to the container, the other children will animate out of the way and then the new one will fade in. Similarly, removing a child from the container will fade it out and then animate the other children around to their final places and sizes.
You can customize the animations and timing behavior of a LayoutTransition object, but the code above allows a very easy way to get reasonable default animation behavior.
One of the constraints in the pre-4.0 version of the class was that it did not account for changes in the bounds of the parent hierarchy. So, for example, if a LinearLayout with horizontal orientation has a layout_width of wrap_content and you want to run a transition that removes an item from that layout, then you might notice the parent snapping to the end size and clipping its children instead of animating all of them into their new positions. The new approach (enabled by default, but disabled by a call to setAnimateParentHierarchy(false)) also animates the layout bounds and scrolling values of the parent layout and its parents, all the way up the view hierarchy. This allows LayoutTransition to account for all layout-related changes due to that view being added or removed from its transitioning container.
setAnimateParentHierarchy(false))
The Android 3.0 release saw huge improvements in the visual capabilities of the platform, as we started enabling hardware acceleration and providing new, more flexible animation capabilities. Android 4.0 continues this trend as we continue to work hard on improving the performance, features, and usability of the Android APIs. We’re not done yet, but the enhancements in this release should help you create more exciting Android experiences and applications.
[This post is by Chet Haase, an Android engineer who specializes in graphics and animation, and who occasionally posts videos and articles on these topics on his CodeDependent blog at graphics-geek.blogspot.com. — Tim Bray]
In an earlier article, Animation in Honeycomb, I talked about the new property animation system available as of Android 3.0. This new animation system makes it easy to animate any kind of property on any object, including the new properties added to the View class in 3.0. In the 3.1 release, we added a small utility class that makes animating these properties even easier.
View
First, if you’re not familiar with the new View properties such as alpha and translationX, it might help for you to review the section in that earlier article that discusses these properties entitled, rather cleverly, “View properties”. Go ahead and read that now; I’ll wait.
Okay, ready?
ObjectAnimator
Using the ObjectAnimator class in 3.0, you could animate one of the View properties with a small bit of code. You create the Animator, set any optional properties such as the duration or repetition attributes, and start it. For example, to fade an object called myView out, you would animate the alpha property like this:
Animator
alpha
ObjectAnimator.ofFloat(myView, "alpha", 0f).start();
This is obviously not terribly difficult, either to do or to understand. You’re creating and starting an animator with information about the object being animated, the name of the property to be animated, and the value to which it’s animating. Easy stuff.
But it seemed that this could be improved upon. In particular, since the View properties will be very commonly animated, we could make some assumptions and introduce some API that makes animating these properties as simple and readable as possible. At the same time, we wanted to improve some of the performance characteristics of animations on these properties. This last point deserves some explanation, which is what the next paragraph is all about.
There are three aspects of performance that are worth improving about the 3.0 animation model on View properties. One of the elements concerns the mechanism by which we animate properties in a language that has no inherent concept of “properties”. The other performance issues relate to animating multiple properties. When fading out a View, you may only be animating the alpha property. But when a view is being moved on the screen, both the x and y (or translationX and translationY) properties may be animated in parallel. And there may be other situations in which several properties on a view are animated in parallel. There is a certain amount of overhead per property animation that could be combined if we knew that there were several properties being animated.
The Android runtime has no concept of “properties”, so ObjectAnimator uses a technique of turning a String denoting the name of a property into a call to a setter function on the target object. For example, the String “alpha” gets turned into a reference to the setAlpha() method on View. This function is called through either reflection or JNI, mechanisms which work reliably but have some overhead. But for objects and properties that we know, like these properties on View, we should be able to do something better. Given a little API and knowledge about each of the properties being animated, we can simply set the values directly on the object, without the overhead associated with reflection or JNI.
Another piece of overhead is the Animator itself. Although all animations share a single timing mechanism, and thus don’t multiply the overhead of processing timing events, they are separate objects that perform the same tasks for each of their properties. These tasks could be combined if we know ahead of time that we’re running a single animation on several properties. One way to do this in the existing system is to use PropertyValuesHolder. This class allows you to have a single Animator object that animates several properties together and saves on much of the per-Animator overhead. But this approach can lead to more code, complicating what is essentially a simple operation. The new approach allows us to combine several properties under one animation in a much simpler way to write and read.
Finally, each of these properties on View performs several operations to ensure proper invalidation of the object and its parent. For example, translating a View in x invalidates the position that it used to occupy and the position that it now occupies, to ensure that its parent redraws the areas appropriately. Similarly, translating in y invalidates the before and after positions of the view. If these properties are both being animated in parallel, there is duplication of effort since these invalidations could be combined if we had knowledge of the multiple properties being animated. ViewPropertyAnimator takes care of this.
x
y
ViewPropertyAnimator
ViewPropertyAnimator provides a simple way to animate several properties in parallel, using a single Animator internally. And as it calculates animated values for the properties, it sets them directly on the target View and invalidates that object appropriately, in a much more efficient way than a normal ObjectAnimator could.
Enough chatter: let’s see some code. For the fading-out view example we saw before, you would do the following with ViewPropertyAnimator:
myView.animate().alpha(0);
Nice. It’s short and it’s very readable. And it’s also easy to combine with other property animations. For example, we could move our view in x and y to (500, 500) as follows:
(500, 500)
myView.animate().x(500).y(500);
There are a couple of things worth noting about these commands:
animate(): The magic of the system begins with a call to the new method animate() on the View object. This returns an instance of ViewPropertyAnimator, on which other methods are called which set the animation properties.
animate()
Auto-start: Note that we didn’t actually start() the animations. In this new API, starting the animations is implicit. As soon as you’re done declaring them, they will all begin. Together. One subtle detail here is that they will actually wait until the next update from the UI toolkit event queue to start; this is the mechanism by which ViewPropertyAnimator collects all declared animations together. As long as you keep declaring animations, it will keep adding them to the list of animations to start on the next frame. As soon as you finish and then relinquish control of the UI thread, the event queue mechanism kicks in and the animations begin.
Fluent: ViewPropertyAnimator has a Fluent interface, which allows you to chain method calls together in a very natural way and issue a multi-property animation command as a single line of code. So all of the calls such as x() and y() return the ViewPropertyAnimator instance, on which you can chain other method calls.
x()
y()
You can see from this example that the code is much simpler and more readable. But where do the performance improvements of ViewPropertyAnimator come in?
One of the performance wins of this new approach exists even in this simple example of animating the alpha property. ViewPropertyAnimator uses no reflection or JNI techniques; for example, the alpha() method in the example operates directly on the underlying "alpha" field of a View, once per animation frame.
The other performance wins of ViewPropertyAnimator come in the ability to combine multiple animations. Let’s take a look at another example for this.
When you move a view on the screen, you might animate both the x and y position of the object. For example, this animation moves myView to x/y values of 50 and 100:
myView
ObjectAnimator animX = ObjectAnimator.ofFloat(myView, "x", 50f); ObjectAnimator animY = ObjectAnimator.ofFloat(myView, "y", 100f); AnimatorSet animSetXY = new AnimatorSet(); animSetXY.playTogether(animX, animY); animSetXY.start();
This code creates two separate animations and plays them together in an AnimatorSet. This means that there is the processing overhead of setting up the AnimatorSet and running two Animators in parallel to animate these x/y properties. There is an alternative approach using PropertyValuesHolder that you can use to combine multiple properties inside of one single Animator:
AnimatorSet
PropertyValuesHolder
PropertyValuesHolder pvhX = PropertyValuesHolder.ofFloat("x", 50f); PropertyValuesHolder pvhY = PropertyValuesHolder.ofFloat("y", 100f); ObjectAnimator.ofPropertyValuesHolder(myView, pvhX, pvyY).start();
This approach avoids the multiple-Animator overhead, and is the right way to do this prior to ViewPropertyAnimator. And the code isn’t too bad. But using ViewPropertyAnimator, it all gets easier:
myView.animate().x(50f).y(100f);
The code, once again, is simpler and more readable. And it has the same single-Animator advantage of the PropertyValuesHolder approach above, since ViewPropertyAnimator runs one single Animator internally to animate all of the properties specified.
But there’s one other benefit of the ViewPropertyAnimator example above that’s not apparent from the code: it saves effort internally as it sets each of these properties. Normally, when the setX() and setY() functions are called on View, there is a certain amount of calculation and invalidation that occurs to ensure that the view hierarchy will redraw the correct region affected by the view that moved. ViewPropertyAnimator performs this calculation once per animation frame, instead of once per property. It sets the underlying x/y properties of View directly and performs the invalidation calculations once for x/y (and any other properties being animated) together, avoiding the per-property overhead necessitated by the ObjectAnimator property approach.
setX()
setY()
I finished this article, looked at it ... and was bored. Because, frankly, talking about visual effects really begs having some things to look at. The tricky thing is that screenshots don’t really work when you’re talking about animation. (“In this image, you see that the button is moving. Well, not actually moving, but it was when I captured the screenshot. Really.”) So I captured a video of a small demo application that I wrote, and will through the code for the demo here.
Here’s the video. Be sure to turn on your speakers before you start it. The audio is really the best part.
In the video, the buttons on the upper left (“Fade In”, “Fade Out”, etc.) are clicked one after the other, and you can see the effect that those button clicks have on the button at the bottom (“Animating Button”). All of those animations happen thanks to the ViewPropertyAnimator API (of course). I’ll walk through the code for each of the individual animations below.
When the activity first starts, the animations are set up to use a longer duration than the default. This is because I wanted the animations to last long enough in the video for you to see. Changing the default duration for the animatingButton object is a one-line operation to retrieve the ViewPropertyAnimator for the button and set its duration:
animatingButton
animatingButton.animate().setDuration(2000);
The rest of the code is just a series of OnClickListenerobjects set up on each of the buttons to trigger its specific animation. I’ll put the complete listener in for the first animation below, but for the rest of them I’ll just put the inner code instead of the listener boilerplate.
OnClickListener
The first animation in the video happens when the Fade Out button is clicked, which causes Animating Button to (you guessed it) fade out. Here’s the listener for the fadeOut button which performs this action:
fadeOut
fadeOut.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { animatingButton.animate().alpha(0); } });
You can see, in this code, that we simply tell the object to animate to an alpha of 0. It starts from whatever the current alpha value is.
The next button performs a Fade In action, returning the button to an alpha value of 1 (fully opaque):
animatingButton.animate().alpha(1);
The Move Over and Move Back buttons perform animations on two properties in parallel: x and y. This is done by chaining calls to those property methods in the animator call. For the Move Over button, we have the following:
int xValue = container.getWidth() - animatingButton.getWidth(); int yValue = container.getHeight() - animatingButton.getHeight(); animatingButton.animate().x(xValue).y(yValue);
And for the Move Back case (where we just want to return the button to its original place at (0, 0) in its container), we have this code:
animatingButton.animate().x(0).y(0);
One nuance to notice from the video is that, after the Move Over and Move Back animations were run, I then ran them again, clicking the Move Back animation while the Move Over animation was still executing. The second animation on the same properties (x and y) caused the first animation to cancel and the second animation to start from that point. This is an intentional part of the functionality of ViewPropertyAnimator. It takes your command to animate a property and, if necessary, cancels any ongoing animation on that property before starting the new animation.
Finally, we have the 3D rotation effect, where the button spins twice around the Y (vertical) axis. This is obviously a more complicated action and takes a great deal more code than the other animations (or not):
animatingButton.animate().rotationYBy(720);
One important thing to notice in the rotation animations in the video is that they happen in parallel with part of the Move animations. That is, I clicked on the Move Over button, then the Rotate button. This caused the movement to stat, and then the Rotation to start while it was moving. Since each animation lasted for two seconds, the rotation animation finished after the movement animation was completed. Same thing on the return trip - the button was still spinning after it settled into place at (0, 0). This shows how independent animations (animations that are not grouped together on the animator at the same time) create a completely separate ObjectAnimator internally, allowing the animations to happen independently and in parallel.
(0, 0)
Play with the demo some more, check out the code, and groove to the awesome soundtrack for 16.75. And if you want the code for this incredibly complex application (which really is nothing more than five OnClick listeners wrapping the animator code above), you can download it from here.
OnClick
For the complete story on ViewPropertyAnimator, you might want to see the SDK documentation. First, there’s the animate() method in View. Second, there’s the ViewPropertyAnimator class itself. I’ve covered the basic functionality of that class in this article, but there are a few more methods in there, mostly around the various properties of View that it animates. Thirdly, there’s ... no, that’s it. Just the method in View and the ViewPropertyAnimator class itself.
ViewPropertyAnimator is not meant to be a replacement for the property animation APIs added in 3.0. Heck, we just added them! In fact, the animation capabilities added in 3.0 provide important plumbing for ViewPropertyAnimator as well as other animation capabilities in the system overall. And the capabilities of ObjectAnimator provide a very flexible and easy to use facility for animating, well, just about anything! But if you want to easily animate one of the standard properties on View and the more limited capabilities of the ViewPropertyAnimator API suit your needs, then it is worth considering.
Note: I don’t want to get you too worried about the overhead of ObjectAnimator; the overhead of reflection, JNI, or any of the rest of the animator process is quite small compared to what else is going on in your program. it’s just that the efficiencies of ViewPropertyAnimator offer some advantages when you are doing lots of View property animation in particular. But to me, the best part about the new API is the code that you write. It’s the best kind of API: concise and readable. Hopefully you agree and will start using ViewPropertyAnimator for your view property animation needs.
