Good morning. How are you guys doing? Good? Okay, so this is HTML5 game development. What I'm going to do in this talk is I'm going to show you how to do 2D games using mostly the Canvas API. I'll also talk about the animation timing specification, which defines a method called request animation frame

that you should be using to correctly do animations in the browser. We'll talk about that. We'll also look at sound. See the sound API, which is pretty bad in HTML5, but we'll see the good and the bad. We'll also talk about local storage and see how you can save user information on a local disk,

such as preferences or high scores and things like that. Okay, so here's the outline for this talk. We're going to start off with a short introduction and a short demo to begin with, and then we're going to talk about animation. Again, we're going to look at the animation timing specification and a method called request animation frame.

We'll see how to repair the background when you do animations. That's really the challenging part, is updating the background as you damage it when you move things around. We'll see three different ways to do that, and we'll compare performance for each of those methods of erasing and maintaining the background.

We'll also talk about how you can calculate frames per second, how you can implement time-based motion. We'll define that first and then show you how to implement that. We'll see how to do scrolling backgrounds. We'll see how to do something called parallax. And then we'll talk about sprites. Sprites are just small animated objects that do things in games,

so we'll see how to implement those. We'll see how to paint sprites, we'll see how to animate sprites, and we'll see how to make them interact. Then we'll talk about sound. We'll look at the sound API. Just a few slides to show you how to do a multi-channel sound so you can play multiple sounds at once in a game.

And then we'll wind up talking about collision detection. We're going to look at some simple collision detection with bounding boxes, and we're also going to look at something called the separating axis theorem, which is kind of the gold standard for collision detection, both in 2D and 3D.

So that's where we're headed in this talk. This talk is all HTML5. The slides are HTML5. All my demos, for the most part, are in the slides themselves. What you're looking at is just one big HTML5 application, and I downloaded that from HTML5rocks.com.

Google put this together. You can download the slideshow. And then I have modified it quite a bit since I downloaded it probably about a year ago, I suppose, now. In fact, here is the previous slide. This is what it looks like in HTML.

So I have a class for slides, and thank you, Google, with the H1 and so forth. Whoops, that must be the wrong slide. Never mind. OK, well, there's an example slide that you could create with HTML5rocks. Some of the game images that I'm going to use are courtesy of Replica Island.

Anybody ever play Replica Island on Android? No? I should get something straight right off the bat here. How many people refuse to raise your hands at a conference? OK. So I'm going to use some images from Replica Island, which is an open source, very popular game on Android.

And the reason I'm up here today is because I just wrote a book. It just came out about three weeks ago on HTML5 Canvas. How many people were at Rob Ashton's WebGL talk? Yeah? OK, Canvas does not suck. OK, and I'm going to show you that today.

So all the examples that you're going to see today are from the book. The book also has a corresponding website, corehtml5canvas.com. You can go there, you can run some featured examples, which is what I'm showing up here. You can download all the code for the book without buying the book. Of course, I don't recommend that you do that, but you can if you want.

You can also download some free chapters from the book. I have three chapters up there that you can download, hopefully to entice you to spend 50 bucks. OK, so let's start off with a demo.

So I have a pinball game, and unbeknownst to me, pinball games are very difficult to implement. There's a lot of stuff going on here, lots of collision detection. Up at the top, I have a curved surface, which is kind of tough for collision detection. In fact, we'll talk about that at the end of this talk.

The ball can potentially move at pretty high speeds. The flippers move at high speeds. Their angular motion is pretty great. So collision detection is pretty tricky with this. Let's see what it looks like. By the way, in the last talk, this game just basically...

Oh, maybe it doesn't play itself. It pretty much plays itself. Every once in a while, I have to activate the flippers. So I have some sound for the flippers. You didn't see that, right?

All right, one more time. It's so much fun. So basically what you're looking at is an image in the canvas background. And I just do that with CSS so I don't have to redraw the pinball image every time. And then what I did was I just took GIMP, and I cut out these pieces and made them brighter.

And then when the ball collides, when I detect that collision detection, that's how I do the bumpers. I momentarily display the brighter version of the bumper. And I have bumpers all over the place. And you can see these are fairly realistic. If it would go through one of those, you could...

Anyway, that's pinball. Okay, so canvas does not suck, right? By the way, canvas did suck. I have to give Rob some credit. And canvas has kind of gotten a bad rap for being slow, which sometimes is true.

If you run this pinball game on iPhone, it's not going to do very well. However, canvas recently has gotten quite a boost. Chrome 18 now has hardware-accelerated canvas. And that's what I'm running here, Chrome 18, or actually Chrome 21.

And a little bit later, I'll show you the difference between hardware-accelerated canvas and non-hardware-accelerated canvas. Okay, so let's start off talking about animation. So here's some basic animation. I'm going to click this button. By the way, before we get too far here, Chrome has a bug that when you start an animation for the first half second,

it kind of jumps around a little bit. And you may notice that if I pause this. Let's just reload this. Yeah, maybe not. Okay. So that's basic animation. All I'm doing is drawing three disks and moving them around the screen.

So, pretty easy to do. So how do I do that? Well, this is how I do not do it. This function's okay up here. This is my animate function. And what I'm going to do is I'm going to clear the canvas. So this is the canvas context. I'm going to clear a rectangle that's the whole canvas. And then I'm going to update whatever that is.

That will update the positions of my three disks. And then I draw the background, and then I draw those three disks. So every time through the animation loop, I erase everything, draw the background, draw the disk. Erase everything, draw the background, draw the disk, over and over and over. The over and over and over part can't be this, right?

Because that's going to lock up the browser. JavaScript runs on a single thread. And if you do something like while true, that'll lock up the browser. And it'll in fact lock up your animation at the same time. So you can't do that. So how do you do animation? Well here's one way to do animation.

So I have the same animate function up here. And I'm going to use setInterval. Anybody know how many frames per second that is? 60. So what I'm going to do is I'm telling the browser every 60, every 1000 divided by 60 seconds,

I want you to call my animate function. So the browser calls that animate function over and over again. You don't want to do animation this way and I'll expand on that in just a minute. One problem with this approach is you are setting the frame rate. You are telling the browser how many frames per second you want.

And that's not a good idea. Here's why. setInterval and setTimeout are general purpose functions, JavaScript functions. They were never meant to support animation. In fact, they're not very precise.

You can tell setTimeout, hey, I want you to call this function in two milliseconds or five milliseconds. And the browser could just say, I'm going to wait 10 or 15. In fact, Firefox will do that. Browsers have a lot of leeway with setInterval and setTimeout

so that they can conserve resources if they need to. So what you're doing with setInterval and setTimeout is you're not telling the browser, hey, call this function in 15 milliseconds. What you're doing is telling the browser, hey, call this function in about 15 milliseconds. And that about is a problem in a game.

You don't want to have a 15 millisecond pause because you only have, as we'll see later, 16.7 milliseconds to do your thing. Okay, the other thing is, as I said before, we're telling the browser what we want to run at, the frames per second, and we don't really know, right? I'm guessing 60 frames per second because somebody else told me that most monitors refresh at that rate,

and I know I want to match the monitor refresh, but I'm just guessing. I have no idea. The browser, presumably, unless it's Firefox, knows better than I do. Sorry. Okay, so the browser knows the best time to draw the next frame, so why not let the browser pick that frame rate for me?

And that's exactly what requestAnimationFrame does. So, like all things in HTML5, this started as a vendor-specific extension, initially by Gecko or Mozilla.

So Mozilla came up with this method, mozeRequestAnimationFrame. And basically, what I'm doing with this function is I'm telling the window, hey, call this function when you're ready to draw the next animation frame. Okay? I'm not telling it draw at 60 frames per second or 100 frames per second.

I'm just saying, hey, when you're ready, let me know, and call my function, and then I'll draw. Very nice. So now the browser is picking the time to call my function, and it's picking the frame rate for me. After Mozilla, WebKit followed suit with pretty much the same implementation.

There are slight variations between these vendor-specific implementations, but they're all pretty much the same. And then we also have an IE version of this. Of course, you know what happens when you have three different versions of a new function, right? What happens? It gets standardized, right? And we have that too. So now we have window requestAnimationFrame, and this is standard, but not many browsers support it yet.

So what do you do? Anybody know? What do we use when we want to use future features, but the browser doesn't support it? We use a what? Anybody know? Starts with a P? No?

No? Okay. Polyfill? Anybody know that? Why didn't you say that? Okay. So we're going to write a polyfill. So a polyfill is a polymorphic backfill. Okay, I'll let you think about that on your own time. But what this is is a function that returns a function that's either one of these functions, if they're there, or it's my function.

So my function is a fallback, and my function uses setTimeout, and I'm doing the best I can here to run at 60 frames per second. But what I really want to do is let the browser take care of that for me.

Using the polyfill is pretty easy, just like that. Now, I've called the polyfill that I wrote requestNextAnimationFrame to distinguish it from requestAnimationFrame. Otherwise, I clobber the original method, and I'm in big trouble. So my version is called requestNextAnimationFrame.

Okay, is that okay? All right, let's talk about frames per second and time-based motion. So if you're going to write a game, you have to monitor your frames per second. You have to know how many frames per second you're running at.

In fact, it's a good idea if your frames per second drop under a certain threshold that you tell the user, hey, this game is running slowly, things might not work exactly right. And let's see how to do that stuff. First of all, let's see how to calculate frames per second. So here I have that same animation, except now up at the top, I'm showing the frames per second.

I'm not using requestAnimationFrame here. I'm using setInterval with an interval of zero, which of course is impossible, right? But the browser is doing the best it can, so my frame rate is wildly jumping all over the place, which is not a good idea. So here's how you calculate frames per second.

This should be easy to figure out, right? What you do is every time your animate's called, you keep track of the last time it was called, subtract off that from the current time, and now you know how much time it took for your last frame to draw. That's your frame rate. If you want, you can average it out over time. I'm just taking the last frame rate.

That's my frame rate. So I have one frame for so many milliseconds. I know this is hard math, but to get to frames per second from frames per milliseconds, you multiply by this, okay? Okay. And here's how I do that.

This is the code from the demo that I just showed you. Here's my animate function. And what I'm going to do is fill some text in the canvas, and I'm going to calculate my FPS. And here is that. Here is that. Okay. Here is that function up there.

What I'm doing is, notice the FPS is 1,000 divided by now minus last time. So that's 1,000 over the milliseconds that it took for the last frame to draw, right? Which is right here. 1,000 over the number of milliseconds gives me frames per second.

Okay. Here's another thing. Games have to run at a constant speed. Your game speed can't be dependent on your frame rate, especially for multiplayer games. You don't want a person with a more powerful computer to have the game run faster on his machine than somebody with a Windows box.

Sorry. So what we want to do is we want to move at a constant rate. Okay. We don't want that rate to fluctuate or be influenced by frames per second. At this point, I'm going to have to go to Firefox.

And I'll tell you why I'm going to Firefox here in a minute. Let's open this. Okay. So here's the same presentation. But now we're running in Firefox.

Now, what I have is I have two identical applications. Pretty simple. I just have a bunch of these disks and I just let them loose and they bounce off the walls. Okay. But I have two identical applications in the same slide here.

Tell me what's going to happen to the frame rate when I click that second button. It's going to go what? Up? Who said up? There's always one in the crowd. It's going to go down, right? I have two applications running at the same time.

So I go down basically from 60 to 40. Now, what I want you to do is watch these disks. Now they're moving fast. Now they're moving slow. Fast, slow. Fast, slow. You see that? It's kind of like being at the eye doctor, right?

Okay. You see that? Do you see the effect? Okay. What I'm going to do instead is I'm going to use time-based motion and I'm going to tell these disks to move at a constant rate no matter what the frames per second is. So now I start up here like before I have 60 frames per second.

I click the second one like before the rate goes down, but notice that the disks did not slow down this time. Watch. It's best just to watch the top part. I have the same animation speed regardless of the frame rate.

Okay? Okay. So that's Firefox. And I told you I'd tell you why I used Firefox. And here's why. Because Canvas in Chrome is now hardware accelerated. So I ran this demo before I left to come over here.

And this is what happened. Okay. Look at that. No drop at all. It just hums right along. So that's pretty impressive. And now, of course, nothing speeds up or slows down regardless of whether I'm time-based or not

because the frame rate never changes. So pretty cool. Okay. So for time-based motion, what we want to do is specify the time in pixels per second. In other words, I want to tell those disks, I want you to move at 10 pixels per second or 50 pixels per second

or whatever I decide. And then I want to calculate how many pixels to move each disk for each frame of the animation. Does that make sense? So what I want to do is calculate pixels per frame given pixels per second, which is how fast the disk moves,

and the frame rate. And so here is the equation to do that. Okay? Of course, the one on the bottom is the same as the one on the top except I switched things around and inverted one of them. Right? If you're familiar with math, you know how that works. But anyway, pixels per frame is pixels per second, which is the speed of the disk,

divided by frames per second, which is the frame rate. So to do this, this is how I did it. I calculated how much to move in the X and Y directions by dividing pixels per second by frames per second, which is what I'm doing right here. And that gives me how many pixels to move each disk

for each frame of the animation. Does that make sense? Yeah? No? Alright, so let's talk about something else. Let's talk about scrolling backgrounds. So a lot of times in games you want to do stuff like this. You want to have some background scrolling by,

maybe you're going to write a side scroller. So how do you do that in Canvas? Well, what I do in this demo is I translate the Canvas coordinate system. This is the original Canvas, or the Canvas that we're looking at.

And what I do is I draw this cloud in the visible Canvas, and then I also draw it outside of the Canvas. Notice where I'm drawing X, Y. I'm drawing this cloud at the width of the Canvas, which means the cloud is off the edge of the Canvas. You can't see it.

And what I'm doing, then I start out with a visible cloud and an invisible cloud, and I translate the coordinate system so that I move this way. When I get all the way to the edge, I go back over here and start over again. So it appears that that thing is just constantly scrolling by.

So, like I said, I draw the cloud twice. I draw it in the visible part of the context, and I draw it outside of the context, the Canvas. And here's how I scroll the background. Ultimately, I call the translate method of the Canvas context,

and I tell it how much I want to translate in the X direction. So I'm just constantly, as I'm animating, translating that context and redrawing the two clouds at the same spot every time. So it looks like the clouds are moving because I'm translating the context. Now, here's one trick that you have to do.

The image, the right side of the image and the left side of the image are identical. These strips of pixels right along this row are identical. Otherwise, when I get to the edge as I'm translating,

and I move back to the beginning, I'm going to have a discontinuity. As long as the drawing has the same pixels on the two edges, then I can keep scrolling, and it just looks like it's that same cloud going by and by and by.

So we know how to scroll the background. Let's look at parallax. So this simulates 3D. And this is parallax, and parallax is pretty simple. Things farther away appear to move more slowly. So you can see that the clouds in the back,

the trees are moving much faster than the clouds. The trees in the front, which are the big bushy trees, are moving faster than the tall skinny trees behind them. So what I have is I have four layers that look like this.

And here's what I do with them. For each of these pieces, I translate the context. First, I save the context, translate it, draw the piece,

and then restore the context. And I do that over and over again for each of the four pieces that I have here. These values are all different. This is small, and this is big, right? And these values get bigger as you go down, because the grass in the front moves much faster than the clouds in the back.

Okay. So this is how much time you have to draw a frame if you're running at 60 frames per second. That's not much time to do your business.

Really, the hard thing, the challenging thing about animation is taking care of the background. It's easy to animate if you don't have to worry about repairing the damaged background, right? You just draw like crazy and get a big mess. The challenging part is maintaining the background.

We're going to look at three different ways to do that. One is with clipping. And with clipping, what I'm doing is I'm erasing and drawing the entire background for every disk, but I'm clipping it to the arc, which is the circle that describes the disk. So I draw the entire background, but the browser restricts it to where the disk is,

which takes the background and puts it in wherever the disk was. Does that make sense? Another way to do it is to copy from an off-screen canvas. Write your background into an off-screen canvas, and then copy from that off-screen canvas on-screen

with the drawImage method of the canvas context. A third way, which we've already seen, is to just redraw everything for every frame. So what is the best approach? Redraw everything, use clipping, or copy from an off-screen buffer. Let's try and see.

So here I have three of these little rings. Let's do 100. And notice that my frames per second is basically 60. That's good. So let's try, go back to three. I get 60 with clipping, I get 60.

Notice clipping is different, right? What I'm doing is, as I'm moving those, I'm replacing or erasing those disks by filling in the background. Normally I would draw the whole background and draw this on top of it, but I wanted you to see the effect of just erasing behind the rings as they move.

So I get 60 frames per second there. If I copy from an off-screen buffer, I also get 60 frames per second. When things get interesting is when I go to 100. So for 100, if I redraw everything, if I redraw everything, I get 60 frames per second.

If I do clipping, I drop all the way down to 20. And if I copy from an off-screen background, I'm somewhere in between, around 40.

Okay, so how do you figure all this stuff out? There's two really useful tools in WebKit. One is profiles. You can profile your code and see how much time your program is idling at the top. You can also do timelines, which tell you all the events and how much time they took, which is very useful.

For three disks, this is no clipping and this is clipping. For three disks, my idle time is almost identical. But for nine disks, notice now when there's no clipping, I'm idling almost 70% of the time, but when I'm clipping, I've lost 10%.

So the more things you have, the less efficient clipping becomes. You can run the profiler in WebKit by clicking on a button, or you can run it programmatically. Not a lot of people know this. You can call the profile method and the profile end method

to run profiles at specific spots in your code. Here's a little animation lab that I put together. This is kind of interesting because when Canvas first came out, well, not when it first came out, but when I first started using it about two years ago, these things made a huge difference.

I have a background. If I take that background image out, watch the frames per second, by the way, up here. If I take the shadow, there's a shadow behind this canvas. If I take that out, I have rounded corners if I make them square, and I'm drawing a grid.

You can barely see it under there, but it's there if I take that out. Notice my frames per second never changed. There used to be huge changes. In frames per second. When you had shadows or rounded corners. But evidently, that's all been fixed now in Chrome. So what that means is some of my recommendations here are already out of date.

Here are some best practices for animations. Use profiles and timelines. Clip when you're animating a small number of objects. Don't double buffer. Canvas is double buffered in every browser on the planet. If you do it, it's just extra work, and it'll just slow you down.

You should avoid CSS shadows and rounded corners and canvas shadows. As you saw in the last demo, that's probably not so important anymore. But this is. And of course, you should always do that and that. Okay?

Okay, let's talk about sprites. So here's sprites. I have a scrolling background. You know how I'm doing that now. And I have a sprite, my Android guy who just sits there doing nothing. But he's a sprite. And I have a toast. A toast is something you present to the user that in more mundane terms could be described as a dialog box.

So here's my sprite. I create sprites. And by the way, sprites are not part of the Canvas API. You can implement your own sprites, which is what I did here. My sprite constructor takes a name, an image, and a set of behaviors. We'll talk about that in a minute.

And here's how I draw my sprite. I just use the drawImage method to draw the sprite's image. Now my sprite has some behavior. He bounces off the walls and the floor and the ceiling. And sprites have an update method.

So sprites maintain an array of objects called behaviors. Behaviors only have one requirement, that they have a method named execute that takes some arguments. One is the sprite. And what I do is the update method of the sprite iterates over those behaviors and calls each behavior's execute method,

passing it the game, the sprite, and the width and the height of the canvas. So here are a couple of sprites, an Android sprite and a bat sprite. And here they have some behaviors. And again, these are all just objects with a single execute method that somehow manipulates the sprite for each animation frame.

Here's the fly behavior. Here's a bounce behavior that bounces the sprite off the walls. Okay, so what you can do is create these behavior objects,

create an array of these behavior objects and attach it to a sprite. And now the sprite can do all the things in the behavior. So here's a sprite animation. Notice when the Android guy gets to the bottom, he does, I don't know what you call that, but he kind of blows up down at the bottom.

So what I'm doing is when I detect that I get below a certain point, I run that sprite through a series of animations with a sprite animation object. And so here's my sprite animation object. What it does is maintain an array of images, and it simply cycles through those images.

So when I start an animation with the sprite animation object, as long as the animation is running, I just increment that image's array. So I set the sprite's image to the image in my array that corresponds with the current index.

If my animation is finished, then I quit animating, I restore the sprite's original image, if that's what the sprite wanted me to do, and then I stop this interval. Here I'm using setInterval instead of requestAnimationFrame because I don't really need this to be millisecond,

down to the millisecond precise. Okay, so sound. Sound in HTML5 is notoriously buggy, but it does work some of the time. What I'm doing here is I'm going to show you

how to use multiple soundtracks so you can play multiple sounds at one time. What I have is an array of audio objects. I just create this many of them and push them onto this array. So I have an array of audio objects, and when you ask me to get the available soundtrack, I iterate over those soundtracks,

looking for the first one that's not playing and send it back to you. If they're all playing, if I'm playing, in this case, 10 sounds at once, you're out of luck and you just have to wait. And finally, here's how I play a sound. I get the available soundtrack, I load the sound, and I play it, and that's it.

Okay, so let's talk about high scores local storage. So here's how you set high scores, how you store them in local storage. And so what I do is I create a key with the name of the game and something,

I forget what it is, underscore high scores or something. Using that key, I pull the high scores out of local storage, I stick in the new high score, and then I stick them back into local storage. Here's how I get the high scores from the local storage.

I'm gonna return the high scores if there are any, and if there are not, I'm just gonna return an empty string.

God, Canvas sucks. Excuse me, let me, you guys have never seen that, have you?

Okay, let's talk about collision detection. So here's a little game that I did, and what I'm gonna do is try and shoot the ball in the bucket. If you go outside the bounds of the canvas

and into the bucket, then you get a three-pointer. Notice I'm keeping track of the score. I don't really have much time. I have to do this, I'm sorry.

Ha, you started. Three-pointer. Okay, sorry. So here's how I did the collision detection. This is really simple. In fact, this is the simplest possible collision detection anyone could ever do. Detect whether two circles are colliding. All you do is look at their radii, and if the centers are closer together

than the combined radii, you have a collision. Does that make sense? That's it, easy stuff. But that's not gonna suffice for everything. This is the separating-axis theorem, and this is what I'm using in the pinball game

for the most part to do my collision detection. The separating-axis theorem is really the gold standard for collision detection in both 2D and 3D, and here's how it works. Mathematically, what we're gonna do is shine a light on two polygons.

On the left, I'm shining the light from the right. On the right, I'm shining it from the bottom. And I'm gonna look at the wall behind the polygons. If there is separation between the shadows, I don't have a collision, agreed? Okay. Now, mathematically, lights and walls

are projections, basically. So what I'm gonna do is I'm gonna project these polygons onto the X and Y axis and look to see if there's any separation between those projections.

If there are, if there is a separation, then I know I don't have a collision. If there is not a separation, I know I do have a collision. Here's how you get a projection. You take a polygon face, you calculate a normal vector to that,

and that is your axis. This can be anywhere in space. It doesn't matter. You could move it up here, but the projection is still gonna be at the same location. What you need to do with SAT is you need to check all the axes of the two polygons that are potentially colliding. And here's how you get all of the axes.

And here's how you test them. So here I have separation between these two projections and that one too, I guess. So I know I don't have a collision here. So what you do with a SAT is you get all the axes for each polygon.

You iterate over those axes, project each polygon onto those axes and see if there's any separation. Once you find separation, you're done. If you find it on the first axis, then you're done. Otherwise you have to iterate through all the axes to make sure there's no collision.

Before I let you go, let me just show you a redux of this pinball game. So let's start this again. The pinball game uses the SAT for almost all of its collision detection except for the flippers. The flippers are moving at high velocity.

It may not look like it, but believe me, they are. And the ball can get moving pretty fast too. So there I augment the SAT with something known as ray casting, which I didn't cover in this talk. But you might notice I have this checkbox up here that says polygons. If I click this checkbox, now it's just gonna show me the collision detection polygons

so I can see exactly how the collision detection is happening, which is really useful when you're debugging. It was really useful to me when I was debugging the flippers. So three days took me to get those flippers to work. But anyway, that's another story. So you can see up at the top, I have triangles forming that rounded dome.

The dots that you see all over the place are the approximate centroid of each polygon. And so what I do is I just shoot the ball, which I represent as a square polygon. And now I'm using the SAT to detect collisions between these polygons.

And of course, if you click on that, then you get the graphics back, which looks a lot nicer. Okay, so that's Canvas and that's HTML5 game programming with Canvas. Thank you guys very much for coming.