Technical Post-Mortem of Runestone Wisp - An HTML5 game

Runestone Wisp

Writing a Game in HTML5

HTML5 is new and exciting and even unfinished, (I'll get to that,) but I've had a lot of fun using it and I want to tell other people about the fun I've had.

First, have a look at the end result. Follow that link up above. It combines logic, puzzle solving and design skills with a bit of magic. You control the wisp, a docile yellow ball of fluff, using magical runestones. The runes combine to form networks that guide the wisp and react to the world around him. I like to call it a Sandbox Design Puzzle Game.

This project really started with Tile Factory, by Jonathon Duerig. I loved that game, but I wanted to play with some new types of memory, which Jonathon assured me would not be done in TF.

So, I set out to make my own game... It took me about a month to plan a new, interesting game I'd want to play. I was committed to the engineering sandbox design genre which Zachtronics has done so well, but I was hoping to broaden the audience. I chose a fantasy setting with magic themes and a charismatic protagonist. That's why the player controls a Will-o-Wisp, instead of something standard, like a zombie or robot, or something creepy like a brain-washed human. (The spells you build in Runestone Wisp are light-hearted, not J.K.Rowling's "Imperio" spell.)

Then I sat down to code it in Flash. After about 3 hours, I gave up. There are probably plenty of Flash developers who can write Flash applications without the Adobe toolset, but I'm not one of them. And I just can't afford that kind of investment for a game that probably won't make any money.

So, I started again, in JavaScript and HTML. I'm very experienced with JS, but not HTML5. Here's what I learned.

Loading and Initialization

This might be the biggest pure-JS project I've done, over 200k of code, so I had to break it up into manageable parts. I wrote the loading system for Runestone Wisp without reference to any guides on modular JS loading, and then read about them later. It turns out, my implementation is relatively close to a normal JS modular loading system, with a couple important twists.

A. My only contribution to the global namespace is the RSW object, so instead of a "module" method in the global namespace, I use rsw.initModule.

B. I dislike wrapping parentheses around anonymous methods. So I don't pass the initialization function into initModule. Instead, my syntax looks like this:

rsw.initModule('Rune','notice','runes','grid', 'menu').init =

function(notice, runes, grid, menu) { ... }

The parameters to initModule are a list of names for modules. In the end, these will all be exposed as properties of the RSW object. The object returned by initModule is an entry in the loading plan for the game, and the init attribute is used to initialize the first module listed. To me, this feels cleaner than the traditional module pattern for JS which makes the function a parameter to a module method.

C. Circular dependencies are a problem for most modular loading paradigms. But I've been

spoiled by Java to expect impossible load orders to be handled gracefully. Fortunately, I found a way to make it work. This is a big departure from the traditional JavaScript loader pattern. Each module springs magically into existence as soon as it's referenced. (It's an empty collection, but that's enough to make closures work.)

Inside initModule, I treat all the module names equally, and if one does not exist, I create it as an empty object literal. Once all the .js files have been parsed, the page finishes loading and I trigger the actual the actual initialization of each module, in three stages:

1. Run the init method, pass it all those empty objects. It can add properties as needed, and then return a collection of properties and methods.

2. Copy that collection onto the previously empty object being initialized.

3. If the list of properties includes a second init method, run it. This time it gets to use all those properties and methods that were returned from the first init.

As a result, I'm free to use circular dependencies with no problems at all.

It's worth mentioning that not all the "modules" can be initialized as empty hashes. rsw.Rune and rsw.State are constructors. Since they're defined directly in the RSW object, initModule will find they already exist the first time someone asks for them and use the existing function instead of an anonymous, empty object. Even though they aren't created by initModule, they're initialized exactly the same way other modules are.

HTML5 Canvas, Canvases and more Canvases

I've seen a lot of canvas demonstrations and even some canvas games, but so far they've all used one or possibly two HTML5 canvas objects. I chose to use a lot of different canvases so I could easily update individual visual elements while leaving other, overlapping elements untouched. The browsers do a brilliantly fast job of compositing canvases together, saving me from doing complex refreshes in JS. My use of canvases may be somewhat egregious, but they all have different lifespans and dimensions that fit how they're used.

1. The runes in the workspace live in lowest z-index canvas. I have a background pattern (the mostly-transparent gray grid) that's used to erase runes when I need to move or remove them. But I can usually avoid that by not drawing anything else to that canvas.

2. There is a second canvas over the workspace which shows all the threads between the runes, the highlights around selected runes and runes that are attached to the mouse pointer being moved. It's highly dependent on mouse movement events, which makes performance particularly important. Having a separate canvas makes this easy without redrawing the entire set of runes.

Both of those canvases are set within a scrolling frame so that everyone has the same workspace size regardless of what screen resolution they're using.

3. The map is drawn in its own small canvas. It gets redrawn completely each time the world state changes.

4. The tutorial has its own canvas covering the entire screen. This allows me to draw arrows pointing to any part of the UI, over the top of the map for instance. Mouse events from the tutorial are passed through to the workspace beneath. Quite a few things which need mouse interaction during the tutorial have higher z-indexes than the tutorial, like buttons and menus.

5. When the mouse pointer moves over the map, there's a second, mostly invisible canvas which contains highlights and the hexagonal cursor for the map. Performance is critically important for mouse move event handlers, so I minimize the amount of things I have to erase and redraw on this canvas. The underlying map doesn't change. This layer's z-index puts it above the tutorial in order to track mouse movements over the map even when the tutorial is active. (I also abused it to display the shadow on the map.)

6. The rune palette where you can grab runes and drag them into play was a late addition, using its own canvas. Its z-index puts it under the radial menu, but just above the tutorial, so it's available during the tutorial if you choose to use it.

7. The radial menu has its own full-screen canvas so it can be quickly erased and redrawn. It's not constrained to the workspace because it must be centered around the rune it's acting on, and those can be right next to the edge of the workspace.

You might think it would take some significant CPU cycles to composite all those transparent canvases, but from what I can tell, when a canvas isn't changing, the browsers can treat it as a transparent image and render it very, very fast.

HTML5 and 3D - Not yet

I want to add a 3D view of the Wisp's world to the game. This would be an "over-the-shoulder" view, if wisps had shoulders. Unfortunately, there are two big obstacles:

1. Art. I've done all the art for RSW myself. (It shows.) And I'm definitely not ready to attempt the higher detail required for 3D rendering.

2. Runtime support. There is no standard for 3D rendering in HTML5 yet. WebGL is getting there, but it's still not enabled by default in Chrome and not available at all in the production release of Firefox.

This is why I'm saying HTML5 isn't really finished. I *could* put together some kind of art and do the linear algebra myself, or pull in some 3rd party 3D library running on a 2D canvas, but it seems like it's just not worth it at this point. Maybe if I do a sequel to RSW, it will have the 3D view, when HTML5 is ready.

User Submitted Levels and a Server

In order to build a community, I needed a place for players to store levels they had created so other people could take up the challenge. Google App Engine provides the perfect platform, with a non-SQL database where levels and credentials can be stored along with hosting for the game files. So far, I've never exceeded any of the thresholds provided with a free Google App Engine account, but I've authorized a few dollars a week just in case.

Now, there is one big limitation for anyone considering using Google App Engine for Java projects. Google charges a fair bit of money to keep your application loaded into a JVM at all times. You can still use it for free, but loading a war file into a JVM takes some time and you should expect that first page turn to take a while.

I avoided that by learning a bit of Python. There are a handful of URLs like this one:
http://runestonewisp.appspot.com/saved/vYYuj.oTEd.kd5N..W4Z0A

That runs through a bit of python that extracts records from the Google Datastore, formats them in JSON and sends out the result. All my client-server communications are handled just like that, with JSON payloads over HTTP. The django library for Python has nice support for JSON in multiple formats, both encoding and decoding.

I don't require any login of any kind for users of Runestone Wisp. But, if someone wants to submit custom levels and then edit them later, I allow them to login using the Google App Engine's login engine, so millions of people already have a password and user name. To keep gaming identities separate from real-world e-mail addresses, I let each person choose a username, and update it each time they submit or edit a level.

There is a tricky bit to using Google Login from inside the game engine. I don't have anywhere to store a user level while the user is logging in, so I can't let them navigate away from the game screen. To get around this, the login link in Runestone Wisp opens the Google Login page in a separate window or tab. Once the user logs in, Google sends it back to a page in RSW which notifies the game that the login has finished, then closes the popup. This seems to work beautifully.

Hard Algorithms and JavaScript Performance

Unlike the games which inspired Runestone Wisp, I wanted to include some significant random elements. As a simple example, that monster that's in your way might be on the left side of the hallway one time, and the right side of the hallway the next time. This forces the player to use sensor components to determine the correct action instead of just scripting a static route through the obstacles. (I also have some levels where clever scripting is the point, but those are the exception.)

That was the idea, but what I saw in beta testing was people declaring victory with solutions that failed 9 out of 10 times. They didn't want to bother to plan for the random elements. So, I had to come up with a way to reject solutions which failed part of the time. This is where the idea originated for the "Murphy's Law mode". The idea is, I want to run each solution over and over again until it fails, and only display the failures to the user. That's actually a hard problem.

One possible type of failure is an attempt that sends the wisp running around in circles without ever reaching the exit. It never finishes, but it also never dies. If you're thinking "Halting Problem" then you remember your CS courses, because it's theoretically impossible in the most general case.

But, I don't have the most general case. My world is a finite state machine with fewer than 2^200 possible states, and far less than that accessible from the starting point. There is a way to apply graph theory cycle detection to detect that solutions that never finish. I would love to use it. Unfortunately, you have to be able to compare two states to determine if they represent the same "node" in the graph of the finite state machine, and for reasons explained below, I cannot do that.

There are plenty of times in a working solution where the wisp has to back up to an earlier location, in the exact same state, waiting for the monsters to wander out of the way. Because monsters change directions randomly, the wisp will eventually have a clear path through to victory. But to the loop detection algorithm, it looks like a loop. The only way to prevent it is to include the seed of the Random Number Generator in the state you compare. And there's the kicker. The loop length for a good RNG is HUGE.

So, I took the lazy route. Murphy's Law is only applied when the user specifically asks for Gold Star Mode (and even then not on every level.) And when Murphy's Law is in effect, there is always a time limit, ranging from 40 simulation steps to 12k simulation steps. If a solution hasn't finished in that amount of time, it will be ineligible for a gold star. I can't apply time limits more broadly because one of the "just-for-fun" challenges in this sort of game is to make the SLOWEST solution possible that actually, eventually works.

Copy and Paste in Pure JS

Engineering games have historically had notoriously bad user interfaces, lacking basic hotkey support and other UI features like copy and paste. I wanted to fix that, the best way possible. In the past, people have added Copy and Paste using the system clipboard to JS by using Flash, but Adobe closed that security hole in Flash 10, so it's no longer an option (and never would have been my preferred method.)

So, how do you copy and paste graphical bits from JS?

First, I don't mess with graphical bits. I already have a serializer that converts a collection of Runes and Threads into a Base64 encoded string for URLs. I update the URL in real-time each time someone moves, deletes or modifies a rune or thread. It's a fast little Base64 implementation that I wrote myself. (And I even wrapped an LZSS implementation on top of it, which I'm not using right now.) So, when you copy runes to the system clipboard, you're actually getting a base64 string containing the binary data about rune locations and types. This is handy for people who want to develop a toolbox of partial solutions to reuse on other levels.

So, how do you stuff text into the system clipboard? And the question which should be harder: How do you retrieve text from the system clipboard? The solution I came up with is, as far as I know, the same one Google uses for Google Docs.

When you press CTRL-C, I intercept the keydown event, serialize the selected runes and write them to a hidden text input area. I then focus that text input and let the default action for a CTRL-C keypress event do the rest. Boom, the runes are on the clipboard.

Similarly, for CTRL-V, I intercept the keydown event, focus the hidden text input and let the default action handle the system clipboard. Then I detect the onchange event for the hidden input, deserialize the text and add the resulting runes to the screen.

It's not 100% effective, since there seem to be some timing issues, but it's darned close. When it fails, the automatic user response is to try again, holding down the buttons a bit longer, which is exactly what it needs to succeed.

Browser Combatibility

Internet Explorer deserves special mention. It IS possible to use canvas in IE, if you avoid using patterns and stick to a bunch of other restrictions, by using a third-party emulation library. It's painfully slow, and the limitations would severely change the interface to RSW, so I decide to ignore IE. Rather than "degrading gracefully", I try to get a halfway decent looking message pointing them to real browsers. I've watched the logs on the server and it looks like the majority of people are getting to the first level, which means they are opening up a browser other than IE. This makes me think I made the right decision.

In Closing...

I feel very good about the current state of Runestone Wisp. I wish it were getting a lot more hits, and preferably a lot of donations, but that's ok. I wrote it mainly for me and I'm very pleased with it. It's helped me connect with people I never expected would play and that's just amazing.

That's it for today, if you have any questions about other parts of the game, please feel free to ask. I haven't mentioned log analysis, topological sorts, tutorial design, promotion, or Google Checkout, but if you're interested, I can tell you what I learned.