Polishing a Game

Luck of the Draw - A Wide-Open Time Line

I'm now a contestant for the Mozilla Game On 2010 competition. It's odd that most people I've heard from were hard-pressed to prepare a game for release. You see, I started this game for fun, and it just happened to coincide with a contest for exactly the kind of game I was writing.

Runestone Wisp was feature-complete roughly by the start of November, after over a month of hard-driven development. That felt like victory, I was done writing my game! I started asking people to test it out at that point, and I thought was done. For a lot of the people now entered in Game On, they were done in that amount of time, because they hit the submission deadline.

So why have I been so busy working on my game for the last two months? While the core game has not changed, the quality of the user experience is far, far more advanced than it was. I have learned more about support during the last two months than I have in 13 years of programming professionally.

The Introduction

My first feedback showed that people were confused about the objectives of the game, overlooking the map and wisp entirely and focusing on the runes and how they interact. To combat that, I added the Introduction. It's just a simple little animation with captions to show you the "heartbeat" of the game. It's done using the same map animation engine the game itself uses. For those who can see the intro, it's a solid start that teaches the motivation for the game.

It's not really enough. It should run in IE to get those users motivated to switch. And I would still like to expand the introduction to a larger video showing how the runes drive the Wisp to do what he does, but I have to stop for now. Before I can do the Introduction justice, I will need to learn more about making internet videos, and it's not in my schedule (yet!)

The Goldilocks Tutorial - Too Much or Too Little

I never knew how delicate it was to create a tutorial for a game until I tried. Runestone Wisp is a HARD game. I don't expect everyone to be able to read a tutorial and jump into any given level and be able to finish it. There are a lot of rune types, each with its own rules to internalize. So in a sense, most of the built-in levels are PART of the tutorial process, adding complexity level by level. There is going to be some work to get into it.

That does not mean it's acceptable to have people asking, "I see a blank grid at the left. Is there supposed to be something there?" (One of my early Opera testers asked me that.)

So I built a tutorial. The original version was very static and told you exactly what to do, in what order and made sure you did nothing else. It was much TOO helpful and took too much of the design and creativity out of the levels covered. While the users ended up knowing exactly how to do the levels in the tutorial, they never got a chance to THINK about it, which ruined the game for them.

I was so disheartened by the user-response to the tutorial, I almost gave up on the whole project. Maybe they were right? Maybe the game just wasn't fun. Instead, I hid most of the tutorial and added warnings to what was left to tell people the tutorial was a spoiler and they shouldn't use it.

Then I moved to improving parts of the game I understood. (Copy-and-Paste for example.) As a lone developer on a project, it's important to be flexible and not let one setback stop you from making forward progress. In fact, when I got the other issues fixed, it gave me the courage to go back and work on the tutorial. It was much harder to build a flexible tutorial than it was to build the simple, static one. I am glad I took the break before diving into it.

The current generation of the tutorial dynamically examines what you've done and advises you, from where you are, how to move forward. And it doesn't try to help you through all the rough spots. Those are important! It just gives you a little shove over the hardest part of the learning curve and then gets out of your way. It's smaller, better, and more sophisticated.

Hotkeys, Copy-And-Paste, Palette

I got high-praise from one fan for the highly polished interface, which never would have happened if not for the immense amount of feedback I've gotten from gamers. I knew going in that I wasn't going to compromise on hotkeys and usability the way other engineering game developers have done. JavaScript makes it too easy to implement a hotkey for every major operation. And with the security holes patched around the system clipboard, JavaScript is now at least as good as Flash at implementing copy-paste that works across tabs, windows and even applications.

I resisted the idea of adding a palette of runes for a very long time because it seemed so redundant, when all the runes are available right there in the menu... just click where you want a rune. So when I finally implemented the palette, I was amazed how much it improved the game. Now it's one of the features I'm most proud of, but in the process of putting it in, I broke a bunch of things, which would have been ok if I'd started it sooner. But late in the process, it was unfortunate that suddenly Moons and Corks stopped working.

The palette also gave me a place to post the available count for limited-quantity rune types. This is a huge boon for copy-paste, because it lets me flag runes which aren't allowed in the level in a very visible and understandable way. It also aids greatly in the next topic.

Implementing Challenge Modes was Worthy of a Gold Star

The same fan who told me I have a polished interface also told me that I have a shallow game. (Not in so many words. And he said it after playing for at least 16 hours straight.)

See, when you put a game out in front of an audience, something odd happens. They play it wrong. In my case, the problem is users aren't trying to engineer a solution. They just hack out something that works. In some cases, it works 1 time in 8 runs, and they call that good enough. That was NOT what I expected! Because it's not what I, as a software engineer, would do.

The fan who was complaining should have known better. He had played engineering games and knew he should have optimized and refined his solutions, but there was no incentive, in Runestone Wisp, to spend time once you completed a level. So he felt let down that the game gave him full credit.

I had to provide him the challenge of doing real optimization and testing. I had to enforce Murphy's Law. But at the same time, my SISTER plays this game, and I didn't want to make all her solutions that barely work obsolete. So that's when I came up with the Gold Star bonuses. Most of these are based on my Murphy's Law implementation (discussed earlier), but I also threw in some Gold Stars based on limiting yourself to very few runes or threads for variety.

You can still finished a level and move on with your flaky, hacked together, unoptimized solution. But, if you're able to go above and beyond the bare minimum, you will see a simple Gold Star on the level select screen. It's all about providing the maximum entertainment hours for the user, and part of that is replayability. You would be surprised how much time a gamer will spend repeating the same levels over and over to get the perfect score, provided you tell them what constitutes a perfect score.

Browser Combatibiilty and Bugs, Bugs, Bugs

Of course the expected outcome of any long testing cycle is the opportunity to track down and fix as many bugs as you possibly can. Since I do the majority of my development in Chromium for the awesome debugger, my first list of browser compatibility bugs always comes from Firefox. (And at least half of them are the lack of a built-in "console" object.) But I've also had the opportunity to work through the issues with Opera and learned some things about a browser I'd never used before.

In the end, the result is a better game, more likely to please the user and less likely to frustrate, and that makes me very happy. Having fans is great. (Money would be nice too.)

No Framework, Just Libraries

Someone asked what kind of framework I used for RSW, so I thought I should add some information here. I didn't use any "framework", because I'm an elitist when it comes to javascript. (You wouldn't know it looking at my code.) Most frameworks have glaring structural problems that I don't want to introduce into my code.

So instead, I'll talk about the "libraries" I used. The first two are written by other people.

Random Numbers
I needed a reseedable random number generator to implement Murphy's Law. I turned down the Mersenne Twist library because of the terrible style, but the ARC4 RNG has a nice structure that fit well into RSW with a couple minor tweaks.

JavaScript Object Notation
I used a JSON library that I found at json.org for formatting. It should also be doing my JSON parsing, but I found it to be a pain in the butt as a parser. Instead, I'm using straight eval. This is reasonably safe from js injection, since all the JSON data is parsed and formatted by the django library on the server.

Base64 and Compression
Originally, I intended to save level data along with solution data in the URL, so I needed a compression algorithm. I didn't like any of the compression libraries I found on the web. One problem was namespace pollution, but I also had high expectations because I had already written a Base64 encoder. You see, most compression schemes depend on something called a "bit-library" to toss bits around in packets other than bytes. My own Base64 implementation is a bit-library, so an intermediate bit-library is redundant.

So, I wrote my own LZSS library on top of my Base64 implementation. It provided wonderful compression for level data, but even with that, I had to admit my levels weren't going to fit in URLs. That's when I added the server-side for storing user-generated levels. I'm still using the library, but only for Base64.

It might not stay dead though. Solution data is now pushing the practical limitations of URL storage. Solution data is too short for effective dictionary encoding, so I need entropy encoding. Either way, with such small data sets, I need to preseed the dictionary/encoding tree, and at the time I settled on a save format, I didn't have enough solution data accumulated to create a good seed. I might try again at some point.

Modular Initialization

As noted previously, I used a modular initialization scheme. It's home-brew, but fairly close to the capabilities found in standard libraries. There are some good modular loading libraries out there, but since I'd already written one, I didn't use them.

Google App Engine

I'm using some fairly basic facilities of the Google App Engine. The only part of Django I use is the JSON library. I save to the BigTable datastore and authenticate using Google accounts. There's a lot more you can do with the App Engine that I have not needed.

Google Checkout

Not one single person has used it yet, but in theory, I'm set up to accept payments via Google Checkout. A word of warning to anyone considering trying this: Google and PayPal both forbid asking for donations unless you have a registered charity. They will take away all your money if you try this. So, I'm very careful to not say I'm accepting donations. I allow people to pay for support for the game. It's a real service, (I released Opera support yesterday for example), so it should be ok, but keep in mind, I'm not a lawyer, and you probably need one if you're going to take in substantial money this way.

Technical Post-Mortem of Runestone Wisp - An HTML5 game

Runestone Wisp

Writing a Game in HTML5

HTML5 is new and exciting and even unfinished, (I'll get to that,) but I've had a lot of fun using it and I want to tell other people about the fun I've had.

First, have a look at the end result. Follow that link up above. It combines logic, puzzle solving and design skills with a bit of magic. You control the wisp, a docile yellow ball of fluff, using magical runestones. The runes combine to form networks that guide the wisp and react to the world around him. I like to call it a Sandbox Design Puzzle Game.

This project really started with Tile Factory, by Jonathon Duerig. I loved that game, but I wanted to play with some new types of memory, which Jonathon assured me would not be done in TF.

So, I set out to make my own game... It took me about a month to plan a new, interesting game I'd want to play. I was committed to the engineering sandbox design genre which Zachtronics has done so well, but I was hoping to broaden the audience. I chose a fantasy setting with magic themes and a charismatic protagonist. That's why the player controls a Will-o-Wisp, instead of something standard, like a zombie or robot, or something creepy like a brain-washed human. (The spells you build in Runestone Wisp are light-hearted, not J.K.Rowling's "Imperio" spell.)

Then I sat down to code it in Flash. After about 3 hours, I gave up. There are probably plenty of Flash developers who can write Flash applications without the Adobe toolset, but I'm not one of them. And I just can't afford that kind of investment for a game that probably won't make any money.

So, I started again, in JavaScript and HTML. I'm very experienced with JS, but not HTML5. Here's what I learned.

Loading and Initialization

This might be the biggest pure-JS project I've done, over 200k of code, so I had to break it up into manageable parts. I wrote the loading system for Runestone Wisp without reference to any guides on modular JS loading, and then read about them later. It turns out, my implementation is relatively close to a normal JS modular loading system, with a couple important twists.

A. My only contribution to the global namespace is the RSW object, so instead of a "module" method in the global namespace, I use rsw.initModule.

B. I dislike wrapping parentheses around anonymous methods. So I don't pass the initialization function into initModule. Instead, my syntax looks like this:

rsw.initModule('Rune','notice','runes','grid', 'menu').init =

function(notice, runes, grid, menu) { ... }

The parameters to initModule are a list of names for modules. In the end, these will all be exposed as properties of the RSW object. The object returned by initModule is an entry in the loading plan for the game, and the init attribute is used to initialize the first module listed. To me, this feels cleaner than the traditional module pattern for JS which makes the function a parameter to a module method.

C. Circular dependencies are a problem for most modular loading paradigms. But I've been

spoiled by Java to expect impossible load orders to be handled gracefully. Fortunately, I found a way to make it work. This is a big departure from the traditional JavaScript loader pattern. Each module springs magically into existence as soon as it's referenced. (It's an empty collection, but that's enough to make closures work.)

Inside initModule, I treat all the module names equally, and if one does not exist, I create it as an empty object literal. Once all the .js files have been parsed, the page finishes loading and I trigger the actual the actual initialization of each module, in three stages:

1. Run the init method, pass it all those empty objects. It can add properties as needed, and then return a collection of properties and methods.

2. Copy that collection onto the previously empty object being initialized.

3. If the list of properties includes a second init method, run it. This time it gets to use all those properties and methods that were returned from the first init.

As a result, I'm free to use circular dependencies with no problems at all.

It's worth mentioning that not all the "modules" can be initialized as empty hashes. rsw.Rune and rsw.State are constructors. Since they're defined directly in the RSW object, initModule will find they already exist the first time someone asks for them and use the existing function instead of an anonymous, empty object. Even though they aren't created by initModule, they're initialized exactly the same way other modules are.

HTML5 Canvas, Canvases and more Canvases

I've seen a lot of canvas demonstrations and even some canvas games, but so far they've all used one or possibly two HTML5 canvas objects. I chose to use a lot of different canvases so I could easily update individual visual elements while leaving other, overlapping elements untouched. The browsers do a brilliantly fast job of compositing canvases together, saving me from doing complex refreshes in JS. My use of canvases may be somewhat egregious, but they all have different lifespans and dimensions that fit how they're used.

1. The runes in the workspace live in lowest z-index canvas. I have a background pattern (the mostly-transparent gray grid) that's used to erase runes when I need to move or remove them. But I can usually avoid that by not drawing anything else to that canvas.

2. There is a second canvas over the workspace which shows all the threads between the runes, the highlights around selected runes and runes that are attached to the mouse pointer being moved. It's highly dependent on mouse movement events, which makes performance particularly important. Having a separate canvas makes this easy without redrawing the entire set of runes.

Both of those canvases are set within a scrolling frame so that everyone has the same workspace size regardless of what screen resolution they're using.

3. The map is drawn in its own small canvas. It gets redrawn completely each time the world state changes.

4. The tutorial has its own canvas covering the entire screen. This allows me to draw arrows pointing to any part of the UI, over the top of the map for instance. Mouse events from the tutorial are passed through to the workspace beneath. Quite a few things which need mouse interaction during the tutorial have higher z-indexes than the tutorial, like buttons and menus.

5. When the mouse pointer moves over the map, there's a second, mostly invisible canvas which contains highlights and the hexagonal cursor for the map. Performance is critically important for mouse move event handlers, so I minimize the amount of things I have to erase and redraw on this canvas. The underlying map doesn't change. This layer's z-index puts it above the tutorial in order to track mouse movements over the map even when the tutorial is active. (I also abused it to display the shadow on the map.)

6. The rune palette where you can grab runes and drag them into play was a late addition, using its own canvas. Its z-index puts it under the radial menu, but just above the tutorial, so it's available during the tutorial if you choose to use it.

7. The radial menu has its own full-screen canvas so it can be quickly erased and redrawn. It's not constrained to the workspace because it must be centered around the rune it's acting on, and those can be right next to the edge of the workspace.

You might think it would take some significant CPU cycles to composite all those transparent canvases, but from what I can tell, when a canvas isn't changing, the browsers can treat it as a transparent image and render it very, very fast.

HTML5 and 3D - Not yet

I want to add a 3D view of the Wisp's world to the game. This would be an "over-the-shoulder" view, if wisps had shoulders. Unfortunately, there are two big obstacles:

1. Art. I've done all the art for RSW myself. (It shows.) And I'm definitely not ready to attempt the higher detail required for 3D rendering.

2. Runtime support. There is no standard for 3D rendering in HTML5 yet. WebGL is getting there, but it's still not enabled by default in Chrome and not available at all in the production release of Firefox.

This is why I'm saying HTML5 isn't really finished. I *could* put together some kind of art and do the linear algebra myself, or pull in some 3rd party 3D library running on a 2D canvas, but it seems like it's just not worth it at this point. Maybe if I do a sequel to RSW, it will have the 3D view, when HTML5 is ready.

User Submitted Levels and a Server

In order to build a community, I needed a place for players to store levels they had created so other people could take up the challenge. Google App Engine provides the perfect platform, with a non-SQL database where levels and credentials can be stored along with hosting for the game files. So far, I've never exceeded any of the thresholds provided with a free Google App Engine account, but I've authorized a few dollars a week just in case.

Now, there is one big limitation for anyone considering using Google App Engine for Java projects. Google charges a fair bit of money to keep your application loaded into a JVM at all times. You can still use it for free, but loading a war file into a JVM takes some time and you should expect that first page turn to take a while.

I avoided that by learning a bit of Python. There are a handful of URLs like this one:
http://runestonewisp.appspot.com/saved/vYYuj.oTEd.kd5N..W4Z0A

That runs through a bit of python that extracts records from the Google Datastore, formats them in JSON and sends out the result. All my client-server communications are handled just like that, with JSON payloads over HTTP. The django library for Python has nice support for JSON in multiple formats, both encoding and decoding.

I don't require any login of any kind for users of Runestone Wisp. But, if someone wants to submit custom levels and then edit them later, I allow them to login using the Google App Engine's login engine, so millions of people already have a password and user name. To keep gaming identities separate from real-world e-mail addresses, I let each person choose a username, and update it each time they submit or edit a level.

There is a tricky bit to using Google Login from inside the game engine. I don't have anywhere to store a user level while the user is logging in, so I can't let them navigate away from the game screen. To get around this, the login link in Runestone Wisp opens the Google Login page in a separate window or tab. Once the user logs in, Google sends it back to a page in RSW which notifies the game that the login has finished, then closes the popup. This seems to work beautifully.

Hard Algorithms and JavaScript Performance

Unlike the games which inspired Runestone Wisp, I wanted to include some significant random elements. As a simple example, that monster that's in your way might be on the left side of the hallway one time, and the right side of the hallway the next time. This forces the player to use sensor components to determine the correct action instead of just scripting a static route through the obstacles. (I also have some levels where clever scripting is the point, but those are the exception.)

That was the idea, but what I saw in beta testing was people declaring victory with solutions that failed 9 out of 10 times. They didn't want to bother to plan for the random elements. So, I had to come up with a way to reject solutions which failed part of the time. This is where the idea originated for the "Murphy's Law mode". The idea is, I want to run each solution over and over again until it fails, and only display the failures to the user. That's actually a hard problem.

One possible type of failure is an attempt that sends the wisp running around in circles without ever reaching the exit. It never finishes, but it also never dies. If you're thinking "Halting Problem" then you remember your CS courses, because it's theoretically impossible in the most general case.

But, I don't have the most general case. My world is a finite state machine with fewer than 2^200 possible states, and far less than that accessible from the starting point. There is a way to apply graph theory cycle detection to detect that solutions that never finish. I would love to use it. Unfortunately, you have to be able to compare two states to determine if they represent the same "node" in the graph of the finite state machine, and for reasons explained below, I cannot do that.

There are plenty of times in a working solution where the wisp has to back up to an earlier location, in the exact same state, waiting for the monsters to wander out of the way. Because monsters change directions randomly, the wisp will eventually have a clear path through to victory. But to the loop detection algorithm, it looks like a loop. The only way to prevent it is to include the seed of the Random Number Generator in the state you compare. And there's the kicker. The loop length for a good RNG is HUGE.

So, I took the lazy route. Murphy's Law is only applied when the user specifically asks for Gold Star Mode (and even then not on every level.) And when Murphy's Law is in effect, there is always a time limit, ranging from 40 simulation steps to 12k simulation steps. If a solution hasn't finished in that amount of time, it will be ineligible for a gold star. I can't apply time limits more broadly because one of the "just-for-fun" challenges in this sort of game is to make the SLOWEST solution possible that actually, eventually works.

Copy and Paste in Pure JS

Engineering games have historically had notoriously bad user interfaces, lacking basic hotkey support and other UI features like copy and paste. I wanted to fix that, the best way possible. In the past, people have added Copy and Paste using the system clipboard to JS by using Flash, but Adobe closed that security hole in Flash 10, so it's no longer an option (and never would have been my preferred method.)

So, how do you copy and paste graphical bits from JS?

First, I don't mess with graphical bits. I already have a serializer that converts a collection of Runes and Threads into a Base64 encoded string for URLs. I update the URL in real-time each time someone moves, deletes or modifies a rune or thread. It's a fast little Base64 implementation that I wrote myself. (And I even wrapped an LZSS implementation on top of it, which I'm not using right now.) So, when you copy runes to the system clipboard, you're actually getting a base64 string containing the binary data about rune locations and types. This is handy for people who want to develop a toolbox of partial solutions to reuse on other levels.

So, how do you stuff text into the system clipboard? And the question which should be harder: How do you retrieve text from the system clipboard? The solution I came up with is, as far as I know, the same one Google uses for Google Docs.

When you press CTRL-C, I intercept the keydown event, serialize the selected runes and write them to a hidden text input area. I then focus that text input and let the default action for a CTRL-C keypress event do the rest. Boom, the runes are on the clipboard.

Similarly, for CTRL-V, I intercept the keydown event, focus the hidden text input and let the default action handle the system clipboard. Then I detect the onchange event for the hidden input, deserialize the text and add the resulting runes to the screen.

It's not 100% effective, since there seem to be some timing issues, but it's darned close. When it fails, the automatic user response is to try again, holding down the buttons a bit longer, which is exactly what it needs to succeed.

Browser Combatibility

Internet Explorer deserves special mention. It IS possible to use canvas in IE, if you avoid using patterns and stick to a bunch of other restrictions, by using a third-party emulation library. It's painfully slow, and the limitations would severely change the interface to RSW, so I decide to ignore IE. Rather than "degrading gracefully", I try to get a halfway decent looking message pointing them to real browsers. I've watched the logs on the server and it looks like the majority of people are getting to the first level, which means they are opening up a browser other than IE. This makes me think I made the right decision.

In Closing...

I feel very good about the current state of Runestone Wisp. I wish it were getting a lot more hits, and preferably a lot of donations, but that's ok. I wrote it mainly for me and I'm very pleased with it. It's helped me connect with people I never expected would play and that's just amazing.

That's it for today, if you have any questions about other parts of the game, please feel free to ask. I haven't mentioned log analysis, topological sorts, tutorial design, promotion, or Google Checkout, but if you're interested, I can tell you what I learned.