Suggestions for Testing: The Author's side
Contact the beta-tester and make your requests clear.
In particular, it’s a good idea to mention some or all of the following points, if they’re important to you:
Time-frame. Mention when you are planning to release the game (and thus what your time-frame is), if you’re working on something that has to come out at a specific time. This doesn’t guarantee that your tester will be able to finish your game in the time you indicate, but it does mean he has a chance to write back and tell you that his schedule isn’t going to fit that.
Format. Say whether you need your feedback in a specific format. By this I don’t mean anything really complicated — there’s usually no point in arranging annotations in a special form — but it can sometimes be useful to ask that testers send you their transcripts from game play. In fact, this is so frequently valuable that I recommend everyone do it. Not only does it make it possible to reproduce bugs that occur under finicky or rare conditions, but it means that you can see things your beta-tester might not actually report as bugs: places where phrasings didn’t work, where the tester was obviously having trouble, where he tried something that you might like to implement. I try to implement responses, as much as is reasonable, for whatever my testers try: any objects or parts of objects that didn’t have descriptions, any verbs that are suggested by the narrative, and so on. The Last Lousy Point in Savoir-Faire comes from an action that a beta-tester tried to perform, and I thought it was funny enough to implement.
Specialty verbs. Some authors add special verbs to their games to make annotation easier, and/or to dump the game state to transcript after a bug has occurred. If you’re using anything like this, you should obviously clue the beta-tester in about them.
Any important disclaimers. I sometimes begin testing the opening phases of a game before the endgame is complete. There are two reasons why: one is that it makes things go faster if I’m working against a competition deadline or something similar; the other is that sometimes tester feedback necessitates structural changes that are easier to implement if the game is not completely finished. Technically speaking, this is pre-beta-testing, and it requires extra understanding and forebearance from the testers — so I always try to let them know what they’re in for. If there are any bits of the game that are unfinished, I let the tester know that this is the case; I also try to put an ending in the game after the final (currently-solvable) puzzle, to indicate to the tester when he’s gone as far as he can go.
Do your best to pre-test the game.
Try not to send your testers something whose bugs are obvious to you even on a trivial playthrough. Even if you’re sending over a game which isn’t complete in every aspect, the parts you’re asking your testers to test should be playable: the puzzles should be (as far as you know) complete and well-clued, the prose should be (to the best of your ability) devoid of typos, and so on. You’ll be wrong about this, naturally, but you should try; there’s no point in asking your testers to tell you things that you already know, and you’ll be wasting their time and patience.
With that in mind, there are a couple of fairly trivial things you may want to check yourself before sending.
- If you’re not strong on spelling, dump your game text to a file and spellcheck it. Your testers may still turn things up that the spellchecker missed, but it will at least lower the number of trivial things they have to concentrate on.
- Check over your default responses, the ones you borrowed from the existing library. (Inform puts these in English.h.) See if there are any that leap out at you as totally inappropriate for your game. Likewise, if you use Inform and you normally use American spellings, be sure and set the constant DIALECT_US so that the library will be consistent with you.
- If you’ve added new verbs with new functionality, consider whether there are any reasonable synonyms you haven’t accounted for. Also, make sure that you’ve accounted for the problem cases: does the verb routine make sense when applied to an object that’s untouchable? out of reach? alive? unmoveable scenery? the player character herself?
- If you’ve added new grammar, is it consistent with the old stuff? One problem I’ve seen — more than once — occurs when the author has added grammar to handle an indirect object (>HIT MAN WITH STICK, say) but hasn’t changed the default response to >HIT MAN. So hitting the man with the stick may be the only solution to a puzzle, but >HIT MAN will turn up “Violence isn’t the answer to this one.” Severely misleading.
- Of course you’re not going to get everything. These are places where it’s particularly easy to generate errors, however. The less your testers have to fight their way through simple errors, the more time and energy they’ll have to devote to the trickier ones.
(You may also be interested in my more recent comments on preparing a game for testing.)
Ask questions.
Once you receive your report, feel free to contact the beta-tester about anything you don’t understand: the circumstances under which your game crashed the interpreter, the reason he doesn’t like your lead NPC, and so on. If he got bored, find out where he got bored. Try to avoid being defensive or justifying your decisions at this stage: you may have a good reason for doing things the way you did, but in most cases I at least find it’s actually counter-productive to convince the beta-tester that I’m right. What I want is to find something that will fix the problems the beta-tester experienced while not ruining my original concept.
Respond to reports.
Different authors have different approaches to this, but wherever possible, I like to respond in some way to the reports I receive even if I don’t have any questions — not just by saying, “thanks, I got it,” but by letting the tester know about any significant changes I made in response to his suggestions. I’m not sure I recall any cases where this was important to the development of the game per se, but I think it does improve tester-author relations. Obviously, it’s not always possible to do this in great detail, especially if you’re doing your testing on a tight, pre-competition schedule. But it’s nice. (I tend to just send my testers email or speak to them in person, where appropriate; another author for whom I beta-tested handled this aspect by sending all his testers a list of changes he’d made when he distributed out the second beta-release of his game.)
Don’t take criticism personally.
This is obvious, but important. Bear in mind that your testers are there to help you. They are on your side. What they say is in aid of your project. If you’re upset or discouraged by their reactions, that’s understandable — but rant to someone else, like your boyfriend or your dog or your journal. Don’t blame your testers for feeling the way they feel. If you really think that someone has criticized your work unreasonably, get second (and third and fourth) opinions on the same issue from your other beta-testers. This may help give you enough perspective to tell whether you should drastically rewrite something, or just ignore that tester’s advice.
Expect to do several rounds of testing.
Once you’ve fixed your first set of bugs, you’ll probably need to test your game again to make sure that you didn’t miss anything and that your changes didn’t themselves introduce bugs. (This happens a LOT. There’s no shame in it.) The more complicated your code is the more likely that you’ll need to iterate this process a number of times. Your original tester(s) may or may not be up for multiple testing sessions; if not, you should be prepared to ask for a second group.
Credit all testers by name.
This should go without saying, but I know of an occasion where it was forgotten and the omission caused some grief. If you’re releasing a competition game under a pseudonym and feel that revealing your testers’ names would be too obvious, you can release a post-comp version with the testers listed — but let them know that that’s what you’re going to do, if you can. Most people will be content as long as they feel their efforts have been appreciated. (Similarly, if you get bug reports after release that you then incorporate into subsequent releases, it’s nice to mention everyone who sent them to you. This will make it more likely that people will send feedback on your subsequent games, if any.)
Take responsibility for any bugs that remain in the game at release-time.
They’re not your tester’s fault, they’re your fault. Never ever blame the testers for missing things. Testing interactive fiction is a very difficult thing to do well, and more or less impossible to do thoroughly.
Cultivate the testers who work well with you.
Some combinations of tester and author just seem to click especially well. Also, some people are better at testing certain kinds of games than others. I have some testers who really like trying to break a simulation, and poke in all the corners looking for extreme cases to experiment with. I have others who like to play through the game as though they were an ordinary player, and then give detailed feedback on how the pacing, structure, and thematic development of the game succeeds. Both of these are incredibly useful to me — the former especially for a game with a lot of finicky materials systems, the latter especially for games with a lot of multilinearity where I might not anticipate the exact experience of Joe Random Player. When I’m inviting testers to work on a new WIP I have in progress, I tend to choose whom to ask first based on my knowledge of their testing style. This is something I couldn’t even have tried to do when I was writing my first couple of games (though, as it happens, I was fortunate enough to accidentally ask people who turned out to be extremely helpful). The more familiar you are with the community, though, the more likely you are to be able to select people who will have useful insights on your work — by, for instance, noticing what sorts of work they’ve themselves written, what they tend to say in reviews, and what other games they have tested.
