Giving useful gamedev feedback

November 10th, 2015

[Updated on 2016.02.17 for clarity and brevity]

We, like many tiny indie studios, don’t have full-time QA staff. And even if we did, feedback from large numbers of players is really what we need. But we know that most players have never beta tested a game, and the fact is that providing useful feedback is super difficult. So here’s a quick prime.

Understand what the feedback is for

Remember: Feedback that isn’t useful wastes everyone’s time.

For example, if the developer hands you a game that is done and scheduled to release a month from now, they may want to know about game-breaking bugs (so they can make an emergency patch) but may not want a laundry list of things that “could be a little better” (it’s too late for that). But all this depends on what the developer is actually looking for.

Developers are often novices when it comes to beta testing, too, and may not think to explicitly tell you what kind of feedback they want. In that case, MAKE THEM TELL YOU!

Okay, so the first step is to focus on exactly what the developer is looking for (in retrospect, DUH, right?). But then the next, harder part is in how to give feedback.

What useful feedback looks like

Say you’re playtesting a game and stumble across something you want to report. How can you do that in the most productive way? Here are the four components of good feedback:

  1. What happened.
  2. Why you think it’s a problem.
  3. The exact steps required to make the thing happen, and how often it happens when you complete those steps.
  4. For 1-3, be as concise and explicit as possible.
Let’s take those one at a time.
What happened? Games are complicated machines: saying that something “didn’t work” is both the most useless feedback there is and the most common we get. There are a hundred ways for any particular thing to “not work.” Those hundred ways fall into two broad categories: (1) you expected something to happen, but something else happened instead, (2) something happened, but you would have preferred that something else happened instead. In either case, the first part of good feedback is to explicitly and clearly state what happened, and what you either expected or wished had happened instead.
Why you think it is a problem. Every gamer has a different play style and different gaming experiences to draw on. Every game is designed with certain intentions that might not match what you, specifically, would prefer. This means that any two people might disagree whether or not something really is a problem. So try to explain as clearly as possible why the thing that happened is a problem to you. Understand that a game isn’t designed for you, specifically, but for a broad selection of different people. Even if an idea is your own personal preference, it might be a great idea. In some cases the problem is so obvious it doesn’t need explaining (e.g. something that crashes the game). Otherwise you should assume that it isn’t obvious.
How to make the thing happen. This is, by far, the most important component. If we can’t reproduce the problem you describe, then we can’t do anything about it. If we can’t reproduce it quickly, then we’ll waste time tracking down the problem. Tell us exactly how to reproduce the problem. If possible, reproduce it yourself at least once before writing your report. Even better, do some experiments to see if you can find multiple routes to the same problem (then tell us all the things you tried and what happened in each case!). For example, if you found a combat bug where striking an enemy caused the game to crash, see if striking a different enemy, or different enemy type, or using a different weapon, all cause the same crash. And then tell us what did and did not work.
Be concise. As a tester, you usually won’t know what information the developers need to know in order to reproduce and fix a problem you find. Therefore you should err on the side of providing a lot of information. But that comes at a steep cost for you and for the developers: long-winded reports take a long time to write and a long time to read! Your goal should be to use as few words as possible while precisely and clearly covering the above points. You don’t need complete sentences, you don’t need to pre-apologize if you’re worried we’ll take offense, and don’t need to give us any backstory. Be a robot 😉
Examples of Feedback
 
Taking all of the above into account, let’s take some made-up examples of Crashlands Beta feedback. For the beta, we considered the game to be content-complete and were only looking to polish the game for launch. This meant that we’re looking for bugs, balance, and usability issues.
Example Feedback #1
 
Bad version. I had been wandering the Savanna for what felt like my entire life, ferociously battling Hewgo and his minions whilst trying to placate the natives by solving their problems. As a native Woanopian, I must say that the developers’ portrayal of the Tendraam is spot-on. Amazing work, you guys. Anywho, having wandered the Savanna — as I was saying earlier — I came across a fishing hole and cast my line deep into the watery depths. I celebrated vigorously as something grabbed the line, only to be dismayed by the fact of its skeleton-like appearance. It didn’t belong there! I am loathe to admit that I found it so distasteful that I had to brush my teeth.
Good version. Savannah Fishing Holes produce skeletonized fatfish ~10% of the time [what happened and how to reproduce it]. These are healing items, but heal for WAY more than anything else, so I suspect they aren’t supposed to be in the Savanna [why it’s a problem].
Example Feedback #2
Bad version. The Spacetime Folder doesn’t work.
Good version. 100% of the time when I click the Spacetime Folder in my hotbar nothing seems to happen, though the cooldown timer starts, implying that it should have done something. [what happened and how to reproduce it, obviously a problem]. I have tried it with and without the presence of a pet, in and out of combat, with various weapons and other hotbar items equipped, and with and without the presence of NPCs on screen. All other aspects of the game seem to behave normally. [more info about how to reproduce it, not worrying about which info is likely responsible]
Example Feedback #3
Bad version. I feel really terrible saying this, and I hope you don’t take any offense. I certainly don’t mean any! I mean, I love the story overall, but I just can’t stand the Omelettes for Burlenon quest.
Good version. The Omelettes for Burlenon quest is too difficult [what happened]. I have enjoyed all other quests so far, but this one is frustratingly impossible. Anger Omelettes require killing Epic Shirks, which don’t exist in the Savanna.  [why it’s a problem and, trivially, how to reproduce it]
head_512_adam

Posted by Adam Coster

Adam is a co-founder at Butterscotch Shenanigans, and programs all sorts of the nifty tools that make our studio special, like BScotchID and the Crashlands Creator. Follow him on Twitter at @costerad