Stardock

What makes a good beta tester?

By on July 10, 2009 6:04:44 PM from Stardock Forums Stardock Forums

Maxpower179

Join Date 06/2008
+8

I've never done this before.

19 Replies
Search this post
Subscription Options


Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 6:16:21 PM from Elemental Forums Elemental Forums

Me neither. Fortunately, you don't actually HAVE to participate in the feedback process: you can just play the game. It's not like they have a limited number of downloads or anything.

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 6:29:12 PM from Elemental Forums Elemental Forums

Yeah, basically if you see something wrong, say it.  It's not like there's an NDA or anything.

 

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 6:42:33 PM from Elemental Forums Elemental Forums

I imagine it's a lot like being a good QA person to begin with. Look to exercise the corner cases. Don't always move, click, etc in the same way. If you can do something on a map or in a dialog, try both. Cast spells. Multi-select things that aren't alike and see what happens. If I select the city then try to multi-select a nearby army, is that legal?

Even if you don't take that kind of approach and instead just play the game and report any bugs, you'll want to stress the game by avoiding ruts. Go military one time, go spell caster another, go all research, or all explore. Choose play styles that make you try out different areas of the game each time. Played zoomed all the way out once, then zoomed way in another time. Do all the map elements show up correctly in both cases? And on, and on

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 6:55:45 PM from Elemental Forums Elemental Forums

Quoting ckessel,
... Even if you don't take that kind of approach and instead just play the game and report any bugs, you'll want to stress the game by avoiding ruts. ...

Your post in general seems very sensible to me, but I wonder if there isn't also value in having some of us work our little ruts as hard as we can, especially if that more or less just means getting as far along in a very large map as you can before the next build is posted.

If Elemental is to be truly good as scaling up for Ludicrous 64-bit maps, it seems like just getting several hundred turns finished is a stress point worth its own consideration, and a familiar rut will get you their much more quickly than mucking around with playstyles that you wouldn't pursue outside of being a beta volunteer.

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 7:05:45 PM from Elemental Forums Elemental Forums

Quoting GW Swicord,
Quoting ckessel, reply 3... Even if you don't take that kind of approach and instead just play the game and report any bugs, you'll want to stress the game by avoiding ruts. ...

Your post in general seems very sensible to me, but I wonder if there isn't also value in having some of us work our little ruts as hard as we can, especially if that more or less just means getting as far along in a very large map as you can before the next build is posted.

If Elemental is to be truly good as scaling up for Ludicrous 64-bit maps, it seems like just getting several hundred turns finished is a stress point worth its own consideration, and a familiar rut will get you their much more quickly than mucking around with playstyles that you wouldn't pursue outside of being a beta volunteer.

There is. You need both people trying to find bugs, and people who just try to play the game and find the obvious things that someone really close to the game won't find.

 

Someone who has never seen it before is in the best position to spot obvious oddness, speaking as a software developer.

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 7:26:18 PM from Elemental Forums Elemental Forums

It also helps to have an old and/or cheap computer, the crappier the better (w/ in reason) to see how the game does on less-than-standard systems.

but I wonder if there isn't also value in having some of us work our little ruts as hard as we can, especially if that more or less just means getting as far along in a very large map as you can before the next build is posted.
Now THERE'S an idea: co-ordinate the beta testing externally so that you don't have 500 people all reporting the same bugs..... I volunteer to do spells.

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 8:02:11 PM from Elemental Forums Elemental Forums

The perfect beta alpha tester:

D is for diligence to continue playing even when it is not fun.

E is for Enthusiasm when you are under the gun

N is for Nitpicky. and also for Nice

R is for Resourceful finding bugs in a trice.

Y is for Yippee! at final release.

U is for Understanding when the devs need some peace,

 

But seriously - anyone can be a good beta tester. From dedicated bug squishers to folks that just want to play as long as they report problems when they see them.

I would say that poor beta testers would be ones that are going to bitch about the art direction, or if something gets implemented that they didn;t like, or if something they REALLY REALLY wanted isn't put in, etc.

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 9:30:46 PM from Elemental Forums Elemental Forums

Basically, play the game and note whenever something goes wrong.  

It is best if you can explain

"what went wrong" (what needs improvement)

"When and where it went wrong"  (How to see what needs improvement)

"How do you reproduce it?  if possible" (Why it needs to be improved)

If you can provide answers to these questions (and do so) you are likely a good beta tester.  Often it is best to explain "what went wrong?" with information that includes screenshots, videos, or crashdumps, in the case that words are not enough to give a bug or glitch justice.

Having the ability to communicate through text and the drive to do so is obviously important.  If many people give bugs they discover with the above questions (or provide feedback with the questions in brackets) then the public beta should contribute very well to the final product.

 

An example of a BAD piece of feedback would be:

"I think the matchmaking system sucks.  It needs to be better"

 

an example of a GOOD piece of feedback would be:

"I find waiting for matches to be very long and boring.  I don't think my attention span lasts for over 5 min, and if that is required to find a match, then at least provide some sort of noise or queue to let me be aware a match was found since I am unlikely to still be watching my screen after 5 min.

Another thing I noticed was that often when it does find a match, a player drops almost every time, and the teams end up being unbalanced.  I know I'm behind a Linksys router with Cox as my ISP and my connection is generally pretty stable, but I wanted to share my information to help narrow down where an issue my exist."

(or at least I imagine that  is good feedback.   I think a dev could provide better clarification)

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 10:13:08 PM from Elemental Forums Elemental Forums

In short from when I betaed Heroes 5: "Try to crash the game any way you can!" That means shifting from zoomed in to zoomed out and rotating the camera as fast as you can while simultaneously moving troops (if that's possible) to playing in a window and alt-tabbing like crazy!

Reason for Karma (Optional)
Successfully updated karma reason!
July 10, 2009 10:28:58 PM from Elemental Forums Elemental Forums

I hope cheats will be included in the alpha/beta. How elese will I be able to easily produce massive numbers of everything and see how long it takes to run out of virtual memory?

Reason for Karma (Optional)
Successfully updated karma reason!
July 11, 2009 1:37:38 AM from Elemental Forums Elemental Forums

As mentioned above, you really need the skill and/or talent to not only discover bugs, but also how to explain a problem and how to reproduce it. Being aware of a bug is helpful and all, but being aware of what causes what helps to determine which lines of code to look at first and can greatly reduce the amount of testing the devs need to do.

Other stuff you would want to be good at is:
-Finding weird stuff
-Finding rare problems
-Figuring out how to break stuff

For instance, at one point in time back in GalCiv 2, the devs were adding in starbase repair modules (to quickly repair damaged starbases). I noticed that each repair added to a repair stat much like you would have for mining. Each repair would add to this number despite the fact the effects was a one time deal. A few screen shots with the stat circled help to explain the problem.

Here is a sample image of what I'm talking about

Reason for Karma (Optional)
Successfully updated karma reason!
July 11, 2009 2:27:40 AM from Elemental Forums Elemental Forums

50 year old crack whores.

 

Too old to get any customers, too strung out to sleep.

Reason for Karma (Optional)
Successfully updated karma reason!
July 11, 2009 8:08:10 AM from Elemental Forums Elemental Forums

When you make your bug reports/feature requests/usability suggestions, a certain amount of precision is useful.

Once you find a bug, the first thing to do is to retrace your steps to reproduce the problem. Next, you should attempt to find the minimum set of actions necessary to trigger the bug (real-life example: do I have to fill out the form and click ok every time to make the program grind to a halt? Or will just opening this menu over and over do the same?). Lastly, report the steps you took and the problem as clearly as you can.

Finally, software testing can get really boring, but still takes some thought. If it starts to grind and you're just going through the same sequence of motions over and over again, then you probably won't find any bugs, so take a break. Or write/record an automated script to do it for you: there are some really good tools out there to do that sort of stuff these days.

Reason for Karma (Optional)
Successfully updated karma reason!
July 11, 2009 5:32:26 PM from Elemental Forums Elemental Forums

A good Beta Tester is someone who can focus as directed, ignore known issues, and can communicate both problems found and why something does or does not feel right.

 

Sammual

Reason for Karma (Optional)
Successfully updated karma reason!
July 13, 2009 7:08:06 AM from Elemental Forums Elemental Forums

A good beta tester is someone who can withstand the constant weekly if not daily downloads and STARTING OVER time and time again to test for old and new bugs and flaws. A good beta tester doesn't just play the game NORMALLY they actually TRY TO BREAK IT. They look for loop holes and exploits and they don't secretly not mention them so they can have or get unfair advantages after the game is released (this usually happens with mmorpgs). A good beta tester is someone willing to give at least 20 hours of their FREE time per week to the game. I've known many who've shouted that they would <begging to be in the beta>, but, never did. What many beta testers wanted was to be able to play the game first and get it for free at the end. I personally was always against this and recommended that testers only get a free game if they turned in so many feedback reports when I beta tested for SSI.

In this form of beta testing many are about to take part in it doesn't really matter or work like the days of old. You've paid for the game you're going to get. You aren't going to get a free version at the end and if you want it to work correctly then you'd better send in your feedback because you (yes the individual you) might be the only one to discover the bugs, exploits or flaws. I kind of like this idea of preorder get to play beta. But, on the other hand I could be the one that puts 110% effort to get the game to work properly and without bugs and flaws and exploits and I'll get no recognition out of it or a free game from my hard work and it IS HARD WORK if you're a GOOD BETA tester.

What Brad and Stardock might or could do is those that really put in a great effort of feedback throughout the beta period they give some kind of reward like perhaps a free game of choice out of their library now or the next game they make give it away for free as long as the person remains part of that beta program.

Reason for Karma (Optional)
Successfully updated karma reason!
July 13, 2009 11:36:13 AM from Elemental Forums Elemental Forums

Speaking as a very long time tester for fun (and pay from time to time) if one goes into doing testing looking for incentive(s) beyond simply wanted to help make a game as good as it could be for the benefit of the community at large, then one is not properly motivated in my opinion.

Reason for Karma (Optional)
Successfully updated karma reason!
July 13, 2009 12:01:27 PM from Stardock Forums Stardock Forums

The best way to Beta Test is to play the game, LOTS.  And report everything odd that you find unless it is listed under some known issues area.  If you don't report something because you assume that everyone else is experiencing the problem then you are a useless Beta Tester.  Related obviously, but check known issues before reporting!

Also in terms of gameplay, try the extremes of things, attempt to do things quickly and see what happens, see if you can buy stupid amounts of one item, see what happens when you fill your inventory and then accept a reward for a quest item.  Save your game and check if everything is exactly how you would expect on Loading.

Reason for Karma (Optional)
Successfully updated karma reason!
July 13, 2009 7:12:11 PM from Elemental Forums Elemental Forums

An ideal beta tester would indeed put in hours and hours of work and feedback reports, even when the game is no fun. However, I would recommend any one to just have fun while testing. We play these games to relax, right? It seems ironic to me that the testers would put un countless hours of no fun in an unfinished product, only to find out that he is done with the game once the final release comes. I would much rather Somewhat report bugs - you know, carefully report anything I find while playing the game - but still focus on having fun and playing.

I test a major title as of now, and in that game an NDA is involved so I cannot go into any details or mention the title or anything. In that game however I carefully report anything I find, keep in touch with the other testers to know what is going on with very one, but in the end I only report a handful of bugs. Some report a few dozen with every new update, I report maybe three... It is all good really, be sure to have fun and if you find anything broken with the game - either bugs or features - be sure to give feedback so that the end product will be as good as it could possibly be.

Reason for Karma (Optional)
Successfully updated karma reason!
July 13, 2009 8:29:45 PM from Elemental Forums Elemental Forums

The best way to test something is to forget nothing, good or bad. If you don't like something but can't quantify WHY then mention it anyway. Its possible that someone else had the same problem, and only by discussing it between the two of you will you be able to give it a name.

Reason for Karma (Optional)
Successfully updated karma reason!
Stardock Forums v1.0.0.0    #108433  walnut3   Server Load Time: 00:00:00.0000875   Page Render Time:
Facebook Twitter YouTube Google+