Quality Guaranteed: What the Game Tester's Work Actually Looks Like (Part 1)

Westwood College's old video ad has become a bit of a joke in the video game world. Two guys, comfortably sitting on the couch, kill controllers in the trash, enthusiastically playing on the Sony PlayStation. A girl comes in and says: “Hey guys, have you finished testing this game? I have one more here. ”

“We just passed the third level, the schedule needs to be pulled up a bit,” one of the guys replies. Then, turning to his friend, he smiles as if he had just won the lottery: "I can’t believe that we play games, and we are still paid for it."
“I know,” the second answers him. “And my mother said that my passion for video games will not lead to anything good.”

That is how for a long time people imagined the life of those who are testing computer games - not as work 5-9 hours a day, but as the dream of all teenagers. Who wouldn’t want to sit on a comfortable sofa and play games all day with short breaks to “pull up” the graphics in the third level?



The reality is slightly different from this picture. The so-called quality control of computer games (QA), that is, their testing (hereinafter, the author mixes together the process components - testing (basic level), quality control and quality assurance - approx. Translator) , is often perceived as "playing games, and you’re still paid for it, "but in reality it could be better described as a process of" breaking "games. This is a low-paying, rarely satisfying and often disappointing job that affects - in one way or another - the quality of modern games, but not in the way you would expect.

A professional tester doesn’t just sit in front of the TV and, drinking some energy like Red Bull, goes through the fifth level of the last shooter. He (or she) spends 14 hours in a row, attacking the walls in these levels in order to verify their integrity. A good video game test is more like solving a puzzle than putting a new record in Donkey Kong, no matter what we are shown in the commercials. “In order to do a good job in the QA world, a specific approach and a special attitude to life are needed,” an experienced computer games tester told me. “This goes beyond the passion for video games and certainly does not coincide with the idea that you play video games and get paid for it.”

Usually testers are underestimated, remembering them only when something goes wrong. QA professionals argue that this work is boring, intense and often seen as an opportunity to get into other areas of game development than the more traditional career path. Often testers work on temporary contracts or for outsourcing companies that interfere with their direct communication with game developers. And when the game has a lot of bugs or it comes out in a raw, almost unplayable form - like many of the latest releases - then usually testers blame it. They are, among other things, those who must guarantee protection, being the last wall between the mistakes of programmers and the money of customers. The name of the process conveys the whole essence: Quality Assurance. In other words,

But are those who test the games really to blame for the fact that they enter the market raw? How is it possible that testers do not find the bugs that we see in games? Why do many servers lie now and then? What do these people do all day long?

In an attempt to explore the world of testing computer games and try to explain what this work consists of and what it is, over the past few months I have been actively communicating with a huge number of people who are currently engaged in testing or were once testers. Many of them chose not to call themselves to protect their careers. Some said they hated testing, while others said they could not imagine themselves in another occupation. Almost everyone agrees that only a few understand exactly what the work of ensuring quality is.

How many games exist, so many bugs live in them. Some are relatively harmless and even become legendary, like the mysterious MissingNoin Pokemon. Others went down in the history of video games: the endless levels of Minus World in the Super Mario Bros. toy, which can be reached by passing through the wall. But the tireless members of the gaming community do not sit still: new bugs are constantly found and vilified, as well as amusing players - glitches in Legend of Zelda: Ocarina of Time, for example, enough for a 17-minute funny video !



All these are rather friendly bugs. Most glitches in video games are annoying at best, and at worst, stop the game. Therefore, each game undergoes a quality control - extensive testing conducted in order to make sure the toy works correctly. The acronym QA came into the video game industry from the world of products - microwaves, machines, conveyors - and in many ways, testing games is no different from checking products. The job of the reviewer is to knock and rummage around every corner of the game and play enough to hell with it until all the glitches are removed - just like a factory worker modifying the last toy.

When it comes to QA, the gaming industry has no standards here: all games are different, and each company has its own approach to testing. But the tester usually spends months behind the game, checking the current options for the product being created in a variety of ways. The more bugs a tester finds, the higher it is valued by the company. This, of course, is not easy: video games are complex sets of interacting systems that must be carefully and methodically tested for bugs. And this may include repeated passage of the same level with minor changes (whether using a new hero, another weapon or choosing a new road) and keeping records of everything that happened to you.

Let's take Grand Theft Auto V as an example. In a huge open world created by developers from Rockstar Games, testers had to divide and conquer. “During the tests, different people were involved in certain missions or tasks, mini-games, etc.,” says the person who helped test the game. “Usually, work went from general to particular. First you go through the main missions in order, then the thefts go, then additional missions and verification of various characters, then you advance to testing the strip club and prostitutes. ”

The same tester said that sometimes they also have to spend a lot of time on tiny sections of the game. For example, when Rockstar designers asked a group of testers to check everything that players could do with the taxi service in the game. They quickly found that if you take a taxi for a new mission, the mission starts even before the player lets the taxi driver go in peace, which led to funny moments when the car circled around or tried to pass back during in-game cut scenes.

“I think that such work on projects makes them much better due to the fact that we find such moments when something really stupid happens,” the tester said. “We found a lot of bugs: talking pigs, now and then humanly standing on their hind legs and going away, simple passers-by, who suddenly skyrocket. Trevor, taking off his pants, did not bother to put them back on - for the rest of the game he ran with his pants hanging out somewhere below. Franklin’s dog died, barely touching the water ... the dog simply fell into the pond and went to the bottom with a stone, as soon as he wet his paws. "



Finding bugs is only the first step. The second, and much more complicated, is to try to reproduce the glitches so that the company's engineers can fix them. A tester cannot just write something like “pants are falling off Trevor” and send this to a team of programmers. What can engineers do with this information? In order to find, isolate and fix a bug, a programmer needs to know how exactly this happened, which can be a difficult puzzle, given the huge number of different factors in video games. Good testers quickly learn to remember their every action - significant and small - so that they can at least try to reproduce any bug they encounter. “I love that the tester’s work is like a paid puzzle solution,” said Rob Hodgson, an experienced tester with 8 years of experience.

The tester’s usual working days can vary significantly depending on the project, role and position in the company. So, a person who got a job through an outsourcing company can spend 10 hours crashing into each wall in the latest version of Call of Duty to find out where the structure can be pierced (a kind of “shock test”). A full-time employee who is engaged in tests can work with a programmer trying to figure out why the frame rate on the Android version decreases in their mobile game. The inconsistent and, as a rule, monotonous in nature work in the field of QA may carry some unexpected tests. For example, testers working on the Rock Band music game said that the sounds emitted by the “plastic” drums infuriated them to such an extent that they had to establish a rule:

During the game, testers record reports using software such as Jira in order to explain what happened and how it happened. Programmers who ideally do not currently work on new content and are exclusively engaged in fixing bugs, analyze reports and answer, if there is such a need, sometimes with questions, problems and sarcastic comments.

When the game on the console is almost complete, it must be certified - a process in which a publisher (EA, for example) asks the console manufacturer (Sony, Microsoft or Nintendo) to check the game for serious bugs. During this certification process, a second wave of QA staff (the guys call it “conformity testers”) go through it again, checking to see if everything meets expectations. Each console manufacturer has its own list, which describes all the requirements - from error messages to achievements, and if the game does not correspond to any element, then the publisher will have to fix it and try to get certified again - to hell with all the deadlines! “Microsoft requires all games to be able to go to the Xbox 360 menu from anywhere in the game,” said one tester. who worked for a major game publisher, checking for compliance. “According to Sony’s rules, in games it should not be possible to skip screens with studio / publisher screensavers at the beginning of the game at the first viewing. “Nintendo does not want obscene language in its games, so all texts have filters that we check and try to break.”

“I haven't played BioShock Infinite for at least two years after the release,” a former tester told me recently. He worked for 2K and spent a lot of time testing this game, he was disappointed with what the product turned into, which, as he noticed, was not worthy of the original version.

“The only thing that made me play the game again was watching the fast-paced passage of BioShock. We spent many nights playing the game quickly. It's interesting to see what players do to cut levels. ”

One evening, the tester saw a video in which the world record for passing the game was broken. At the eleventh minute he just lost his temper.



“I got mad because they use a bug to get out of the level and automatically move forward. I HAD TO FIND THIS GLUT! ” - the soap tester wrote to me.

For a very long time, the video game industry has exposed testing as a dream job: hey kids, play games all day, and we will pay you for that! But in recent years, various “horror films” about this profession of dreams have crept to the surface: testers began to share stories about monotonous exhausting work and the poor attitude towards them from companies that perceive them as consumables in a huge developing machine.

This is also reflected in low salaries. Such work does not have high requirements - usually in order to get the position of an entry-level tester, you do not need to have experience or a diploma. At the same time, many want to get this job, which is why salaries are average. In 2014, the results of a study on the salaries of an average beginner tester were published. It turned out that the annual salary of such an employee was about 55 thousand dollars (apparently, this is the salary before taxes - approx. Translator), but this is the salary of full-time employees, while most testers are contractors who work either directly with the developer or for companies that accept test orders from many publishers. Many of these contractors told me that their salaries vary from 10 to 15 dollars per hour - an average of 21-30 thousand dollars a year.

Testers also say that they feel disrespectful in the workplace. Many of them (especially contractors) are forbidden to communicate directly with developers, and all communication is carried out exclusively through written reports on identified bugs. “It was a bit of an unwritten rule - we couldn’t have direct contact with the developers,” one tester told me. - All communication was usually carried out through QA leads. All communication with the developers came down to comments in the error database, which cannot be called the ideal form of interaction, in which it is easy to incorrectly interpret the comment / question of the developer about the error as a taunt or annoying remark. "

So not in every studio: “When you provide testers with benefits, career opportunities, respect and the absence of fear of dismissal, it attracts the right people,” says Ariel Smith, who is testing MMO games at Cryptic. She told me that she loved her job, but disrespect for testers really became fashionable. Several testers told me that they have to use the side entrances to enter the offices where they work, and that they are forbidden to communicate with other employees. Others say developers often mock them to one degree or another. For example, one situation is known when a quality engineer corrected another bug, which did not stop him from constantly sending a message like “Unable to play” to the tester. In a typical studio, testers are considered the lowest layer of the hierarchy. This is partly due to the nature of the work - the tester shows others where they screwed up. It always hurts someone’s pride.

“Those who test the games only think about finding bugs, the developers think only about fixing these errors,” one tester told me: “They are not a team and do not work together. It is almost like a game of tennis. Testers are generally interested in making the game buggy, otherwise they will have nothing to do. Therefore, the two sides, in a sense, work to the detriment of each other, which cannot be called a healthy workflow. ”

In some game companies, the authorities set the standards for the testers for the errors found, and if there are fewer bugs than the norm, then the testers may face a reduction. This creates a strange tension when testers begin to compete for who first finds the biggest mistakes. Sometimes such employees show ingenuity and find ways to work more, get more and look more valuable employees for the company. “There were testers who detected bugs at such a time to be able to work overtime. If no extra hours are planned for the weekend, they report a lot of errors on Friday afternoon. In some cases, this will entail overtime hours, ”one tester told me.

Testers also have to deal with other not-so-pleasant issues in the game industry, mainly related to obligations and frequent layoffs. Large developers usually hire dozens of testers before the end of a large project and say goodbye to them as soon as the game comes out. Instead of celebrating the successful completion of the project with the developers, they are forced to look for a new job.

Given all of the above, an outsider may get the impression that this is a terrible dog job, but it also has positive aspects. Many testers tell me that, despite many problems at work, testing video games can be satisfying and unique in its way.

“I enjoyed doing the testing, and I would do it again if I needed to,” says Obed Navas, a former tester who worked on titles such as BioShock and Call of Duty. “Despite the fact that the tester is not the most glamorous title, and with such work you risk losing all interest in video games outside the working hours, in the end, the opportunity to see your name in the credits is worth a lot. It’s also cool to have some things related to projects that you can’t get anywhere, and to my friends questions about where I got them, I’m proud to say “I worked on this game”.

The second part is here .

PS Do you work as a tester? Do you agree with the opinion of the author of the original article? Please tell us about your work, its pros and cons - the way you see it.