QA stands for …
Quality Assurance. Not quality testing, not bug tracking, not feedback. Quality assurance. Assurance of Quality.
When you look at a painting, read a book, drink a fine wine, you engage more-or-less directly with what the creators made. When you use a piece of software there is a middleman. When you play a game it is dominated by the software. And yet the software you experience is indirect. The authors create source code which goes on to become the product you come into contact with.
The art you experience is conveyed via the software: the software is the canvas, the paint, the brush, the lighting, the lense, the camera and the projector while the actual artists themselves become draft sketchers because the art you see in-game is ultimately drawn and animated by the software.
If software were cuisine, the bulk of it would be created by a chef who can prepare and cook individual ingredients, but the meal would be combined, seasoned and presented by the plate and the cuttlery.
Most software disciplines go through an amoebal stage where applications are born of a single programmer. Obviously gaming is no-longer in that phase, but it hasn’t completely left the egg yet. So it’s been my observation that the attitude still exists within the game programmers that the game is born of them. But it also seeps upwards.
Unfortunately, game development has grown and this vestigial mentality can crush a product. A modern game is largely a corporate product. The concept is prepared, a design outlined and people brought onboard to make the product. Then, like a cell, it divides into the corporate product and the game…
The game is developed by the programmers. The design, the art, the animation, the sound, the gameplay are all processes and artifacts and controlled by the software. This would give the programmers a sense of proprietariness over the product, but in a more modern game development house the programmer is no-longer the rockstar.
This affects their relationship with quality control. Programmers are sufficiently displaced from the corporate concept of the product that they become focused on their own scope, and as games become larger and more complex, this is a bad thing.
In ye olde days QA were the guys who tested for defects before shipping. That’s not QA. That’s defect testing. The QA divisions have begun to climb the ladders of reporting structures in recent years but only a few companies seem to have realized that gaming – as a software development discipline – needs to promote QA all the way. They shouldn’t be an appendage, the clue is in the title: Quality Assurance.
If someone is stumping up $50m for you to build a game, even if it is your own board, surely your very first concern and priority should be ensuring that you are meeting the quality of the downpayment?
As happens when a discpline makes this transition, many developers still think that you can’t worry about QA until you have something for them to test. Wrong: from the outset they need to be putting things in place to make sure that quality standards are in place without hindering everything. They need to be making sure that your product is self-documenting not just in terms of comments in the code but in terms of stock taking. Is your bug database tracking variables that will allow you to detect and cure recurring problems in the codebase? Is your issue tracking robust enough to detect some implementation or tool that keeps costing time, without innundating the dev team in useless information?
QA systems can be a pain in the butt. Been there, felt that. But so is pretty much anything you tack on to a project late in the day. If QA is not just there near the start but breaks ground along with the code, then the QA team won’t be playing continual catch up and they can work on being a symbiont rather than a parasite. In those early days, when there’s not a great deal for them to test, they can expend efforts of honing their analytical tools and providing relevant and helpful facilities for the coders.
The QA team is where the buck should stop, so it makes no sense that they are some department over there or some sub-division of development. It should start with QA and stop with QA. The project needs to be QA’s baby because their role is to Assure Quality and if they don’t have control they can’t do that.
I’ve seen a few game companies try this approach when a project has started going sour. Which is why those experiments usually fail. That’s like blaming the band when you provided the music.
“Testers on top” is likely to sit wrong with a lot of coders. You can’t be answering to a bunch of test monkeys that first saw the project at alpha.
That’s not QA, those guys aren’t assuring, they are Quality Chasers in a perpetual uphill battle to catch up with the project.
Testers like are little more than line inspectors checking for obvious blotches. Real QA is about the project. Which isn’t just the end result but the process and the tools as well.
Software Quality = Effort Taken / Quality of Tools
The lower the quality of your tools, the more effort taken to achieve the same quality of software.
If you ground your project on QA it will be pervasive. The difference between pervasive and intrusive is how early and how well you integrate it Solid, concrete Quality Assurance systems should cover more than just your end-user product but your internal systems and tools and processes. That coverage should present itself as a utility to the development team.
Ultimately the success of a QA team/system hinges on one thing: it’s ability to assure the quality of the service it delivers to the teams operating under it and the people it reports to. In programming terms, a simple feedback loop. If you throw the QA concept in at alpha, it’s going to have a very hard time adding that kind of self-evaluation into its metrics because your team and your project already has a body of work with no useful metrics or data.
I originally wrote this draft back in March but could never quite figure how to finish it. I have some friends who worked on an AAA title that launched recently and did really badly and after hearing the inside skinny (which I don’t have the OK to share) I decided to dig up this little post… Their root problem seems to have been with QA being subordinate to the coding team and effectively being little more than testers and bug reporters. Most of the player complaints about the product are in their ticket database with a closed status of “won’t fix” from a coder. The actual quality decisions were being made by the coders or designers above them.
Full disclosure: I can only profess to hands on experience of these kinds of problems in a gaming environment at Playnet; in terms of gaming most of it is second hand discussed in conferences and in conversations/emails with other devs. It’s not a unique experience to gaming, it’s merely gamings turn to pass thru the process of maturing into a quality-based discipline.