Want Better Quality? Fire Your QA Team.

Seriously. I recently spoke with a client who swears that software quality improved once they got rid of the QA team. Instead of making QA responsible for quality, they put the responsibility squarely on the backs of the developers producing the software. This seems to go against conventional wisdom about quality software and developers: Don't trust developers. Or, borrowing from Ronald Reagan, trust but verify.

This client is no slouch, either. The applications provide real-time market data for financial markets, and the client does more than 40 software releases per year. If the market data produced by an application were unavailable or inaccurate, then the financial market it serves would crumble. Availability and accuracy of information are absolute. This app can't go down, and it can't be wrong.

Why Does This Work?

The client said that this works because the developers know that they have 100% responsibility for the application. If it doesn't work, the developers can't say that "QA didn't catch the problem." There is no QA team to blame. The buck stops with the application development team. They better get it right, or heads will roll.

As British author Samuel Johnson famously put it, "The prospect of being hanged focuses the mind wonderfully."

Can This Work For You?

This can work if the application development team is held accountable for software quality. The consequences for bad quality must be so grave that the application team has no choice other than to focus their minds wonderfully on design, development, and testing. Most of the software I have developed over the years was sans QA team. When you develop mission-critical applications for large enterprises as a consultant, you better deliver quality or heads will indeed roll. You won't get paid.

Responsibility isn't enough. To be successful, app dev teams must create test harnesses, create test plans, and use testing tools. But, because the developers know where the bodies are buried in the code, they can focus their efforts on the most brittle parts of the applications. The level of effort will depend on the type of software. The end result could be better quality and faster release cycles.

Comments

Testing v QA

I am certain I made the error of thinking that this post meant QA to be "the testers", so I apologize for that. And I fully agree that some separate set of QA druids hovering over a development effort is silly and redundant. And even dangerous.

However, in re-reading the initial post, the line about the "bodies being buried" was the one that most likely led me astray. I agree with others who say that TESTING, as opposed to QA, needs to do two things:

- ensure continuing quality of code

- ensure that the software meets user needs (as Lisa has so eloquently stated)

Testing

Most of the companies involved in Offshore software development does the testing at their end before delivering the projects to their customers, it doesn't matter that the client have asked for it or not. Companies wants to deliver best to clients, that's why they are spending so much on QA .

Sorry, can't applaud "out of the box" thinking here

IT has come a long way in 40 or 50 years. Best practices HAVE evolved, for the better, including formalized testing, and projects can indeed be brought in on time and on budget, with a much greater degree of structure than existed a few decades ago. It's simply ludicrous and counterproductive at this point to characterize systems development as equivalent to playwriting, an argument that basically claims that oh, yes, "we're ARTISTS, not engineers! Don't cramp our style!". This attitude strikes me, as a seasoned practitioner in the IT arena, as both misinformed and profoundly naive. And I'm afraid I simply can't couch that in any more palatable way: as I tweeted earlier this week, the notion of firing one's QA department and putting the entire responsibility for error-free code on the developers is "painfully wrongheaded". Thinking that upping the stakes (i.e., making the "consequences" for error be harsh) is going to motivate people to do better is insulting to everyone involved (see one emotional response in one respondent's blog post: http://rootwyrm.us.to/2011/02/firing-your-qa-worst-idea-evar/ ). It brings to mind the old quip often posted in abusive work environments: "The floggings will continue until morale improves." So more than IT is being misunderstood here: basic principles of leadership and motivation are as well.

There are reasons that QA evolved as a discipline and as a useful function, and these are far too great to enumerate here. And some of the comments above, particularly the one that equates development to playwriting, or tweets that label people as having a "dogmatic devotion to QA" (!) reflect a sadly confused and equally wrongheaded view of the essence of systems development.

In fact, as cited here (http://www.guardian.co.uk/business/2004/may/02/theobserver.observerbusin...):
"Much of the blame for the failures [lies] on management shortcomings, in particular the failure of both customers and suppliers to follow known best practice. Contrary to many assertions, big IT projects have much in common with other large-scale engineering schemes. 'It's project management,' says the Dr Mike Rodd of the British Computer Society (BCS). 'There's not much difference between a big IT project and building a second Severn crossing.'

Professor John McDermid of York University, one of the report's authors, laments that 'in IT best practice is rarely practised. Projects are often poorly defined, codes of practice are frequently ignored and there is a woeful inability to learn from past experience'."

Posts like yours, Mike, unfortunately contribute to that "woeful inability to learn from past experience." Citing one person's experience where QA didn't work well is a poor excuse to throw the baby out with the bathwater. No one opposes accountability, and thinking that the presence of a QA function absolves any developer of accountability sorely misunderstands how things work in modern systems development.

Engineers (or "artists") versus automatons

Peter, You make some excellent points, among them that strong project management and leadership are lacking in many IT projects. One of those lackings is that some managers view developers as automatons on an assembly line rather than engineers. So, I agree with you that developers should be engineers rather than coders. Coders implies a typist just doing what they are told. The industrial metaphor for software engineering is a failure. Software development is more akin to an engineering project or, yes, making a movie (perhaps just writing the play is too narrow - it is the entire production). Application developers who consider themselves engineers or playwrights will have more self-respect for their work and therefore more pride in their work. The result: better quality.

Clarifying point:I am not advocating eliminating testing. I am suggesting that developers must be more accountable for quality and that many separate QA teams do routine testing rather than having the skill to find the real showstoppers.

Summary: I agree that IT projects have much in common with engineering projects like building a large, unique bridge. Sadly, IT best practices often come from an industrial metaphor like building cars on an assembly line where developers are working the line doing the same thing over and over again.

QA and QA

I think that the people who react here have a different interpretation of QA. Are we talking about QA activities or are we talking about a QA group / QA department?

Of course, QA activities should always be done. In small companies, in large organizations. No doubt about that. To me the main issue of the blog was: should QA activities be executed by a seperate QA team. To me the latter has many disadvantages.
Therefore I say: only a small part of all of the (very many) QA-activities that should be executed are executed best by a seperate independent QA-team. All the other QA activities may best be performed by and are a responsibility of the developing teams them selves.

Sticks and stones and real issues

For someone so passionate I would have thought Kretzman would engage with the issues raised instead of calling us names.

My playwriting analogy is a description of what is, not what ought to be. I'm trying to illuminate why software is still in crisis. Yes let's make software development a real profession but don't kid anyone that it is anything like one as yet. It remains more art and craft than engineering.

Kretzman refers to "known best practice". What would that be exactly? Respondents on this thread - despite the strength of their convictions - arent even using the same terminology! It's embarrassing isn't it?

Certainly we should all wish that big software projects are managed like a second Severn crossing but these endeavours remain poles apart. Software is different from the stuff of any other engineering. No big bridge project ever changes spec half way through. And as I tried to explain, the reliability of civil engineering efforts comes from hundreds or years of experience - and gravity.

Process is the only thing that confers any structure and strength at all on a piece of code. Serious analysts in this field need to be a little more receptive to constructive criticism of sacred processes.

Yes, real issues

Wow, I'm reluctant to respond to this sort of ad hominem attack, but here goes…

I'd note, first, that Mr. Wilson is throwing stones from a glass house when he complains of anyone calling him names, since his initial entry into the discussion consisted of declaring opposing views (mine) as both "dogmatic" and "daft". (http://twitter.com/#!/Steve_Lockstep/statuses/40169432634359808)

Mr. Wilson also states that I failed to engage on the issues, which both mystifies me and then leads me to conclude that he didn't actually read my five-paragraph reply in its entirety, which cited two external links relating to the debate, as well as industry practice in general. Meanwhile, Mr. Wilson cites no supporting evidence whatsoever, other than his own strong opinions and comparisons of software to soil and gravel.

I stand by my statement that the subject of the article, firing the QA department, is "painfully wrongheaded". Dispensing with a separate QA department, after all the lessons learned in the industry over the years, is anything but a mainstream view. Not all opinions are created equal, and those that think so far "out of the box" as to throw away decades of industry learning in doing so certainly should be considered as bearing a heavy burden of proof, proof that goes beyond simple statement and restatement. There's a reason that the article evoked such derision among many of the commenters. It's simply far afield of accepted industry practice.

Mr. Wilson wonders what "known best practice" could possibly be. There's really no need to quarrel deeply about what is meant by testing, etc., or "known best practice". The fact that some people in this thread don't have their terminology clear is hardly an indicator that best practices, or clear definitions of terms, somehow don't exist. My own career of several decades has shown me countless examples of the value of the separation of duties when it comes to development and testing. And this is widely recognized across various development methodologies. I'd recommend referring, for example, to what is considered a classic in the field, Kaner's Testing Computer Software, particularly Chapter 15, "Managing a Testing Group." Kaner presents a nuanced and balanced view of the different kinds of testing groups (QC, QA, Testing Services, and Development Services), and discusses the evolution, including pros and cons, of "all testing done by programmers" into having a specialized testing group.

Equally (and I'll stop at two just for reasons of space; there are many more), refer to Steve McConnell's well-known and acclaimed book, "Code Complete," Chapter 22, "Developer Testing". He cites studies showing the specific issues that appear when developers test (pp. 500-504), and presents a recommended breakdown of testing phases. After presenting an itemized list entitled "Limitations of Developer Testing," he concludes "none of these points reduce the value of developer testing, but they do help put developer testing into proper perspective. As valuable as developer testing is, it isn't sufficient to provide adequate quality assurance on its own and should be supplemented with other practices, including independent testing and collaborative construction techniques."

And jumping back up now to the subject of system development failures, the problem, of course, is not that better methods of development don't exist, the problem is (as I cited above) that they are often simply not exercised -- and often, that's because someone (yes, wrongheadedly) has insisted that prior art is irrelevant, or that systems development is really more like playwriting, and that rolling one's own approach based on strong opinions and little experience will be just fine.

This can work

We can do away with QA team as long as we do not do away with QA activity per-se. And passing on the responsibility to the developers also will not ensure good code as they might already be hard pressed for meeting timelines. Instead they way it might work is if we can entrust the team lead or module lead with testing of the application. Instead of using his/her time in filling up and preparing metrics which hardly are required - he/she can focus on helping the team technically and guiding in development and once the product is build he/she is the one who is supposed to test it . Advantage is that since this person is involved with client requirement discussion from the beginning understand the customer's need better plus being a senior member himself understands the business and technology better.

What QA team? We don't have

What QA team? We don't have one. Two web developers who do their own QA and that's it for our IT staff.

Software Testing Service

Hi,
Testing is the most widely accepted testing practice in software development life cycle...Admiring the hard work you put into your website and detailed information you present in this post...
Thanks a lot......

Dan

Mike, your article is very dangerous. If all managers decide to fire QA teams, very many people will be unhappy. I tell about testers, QA engineers and other specialists of "quality industry". Some of them will hate you very much... I'm a tester and I don't want to be fired. Please, don't write such articles anymore.