Position Statement
on Agile Testing for Quality Week Debate with Ross Collard
Bret Pettichord
Pettichord
Consulting LLC
www.pettichord.com
Quality Week, September 2002, San Francisco
Agile software development is based on a hierarchy of
values. Individuals and interactions are valued over processes and tools.
Working software is valued over comprehensive documentation. Customer
collaboration is valued over contract negotiation. Responding to change is valued
over following a plan [1]. Extreme programming is the most popular agile
development practice, but any practice that follows theseadheres to these
values can properly be called agile.
Agile software development is a rejection of the
traditional ideas about software development. These have focused depend on a
phased approach, often known as a “waterfall”:
first requirements, then analysis, then design, then coding and finally testing
and deployment. Each step results in a document or deliverable that is used to
define the next. When followed fastidiously, testers receive software with
complete design and requirements specifications. More often, testers get
software with incomplete documentation. Even so, they often find bugs resulting
from errors in design or analysis. These bugs are found relatively late in the
process. They These can often
lead toresult in
unexpected delays as the bugs are found late in the process.
Two significant recent revisions to the
traditional approach deserve mention. One is the spiral
model, which runs through each of these phases multiple times
in shorter time frames or iterations. Another is the V-model,
which has
testers prescribes testing check the deliverables at
the end of each phase so that problems can be fixed before to
find bugs in the deliverables so that they are fixed before moving
on to the next.
Both models find ways to involve testers earlier in the process.
Agile methods, on the other
hand, take a more radical approach. They discourage the detailed
documentation of requirements and design, focus emphasizeon
writing code as soon as possible, and encourage frequent design changes
of design. These ideas scare a lot of people, including my friend
and colleague Ross Collard.
Many
of these people have been warning of the dangers of ill-defined requirements,
poor planning and cowboy coding for years. Too often, in their view,
the results have been buggy software, schedule delays and software systems that
didn’t perform as expected.
Many of themThese critics
have been frustrated by their lack of impact even when they were getting little
public resistance to their warnings. Now a vocal group of influential
developers are boldly encouraging these “dangerous” practices and giving
programmers a license to hack.
That’s how I understand their concern and the context
behind it. I take a very different view.
In my
experience over the past 14 years of working in
software developmenton various software projects, I’ve observed
noticed
that speed has often been of the essence. Building software
fast was not only essential to meeting market windows, but it was also
necessary to maintain project inertiamomentum.
Without it, projects easily floundered with dueling designs and grandiose
plans.
On the other hand, I’ve also seen
projects buried under the weight of their documentation, too vast to read
and too troublesome to revise, quickly becoming out of date.that
was often too vast to be read by everyone and was often allowed to become out
of date. Effective communication requires concise documentationIn
order to communicate effectively, project documents must
be concise and, sadly, ponderous tomes have often been
created only to go unread. A key One of the insights
of agile programming is that it’s not the documentation that matters: it’s the
communication. Documentation is no good if no one reads itit goes unread.
It’s important to
distinguish between programmers who are careless and undisciplined and
programmers who are honestly trying to follow the new agile methods. They require discipline,
although a
different kind of discipline than traditional development. For example,
A major difference
between agile programmers and the sloppy programmers that quality professionals
have been complaining about for years is their attention to testing. aAgile
programmers are
expected to practice strongly encourage test-first
design all the
time. This practice has programmers writing their tests
before they write theirwriting code.
Testers have been encouraging thissaying this
for over at least a decade.;
Nnow
the agile programmers are saying it too!
Agile programmers report that
test-first design makes speeds coding
go faster and also and improves design
results in better designs (with less
coupling and more cohesion). Skeptics, accustomed to programmers who claim to
do more testing than they actually do, might wonder whether agile programmers
actually follow through. In fact,
Aagile
programmers have recently releasedgenerated
a cornucopia of open-source testing frameworks over the past few years. They are
very serious about testing, and are creating the tools needed to do it well.
There are more testing frameworks available than I
can begin to keep track of.
Another reason for
tThey are alsoir
interested
in automated test suites is to support refactoring.
Refactoring is the practice of changing the structure of code without changing
its behavior. Traditional
development tries to understand how all the code will work together in advance.
This is the design. With agile methods, this difficult process of imagining
what code might look like before it is written is avoided. Instead, the code is
restructured as needed to maintain a coherent design.Frequent
refactoring allows less up-front planning of design. Agile methods
replace high-level design with frequent redesign (refactoring). Successful
refactoring But it also requires a way of ensuring
checking
whetherthat the behavior whasn’t
inadvertently changed. That’s where the tests come in.[2] That’s
a big reason why agile programmers want good test suites.
Yet another reason is that they often
prefertheir preference for tests (code) to text (words) for
describing system behavior. Tests are more precise than human language and they
are also a lot more likely to be updated when the design changes. How many
times have you seend
design documents that no longer accurately described the current workings of
the software?
Out-of-date design documents look pretty much like up-to-date documents.
Out-of-date tests fail.
Many testers are
unhappy with agile development not so much because they think it is a bad
practice but simply because it fails to give them the documentation they need
for testing. This is an age-old complaint. Some testers claim to be unable to test a system
without detailed and authoritative specifications describing how it should
work. I take a different view. Testers complain
that they can’t test the software because they don’t have adequate
specifications telling them how it should work.
I’ve never had a
lot of sympathy with this
complaint. I often find many sources of
expectations and intentions regarding software behavior. There may be gaps,
conflicts and ambiguities, but they often provide enough to test from, reporting any surprises. Bug reports may or may
not report bona fide defects. Nonetheless testers are most
useful when they report discrepancies and let the team sort out right from
wrong.
Testers need to ask questions, and they need to be open-minded when their bug
reports are rejected. In my view there
are usually lots of places to look for information about how the system should
work. It takes some digging up and interpretation, but that’s fine. Sometimes
it also takes some direct questions to various people on the team. Now
we have a bunch of programmers who are encouraging this kind of reaction. I’m all for it.
I believe that aAgile pProgramming
is a major
step forwardin general an improvement in development practices.
You may disagree. I believe that testers can work effectively with Agile
Programming. You may
disagree. But regardless, more than anything, I think that Aagile pProgramming
is the wave of the future. The practices will develop and some of the extreme
edges may be worn off, but it’s only growing in influence and attraction. Some tTesters
may not like it, but those who don’t figure out how to live with it are simply
going to be left behind.
Some testers are still upset that they don’t have the
authority to block the release of buggy software that still has
bugs in it. Do they think that they
now have the authority to block the adoption of these new
development methods? They’ll I think I’ll have
the most influence if I get onneed to get on this ship and if they want to try
to keep it from the shoals. Stay on the dock if you wishll.
Bon Voyage!
1.
1. Agile
Manifesto. 2001. http://www.agilemanifesto.org/
2. Martin Fowler, Refactoring: Improving the Design of Existing Code (Addison-Wesley: 2000). The book uses 17 sound bites to summarize the key ideas for effective refactoring. Of them, 9 address issues relating to tests and testing.