[prog] State of software engineering profession

[Kathryn Andersen]

On Tue, Apr 15, 2003 at 05:39:43AM -0500, Robert J. Hansen wrote:
> > rely on an external QA group to test your software.  An external group
> > will never be able to test the software as well as the group that
> > developed it.  Usually, black box testing can only verify the use case is
> 
> This is, as near as I can tell, in contradiction to the best current
> practice.  Every shop I've worked in has kept QA separate, precisely
> because developers are so lousy at QA.  We often received builds which
> wouldn't compile and we had to fix it ourselves.  Just making sure the
> code compiles is the simplest, most basic of checking, and developers
> would repeatedly not do it.

Ouch.
 
> > 6a.  Know HOW you plan to test the code when you design and implement it.
> 
> This leads to writing test cases which are meant to pass the design, not
> to test the code.  "We plan to test the code by entering up to
> 80-character ASCII strings" leads to some developer thinking "okay, so
> all I need are 80 bytes and I'm safe", instead of "I don't know how long
> this string is going to be, so I'd better resize it on the fly".
> 
> > Often, testing is passed to someone else, so very little thought is given
> > to the testing procedure.
> 
> Which is insulting to those of us who've worked in QA, and who devoted
> quite a bit of thought to figuring out new and interesting ways to make
> developers cry.  :)  

Too right, mate.  A good QA person is worth their weight in gold.

The thing is, what is really needed is *both*.
You need the programmers to do "unit testing" -- compile the code, run
it, and if possible, write and run regression tests for the bitsy things
that the program can do.  Every time a bug is fixed, while the bug is
being worked on, add another regression test which reproduced the bug --
that way, one can test whether the bug was fixed, and test whether the
bug remains fixed in future.
Of course some software is more amenable to regression-test writing than
others.

*Then* you get the QA people (at our small company it's the support
people who do it, we don't split it out) to bash at the program like a
user would.  You *need* that, because the programmer is too close to the
system -- they know how it's supposed to be used, and unconciously avoid
the corner cases or the randomness that a user would do ("Hey, let's
press that button and see what it does.  Ooops, it crashed.")

A testing person shouldn't know anything about the implementation, they
should just know what the requirements for the program are, and test
them.  You need someone with a user-eye view, because the programmers
are always going to have a programmers-eye view of it.  Fresh eyes are a
good thing.

But it sounds like, in both your cases, that testing was simply not
given a high enough priority from either side.

Kathryn Andersen
-=-=-=-=-=-=-=-=-
Spock: Excuse me, Doctor.  I'm getting a number of distress calls.
McCoy: I don't doubt it.
	(Star Trek IV: The Voyage Home)
-- 
 _--_|\	    | Kathryn Andersen	<http://www.katspace.com>
/      \    | 
\_.--.*/    | GenFicCrit mailing list <http://www.katspace.com/gen_fic_crit/>
      v	    | 
------------| Melbourne -> Victoria -> Australia -> Southern Hemisphere
Maranatha!  |	-> Earth -> Sol -> Milky Way Galaxy -> Universe