[techtalk] Testing (Was: Re: Technical issues)

jenn at simegen.com jenn at simegen.com
Tue Nov 21 23:38:57 EST 2000

Laurel Fan wrote:
> Excerpts from linuxchix: 21-Nov-100 [techtalk] Re: Technical is.. by
> jenn at simegen.com
> > And not just free of known bugs, but tested well.
> So, how do you test well?  What testing procedures and methodolgies
> should be used?  And how can adequate coverage be ensured?

Hoooboy .. this is a 'write a book' set of questions.

How you test depends on your personal philosophies and your ability, as
well as everything else.

My way tends to be:
Test each component in isolation.
Test the interactions between components.
Test the unit as a whole.

Test according to the design split (how it's coded), AND
Test according to the feature split (how it's used).

Test for good input, bad input, and boundary cases.
Test for various different sorts of bad input.
Test for various different sorts of good input.

Test on multiple hardware platforms (a misused pointer can work fine on
one platform, crash on a second, and cause wierd behaviour on a third).

Code to catch all user inputs and check that they're valid before use,
and fail gracefully in some way if they're not (usually with a
meaningful error message and a chance to re-input).
If it's a mission-critical or life-critical application, code to check
that the inputs from other computer functions are valid before use, as

> Scenario: 
> All of your fellow developers recognize the wisdom
> of testing, but some have no experience in more formal (meaning
> 'organized', not necessarily proofs of correctness) testing.  The
> project is written in C++, and makes extensive use of external
> libraries (in C and C++).  The application has a gui, and has a
> network component talking a well documented, standard protocol.  You
> are roughly in the beta stage, and your project is used by a few
> hundred people.
> What factors would you take into consideration, and what
> would your plan look like?

Ack! It's morning! Don' wanna...

* I'd document the test procedure extensively, to help the inexperienced
developers. I'd include definitions of 'boundary' 'bad input' and 'good
input' testing, and explain how extensive I expect it to be.
* I'd have people test /each other's/ code, as well as their own (or
perhaps instead of). People subconsciously know how their own code
works, and overlook the things they tend to overlook when writing code.
* Whether or not I tested the external libraries would depend on how
'trusted' they were - for the standard C/C++ libraries I'd not bother.
* I'd sit down with the protocol's standard, and thoroughly test all the
MUST and SHOULD cases, at minimum. I'd also test that all MAYs were
supported, at least to fail-gracefully should they be received.
* I've no experience per se in GUI testing, except at the user 'I know
what I like and know what I hate' level. I'd fall back to my standard
'test each subroutine, test each feature, in isolation and in groups'

My test plans tend to be big long checklists that developers hate. If
anyone's got a better way, PLEASE say so...

Jenn V.
       "Do you ever wonder if there's a whole section of geek culture 
		you miss out on by being a geek?" - Dancer.

jenn at simegen.com        Jenn Vesperman       

More information about the Techtalk mailing list