lfd@cbnewsm.att.com (leland.f.derbenwick) (12/19/90)
In article <1790@taurus.cs.nps.navy.mil>, shimeall@taurus.cs.nps.navy.mil (timothy shimeall) writes: > In article <278@smds.UUCP> rh@smds.UUCP (Richard Harter) writes: > >Well my humble view is that testing is drudge work. That's why you have > >machines do it. People are supposed to design and implement the procedures > >that let the machines do it. > > A worthy goal, but there's one small fly in that soup. > [ ... several important flies :-) deleted ... ] > Given > that you don't want to do "drudge work" in the first place, are you > willing to give over several years of your life to automating that > very "drudge work" (and testing your automation)? And willing to repeat > that effort every time you change applications? There seems to be an underlying assumption that the system is going to be tested once, and then it's done and you move on to something else. If your application is at all successful, you are going to test it over and over and over again. Each new release requires 1. testing everything new to that release, and 2. regression testing the old features to make sure they weren't broken. For lots of reasons (including the "flies" I omitted above), it can be very hard to have automated tests for #1. Some mix of manual and automated tests is usually feasible, but a large portion is probably going to be manual. But #2, in its simplest form, merely requires generating a fixed input script and recording the outputs from a known working version (which was presumably tested manually; the recorded outputs should also be checked manually). Then, those same inputs can be provided to a later, enhanced version, and the output compared with the recorded output. This provides a cheap way of doing lots of regression testing, _every_ release. In practice, it is usually more complicated than that: date strings in the output (and maybe the input) must be different for tests run at different times; measuring performance or timeouts may be required, etc., etc. And programmed testing can do much more than simple record-the-output-and-compare-it. But since each test is going to be run many, many, times, it can justify the extra work to develop it. A personal example: At the moment, my project puts out a maintenance release about every 6 weeks, with feature releases several times a year. A purely manual regression test would take at least several months; we would have to resort to occasional spot checks or combine our frequent, reasonably sized releases into huge annual or semiannual reissues. (That was the tradition.) Instead, we have an automated regression test system, capable of testing our core functionality and most features, that runs a complete cycle of tests into several machines (which may be running different versions of software) every two weeks. -- Speaking strictly for myself, -- Lee Derbenwick, AT&T Bell Laboratories, Warren, NJ -- lfd@cbnewsm.ATT.COM or <wherever>!att!cbnewsm!lfd P.S. With regards to the original thread, I see software testing as being thoroughly part of software engineering, so the only reason to form a new group is if the testing-related traffic gets so high that everyone else wants us kicked out. (Or until the total traffic is high enough that we must subdivide intoseveral groups.)