gaf@uucs1.UUCP (gaf) (11/17/89)
I've seen the X Testing Consortium notice from time to time, and wondered about the disclaimer saying that there are as likely to be bugs in the test suites themselves as in the software being tested. Has anyone actually used this test suite? Have bugs been reported against it and, if so, are the fixes publicly available? I sure hope it isn't an orphan; a working validation suite would be a great help to X porters (like me). Would anyone connected with the test suite care to comment on the state of the software and what we can expect in this area in the future (R4)? -- Guy Finney It's that feeling of deja-vu UUCS inc. Phoenix, Az all over again. ncar!noao!asuvax!hrc!uucs1!gaf sun!sunburn!gtx!uucs1!gaf
rws@EXPO.LCS.MIT.EDU (Bob Scheifler) (11/18/89)
Has anyone actually used this test suite?
Yes, many people have. Certainly we have at MIT.
Have bugs been reported against it
Yes, a few. Others are known but haven't been written down, due to lack of time.
and, if so, are the fixes publicly available?
Not at present.
I sure hope it isn't an orphan;
Me too. But, you must realize that verifying such a suite is a rather
substantial task.
Would anyone connected with the test suite care to comment on the state
of the software
The most important aspect of the suite is that it is under-reviewed. Until
a reasonable evaluation of the suite takes place, there's no good way to know
how good or bad it is. Some evaluation has taken place, but not by enough
people or in enough detail to be satisfactory. If you have any comments on
the suite, I'd be glad to collect them. My own guess is that it's rather good
compared to many test suites, but considerably lacking in terms of the coverage
needed to really provide good confidence in an implementation. For example, we
ran the suite against a pre-release of ours and the server didn't dump core, but
the very first operation with a particular window manager on the same server
did cause a core dump.
and what we can expect in this area in the future (R4)?
You won't see anything in R4. The testsuite is considered a separate
distribution. Work on the suite is on my agenda, but I can't really give
you any more information at present.klee@chico.pa.dec.com (Ken Lee) (11/18/89)
In article <215@uucs1.UUCP>, gaf@uucs1.UUCP (gaf) writes: > Has anyone actually used this test suite? Have bugs been reported > against it and, if so, are the fixes publicly available? I sure hope it > isn't an orphan; a working validation suite would be a great help to > X porters (like me). I've used the test suite (alpha release). Yes, there are some bugs. I've reported a dozen or so. I think the Testing Consortium plans to make the test suite much more robust for the official release. Ken Lee DEC Western Software Laboratory, Palo Alto, Calif. Internet: klee@decwrl.dec.com uucp: uunet!decwrl!klee