pc@hillside.co.uk (Peter Collinson) (06/26/91)
Submitted-by: pc@hillside.co.uk (Peter Collinson) USENIX Standards Watchdog Committee Stephen R. Walli <stephe@usenix.org>, Report Editor Report on 1003.3: POSIX Test Methods and Conformance Andrew Twigger <att@root.co.uk> reports on the April 15-19, 1991 meeting in Chicago, IL: Summary The POSIX.2 (Shell and Utilities) working group made good progress writing test assertions this week, with POSIX.3's (Test Methods and Conformance) help. Many working groups, however, are discovering that writing test assertions requires a non-trivial effort. This week also saw the delivery of the newly published ``IEEE 1003.3-1991 - Test Methods for Measuring Conformance to POSIX''. Concerns are still being raised over NIST's certification policies. Report Chicago will probably go down in history as the meeting where test methods invaded the POSIX working groups with a vengeance. After years of mild abuse and jesting (mostly aimed at NIST), the SCCT (Steering Committee on Conformance Testing) seems to be succeeding in the goal of ensuring that POSIX standards are balloted with test method specifications. Despite rumours during the week that a wake had been arranged for the SCCT Chair, most of the screams were heard from working groups, who having been previously informed that test methods would be easy to write and would only take a couple of meetings, were finding that this was a far from straightforward task. While most of the remaining members of the original POSIX.3 working group continued work with the remaining members of POSIX.2 in generating assertions for the POSIX.2 standard, a few of the POSIX.3 elders started helping other working groups to develop test methods for their standards. The POSIX.3.2 group (i.e POSIX.3 + POSIX.2) met for three days during the week and spent all of that time writing assertions in small groups of three or four people. Some of the more difficult aspects of POSIX.2 were tackled, specifically Basic Regular Expressions and the Make utility. Most of the smaller utilities have assertions written already although most of these need to be updated to align with the current draft. It is hoped that enough of the work will have been completed after the October balloting commencing in the first half of 1992. Other working groups that have started producing test methods include POSIX.4, POSIX.6, POSIX.8, POSIX.15, POSIX.17 and P1224.1, P1224.2. Most of these groups are at an early stage in their test method development and are producing a wide variety of problems for the ``experts'' to address. Several of these groups have noted that the formal process of producing test assertions has uncovered a variety of deficiencies in their drafts; so perhaps there is some benefit in test methods after all! The highlight of the week was the arrival of the latest of the series of POSIX standards, IEEE 1003.3-1991. This document was made available at the extraordinary discounted price of $15.00 per copy, which works out to 30 cents a page! Still I suppose that considering the number of committee hours that went into the document, it's a real bargain. (One working group member calculated an industry cost in excess of $5,000 per page.) Other concerns which arose during the week relate to NIST's adopted certification policies and procedures. Many working groups continue to be concerned about these. This has been a long running battle involving both accredited testing centres and implementation suppliers in assisting NIST in the refining of their policies. The current major cause for concern is whether there would be equality in the certification process or whether a particular implementor would gain advantage from receiving the first conformance certificate. NIST was not explicit as to the procedures that they would employ to deal with the initial surge of certification requests, but made assurances that everybody would be satisfied when the process was completed. This seemed to satisfy nobody! We'll have to wait until Santa Clara to see whether NIST is really here to help us. Volume-Number: Volume 24, Number 23