pc@hillside.co.uk (Peter Collinson) (06/06/91)
Submitted-by: pc@hillside.co.uk (Peter Collinson)
USENIX Standards Watchdog Committee
Stephen Walli <stephe@usenix.org>, Report Editor
Report on IEEE 1003.2: Shell and Utilities
David Rowley <david@mks.com> reports on the April 15-19 meeting in
Chicago, IL:
Background
A brief POSIX.2 project description:
- POSIX.2 is the base standard and deals with the basic shell
programming language and a set of utilities required for the
portability of shell scripts. It excludes most features that
might be considered interactive. POSIX.2 also standardizes
command-line and function interfaces related to certain POSIX.2
utilities (e.g., popen(), regular expressions, etc.). This part
of POSIX.2, which was developed first, is sometimes known as
``Dot 2 Classic.''
- POSIX.2a, the User Portability Extension or UPE, is a supplement
to the base POSIX.2 standard and standardizes commands, such as
vi, that might not appear in shell scripts but are important
enough that users must learn them on any real system. It is
essentially an interactive standard, and it will eventually be an
optional chapter to a future draft of the base document. This
approach allows the adoption of the UPE to trail Dot 2 Classic
without delaying it.
Some utilities have both interactive and non- interactive
features. In such cases, the UPE defines extensions from the
base POSIX.2 utility. Features used both interactively and in
scripts tend to be defined in the base standard.
- POSIX.2b is a newly approved project which will cover extensions
and new requests from other groups, such as utilities for the
POSIX.4 (Realtime) and POSIX.6 (Security) documents.
Together, Dot 2 Classic and the UPE will make up the International
Standards Organization's ISO 9945-2 -- the second volume of the
proposed ISO three-volume POSIX standard.
Summary
POSIX.2 (Shell and Utilities) closed its recirculation ballot on 29
March. The Chair feels there will only be a small number of changes to
the current draft, probably circulated as change pages. There were
some ballot concerns over the internationalization areas, but these
will likely remain intact due to current support. There was also a
concern raised over the ballot period for a 900+ page document.
POSIX.2a closed its recirculation ballot on 24 April.
POSIX.2b has been approved after a number of scope change
recommendations from the PMC. The POSIX.2 group requests technology
for both a new archive format, and also a new family of compression
utilities. Much of the Chicago meeting was spent with POSIX.3.2 (Test
Methods for POSIX.2) creating test assertions for the document.
Status of POSIX.2 Balloting
The Draft 11 Recirculation Ballot for POSIX.2 closed March 29th. Some
balloters seemed to forget that it was a recirculation ballot, as
there were a large number of objections on items other than changes.
These were ruled unresponsive.
Hal Jespersen, the POSIX.2 Chair and Technical Editor, believes that
there will only be a small number of actual modifications to the
draft. Draft 12 (which may possibly be called Draft 11.1) will likely
be distributed as a set of changed pages, which he estimates to number
about 200. (More recent estimates suggest the number of pages to be as
low as 50). Expect it sometime around July.
There were a number of objections to the internationalization part of
POSIX.2, but since Hal has support from X/Open, WG15, etc, he thinks
the current specification will remain largely intact.
There was a problem with the Draft 11 distribution. POSIX.2 is now
over 900 pages. It's balloting period was 30 days, although with a
mailing lead it was about 6 weeks. Due to postal services, some
members of the balloting group only received their ballot copies two
weeks prior to the closing deadline. Hal Jespersen was as
accommodating as he could be on the deadline, but the extent of these
submissions was definitely affected. The question rears its head
again. Should we be balloting POSIX standards the same as we ballot
smaller hardware standards?
The ISO standards process sees a document move through three phases on
its way to standardization -- Committee Document, Draft International
Standard, and finally International Standard. Draft 9 of POSIX.2 is
currently being used as a committee document. The ISO has requested
the U.S. Member Body to forward to them another draft once it has
become more stable. The next draft (Draft 12 or Draft 11.1) will be a
likely candidate. The ISO has delegated responsibility for producing
the POSIX draft standards documents to the U.S. Member Body, ANSI,
which in turn delegated the responsibility to the IEEE.
Status of POSIX.2a Balloting
The Draft 6 Recirculation Ballot of POSIX.2a (UPE) closed April 24th.
Unfortunately the deadline for comments came a mere three days after
the end of the April meeting. There were quite a few comments on the
unfortunate timing of the ballot close. Work on ballot resolution is
ongoing.
New PARs
The Project Management Committee (PMC) has recommended the proposed
PAR (Project Authorization Request) for 1003.2b be split into two
parts. One part will cover extensions and new requests from other
groups, such as the tar, cpio and new pax file formats from POSIX.1
(Kernel) and utilities from POSIX.4 (Realtime) and POSIX.6
(Security). The timing and alignment issues with the ISO 9945-2
balloting process will be covered by the other part.
The scope of this second PAR is restricted to standardization of
existing industry practice. The group does not want to get into
designing new utilities.
There is also concern over draft stability when discussing the
commands to access the features from the POSIX.4 and POSIX.6
standards. How mature does the feature have to be before the POSIX.2
group goes to the effort of defining a command interface to it ?
Discussion
Donn Terry, the chair of POSIX.1 officially handed off responsibility
of the pax file formats, including the new format currently being
designed, to the POSIX.2 group.
A proposal was examined to reserve the utility status return code 126
to indicate that a utility was found but could not be successfully
invoked. This would be especially useful in systems with limited
resources, where execution can not be assured even though the utility
has been found. The group generally agreed that this was reasonable.
There was a question as to whether the warning message for getopts
should be specified in the standard or not, so that filters could
parse it. It was decided to not specify the error format, since there
is no precedent for stating the format of something written to
standard error.
There was discussion on removing -e from pax, since charmaps were not
really intended to be used in this manner. The -e option was intended
to allow filenames to be written out using only characters from the
portable character set. OSF had already implemented this in their
pax, and agreed that it didn't work out too well.
The $(( notation in the Korn Shell currently can have two widely
different meanings: either spawning a subshell or expressing an
arithmetic operation. The working group agreed that disambiguating by
placing a space between the two parentheses, though inelegant, was the
best approach.
There was some discussion on a proposal on User Controllable Limits,
and whether or not it had relevance to POSIX.2. The group felt that
there should be a command interface to these services, with the
functional interface to be provided by POSIX.1. A proposal for the
POSIX.2 interface is now being solicited.
We also discussed the the test command. David Korn proposed fixing the
functionality of test based on the number of arguments given (1, 2 or
3). Invocations with greater than 3 arguments would not be portable.
We generally agreed on this approach.
Richard Hart from POSIX.7 (System Administration) presented the syntax
for a new printing command based on the MIT/Athena Palladium network
printing services. Everyone in the POSIX.2 group agreed that the
proposed syntax was awkward:
prpr -print-quality draft
means use draft if you can
prpr =print-quality draft
means you must use draft
prpr =p-q draft
means the same thing, but ``print-quality''
has been abbreviated.
The abbreviation mechanism is particularly poor, since it is likely
that new extensions will cause namespace conflicts.
Requests for technology
There is an opportunity now to propose a new archive format. The only
prerequisites are that it is ISO 1001 tape format compatible, and uses
the ISO 646 character set. This consists of the invariant codeset
from a variety of character set standards, largely 7-Bit ASCII minus
about 10 contentious characters. Here's a list of requirements:
o Should be textual (mailable) if members are all textual
o Should support filename and file contents mapping (eg.
for dynamic encryption or compression)
o Should be extensible
Personally I don't understand why the ISO 1001 tape format needs to be
a restriction. Archive formats have many other uses besides tape
backup and transfer. To embed the tape format in what could otherwise
be a general-use archive format seems overly complex and restrictive.
The group is also looking for a new family of compression utilities,
now that the Lempel-Ziv-Welch family of commands have been removed
from the standard. The main requirements for a substitute are:
o The algorithm should be expressed (expressible) in a
language independent form
o The algorithm should be free of patent issues
Test Plans and Assertions
A test plan for POSIX.2 and POSIX.2a has been written, and has been
passed to POSIX.3.2 (Test Assertions for POSIX.2) for comment and
approval.
Tuesday to Thursday were spent writing test assertions in a joint
meeting between POSIX.2 and POSIX.3.2 Commands tackled included make,
regular expressions, ln, cp, rm, mv, pax, pathconf, echo and read.
Some members volunteered test assertions written by their companies,
usually to a previous draft. They were almost always unusable, either
because they were out of date (based on previous drafts), or of poor
quality. Writing good test assertions is very difficult, and quickly
points out (the many) ambiguous wordings in the draft.
The POSIX.3.2 group would like to go to a mock ballot after the
October meeting in Parsippany, New Jersey.
Volume-Number: Volume 23, Number 97randall@Virginia.EDU (Randall Atkinson) (06/12/91)
Submitted-by: randall@Virginia.EDU (Randall Atkinson) In article <1991Jun12.034606.16284@uunet.uu.net>, Peter Collinson <pc@hillside.co.uk> writes: >There was a problem with the Draft 11 distribution. POSIX.2 is now >over 900 pages. It's balloting period was 30 days, although with a >mailing lead it was about 6 weeks. Due to postal services, some >members of the balloting group only received their ballot copies two >weeks prior to the closing deadline. Hal Jespersen was as >accommodating as he could be on the deadline, but the extent of these >submissions was definitely affected. The question rears its head >again. Should we be balloting POSIX standards the same as we ballot >smaller hardware standards? This is a very real problem. I ended up abstaining on both the Ada-bindings and Threads ballots because I didn't have sufficient time to review the lengthy &/or complex ballots in the time allotted. It seems to me that the SEC should consider mandating that future ballots for the POSIX arena have a longer minimum balloting period than the IEEE requires. This would not conflict with the IEEE requirements and would get better standards in the final result. >New PARs > >The Project Management Committee (PMC) has recommended the proposed >PAR (Project Authorization Request) for 1003.2b be split into two >parts. One part will cover extensions and new requests from other >groups, such as the tar, cpio and new pax file formats from POSIX.1 >(Kernel) and utilities from POSIX.4 (Realtime) and POSIX.6 >(Security). The timing and alignment issues with the ISO 9945-2 >balloting process will be covered by the other part. > >The scope of this second PAR is restricted to standardization of >existing industry practice. The group does not want to get into >designing new utilities. Such a scope is the only appropriate one. In particular the experience of other standards (e.g. ISO Network Protocol interoperability problems) have made it clear that it is best to standardise only existing practice and let there be actual implemented experience with new utlitites or features before putting them in the standard. I am glad to see this made explicit in the POSIX.2b PAR as it has been an ongoing concern of mine (in particular with regard to windowing interfaces that there isn't enough experience with). >There is also concern over draft stability when discussing the >commands to access the features from the POSIX.4 and POSIX.6 >standards. How mature does the feature have to be before the POSIX.2 >group goes to the effort of defining a command interface to it ? It seems to me that if a draft hasn't even reached balloting that it is not sufficiently stable for the POSIX.2 WG to make much effort devising a suitable command interface for it. There have been a lot of cases where changes occurred in balloting as well, so it might not even be stable enough at ballot-time. >Discussion >Richard Hart from POSIX.7 (System Administration) presented the syntax >for a new printing command based on the MIT/Athena Palladium network >printing services. Everyone in the POSIX.2 group agreed that the >proposed syntax was awkward. The posted examples are pursuasive. The printing commands should all be based on widespread existing practice in UN*X systems. The MIT/Athena Palladium services are not widespread enough to justify their adoption. >Requests for technology > >There is an opportunity now to propose a new archive format. Why is yet another archive format needed ?? Volume-Number: Volume 23, Number 103
brnstnd@KRAMDEN.ACF.NYU.EDU (Dan Bernstein) (06/16/91)
Submitted-by: brnstnd@KRAMDEN.ACF.NYU.EDU (Dan Bernstein) In article <1991Jun12.034606.16284@uunet.uu.net> pc@hillside.co.uk (Peter Collinson) writes: > The group is also looking for a new family of compression utilities, > now that the Lempel-Ziv-Welch family of commands have been removed > from the standard. The main requirements for a substitute are: > o The algorithm should be expressed (expressible) in a > language independent form > o The algorithm should be free of patent issues My compression method, Y coding, appears to be free of all patents, including LZW. There's nothing language-dependent about the method or current file format. My reference implementation, yabbawhap (see comp.sources.unix volume 24), works on a variety of UNIX and non-UNIX systems, and produces better results than compress at reasonable speed. Another possibility is Ross Williams' LZRW1, which produces worse results than compress but is blindingly fast. ---Dan Volume-Number: Volume 24, Number 9