[net.unix-wizards] Undocumented features

ptw@vaxine.UUCP (P. T. Withington) (08/25/83)

We serendipitously discovered that 4.1BSD find(1) implements the -cpio switch
documented in System III.  Lucky us, we had a cpio to read the tapes back with
too!

			     't`   --Tucker (ptw@vaxine.UUCP)
			      ~

guy@rlgvax.UUCP (Guy Harris) (08/27/83)

Actually, even the V7 "find" has the "-cpio" option - interesting, because
if you don't have a PWB descendant you don't get "cpio" and, as you point out,
once you've made the tape the only thing you can do with it is give it to
somebody who *does* have "cpio".  But if you need to make a "cpio" tape and
you don't have "cpio" but do have a V7 or later "find", there's the way to
do it...

	Guy Harris
	{seismo,mcnc,we13,brl-bmd,allegra}!rlgvax!guy

rcd@opus.UUCP (05/25/84)

Guy Harris, commenting on another netter's discovery that a needed feature
actually exists but is undocumented:
>Yup.  Welcome to the wonderful world of UNIX; a treasure hunt each day.
>The annoying thing is that some of the stuff are real treasures, but the
>authors have seen fit to bury them deeply...

But look things over carefully before you use them.  I remember chasing
fopen(3) in V7 - I wanted to be able to open a file for both reading and
writing, so I set out to add code to do this.  Turned out that the code was
already there - in addition to the documented "r", "w", and "a" options,
you could add "+" to get read/write, as in "w+".  However, it didn't
work...so was it undocumented because whoever wrote it not only blew it but
didn't document it, or (more likely) because it wasn't done at the time the
system was shipped?  In any case, I had only myself to blame for using an
undocumented feature.
-- 
...Stop to smell the flowers.				Dick Dunn
{hao,ucbvax,allegra}!nbires!rcd				(303) 444-5710 x3086

hansen@pegasus.UUCP (05/26/84)

When I saw the note about the file sync'ing undocumented feature, I thought
"Great! The people we have working on databases may have seen this, but if
they haven't, I'll pass the note on to them." They're replies:

> Person A:
> 
> Person B will correct me if I'm wrong, but I believe this is a bit we had
> already heard about.  It would be quite useful, except for one minor
> problem - it was put in for kernel use, and while it writes the data block
> synchronously, it does NOT write the inode before returning to the user.  It
> was too bad, we thought we had found something useful before Person B spent
> a few hours on the phone to [the USG UNIX people].
> 
> Person B:
> 
> Person A is correct about the utility (or lack thereof) of this feature.
> Thanks for the information though.

So BEWARE! If you use this "undocumented feature". There's a reason for it
being undocumented!

					Tony Hansen
					pegasus!hansen

guy@rlgvax.UUCP (Guy Harris) (05/28/84)

> Take for instance fseek(3S) in 4.1 (same thing or very similar in system III):
> 	....
> 	Rewind(stream) is equivalent to fseek(stream, 0L, 0).

> 	SEE ALSO
> 		lseek(2), fopen(3)

> 	DIAGNOSTICS
> 		Fseek returns -1 for improper seeks.

> System III documentation says that fseek returns non-zero for improper seeks,
> zero otherwise. The truth of the matter is: rewind will return negative on
> PROPER seeks. If you want to test your rewind status, use fseek(stream, 0L, 0).
> This difference hardly makes rewind "equivalent" to fseek(stream, 0L, 0).

The S5 documentation clears up this; it says "Rewind(stream) is equivalent...
except that no value is returned."  Better late than never, I guess (grumble,
grumble, grumble)...

By the way, while we're on the topic of undocumented features:

In the System III "TTY(4)" manual page, it mentions the TCXONC "ioctl";

	TCXONC	Start/stop control.  If "arg" is 0, suspend output;
		if 1, restart suspended output.

They don't tell you that if "arg" is 2, it suspends *input* (i.e., sends an
XOFF to the other side), and if "arg" is 3, it restarts suspended *input*
(i.e., sends an XON).  If they didn't document it because it didn't work,
fine, *except that the code is still there in System V* and it's pretty
simple - if it doesn't work, "IXOFF" mode (what was called "tandem" mode
in V7) probably doesn't work either.  And no, it's still not documented in
System V (quite aside from the fact that the tty driver isn't documented
*at all* in the S5 UNIX User's Manual - it's only documented in the
*Administrator's* Manual.  Obviously, nobody but a system administrator needs
to know the characteristics and device-dependent "ioctl"s for the various
devices attached to your machine, right?  No mere programmer should be allowed
to write a screen editor... The (second) reorganization of the UPM/UUM
in System V Release 2 doesn't correct this botch - there's now a User's Manual
(mainly section 1), a Programmer's Manual (mainly sections 2 and 3), and
an Administrator's Manual (the system maintenance stuff from the other manuals).
Unfortunately, section 7 (which used to be section 4) - devices - is *still*
in the Administrator's Manual.  Put it back in the Programmer's Manual, where
it belongs.  *NOW*.

	Guy Harris
	{seismo,ihnp4,allegra}!rlgvax!guy

guy@rlgvax.UUCP (Guy Harris) (05/28/84)

> When I saw the note about the file sync'ing undocumented feature, I thought
> "Great! The people we have working on databases may have seen this, but if
> they haven't, I'll pass the note on to them." They're replies:

> > Person A:
> > 
> > Person B will correct me if I'm wrong, but I believe this is a bit we had
> > already heard about.  It would be quite useful, except for one minor
> > problem - it was put in for kernel use, and while it writes the data block
> > synchronously, it does NOT write the inode before returning to the user.  It
> > was too bad, we thought we had found something useful before Person B spent
> > a few hours on the phone to [the USG UNIX people].
> > 
> > Person B:
> > 
> > Person A is correct about the utility (or lack thereof) of this feature.
> > Thanks for the information though.

> So BEWARE! If you use this "undocumented feature". There's a reason for it
> being undocumented!

This is correct.  The problem is that "writei" calls "bwrite" instead of
"bdwrite" if the FSYNC flag is set in the file descriptor, *but* that's
not enough.  If the B_ASYNC (asynchronous write - "bawrite") or B_DELWRI
(delayed write - "bdwrite") flag is already set in the buffer, the write
will be treated as an asynchronous or delayed write.  For this to work,
you'd have to clear both those flags in the buffer before "bwrite"ing it.

The FSYNC bit is used only when writing superblocks in "update", and directory
entries in "unlink" (to make sure the directory entry is reamed out before
the inode it refers to is); since 1) you can't open a directory for writing
and 2) you obviously aren't going to "fsck" a cooked device corresponding
to a mounted file system, presumably that block of the file system will only
be written with a "bwrite".  Unfortunately, it ain't so; a "link" system
call calls "wdir" which does a "bdwrite".  This isn't a problem for "link",
as the S5 "link" code makes sure the inode is written to disk before writing
the directory entry, but could surprise a later "unlink" if that block
remains unwritten in the cache with B_DELWRI on.  4.2BSD (and, I believe,
4.1BSD, whose file system is the same V7 file system as S3 and S5 use)
flatly says "ALL WRITES TO DIRECTORY FILES WILL BE SYNCHRONOUS.  PERIOD."
As such, I'd vote for S5 turning off any B_ASYNC or B_DELWRI bits if the
descriptor has the FSYNC flag set, and then making the FSYNC flag a documented
and official bit with an O_FSYNC flag for "open" and "fcntl".  The only side
effect might be occasional performance degradation on directory I/O (less
overlap), but more file system integrity *and* the ability to provide database
integrity.  A pretty good tradeoff, in my opinion.  Besides, any such overlap
due to a directory being (mistakenly) written with a "bdwrite" is an accident
anyway.

	Guy Harris
	{seismo,ihnp4,allegra}!rlgvax!guy

lied@ihlts.UUCP (Bob Lied) (05/30/84)

[]
	"It is the glory of God to conceal a thing;
	 but the honour of kings is to search out
	 a matter."
		- Proverbs 25:2

guy@rlgvax.UUCP (Guy Harris) (06/01/84)

> Guy Harris, commenting on another netter's discovery that a needed feature
> actually exists but is undocumented:
> >Yup.  Welcome to the wonderful world of UNIX; a treasure hunt each day.
> >The annoying thing is that some of the stuff are real treasures, but the
> >authors have seen fit to bury them deeply...

> But look things over carefully before you use them.  I remember chasing
> fopen(3) in V7 - I wanted to be able to open a file for both reading and
> writing, so I set out to add code to do this.  Turned out that the code was
> already there - in addition to the documented "r", "w", and "a" options,
> you could add "+" to get read/write, as in "w+".  However, it didn't
> work...so was it undocumented because whoever wrote it not only blew it but
> didn't document it, or (more likely) because it wasn't done at the time the
> system was shipped?  In any case, I had only myself to blame for using an
> undocumented feature.

Oh yes, there are definitely zingers like that.  One real prize is the
"substr", "length", etc. functions in the "expr" command.  I first found
them in V7, where "expr" was YACC-based.  Then I looked at the S3 version,
which had been rewritten not to use YACC, but the rewriter took care to
preserve all those undocumented (and useful) functions.  (Or maybe the
S3 version came first, from UNIX/TS 1.0 vintage perhaps, and the V7 version
came later.  Whatever.)  Then I looked at the S5 version; guess what had
disappeared!  I suspect the problem was that that feature made "substr", etc.
reserved words and that tripped up some shell files.  (Anybody know the
real reason they disappeared?  Some of them were quite useful...)

The V7 version of "[rwa]+" worked if the (documented) S3 version did, except
if your umask took away your own read or write permissions, because the V7
and S3 code was the same except for a fix in S3 for a bug where such a umask
would prevent stdio from reopening the file after creating it.  (They didn't
remember that using the USG (and 4.2BSD) "open" you can create a file and
have the resulting descriptor open for reading and writing (and about damn
time, too!) until S5.)  The 4.xBSD (and possibly 2.xBSD) versions would have
worked, too, except that there was a bug in "fseek" (fixed in 4.2) that
screwed you if you had a stream open for reading and writing.  (The fix
has been posted on several occasions.)

Of course, just because something is a documented feature doesn't guarantee
that it works either...

By the way, the "defined()" operator and the __LINE__ and __FILE__ pseudo-
macros in the BTL UNIX C preprocessor, which have been around since V7,
are now documented in System V so they're safe to use.

	Guy Harris
	{seismo,ihnp4,allegra}!rlgvax!guy

dave@rlgvax.UUCP (Dave Maxey) (06/03/84)

As Guy Harris put it:
> Of course, just because something is a documented feature doesn't guarantee
> that it works either...

Take for instance fseek(3S) in 4.1 (same thing or very similar in system III):
	....
	Rewind(stream) is equivalent to fseek(stream, 0L, 0).

	SEE ALSO
		lseek(2), fopen(3)

	DIAGNOSTICS
		Fseek returns -1 for improper seeks.

System III documentation says that fseek returns non-zero for improper seeks,
zero otherwise. The truth of the matter is: rewind will return negative on
PROPER seeks. If you want to test your rewind status, use fseek(stream, 0L, 0).
This difference hardly makes rewind "equivalent" to fseek(stream, 0L, 0).

			- Dave Maxey (alias tbm)
			{seismo,mcnc,brl-bmd,allegra}!rlgvax!dave