[news.software.b] Truncated postings

paul@devon.UUCP (Paul Sutcliffe Jr.) (02/28/88)

[ Originally posted in comp.sys.ibm.pc ]

In article <3816@ihlpf.ATT.COM> gmd1@ihlpf.ATT.COM (Doughty) writes:
> In article <597@slvblc.UUCP>, dick@slvblc.UUCP (Dick Flanagan) writes:
> 
> > The first fourteen parts of this posting consist of 800 uuencoded
> > lines.  Part 15 consists of 52 uuencoded lines.  Part 1 also contains
> > a 'begin' line, and Part 15 also contains an 'end' line.
> 
> Parts 2 and 8 arrived at out site truncated to 792 and 761 lines respectively.
> Did anyone receive them intact?

[ Of course, we see lots of these kinds of postings every week. ]

My site also received the referred-to articles truncated.  Since the
news software and transport mechanisms normally pass articles along
without munging them, something must occasionally cause this.  In the
above case, the missing lines are somewhere in the middle of the
article (determined by the fact that the end of each short article
contained the same .signature as the rest of the parts posted by the
same person).  So the truncation is not just "chopping off the end"
of an oversized message.

My theory is that the problem occurs when inews is processing inbound
news and runs out of space.  [Note: I have not taken the time to read
the 2.11 source to see if this is true.]  For example, someone at
site "a" posts an article.  Site "a" passes it on to site "b".  While
inews is unbatching the article (on site b), it encounters a write
error, probably ENOSPC.  Then, while it is busily logging the write
error (which may fail too if the errlog file is on the same filesystem
as SPOOLDIR, but I digress), another process does "something" to free
up some space.  Inews, back from logging the first error, continues to
write the rest of the message successfully, since there is now room.
But what happened to the buffer full of text that caused the write
error in the first place?  Lost in the ozone layer, perhaps?  This
could explain why an article would have it's beginning and end intact,
but would be missing stuff from the middle.

Again, I cannot say that this is inews' behavior.  My theory is based
on observations from my own system when, yesterday, inews ran out of
space in the /usr/spool filesystem here.  While attempting to clean
up the damage, I ran an "expire -r" to rebuild the history file (as
some of the error messages said "history write failed").  Expire
reported several "Garbaged article in ..." while running.  I've traced
these (before) to articles with zero length or badly munged headers.

I guess the correction to this could be either of two things.  One,
have inews retry the failed write (after sleeping) until no error
is reported, or two (better), have inews keep an eye on free space
and stop processing when space gets low -- BNU (HoneyDanBer) UUCP does
this.

Discussion anyone?

- paul

-- 
Paul Sutcliffe, Jr.				       +----------------------+
						       |  THINK ...           |
UUCP (smart):  paul@devon.UUCP			       |            or THWIM  |
UUCP (dumb):   ...rutgers!bpa!vu-vlsi!devon!paul       +----------------------+