[net.news] getting rid of Orphaned Response

phil@amd.UUCP (Phil Ngai) (10/15/84)

Here's an idea I just had. My understanding of the reason news users
get articles with "Orphaned Response" for a subject line is that
when a msg comes into a notes system, it gets sent out with
an "Orphaned Response" subject line if the parent (base) note isn't
there. When the base note does come in, then users on the notes system
get the correct subject line but their news neighbors don't edit the
followup article's subject line even when the parent article comes in.

One approach would be for notes to hold the followup until the base
note came in, then the followup would have the correct subject.

Another approach would be for news to go edit the followup article's
subject line when the base note came in.

Finally, notes could avoid stripping the subject line on the followup
when it entered the notes system. I assume that the followup has the
correct subject line at some point because it had to be generated as
a response to a base note and both news and notes can do that correctly.

Then again, I could be all wet in my understanding of notes.

It would help if people would run a modern version of uucp that
sends things out in the order generated.

from the rice eater
-- 

 Phil Ngai (408) 982-6554
 UUCPnet: {ucbvax,decwrl,ihnp4,allegra,intelca}!amd!phil
 ARPAnet: amd!phil@decwrl.ARPA

gordon@sneaky.UUCP (10/19/84)

> Here's an idea I just had. My understanding of the reason news users
> get articles with "Orphaned Response" for a subject line is that
> when a msg comes into a notes system, it gets sent out with
> an "Orphaned Response" subject line if the parent (base) note isn't
> there. When the base note does come in, then users on the notes system
> get the correct subject line but their news neighbors don't edit the
> followup article's subject line even when the parent article comes in.
> 
> One approach would be for notes to hold the followup until the base
> note came in, then the followup would have the correct subject.

Contrary to some of the wording in the notes documentation, Orphaned
Responses rarely occur because "the response arrives before the base
note".  Notes does not have the equivalent of the "L" flag.  In order
to respond to a note, the note has to be at the system the response is
entered on.  From then on, the response should travel to other systems
along with or behind the base note, since if a system is going to send 
the response, it will also send the base note along with it, unless
it already sent it there earlier, the base note was written on the
destination system, or the base note was received from the destination.
The exceptions all supposedly mean that the destination already has
the base note.

There are two main reasons I know of for Orphaned responses:

Either the base note is never going to get there, because uucp or something
else trashed it, or the base note was there, but got expired before the 
response arrived.  

If everyone expires notes at two weeks, and it takes 8 days for a note 
to go across the notes net, and the response takes 8 days to get back, the 
original note will have vanished on the system it was posted on, assuming
nobody else replied (Notes expires notes based on the arrival date of the 
note or latest response.  Even with a 2-week expiration date, hot topics 
in net.flame with lots of responses can stick around for months).  The 
response shows up there as orphaned.  If that system or one close is a 
gateway, it goes into news orphaned.  Flames sent to the author of the 
response about writing an orphaned response are not going to help any, 
because the author has no idea it went out that way, nor can he/she control 
it.  

Lots of the notes systems do not run news at all.  I THINK I know who 
gateways my stuff into news, but I can't prove it.  I am not absolutely
sure that anyone does.  This also means that if my article is gatewayed by 
very dumb gateway software, I can't change it by updating my site.
(Someone want to send me a copy of this article's headers, once it gets
into news?)

Orphaned responses will occur momentarily if uucp manages to shuffle the
order of transmission of batches of notes.  However, unless "newsoutput"
(the notes -> news gateway program that calls inews) is run between the 
arrival of the second and first batch, these won't be transmitted into the 
news system as an "Orphaned Response", they will be transmitted with the 
proper title.  I doubt anyone runs newsoutput more than once an hour.
If I had to guess, I would estimate about 4 times a day.

> Another approach would be for news to go edit the followup article's
> subject line when the base note came in.

This sounds incredibly messy.  You would have to keep track of whether the
article had been orphaned, but wasn't any more, and whether the orphan had
been sent into news (no point editing if it wasn't).  You would need some way
to find the article in the news system.  Also, it would tend not to work, 
given the existing nature of news.  If inews is queueing a copy of the 
article immediately, (no batchers or U flag), then editing the header in the
gateway system would fix it for that system only.  News has no provision
for sending an update to an article using the same article-id, it would
just get dumped as a duplicate article.  If batchers or the U flag are 
being used, then the base note needs to show up before anyone transmits a 
batch or picks up the articles.  Same problem, except a bit longer window 
during which the title could be properly inserted.

Somehow I doubt that sending out a cancel message and transmitting a new 
copy of the message would be effective.  (Cancel messages tend to be 
ineffective for the same reason that updating an article would be:  you 
can take the article out of the news directory, but you can't zap the copy 
already in the uucp spool directory.  If the cancel message manages to get 
ahead of the article, it will cancel nothing, and then the article shows 
up, and stays on the system.) Mostly people would complain about seeing 
articles twice.  

> Finally, notes could avoid stripping the subject line on the followup
> when it entered the notes system. I assume that the followup has the
> correct subject line at some point because it had to be generated as
> a response to a base note and both news and notes can do that correctly.

The notes format does not contain a place for a subject line in responses.
Unless you want it duplicated in the text, which is almost as bad as
the "#" line notes used to (I think the new gateway stuff fixed this)
stick into the text of articles, there just isn't a place to put it.
All it has is a pointer to the parent article.  Changing this would cause 
considerable compatability problems and disruption because systems 
continued to run old versions.  Does this sound familiar to anyone in
news-land?  What would happen if, to meet a new standard, every news system
had to add a Line-Eater-Bug-Not-Present-At: line, or the article would
get dumped at the next site down the line?  What happens if your neighbor
insists on running news 2.6?

Also, the title is redundant information, and increases transmission time 
and disk storage.  Does news want to add Parent-Article-Subject: headers?
Also, nobody has time to modify the code, and various other excuses.

> Then again, I could be all wet in my understanding of notes.

Well, your suggestions weren't unreasonable for someone not familiar with
the innards of notes.

> It would help if people would run a modern version of uucp that
> sends things out in the order generated.

If all the files get transmitted in the same call, I don't think that
this problem causes many Orphaned Responses.  It is irritating to see 
News-originated articles with the response article immediately preceeding
the parent, (notes can't pair up an article and a reply if it wasn't a
followup, or the gateway software is stupid) but that's a different problem.
It bothers notes and news users alike.

>  Phil Ngai (408) 982-6554
>  UUCPnet: {ucbvax,decwrl,ihnp4,allegra,intelca}!amd!phil
>  ARPAnet: amd!phil@decwrl.ARPA


				Gordon Burditt
				convex!ctvax!trsvax!sneaky!gordon
				microsoft!trsvax!sneaky!gordon

crl@pur-phy.UUCP (Charles LaBrec) (10/22/84)

We once ran notes, but dropped it when it became too much of a hassle
to keep up.  (I won't go into the reasons now, because all this does
is promote a flaming session.  Let me just say that we felt the huge
advantage then of the user interface was not greater than the other
disadvantages.)  I make this statement to show that I at one time had
knowledge of the internals of notesfiles.

Now, I just wanted to address the problem of orphaned responses.  I see
no problem with eliminating these titles.  When an "orphaned" response
comes in, notesfiles generates a "fake" basenote for it to be a response
of (at least, this was my understanding back then).  Therefore, it should
be possible to create a "fake" basenote, with a "proper" title.

As a side note, now with the advent of "rn", I REALLY see no reason
to run notesfiles.  Rn gives you everything you want, plus more.  Plus,
it is completely compatible with the news system used in the majority
of sites, since it is only a news reader, and relies on inews, etc., for
the hard work.  Hat's off to Larry Wall!!!

Charles LaBrec
UUCP:		pur-ee!Physics:crl, purdue!Physics:crl
INTERNET:	crl @ pur-phy.UUCP

gordon@sneaky.UUCP (10/25/84)

> Now, I just wanted to address the problem of orphaned responses.  I see
> no problem with eliminating these titles.  When an "orphaned" response
> comes in, notesfiles generates a "fake" basenote for it to be a response
> of (at least, this was my understanding back then).  Therefore, it should
> be possible to create a "fake" basenote, with a "proper" title.

Have you got a program that takes the text (note: NO HEADERS) of a response,
who originated it, when it was posted, what group it's in, and a notes
id number of the base note, which is NOT available for reference, and
generates a title?  The transmission of a response does NOT include a title.  
If it did, the problem would never have appeared in the first place.
I'd be interested in seeing a program that assigns titles.  It might even do a 
better job than the humans and AI projects do giving titles to their 
own articles.  Or is it your suggestion that "Re: Orphaned Response" simply 
be changed to "Re: 'Proper' Title"?  How about a Random Title Generator?

I can think of a few degenerate cases where you could generate a reasonably 
appropriate title.  In net.general, ALL titles should be of the form 
"Inappropriate posting from kremvax!kgb", since I have yet to see anything of 
truly general interest in net.general.  Most of Mark Horton's stuff comes
close, but it still doesn't fit.  I can't even think of anything that 
IS of general interest to such a varied audience.  No, not even "Nuclear War 
Destroys World", or "USENET confiscated for copyright violations".  In 
net.flame, most titles could just as well be "Yet more drivel from 
myvax!umberhulk".

> As a side note, now with the advent of "rn", I REALLY see no reason
> to run notesfiles.  Rn gives you everything you want, plus more.  Plus,
> it is completely compatible with the news system used in the majority
> of sites, since it is only a news reader, and relies on inews, etc., for
> the hard work.  Hat's off to Larry Wall!!!

It's rather difficult to run news if you don't have any news feeds within
affordable distance (local call in 817 area code), because everyone else is 
running notes, and all you have to interface news with is the really old 
and horrible notes/news interface.  (Why isn't news compatable with the 
notes transfer format? :-) ) Running both takes up a lot of disk space.  
The notes/news interface requires some hacking to make it ship stuff that 
came from news back into it, because it doesn't understand that this news 
doesn't have anyone else to connect to.  (Offers of long-distance news 
connections appreciated but not accepted, I can't afford the phone bills).  
Also, my version of rn still dumps core occasionally.  Larry Wall did a 
fantastic job in trying to make rn portable, and from what I have seen, the 
user interface is excellent, too.  I think there are a few bugs still lurking 
around, and I'm still not sure it beats notes.  I don't have it working to 
the point I can believe it really works grouping topics together as well as 
notes does.

There are a few things I'd really like to know how to do with news.  Supposing
I had several news connections, and I get mail from one of them saying,
"My disk overflowed.  Please resend everything from last night on.".  This
is likely to happen often, considering the small disks on many of the systems
around here.  How would I do this?  And how do I get around the 65535 inodes 
per filesystem restriction in v7 or 4.1bsd, which news is likely to keep 
exceeding if I try to save certain groups for a long time?  I'm sure someone 
has gotten around these, but I haven't learned all the tricks that aren't 
stated in the documentation, but well known to veteran news administrators.

> Charles LaBrec
> UUCP:		pur-ee!Physics:crl, purdue!Physics:crl
> INTERNET:	crl @ pur-phy.UUCP

				Gordon Burditt
				convex!ctvax!trsvax!sneaky!gordon
				microsoft!trsvax!sneaky!gordon

crl@pur-phy.UUCP (Charles LaBrec) (10/29/84)

>Have you got a program that takes the text (note: NO HEADERS) of a response,
>who originated it, when it was posted, what group it's in, and a notes
>id number of the base note, which is NOT available for reference, and
>generates a title? The transmission of a response does NOT include a title.  
>If it did, the problem would never have appeared in the first place.

That's true, I had forgotten about the case where notesfiles sites transfer
articles between one another in notesfiles format.  However, I think that
the majority of the problem occurs at the notes/news interfaces, since I
feel that it is extremely simple for replies to arrive before the basenote
from news rather than notes.  It could at least be fixed at these points.

>It's rather difficult to run news if you don't have any news feeds within
>affordable distance (local call in 817 area code), because everyone else is 
>running notes, and all you have to interface news with is the really old 
>and horrible notes/news interface.

True, if you have no choice, you're stuck.  We kept both versions around
for about a year, and two copies of the junk is just too much wasted disk
space.

>Also, my version of rn still dumps core occasionally.  Larry Wall did a 
>fantastic job in trying to make rn portable, and from what I have seen, the 
>user interface is excellent, too. I think there are a few bugs still lurking 
>around, and I'm still not sure it beats notes.  I don't have it working to 
>the point I can believe it really works grouping topics together as well as 
>notes does.

Bugs #1-#19 have been posted that fix most of the problems found.
(One more is lurking, waiting for the "official" fix.  Raise the declaration
of filename[] in filexp() in intrp.c to LBUFLEN from MAXFILENAME until
something better comes along.)  We run on a PDP-11/44 and right now rn is
very stable.  As to whether it beats notesfiles, I think so, but that's
just my opinion.

>				Gordon Burditt
>				convex!ctvax!trsvax!sneaky!gordon
>				microsoft!trsvax!sneaky!gordon


Charles LaBrec
UUCP:		pur-ee!Physics:crl, purdue!Physics:crl
INTERNET:	crl @ pur-phy.UUCP

berry@zinfandel.UUCP (Berry Kercheval) (10/31/84)

In article <-1025373@sneaky.UUCP> gordon@sneaky.UUCP writes:
>
>Have you got a program that takes the text (note: NO HEADERS) of a response,
>who originated it, when it was posted, what group it's in, and a notes
>id number of the base note, which is NOT available for reference, and
>generates a title?  The transmission of a response does NOT include a title.  
>If it did, the problem would never have appeared in the first place.

Yes, transmission of a response from a notes system to anther notes system
does not include a title.  That is not the problem here!  

The problem is that when a response goes out into the NEWS world, it has a 
title "Re: Original Subject - (nf)".  When it goes into another NOTES
system, newsinput (the news/notes gateway program) says to itself,
"Aha!  This news article is a NOTES response.  I don't have a base note for it
and will create an empty one.  Since this is an orphaned response, I will title
the new empty base note "Orphaned Response" and THROW AWAY THE PERFECTLY GOOD
TITLE I ALREADY HAVE"

This is very easy to fix.  I can send diff listings to any system
administrators who request them.  

-- 
Berry Kercheval		Zehntel Inc.	(ihnp4!zehntel!zinfandel!berry)
(415)932-6900

essick@uiucdcsb.UUCP (11/01/84)

A new version of the notesfile/news gateway programs "newsinput" and
"newsoutput" written mostly by Tw Cook at HP and Lou Salkind at NYU
manages to keep the titles.

newsinput was originally a quick, cheap and very dirty (probably even
filthy) program to "get the job done".  It worked with A-news format
and made assumptions about things arriving in order. "Newsoutput" was
almost as bad (although it was more coherent).

The new version is much better organized, understands and generates
USENET compatible headers, and doesn't through away as much information
as it used to.  Tw and Lou did a wonderful job of fixing it up.

I will probably be posting the current notesfile source & documentation
to net.sources in the next month.

-- Ray Essick, University of Illinois