[news.admin] References line

jeffery@jsheese.FIDONET.ORG (Jeff Sheese) (05/11/89)

After a quick (read - not thorough) browse through RFC1036, the following  
excerpts make me ask if the 'References' is really required by RFC1036.  Now  
I'm not questioning the need or usefulness of the line, but I read this as it  
being an optional header line.  If it is indeed optional, at least until  
required by an updated RFC, does this warrant shutting out registered systems  
from netnews?

Excerpts follow:

2.  Message Format
    Certain headers are required, and certain other headers are
    optional.  Any unrecognized headers are allowed, and will be passed
    through unchanged.  The required header lines are "From", "Date",
    "Newsgroups", "Subject", "Message-ID", and "Path".  The optional
    header lines are "Followup-To", "Expires", "Reply-To", "Sender",
    "References", "Control", "Distribution", "Keywords", "Summary",
    "Approved", "Lines", "Xref", and "Organization".  Each of these
    header lines will be described below.

2.2.  Optional Headers
2.2.3.  Followup-To
2.2.5.  References
    This field lists the Message-ID's of any messages prompting the
    submission of this message.  It is required for all follow-up
    messages, and forbidden when a new subject is raised.


--  
Jeff Sheese - via FidoNet node 1:109/116
UUCP: ...!netsys!jsheese!jeffery
ARPA: jeffery@jsheese.FIDONET.ORG
(I am sole owner.  My opinions represent my company.)
(Send all flames to null@jsheese.Fidonet.ORG)

brad@looking.UUCP (Brad Templeton) (05/12/89)

No, the References: line is not required in an article -- for one reason
original 'root' articles aren't supposed to have it.

All I am pointing out is the non-obvious fact that once you write software
to deal with the References line, you find out that it is all or nothing,
like the 'great renaming'.   If 5% of sites refused to do the great renaming,
then they would have either had to leave the net or the great renaming would
not have worked.

The same is true with References -- Almost all sites have to put out
References or References will not work.  I am not saying it is utterly
crucial that References work, but I do think it is a good idea, and I
think that most people also feel this way.  We have been supporting it for
5 years to no end and it still can't be used to kill a chain of articles.

Almost nothing on usenet is required other than agreement on an article
format.  In fact, in many ways the definition of Usenet is 'sites that
use the following article format.'   By and large what people do on
their own sites is their own business -- the one exception is this format.
It's the only thing that we have to agree to as a group -- even the newsgroup
list (which we've argued so much about) doesn't *have* to be done that way.

There is no rule, for example, against setting up a gateway so that every
message is entitled "Bitnet mail follows," to give an example from the
past.  But it really ticks people off, and shows up in kill files.

In general, the decision of what to read should be up to the reader, but
to do this by software, the articles must be well and correctly classified.
-- 
Brad Templeton, Looking Glass Software Ltd.  --  Waterloo, Ontario 519/884-7473

lear@NET.BIO.NET (Eliot Lear) (05/12/89)

From RFC 1036:

2.1.4.  Subject

    The "Subject" line (formerly "Title") tells what the message is
    about.  It should be suggestive enough of the contents of the
    message to enable a reader to make a decision whether to read the
    message based on the subject alone.  If the message is submitted in
    response to another message (e.g., is a follow-up) the default
    subject should begin with the four characters "Re:", and the
    "References" line is required.  For follow-ups, the use of the
    "Summary" line is encouraged.
-- 
Eliot Lear
[lear@net.bio.net]

jeffery@jsheese.FIDONET.ORG (Jeff Sheese) (05/12/89)

In an article of <11 May 89 17:09:22 GMT>, brad@looking.UUCP (Brad Templeton) writes:

 >All I am pointing out is the non-obvious fact that once you write 
 >software to deal with the References line, you find out that it is all or
 >nothing, like the 'great renaming'.   If 5% of sites refused to do the 
 >great renaming, then they would have either had to leave the net or the 
 >great renaming would not have worked.

In that case I have a suggestion.

I'm reading and entering news on an IBM PC/AT compatible using the UFGATE  
version 1.03 software and MSGED.  UFGATE is a gateway between Fidonet and  
Usenet.  MSGED is a Fidonet (FTSC) compatible message editor.  Simply put,  
UFGATE converts my messages to news messages, and incoming news to FTSC  
messages.  Since I don't have the source code for UFGATE, I can't change it.   
Since it's shareware, I'm not gonna hound the authors to change it overnight.

However, the editor I use has the capability of putting message lines at the  
header of every message.  It's completely user configurable, and the source  
code is available for personal modification.

Now the kicker.  When I export an FTSC message from MSGED to usenet using  
UFGATE, the headers are automatically added.  Everything else is considered  
the body of the message, therefore a blank line is added between the header  
lines added from UFGATE.  Anything I add at the top of the message appears  
after this blank line.

If RFC1036 were changed to relax the restriction of all the header lines  
appearing before the blank line, I could be compatible by configuring my  
editor to place these lines in there automatically.

I'm not suggesting that everyone change to meet my specifications, but I'm  
sure that I'm not running the only message editor that's so user configurable.
 
--  
Jeff Sheese - via FidoNet node 1:109/116
UUCP: ...!netsys!jsheese!jeffery
ARPA: jeffery@jsheese.FIDONET.ORG
(I am sole owner.  My opinions represent my company.)
(Send all flames to null@jsheese.Fidonet.ORG)

peter@ficc.uu.net (Peter da Silva) (05/12/89)

In article <3236@looking.UUCP>, brad@looking.UUCP (Brad Templeton) writes:
> We have been supporting it for
> 5 years to no end and it still can't be used to kill a chain of articles.

If that was the only purpose for the references line your argument would
make sense. But the far more common use of the references line is to
find the parent of an article... and for the vast majority of articles
this works.

Like I said, you're throwing the baby out with the metaphor.
-- 
Peter da Silva, Xenix Support, Ferranti International Controls Corporation.

Business: uunet.uu.net!ficc!peter, peter@ficc.uu.net, +1 713 274 5180.
Personal: ...!texbell!sugar!peter, peter@sugar.hackercorp.com.

wcf@psuhcx.psu.edu (Bill Fenner) (05/12/89)

In article <62.246A2DCC@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG (Jeff Sheese) writes:
|However, the editor I use has the capability of putting message lines at the  
|header of every message.  It's completely user configurable, and the source  
|code is available for personal modification.
|
|Now the kicker.  When I export an FTSC message from MSGED to usenet using  
|UFGATE, the headers are automatically added.  Everything else is considered  
|the body of the message, therefore a blank line is added between the header  
|lines added from UFGATE.  Anything I add at the top of the message appears  
|after this blank line.
|
|If RFC1036 were changed to relax the restriction of all the header lines  
|appearing before the blank line, I could be compatible by configuring my  
|editor to place these lines in there automatically.

That would take quite a lot of changing on most of USENET's part.  Maybe
a better thing would be to ask the UFGATE folks to allow misc. header lines
that are at the top of the article to pass through in the header.  How
do you tell a header line from something like
First: I don't ...
or whatever?  I don't know.  Maybe keep a list of all the ones allowed
by RFC1036, or something.

  Bill
-- 
   Bitnet: wcf@psuhcx.bitnet     Bill Fenner       | "Yesterday starts
  Internet: wcf@hcx.psu.edu                        |  tomorrow; tomorrow
 UUCP: {gatech,rutgers}!psuvax1!psuhcx!wcf         |  starts today"
Fido: Sysop at 1:129/87 (814/238 9633) \hogbbs!wcf |       -- Marillion

henry@utzoo.uucp (Henry Spencer) (05/12/89)

In article <62.246A2DCC@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG (Jeff Sheese) writes:
>If RFC1036 were changed to relax the restriction of all the header lines  
>appearing before the blank line, I could be compatible by configuring my  
>editor to place these lines in there automatically.

Uh, can you explain how headers would be distinguished from message text
then?  The requirement that headers appear before the blank line is not
a "restriction", it is the fundamental top-level syntax rule for messages,
and vast amounts of code depend on it.
-- 
Mars in 1980s:  USSR, 2 tries, |     Henry Spencer at U of Toronto Zoology
2 failures; USA, 0 tries.      | uunet!attcan!utzoo!henry henry@zoo.toronto.edu

jeffery@jsheese.FIDONET.ORG (Jeff Sheese) (05/13/89)

References: <1222@psuhcx.psu.edu>

In an article of <12 May 89 14:50:50 GMT>, wcf@psuhcx.psu.edu (Bill Fenner)    
writes:

 >That would take quite a lot of changing on most of USENET's part.  Maybe
 >a better thing would be to ask the UFGATE folks to allow misc. header 
 >lines that are at the top of the article to pass through in the header.
 >How do you tell a header line from something like
 >First: I don't ...
 >or whatever?  I don't know.  Maybe keep a list of all the ones allowed
 >by RFC1036, or something.

I'd have to refer back to RFC1036, but maybe those misc header lines have no    
more than one or two words followed by a colon.  Then the s/w can ignore    
anything past the blank line that follows.  How can the s/w tell the    
difference between a valid misc header line and 'First: I don't...' in the    
first line of the message?  Well as we say in Fidonet, RTFD and 'Caveat    
Emptor'.

I'll request the source for NEWSOUT just to see if I can make the necessary    
adjustments.  At least it will give me a starting point for the software    
necessary to make myself compatible.

OBTW - I entered a sample References line to the first line in the body of   
this message.  I'd like to request a few people to test it's validity with   
their news software.  Please respond via mail *only* if it works and I'll   
summarize.  If it doesn't work then don't respond.

In the meantime I'll examine the source for the PC implementation of SED to   
see if I can convince it NOT to put cr/lf pairs in it's output files.  Maybe   
that would be a quick kludge.
   
--  
Jeff Sheese - via FidoNet node 1:109/116
UUCP: ...!netsys!jsheese!jeffery
ARPA: jeffery@jsheese.FIDONET.ORG
(I am sole owner.  My opinions represent my company.)
(Send all flames to null@jsheese.Fidonet.ORG)

mju@mudos.ann-arbor.mi.us (Marc Unangst) (05/13/89)

In article <1222@psuhcx.psu.edu>, wcf@psuhcx.psu.edu (Bill Fenner) writes:
 >That would take quite a lot of changing on most of USENET's part.  Maybe
 >a better thing would be to ask the UFGATE folks to allow misc. header lines
 >that are at the top of the article to pass through in the header.  How
 >do you tell a header line from something like
 >First: I don't ...
 >or whatever?  I don't know.  Maybe keep a list of all the ones allowed
 >by RFC1036, or something.

I also use the UFGATE-and-MSGED combination, and have run into this problem,
too.  I've been lobbying for the UFGATE folks to allow you to add whatever
headers you want at the beginning of your message, followed by a blank
line, and then the body of the article.  If you didn't want to add any
header material, you would just make the first line of your message
blank.  (If you use the method you suggested (keep a list of headers
allowed), you would run into the problem of people who want to use
things like X-Phone: and X-Fax:.)

Unfortunately, because I have not had much luck convincing them to change
another annoying aspect of UFGATE's behavior(*), I doubt that this
change will happen soon.

(*) - UFGATE has a nasty habit of following the RFC-976 standard too closely.
For example, because every componant in an address with either "%" or "@"
in it is assumed to be a domain-ized host name (except the first componant,
which is assumed to be the user name), it insists on adding a period to
the end of "host1" in the address "user%host1@host2.with.domain", turning
it into "user%host1.@host2.with.domain".  This confuses sendmail to
no end, who (after querying the nameservers for "host1."), dutifully
bounces the mail.  John Galvin, the author of mailout (the mail exporter),
maintains that it is simply trying to make "host1" into a fully-qualified
domain-ized name.  Because of this behavior, I frequently have to add
".UUCP" to the end of a hostname, to prevent mailout from mangling it.

--  
Marc Unangst
UUCP smart    : mju@mudos.ann-arbor.mi.us
UUCP dumb     : ...!uunet!sharkey!mudos!mju
UUCP dumb alt.: ...!{ames,rutgers}!mailrus!clip!mudos!mju
Internet      : mju@mudos.ann-arbor.mi.us

jeffery@jsheese.FIDONET.ORG (Jeff Sheese) (05/13/89)

References: <1989May12.152728.20982@utzoo.uucp>

In an article of <12 May 89 15:27:28 GMT>, henry@utzoo.uucp (Henry Spencer)  
writes:

 >>In article <62.246A2DCC@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG 
 >>(Jeff Sheese) writes:
 >>If RFC1036 were changed to relax the restriction of all the header lines 
 >>appearing before the blank line, I could be compatible by configuring my 
 >>editor to place these lines in there automatically.
 >
 >Uh, can you explain how headers would be distinguished from message text
 >then?  The requirement that headers appear before the blank line is not
 >a "restriction", it is the fundamental top-level syntax rule for 
 >messages, and vast amounts of code depend on it.

Just a suggestion, if not an 'off the wall' suggestion.  I'm just a small  
system, and a relatively new news user, that's pushing around ideas.  Your  
comments make sense.  It would take too much time and code to parse the  
'acceptable' header lines from the rest, if they were not followed by a blank  
line.  Therefore I should be able to configure my system to conform to the  
already accepted standard.

Thanks, Brad, for getting my mental processes to work on this. ;-)

PS:  I do not represent Fidonet.
 
--  
Jeff Sheese - via FidoNet node 1:109/116
UUCP: ...!netsys!jsheese!jeffery
ARPA: jeffery@jsheese.FIDONET.ORG
(I am sole owner.  My opinions represent my company.)
(Send all flames to null@jsheese.Fidonet.ORG)

sl@unifax.UUCP (Stuart Lynne) (05/14/89)

In article <1222@psuhcx.psu.edu> wcf@psuhcx (Bill Fenner) writes:
>In article <62.246A2DCC@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG (Jeff Sheese) writes:

>That would take quite a lot of changing on most of USENET's part.  Maybe
>a better thing would be to ask the UFGATE folks to allow misc. header lines
>that are at the top of the article to pass through in the header.  How
>do you tell a header line from something like

The editor can assume that any lines until the first blank line, or first
line that starts with a word not followed with a colon; are header lines:

	Line 1		This: is a header line.
	Line 2		Another: header line.
	Line 3		This is not a header line:

Or:
	Line 1		This: is a header line.
	Line 2		Another: header line.
	Line 3
	Line 4		This: is not a header line.

You only have to worry if you absolutely must start your mail/news message
with a first line that has a colon following the first word.

-- 
Stuart.Lynne@wimsey.bc.ca uunet!van-bc!sl 604-937-7532(voice) 604-939-4768(fax)

wcf@psuhcx.psu.edu (Bill Fenner) (05/16/89)

In article <71.246B9252@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG (Jeff Sheese) writes:
|I'd have to refer back to RFC1036, but maybe those misc header lines have no
|more than one or two words followed by a colon.  Then the s/w can ignore    
|anything past the blank line that follows.  How can the s/w tell the    
Only if you put a blank line after the headers you put in.  I suppose that's
an OK restriction.  I suggest that this discussion move to the UFGATE echo;
unless you still want others' input.

|difference between a valid misc header line and 'First: I don't...' in the    
|first line of the message?  Well as we say in Fidonet, RTFD and 'Caveat    
|Emptor'.
I've never heard RTFD... Document?

|I'll request the source for NEWSOUT just to see if I can make the necessary    
|adjustments.  At least it will give me a starting point for the software    
|necessary to make myself compatible.
I thought that (forgot his name; author of newsout) said he was coming out
with a new version soon -- we could probably convince him to add it; shouldn't
be hard.

  Bill
-- 
   Bitnet: wcf@psuhcx.bitnet     Bill Fenner       | "Yesterday starts
  Internet: wcf@hcx.psu.edu                        |  tomorrow; tomorrow
 UUCP: {gatech,rutgers}!psuvax1!psuhcx!wcf         |  starts today"
Fido: Sysop at 1:129/87 (814/238 9633) \hogbbs!wcf |       -- Marillion

david@ms.uky.edu (David Herron -- One of the vertebrae) (05/16/89)

IMHO --

Continue developing the HyperText stuff for news.  Make sure that
it has some nifty keano features.  Sell the people on *existing*
software with *existing* nifty keano features.  But until you have
some software over on the shelf to point to and complain about how
other people are breaking these *existing* nifty keano features you
don't have much room for saying anything.

(Wish I had time to help in developing these nifty keano features ...)
-- 
<- David Herron; an MMDF guy                              <david@ms.uky.edu>
<- ska: David le casse\*'      {rutgers,uunet}!ukma!david, david@UKMA.BITNET
<- By all accounts, Cyprus (or was it Crete?) was covered with trees at one time
<- 		-- Until they discovered Bronze

david@ms.uky.edu (David Herron -- One of the vertebrae) (05/16/89)

In article <62.246A2DCC@jsheese.FIDONET.ORG> jeffery@jsheese.FIDONET.ORG (Jeff Sheese) writes:
>In an article of <11 May 89 17:09:22 GMT>, brad@looking.UUCP (Brad Templeton) writes:
>If RFC1036 were changed to relax the restriction of all the header lines  
>appearing before the blank line, I could be compatible by configuring my  
>editor to place these lines in there automatically.

This is a restriction inherited from RFC-822 mail and is required for
news messages to be treatable as mail.
-- 
<- David Herron; an MMDF guy                              <david@ms.uky.edu>
<- ska: David le casse\*'      {rutgers,uunet}!ukma!david, david@UKMA.BITNET
<- By all accounts, Cyprus (or was it Crete?) was covered with trees at one time
<- 		-- Until they discovered Bronze