[news.admin] serious problem - mod articles not sent to mailing site

dave@lsuc.UUCP (03/09/87)

I recall this problem being mentioned a year or so ago, and I
thought it had been resolved at that time. Obviously it hasn't,
with respect to at least one newsgroup.

Symptom: I reply to a mod.ai article by mailing to
mnetor!seismo!sri-stripe.arpa!ailist.  The article gets
posted with a Path: line that includes my site (lsuc);
as a result, lsuc is never sent it. Not only do I not
see it (not so serious), but our downstream feeds never
get it either. (I've confirmed that this has happened with
others' postings. It happened to me with mod.announce too.)

If the mail path used to reach the moderator includes major
Usenet backbones, the chances are that other large chunks of
the net won't see the article either.

What can be done? I know we want the From: lines to indicate
the real author, but perhaps if the Path: lines stick to starting
with the moderator, rather than reaching back to the person who
mailed the submission, the problem could be solved.

David Sherman
The Law Society of Upper Canada, Toronto
-- 
{ seismo!mnetor  cbosgd!utgpu  watmath  decvax!utcsri  ihnp4!utzoo } !lsuc!dave

ted@blia.UUCP (03/13/87)

This has happened to me too, using mod.computers.vax. However, the problem
stopped about the time we changed our newsfeed node from bigbang to voder.
In both cases, the path begins at my node so I assume that some sites have
fixed their software to pass a message on even if the path indicates that
it came from there.

Note, I am not a USENET wizard or even our site manager so alot of this is
just educated guesswork.

===============================================================================
            Ted Marshall
            Britton Lee, Inc.
p-mail:     14600 Winchester Blvd, Los Gatos, Ca 95030
voice:      (408)378-7000
uucp:       ...!ucbvax!mtxinu!blia!ted
ARPA:       mtxinu!blia!ted@Berkeley.EDU
disclaimer: These opinions are my own and may not reflect those of my employer;
            I leave them alone and they leave me alone.

dave@lsuc.UUCP (03/17/87)

In article <1830@blia.BLI.COM> ted@blia.BLI.COM (Ted Marshall) writes:
>
>This has happened to me too, using mod.computers.vax. However, the problem
>stopped about the time we changed our newsfeed node from bigbang to voder.
>In both cases, the path begins at my node so I assume that some sites have
>fixed their software to pass a message on even if the path indicates that
>it came from there.

Nope, that can't be it, since articles should not be sent to a
site which actually has had them once AS USENET ARTICLES. Otherwise
they'd go around the net forever.

The fix is for moderators and automoderator-maintainers to fix
their posting software so that the Path: line DOES NOT include
anything which identifies the person who mailed the submission.
The From: line should indeed include that information, but not
Path:, or the article will never get downstream from the mailing
site and any other sites showing up in the mail information.

Please, anyone who is maintaining a mod.* group, check whether
you are including the mailer's name on the Path: line, and if
so, stop it at once! Thanks.

David Sherman
The Law Society of Upper Canada, Toronto
-- 
{ seismo!mnetor  cbosgd!utgpu  watmath  decvax!utcsri  ihnp4!utzoo } !lsuc!dave

ted@blia.BLI.COM (Ted Marshall) (03/19/87)

In article <1644@lsuc.UUCP>, dave@lsuc.UUCP (David Sherman) writes:
> In article <1830@blia.BLI.COM> ted@blia.BLI.COM (Ted Marshall) writes:
> >
> >This has happened to me too, using mod.computers.vax. However, the problem
> >stopped about the time we changed our newsfeed node from bigbang to voder.
> >In both cases, the path begins at my node so I assume that some sites have
> >fixed their software to pass a message on even if the path indicates that
> >it came from there.
> 
> Nope, that can't be it, since articles should not be sent to a
> site which actually has had them once AS USENET ARTICLES. Otherwise
> they'd go around the net forever.

I agree with your logic. However, it happens! The following is a header from
an article that I mailed to INFO-VAX@SRI-KL.ARPA (moderator address for
mod.computers.vax) as it arrived back at my machine:

> From ted@blia.UUCP (Ted Marshall) Thu Mar 12 09:51:49 1987
> Path: blia!voder!pyramid!decwrl!decvax!ucbvax!blia.UUCP!ted
        ----                                    ----
> From: ted@blia.UUCP (Ted Marshall)
> Newsgroups: mod.computers.vax
> Subject: Re: Info-Vax Digest (suppressing error messages)
> Message-ID: <8703121751.AA20496@blia.BLI>
> Date: 12 Mar 87 17:51:49 GMT
> References: <8703120316.AA11929@ucbvax.Berkeley.EDU>
> Sender: daemon@ucbvax.BERKELEY.EDU
> Distribution: world
> Organization: The ARPA Internet
> Lines: 26
> Approved: info-vax@sri-kl.arpa

Note the path. Even though it originated at blia.UUCP (my site), it still
returned.

Also, as I understand the workings of USENET, the don't-send-back-to-a-node-
on-the-path rule is un-needed because of the history file, the same way that
a node with two news feeds doesn't get two copies of everything.

-- 
===============================================================================
            Ted Marshall
            Britton Lee, Inc.
p-mail:     14600 Winchester Blvd, Los Gatos, Ca 95030
voice:      (408)378-7000
uucp:       ...!ucbvax!mtxinu!blia!ted
ARPA:       mtxinu!blia!ted@Berkeley.EDU
disclaimer: These opinions are my own and may not reflect those of my employer;
            I leave them alone and they leave me alone.
fortune for today:
THE GOLDEN RULE OF ARTS AND SCIENCES
	The one who has the gold makes the rules.

dave@lsuc.UUCP (03/19/87)

In article <1865@blia.BLI.COM> ted@blia.BLI.COM (Ted Marshall) writes:
>In article <1644@lsuc.UUCP>, dave@lsuc.UUCP (David Sherman) writes:
>> Nope, that can't be it, since articles should not be sent to a
>> site which actually has had them once AS USENET ARTICLES. Otherwise
>> they'd go around the net forever.
>
>I agree with your logic. However, it happens! The following is a header from
>an article that I mailed to INFO-VAX@SRI-KL.ARPA (moderator address for
>mod.computers.vax) as it arrived back at my machine:
>
>> From ted@blia.UUCP (Ted Marshall) Thu Mar 12 09:51:49 1987
>> Path: blia!voder!pyramid!decwrl!decvax!ucbvax!blia.UUCP!ted
>        ----                                    ----

Don't you see the difference yourself? The news software which
makes the decision not to send an article to a site doesn't
know about domains. It just does a strcmp on stuff between '!'s.
"blia" and "blia.UUCP" are therefore not equal, and you get the
article back.

> Also, as I understand the workings of USENET, the don't-send-back-to-a-node-
> on-the-path rule is un-needed because of the history file, the same way that
> a node with two news feeds doesn't get two copies of everything.

Absolutely not! Rejecting an article because it's in the history file
requires you to receive it first, which means you would have to spend
megabucks to get back a full newsfeed of stuff you'd just sent to each
neighbour, only to decide that you already have it. That's why inews
will decline to send an article to any site which appears in the
Path: line.

As stated before, the fix is for all moderators to make sure nothing
containing mailing information appears in the Path: line.  Please!

David Sherman, The Law Society of Upper Canada, Toronto
-- 
{ seismo!mnetor  cbosgd!utgpu  watmath  decvax!utcsri  ihnp4!utzoo } !lsuc!dave

jerry@oliveb.UUCP (Jerry F Aguirre) (03/19/87)

In article <1865@blia.BLI.COM> ted@blia.BLI.COM (Ted Marshall) writes:
>> From ted@blia.UUCP (Ted Marshall) Thu Mar 12 09:51:49 1987
>> Path: blia!voder!pyramid!decwrl!decvax!ucbvax!blia.UUCP!ted
>        ----                                    ----
...
>Note the path. Even though it originated at blia.UUCP (my site), it still
>returned.

No, "blia" does not equal "blia.UUCP", at least not in the news software.
You "lucked out" as a result of having inconsistant host names in your
mail and news software.  Also some sites use procedures that bypass the
"don't send back" rule.  For instance I feed 4 "leaf" sites and send the
same batches of news to all of them.  It is not worth the overhead to
filter out the <1% of the articles that came from each one.  So they get
their articles back and reject them because of the history file.

>Also, as I understand the workings of USENET, the don't-send-back-to-a-node-
>on-the-path rule is un-needed because of the history file, the same way that
>a node with two news feeds doesn't get two copies of everything.

The problem with this is that you have already payed for it.  The
orriginating site batched, compressed, and queued it.  Someone payed for
the transmission (phone) costs to send it.  And the receiving site
uncompressed, unbatched, and checked each article against the database.
An expensive proposition if your just going to throw it away.

Without this "rule", each site would send every article back to the site
it just came from, doubling the transmission costs.

The "Path:" line is supposed to reflect the NEWS systems the article
went through, not systems it was mailed through.  Moderators fix your
software!

					Jerry Aguirre

fair@ucbarpa.Berkeley.EDU.UUCP (04/13/87)

This is a slightly abridged version of a letter I sent to Dave Sherman
explaining what is going on with this, and what to do about it.

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.berkeley.edu

From fair Sun Mar 29 09:48:19 1987
To: mnetor!lsuc!dave@seismo.CSS.GOV

OK, here's that explanation you've been waiting for:

In netnews there is this #define called INTERNET.

If you have it on, and a user hits "r" in readnews (or whatever),
the software will take the address found in the "From:" line and
attempt to use that as a mail address. For this to work properly, the
local mailer must understand what to do with an address of the form:

	user@host.UUCP

where "host" is not necessarily one that it speaks directly to (i.e.
it's gotta generate a path to get there). In fact, the local mailer
has to be prepared to deal with just about anything in the "From:"
line, which is a lot to ask, really. This is why the smart mailers for
UUCP land too so damn long to write. Well, that and the fact that we
were relying on volunteer labor.

So you probably do what most people do: you leave #define INTERNET
off. So what happens when a user hits "r" to mail a reply? Well the
"Path:" line is taken, en toto, as the UUCP path to get to the
author. Neat thing about this is that it makes no demands of the local
mailer at all, because the first site in that path will be you, the
second one will be one that you speak directly to (since you trade
news with them) and so on.

OK, with the background firmly in mind, here's the problem:

You're writing a gateway for stuff from the ARPA Internet mailing
lists, and you have this pathological case to deal with:

	foo!bar!blatz!user@gateway.domain

This is some poor schmuck from UUCP land who has mailed in his message
to the mailing list through a random UUCP gateway on the Internet, and
now you have to make this work from a USENET perspective, so what do
you do? Well, the "From:" is easy:

	From: user@blatz.UUCP

But now we have the "Path:" to deal with. Uh, oh. And this is what the
majority of the USENET uses for mail replies. Well, given that the
netnews software won't send the message to his site if you include the
bare UUCP name in the Path:, you tack on a domain name, so that
they're not equal, that is:

	Path: ucbvax!blatz.UUCP!user

rather than

	Path: ucbvax!blatz!user

and you count on the fact that blatz != blatz.UUCP so that this USENET
article will get to site "blatz" (unfortunately, this was not true
for releases prior to 2.11, but it is trivially fixed: remove the "."
character from the NETCHRS definition and recompile).

Now then, given the mechanics of a mail reply with INTERNET off, whose
mailer has to be smart? Why, yours, Mr. Gateway author! After all, what
you send out will eventually come back as mail of the form:

	XQT (rmail blatz.UUCP!user)

All clear?

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.berkeley.edu