[comp.mail.headers] My gawd...

page@ulowell.UUCP (02/16/87)

Just when I thought I'd seen every way possible to mung a From: line:

  From: wisc.EDU>' -P 'adm!ins_adsk%JHUNIX.BITNET@wiscvm.wisc.EDU'
  Newsgroups: comp.unix.wizards
  Subject: [To]:     UNIX-WIZARDS@brl.arpa
  Message-ID: <4468@brl-adm.ARPA>
  Date: 15 Feb 87 17:14:35 GMT
  Sender: news@brl-adm.ARPA

I've been thinking that there should be some organization that made sure
sites had decent mailers - so many people violate the most fundamental
parts of the major RFC's, for various reasons.  Some sites, like
seismo, rutgers and csnet-relay do it because the admins there say
"if we were 100% compliant, many sites could not respond, since THEY
are not RFC compliant."  I say screw the sites that can't comply;
this is the fastest way to get them to shape up.

Far worse are the mailers that just plain mung addresses into an
unusable format - I've even seen a MAJOR Usenet site smash Cc:
lines, not to mention the From: lines.

So what can be done?  If the CSNET-relay generates (technically)
wrong addresses, should we expect them to police their member sites?
How about BITNET and Internet net police forces?

Lastly, how about UUCP sites?  No organization can do it, and I don't
know if any organization would want to.

..Bob
-- 
Bob Page,  U of Lowell CS Dept.      ulowell!page,  page@ulowell.CSNET

jim@strath-cs.UUCP (02/18/87)

In article <1061@ulowell.cs.ulowell.edu> page@ulowell.cs.ulowell.edu (Bob Page) writes:
>I've been thinking that there should be some organization that made sure
>sites had decent mailers - so many people violate the most fundamental
>parts of the major RFC's, for various reasons.  Some sites, like
>seismo, rutgers and csnet-relay do it because the admins there say
>"if we were 100% compliant, many sites could not respond, since THEY
>are not RFC compliant."  I say screw the sites that can't comply;
>this is the fastest way to get them to shape up.

Oh, that it were true! What proportion of sites on USENET are stuck with
V7-style mailers that can only cope with UUCP bang-style addresses? They
won't comply with RFC822 and I doubt if many of them could become RFC822
compliant even if they wanted to. [Anybody fancy putting sendmail or MMDF
up on a XENIX box with a 20 Mbyte disk? :-)] 

Forcing the bad sites to "shape up" is a reasonable idea in theory, but
it's somewhat impractical. I dread to think how much stuff ends up in the
postmaster's mailbox at somewhere like seismo. If they started being much
more fussy about mail addresses, could you imagine how much worse that
situation would get? The choice becomes straightforward: stick rigidly to
the protocols and watch megabytes of mail bounce or break the spec. and
keep the mail flowing. If you were a backbone administrator, what would
you do? (Especially if you have to pay the cost of returning failed mail.)
Even if you ignore the cost aspects and the "mail must get through" argument,
violating RFC822 at the likes of seismo and csnet-relay for some mail must
reduce the workload on the admins. there. This can be no bad thing.

USENET relies on co-operation. Unlike networks like CSnet or ARPA, there
is no real central authority to enforce standards and, more importantly,
nobody funding the development of software to comply with these standards.
Without a centrally-directed and funded network administration, it's
unlikely that USENET could make progress in this area.

>Lastly, how about UUCP sites?  No organization can do it, and I don't
>know if any organization would want to.

USENET has no organisation - it's almost total anarchy! :-) The backbone
admins already have enough to do without policing RFC822 conformance in
every piece of mail that they get. Are you volunteering to do this?

		Jim

ARPA:	jim%cs.strath.ac.uk@ucl-cs.arpa, jim@cs.strath.ac.uk
UUCP:	jim@strath-cs.uucp, ...!seismo!mcvax!ukc!strath-cs!jim
JANET:	jim@uk.ac.strath.cs

"JANET domain ordering is swapped around so's there'd be some use for rev(1)!"

edward@ukecc.UUCP (02/18/87)

In article <1061@ulowell.cs.ulowell.edu> page@ulowell.cs.ulowell.edu (Bob Page) writes:
>I've been thinking that there should be some organization that made sure
>sites had decent mailers - so many people violate the most fundamental
>parts of the major RFC's, for various reasons.  Some sites, like
>seismo, rutgers and csnet-relay do it because the admins there say
>"if we were 100% compliant, many sites could not respond, since THEY
>are not RFC compliant."  I say screw the sites that can't comply;
>this is the fastest way to get them to shape up.

I wish that we could take this attitude. Unfortunetly there are sites
that, for one reason or another, can't upgrade to compliant mailers.
We, those of us with smart mail software, must bend the rules occasionally
to keep things running.

>Far worse are the mailers that just plain mung addresses into an
>unusable format - I've even seen a MAJOR Usenet site smash Cc:
>lines, not to mention the From: lines.

So write to the postmaster and tell them.

>So what can be done?  If the CSNET-relay generates (technically)
>wrong addresses, should we expect them to police their member sites?
>How about BITNET and Internet net police forces?

If you know of a site that doesn't conform, call or write the sysadmin.
Talk to them. They probably aren't even aware of the problem. Explain
the importance of following the rules.

>Lastly, how about UUCP sites?  No organization can do it, and I don't
>know if any organization would want to.

Mark Horton and the UUCP Map Project are trying. It's a huge job. Help
them out.


-- 
Edward C. Bennett			UUCP: cbosgd!ukma!ukecc!edward
					CSNET: edward@engr.uky.edu
"Goodnight M.A."			BITNET: edward%ukecc.uucp@ukma
		Change the world. Make a new friend today.

page@ulowell.UUCP (02/20/87)

jim@cs.strath.ac.uk wrote in article <394@stracs.cs.strath.ac.uk>:
>page@ulowell.cs.ulowell.edu (Bob Page) writes:
>>I say screw the sites that can't comply; this is the fastest way
>>to get them to shape up.

>fancy putting sendmail or MMDF up on a XENIX box with a 20 Mbyte disk? :-)

You don't need MMDF or sendmail, smail will do it.

>If [somewhere like siesmo] started being much more fussy about mail
>addresses

Did I suggest that?  The mailers should be as *robust* as possible,
and handle every garbage address sent to it, but *don't* *ever* change
the headers.  As it happens now, some mailers mung headers and some
don't, so many addresses are unusable anyway, regardless of what some
good-intentioned sites do.  Just because rutgers (for example) is being
nice and putting its name on everything doesn't mean that some mailer
betwen there and the destination isn't going to mess it up, or that
the mailer at the end of the line will be able to respond to the
getting-more-complex-every-hop address.

>The choice: stick rigidly to the protocols and watch megabytes of mail
>bounce or break the spec. and keep the mail flowing. If you were a backbone
>administrator, what would you do? (Especially if you have to pay the cost
>of returning failed mail.)

Addresses that don't conform get sent along where possible, but *my site*
doesn't mung headers.  If a mailer downstream breaks because it's not
robust enough, it breaks. If the recipient can't reply because the
address is unusable, what can I do about that?  What should I do?  I
didn't change the headers, I just forwarded it.

>violating RFC822 at the likes of seismo and csnet-relay for some mail must
>reduce the workload on the admins. there. This can be no bad thing.

This is no justification.  Hiding the problem only prolongs it.  Why
have RFC822 when nobody is following it?

>Without a centrally-directed and funded network administration, it's
>unlikely that USENET could make progress in this area.

Again, I disagree with you, and point out that to hint about having
such a beast is A Bad Thing.  The last thing that should happen is that
you have to pay a fee to *join* Usenet ... just so that some central
authority can 'direct' you!

>The backbone admins already have enough to do without policing RFC822
>conformance in every piece of mail that they get. Are you volunteering
>to do this?

I did not suggest that anyone bounce mail.  How about a daemon that
sits someplace in the ever-growing mailer chain, scanning headers?
If it sees one that is not compliant, it flags it and takes action?
The action could be a number of things, anywhere from the MMDF
"parse error" warning that it imbeds in the message header, to
sending mail to the postmaster of the offending system, saying "look
at this, it is bogus" (think of all the mail some sites would get!),
to something as radical as moving the offending line and writing a
new one, like:
	From: (mumble)
		becomes
	X-Bogus-From: (mumble) rewritten at myvax.EDU
	From: (good mumble address)

I DO NOT suggest the latter, only that is is possible.  Site admins
could pick their algorithm, if more than one were supplied, based on
their individual requirements.

Since I opened my big mouth about this, I will take on the task of
writing such a beast.  Volunteers to help (preferably versed in RFC822)
are welcome.

..Bob
-- 
Bob Page,  U of Lowell CS Dept.      ulowell!page,  page@ulowell.CSNET

mark@cbosgd.UUCP (02/20/87)

In article <394@stracs.cs.strath.ac.uk> jim@cs.strath.ac.uk writes:
>Oh, that it were true! What proportion of sites on USENET are stuck with
>V7-style mailers that can only cope with UUCP bang-style addresses? They
>won't comply with RFC822 and I doubt if many of them could become RFC822
>compliant even if they wanted to. [Anybody fancy putting sendmail or MMDF
>up on a XENIX box with a 20 Mbyte disk? :-)] 

Actually, there are easy ways to do this.  Smail runs very nicely on a
PC - I've used it on 20MB disk machines such as the AT&T 6300 PLUS.
I expect it works fine on Xenix, too, but I haven't tested it.  It
does not require putting up sendmail, MMDF, pathalias, or the like,
nor does it require sources to modify /bin/mail.  It does require a
C compiler, and it's most useful if you have a legal domain name
registered.

I suspect the same is true of some other software packages, like uumail.
For MS DOS machines, you can get RFC 822/976 compliance from UULINK.
I run it on a laptop Toshiba machine with only 720K floppy disks; I think
it can also be used from 360K floppies.

The real reason why many people will continue to run bang mailers is that
the typical systems sold by the major vendors only come with bang mailers.
Many people only run whatever comes in the box, because they don't know
any differently.  Some other places have the expertise to install smarter
software, but local politics or regulations forbid it.  Smail only came
out last August, and UUMAIL is also fairly recent.  The version of smail
that runs well on small machines was just posted to mod.sources this
week.  It's too soon for major systems to have them; it's even too soon
for market demand to develop.  AT&T and Microsoft and Berkeley will only
put such software in if their customers ask for it.

	Mark

jeff@cideq3.UUCP (02/22/87)

In article <394@stracs.cs.strath.ac.uk> jim@cs.strath.ac.uk writes:
>Oh, that it were true! What proportion of sites on USENET are stuck with
>V7-style mailers that can only cope with UUCP bang-style addresses? They
>won't comply with RFC822 and I doubt if many of them could become RFC822
>compliant even if they wanted to. [Anybody fancy putting sendmail or MMDF
>up on a XENIX box with a 20 Mbyte disk? :-)] 

I don't have sendmail or MMDF running, but smail version 2.3 seems to run
just fine on my Xenix 2.1.3 system (and it's on a 20mb disk! :-). Seriously,
smail does at least generate the proper headers, and it didn't take very
long to install.

Send e-mail if you want the installation procedure - but please make sure
your message has RFC822 headers so I can reply! :-)

-- 
Jeff Marraccini	- cideq3 System Administrator     | cideq3: Public Access Xenix
UUCP:     ...!ihnp4!mb2c!fmsrl7!cideq3!jeff       | 313/623-6309, 1200/2400 bps
INTERNET: jeff@cideq3.cidnet.com                  | Login with 'help' for info.
PHONE:    +1 313 623 7115                         |____________________________

allyn@sdcsvax.UUCP (02/25/87)

In article <394@stracs.cs.strath.ac.uk>, jim@cs.strath.ac.uk writes:
>                                   [Anybody fancy putting sendmail or MMDF
> up on a XENIX box with a 20 Mbyte disk? :-)] 

yes.  i have sendmail running on PC/IX systems (on IBM PCs), some
with 10 Mb disks!  PC/IX is just another UNIX (System III) port
for the IBM PC.

porting sendmail was trivial (<3 hours including installing our own
internally developed networking for smtp).  i wish all programs ported 
as easily as sendmail to hostile environments (16 bit ints).  
(good work, eric, you've heard of lint).

of course, getting the configuration file correct took much longer...

-- 
 From the virtual mind of Allyn Fratkin            allyn@sdcsvax.ucsd.edu    or
                          EMU Project              {ucbvax, decvax, ihnp4}
                          U.C. San Diego                         !sdcsvax!allyn

lyndon@ncc.UUCP (02/27/87)

In article <394@stracs.cs.strath.ac.uk>, jim@cs.strath.ac.uk (Jim Reid) writes:
> In article <1061@ulowell.cs.ulowell.edu> page@ulowell.cs.ulowell.edu (Bob Page) writes:
> >I've been thinking that there should be some organization that made sure
> >sites had decent mailers - so many people violate the most fundamental
> >parts of the major RFC's, for various reasons.  Some sites, like
> >seismo, rutgers and csnet-relay do it because the admins there say
> >"if we were 100% compliant, many sites could not respond, since THEY
> >are not RFC compliant."  I say screw the sites that can't comply;
> >this is the fastest way to get them to shape up.
> 
> Oh, that it were true! What proportion of sites on USENET are stuck with
> V7-style mailers that can only cope with UUCP bang-style addresses? They
> won't comply with RFC822 and I doubt if many of them could become RFC822
> compliant even if they wanted to. [Anybody fancy putting sendmail or MMDF
> up on a XENIX box with a 20 Mbyte disk? :-)] 

I can see the immediate comeback:

	"Why not install [smail, uumail, gnumail, ...]?"

In most cases this is a very nice solution to the problem (we
have run smail for close to a year now).

Unfortunately, Bobs answer is not universally possible (in addition
to being in bad taste...), and this problem will grow larger over
the next few years. Why? Well, it seems that AT&T (in their infinite
wizdumb) has decided to unbundle UNIX. As a result, many customers
will not have the "Programmers Toolkit" (or whatever it is they call
cc these days) and will therefore be unable to compile a mailer that
complies with RFC822. Screw them? Sorry, someone has beaten you to it.

The fact that large numbers of sites are breaking the rules is largely
due to a poor decision made by the sendmail implementors, who I doubt
had any idea of what would happen to UUCP routing "down the road".
I had hoped (well, just a *little*) that 4.3 sendmail would acknowledge
the existance of domain addresses in the From: header as the default
case, but I gather that this did not happen. Looking at the sendmail
documentation, code, and those awful .cf files, I don't wonder why
noone bothers to change things.

So what are we to do? I think the first step is to try to get as
many UUCP sites running a "standard" mailer package. (I hear screams
in the audience). "Never!" they say. Perhaps not, but look at the
near universal acceptance of the Bnews software. I'm not implying
that it would be easy, just that it wouldn't be *impossible*.
There are quite a number of independent pieces of software available
on the net that handle various aspects of the problem (i.e. routers:
smail, uumail  UA's: elm, others_I_can't_remember). With considerable
effort, we could probably amalgamate the best of these into a single
integrated source distribution. Perhaps the authors of some of these
packages would care to comment on this?

This still doesn't help the poor sucker who doesn't have a C compiler
though. I don't know if there really is a solution here, other than to
send the mailer source to all the systems vendors with an plea that they
include it as part of their port. Actually, maybe it should be included
as a *free* *bonus* add-on, lest AT&T sue the USENET community at large
for not complying with SVID... (no smiley as I'm not being funny here)

-- 
Lyndon Nerenberg - Nexus Computing Corp. - lyndon@ncc.UUCP
UUCP: {ihnp4,ubc-vision,vax135,watmath}!alberta!ncc!lyndon