[net.mail] Mail overload and so-called "smart" mailers

robert@fear.UUCP (Robert Plamondon) (12/31/85)

The ihnp4->tektronix discussion brings several problems (and
solutions!) to light:

1.  If you use so-called smart mailers to route mail passing
through your site, you *WILL* introduce errors and piss off users.
This is made worse by the extremely simple-minded way most "smart"
mailers operate.  Most errors can be avoided by simply refusing to
do any rewriting without good reason, rather than just running
everything through a pathalias router.

2.  The pathalias entries of overloaded systems are partly
responsible for the own overload.  For instance, many of ihnp4's
links are shown DEDICATED, DIRECT, or DEMAND, although the speed of
the serial link is irrelevant when mail sits in the queue for days
before being processed.  If ihnp4 downgraded its links, it would
reflect reality better, and mail generated by sites using pathalias
would use alternate routes, giving better transit times and reducing
the load on ihnp4.  Here in the SF Bay Area, dual is in a similar
predicament, with good connectivity but long queue times.  I've
edited the pathalias entry to give a 100*LOW penalty for all of
dual's connections, which has caused most paths through dual to find
alternate routes.

In other words, system management can control mail volume through
pathalias postings.  If your mail load is too high, I suggest that
you downgrade your connections a bit until it tapers off to a level
you can tolerate.  Call your DEMAND connections HOURLY, etc.
-- 

		Robert Plamondon
		UUCP: {turtlevax, resonex, cae780}!weitek!robert
		FidoNet: 10/624 robert plamondon

gjm@packard.UUCP (Gary J. Murakami) (01/07/86)

Robert Plamondon is oversimplifying the problem without expressing due
consideration to some of us who have lived with mail routing problems
for years.

While some individuals may argue (somewhat justifiably) that routing
(optimization) on ihnp4 is arbitrary, the decisions have been carefully
weighed to try to provide the most benefit for both internal company use
and for many friends and even competitors AT NO COST.  Many people have
expressed their gratitude via mail for the services provided by ihnp4,
but some individuals appear to be ungrateful.

Robert is correct in that pathalias data can be adjusted to alter your
load and change routes in the network.  He also correctly identifies the
a problem in the lack of a perfect way to reflect load and/or queuing
delays. Many of us have recognized and discussed this since the earliest
days of pathalias.  However the problem is more complicated since:

	changes in pathalias costs also affect your neighbors.

An increase in ihnp4's pathalias costs would shift more burden to some
of its neighbors.  We've cranked the data through pathalias before, and
even the recent pathalias errors point to the drastic effect that can be
caused by changes in input data.  We've seen many scenarios (e.g.
allegra, cbosgd, packard, or topaz flooded, ucbvax bypassed).

ihnp4 has several networks with different characteristics, and the costs
have to be weighed carefully to end up with the correct balance
incorporating hops, media capacity/load, neighboring host capacity/load,
mail delivery delays, and local capacity/load.

Quite a while ago, we downgraded ihnp4's cost to HOURLY for 2400/1200
bps connections to non-AT&T sites.  This meant that ihnp4's AT&T
neighbors had to make similar adjustments.  Fortunately the AT&T data is
centrally collected (unfortunately, I still end up doing it).  I'm
willing to take constructive suggestions on what costs to use (in place
of HOURLY).  Keep in mind that we dont want to touch off a pathalias cost
war...

-Gary

avolio@decuac.UUCP (Frederick M. Avolio) (01/07/86)

In article <314@fear.UUCP>, robert@fear.UUCP (Robert Plamondon) writes:
> 1.  If you use so-called smart mailers to route mail passing
> through your site, you *WILL* introduce errors and piss off users.

No... but you *might* do both.  For any HOSTA, rewriting/rerouting
should usually only be done if the "next" host in the path is not
directly connected to the HOSTA. (This makes the path longer.)
In this case, a routing table should be used (since the alternative
is undeliverable mail).  The only other instance when it might be
reasonable is when some host deeper in the path is directly connected
to HOSTA.  For example, if we get mail for "host1!host2!aplvax!user"
we don't send it to "host1," but rather right on to aplvax -- a local
call away.

-- 
Fred @ DEC Ultrix Applications Center    {decvax,seismo,cbosgd}!decuac!avolio

robert@fear.UUCP (Robert Plamondon) (01/07/86)

In article <393@packard.UUCP>, gjm@packard.UUCP (Gary J. Murakami) writes:

> 
> While some individuals may argue (somewhat justifiably) that routing
> (optimization) on ihnp4 is arbitrary, the decisions have been carefully
> weighed to try to provide the most benefit for both internal company use
> and for many friends and even competitors AT NO COST.

I never claimed that ihnp4's pathalias file was arbitrary, just that
it's inaccurate.  I agree with your reasons for setting it up the way
you did, by the way.  The idea of routing mail through a
"sacrificial" VAX with no users to spare the VAXen with users is a
good idea, if you can afford it.

But while I agree with both your motives and methods, I feel no
corresponding obligation to route my mail through ihnp4 and suffer
both low delivery speed and occaisional misrouting that this entails.

> Many people have
> expressed their gratitude via mail for the services provided by ihnp4,
> but some individuals appear to be ungrateful.

I'm appreciate the efforts of all the people who are maintaining and
paying for the net.  I thought that went without saying, but I guess
not.

> 	changes in pathalias costs also affect your neighbors.
> 
> An increase in ihnp4's pathalias costs would shift more burden to some
> of its neighbors.  We've cranked the data through pathalias before, and
> even the recent pathalias errors point to the drastic effect that can be
> caused by changes in input data.  We've seen many scenarios (e.g.
> allegra, cbosgd, packard, or topaz flooded, ucbvax bypassed).

I've run combinations, too. My results indicate that the shift would
be to other AT&T machines, and that this is largely an effect of
having very similar connectivity and path costs for these machines.
This suggests that you could downgrade ihnp4's numbers to reflect
reality, and then downgrade the other systems' numbers enough to
prevent them from being flooded.

Anyway, There are several ways to look at the problem.  I mostly
approach it from the perspective of a user trying to get mail safely
and quickly form one place to another.  When looking at things this
way, sites like ihnp4, with its unbelievable mail load, and dual
(which has been characterized as "well-connected, but the phones are
always busy") are more of a hazard than help.  I mean no slight on
the people running these sites -- but the fact is, mail moves through
them very slowly, and they are best avoided if you want your messages
delivered quickly.

-- 

		Robert Plamondon
		UUCP: {turtlevax, resonex, cae780}!weitek!robert
		FidoNet: 143/12 robert plamondon

mikel@codas.ATT.UUCP (Mikel Manitius) (01/11/86)

> Anyway, There are several ways to look at the problem.  I mostly
> approach it from the perspective of a user trying to get mail safely
> and quickly form one place to another.  When looking at things this
> way, sites like ihnp4, with its unbelievable mail load, and dual
> (which has been characterized as "well-connected, but the phones are
> always busy") are more of a hazard than help.  I mean no slight on
> the people running these sites -- but the fact is, mail moves through
> them very slowly, and they are best avoided if you want your messages
> delivered quickly.
> 
> 		Robert Plamondon

I have been moving mail trough ihnp4 (amongst other gateways) for some
time, and haven't really noticed any long delays. I am currious to see
any statistics anyone may have about the volume of trafic that goes
through such machines (mainly ihnp4), and how many lines such a site
may have dedicated to gateway.

(I for one, am very appreciative of the service ihnp4 offers)
-- 
			Mikel Manitius @ AT&T-IS Altamonte Springs, FL
			...{ihnp4|akgua|bellcore|clyde|koura}!codas!mikel

gjm@packard.UUCP (Gary J. Murakami) (01/15/86)

> I am currious to see any statistics anyone may have about the volume
> of trafic that goes through such machines (mainly ihnp4), and how many
> lines such a site may have dedicated to gateway.

> ... Mikel Manitius ...

Sunday's UUCP traffic on ihnp4

H0 Id.,[Line],    ...... total total total    in    in    in   out   out   out
H1 Sys,sys!User   ......   num    KB   Bps   num    KB   Bps   num    KB   Bps
H2 -------------- ------ ----- ----- ----- ----- ----- ----- ----- ----- -----
I ihnp4 ................  4944 30925   103  2500  9420   121  2444 21504    97

Monday's UUCP traffic on ihnp4, plus some top UUCP neighbors:

I ihnp4 ................  7312 21886   100  3713  7892    97  3599 13994   101
S ucbvax ...............   545   719    62   447   639    60    98    79    82
S seismo ...............   464   668    66   300   421    61   164   247    76
S watmath ..............   208   543    94   138   314    85    70   229   108
S pur-ee ...............   199   293    96   105   120    82    94   172   108
S alberta ..............   164  1024   100    68    85    91    96   938   101

ihnp4 averages about 25 Mbytes/day via UUCP (plus similar amounts on two
other networks).  I've seen as many as a dozen simultaneous uucico's.

Networking hardware info follows.

-Gary

Processor:
        AT&T 3B20S Mod I

Networks:
        NSC HYPERchannel        IHCC General Purpose NSC network
        (50 Mbps CSMA/CD LAN)                63 hosts.nsc

        Datakit(r) VCS          AT&T Interlocation network (100+ nodes)
        (8 Mbps/node VCS WAN)               307 hosts.dk

        RJE/BLN                 Bell Labs Network (BLN) (ISO-like host PS)
        (RJE + 56Kbps backbone)             448 hosts.asp

        Phone                   AT&T CORNET, AT&T Communications
        (1200/2400 bps modems)             1375 hosts.acu

        - Phone/CORNET          - AT&T CORNET internal network
          (+ATTCOM access)                 1061 hosts.cor

        - Phone/ATTCOM          - AT&T Communications
                                            314 hosts.ext
Network-Interfaces:
        nusend
                HYPERchannel    1 x NSC interface
        UUCP (out)
                Datakit         1 x HS mux interface (pending)
                Datakit         3 x 9600 bps (+ Phone access)
                Datakit         5 x autobaud
                Phone/CORNET    2 x 1200 bps (+ Phone/ATTCOM access)
                Direct          2 x 9600 bps (ihnp1, ihnp3)
        UUCP (in)
                Datakit         11 x 9600 bps
                Phone/CORNET    8 x 1200 bps
                Phone/ATTCOM    8 x 1200 bps
        usend
                RJE/BLN         1 x 19.2 Kbps RJE
-----
HYPERchannel is a trademark of Network Systems Corporation
Datakit VCS is a registered trademark of AT&T

greg@ncr-sd.UUCP (Greg Noel) (01/16/86)

In article <397@packard.UUCP> gjm@packard.UUCP (59455-GJ Murakami) writes:
>H0 Id.,[Line],    ...... total total total    in    in    in   out   out   out
>H1 Sys,sys!User   ......   num    KB   Bps   num    KB   Bps   num    KB   Bps
>H2 -------------- ------ ----- ----- ----- ----- ----- ----- ----- ----- -----
>I ihnp4 ................  4944 30925   103  2500  9420   121  2444 21504    97

What does "bps" mean in this table?  Initially, I thought it meant "bits-per-
second" and I was couldn't believe that it was that bad.  But it occurs to me
that it might be "bytes-per-second," in which case that's 1,000 bits-per-second
via an asynchronous modem, which is \very/ respectible as an average.

Oh, and if "num" is the number of calls, 2500 calls per day is amazing.....
That's something like two calls per minute -- and these were statistics for
Sunday, which I presume to be a light day.
-- 
-- Greg Noel, NCR Rancho Bernardo    Greg@ncr-sd.UUCP or Greg@nosc.ARPA

ulmo@well.UUCP (Brad Allen) (01/27/86)

> <761@decuac.UUCP>


Let me suggest, perhaps sending mail on paths such as this:
...!ihnp4.notouch!etc
would tell ihnp4 not to touch the rest of the path, and paths
like
...!ihnp4.auto!...   or ...!ihnp4.touch!... would request ihnp4 to
send the path through an alias file.

Of course, perhaps it would do better to say "ihnp4.uucp.notouch"?
I am no network wizzard, but this seems like a feasable and simple
solution, provided that the sender know of it, to the pathalias problems.