[comp.mail.misc] How do you handle large mailing lists?

zemon@felix.UUCP (Art Zemon) (07/17/87)

I have a relatively large mailing list, info-ultrix, which
has outgrown the alias facility in sendmail.  Even using the
:include: construct only postponed what I supposed I should
have forseen.  How do you handle large mailing lists?  Right
now, info-ultrix is well over 100 people and growing daily.

	-- Art Z.
--
	-- Art Zemon
	   FileNet Corporation
	   Costa Mesa, California
	   ...!hplabs!felix!zemon

page@ulowell.cs.ulowell.edu (Bob Page) (07/19/87)

What is it you're trying to do?  There's no difference between small
and large mailing lists except that one creates a lot more mail
traffic on your machine, and on your neighbors' machines.

Sendmail's :include: feature works fine, although it's a bit slow.
When somebody sends a message to info-sendmail@ulowell, the load
average on ulowell (a vax 750) goes up to about 8 for ten minutes
or so, and the phone lines get tied up for a while, but other than
that, I don't see any difference from smaller mailing lists, like
amiga-minix@ulowell.

If you're talking about the administrative details, that's another story.

..Bob
info-sendmail-request@ulowell.{edu,uucp,csnet}
amiga-minix-request@ulowell.{edu,uucp,csnet}
postmaster@ulowell.{edu,uucp,csnet}
-- 
Bob Page, U of Lowell CS Dept.   page@ulowell.{uucp,edu,csnet} 

nowicki%rose@Sun.COM (Bill Nowicki) (07/20/87)

In article <3600@felix.UUCP>, zemon@felix.UUCP (Art Zemon) writes:
> 
> I have a relatively large mailing list, info-ultrix, which
> has outgrown the alias facility in sendmail.  

Most early versions of sendmail had bugs and performance problems with
very large mailing lists.  Here at Sun we have a list with four
thousand people on it, and several with over three thousand.  Usually
each of these gets a few messages per day.  Needless to say, we fixed
many bugs with long lists.  We fed bug fixes back to Berkeley, and the
4.3BSD sendmail is much better than the 4.2BSD one.  I am biased, but I
think the SunOS 3.4 (and SunOS 4.0!) sendmails are even better.

	-- WIN

zemon@felix.UUCP (Art Zemon) (07/23/87)

Info-ultrix is over 100 people and the Ultrix v2.0 field
test sendmail (I haven't upgraded to the production version
yet) chokes after about 30 addressees.  It delivers that
many and queues the message for later processing.  When it
processes the message again, the same thing happens, ad
nauseum.

Maybe the production version of sendmail will help.

	-- Art Z.
--
	-- Art Zemon
	   FileNet Corporation
	   Costa Mesa, California
	   ...!hplabs!felix!zemon

page@ulowell.cs.ulowell.edu (Bob Page) (07/24/87)

zemon@felix.UUCP (Art Zemon) wrote that sendmail delivers about 30
addresses out of 100, queues the rest for later, etc.

It is possible that the sendmail.cf has the x option enabled.  A 'Oxn'
line in the config file or a '-oxn' flag in the argument list to
sendmail will tell it to stop processing messages when the load
average gets above 'n'.  Also be aware of the c, q, s, X, y, Y, z and
Z options, which can affect sendmail performance on large mailing
lists.

A temporary solution is to comment out all of these options, refreeze
the .cf file and restart sendmail.  Then try it again.

The problem is that large mailing lists push up the load average, and
sendmail will defer processing until the load average gets back down
to a reasonable level.

..Bob
-- 
Bob Page, U of Lowell CS Dept.   page@ulowell.{uucp,edu,csnet} 

sob@academ.UUCP (Stan Barber) (07/25/87)

In article <23828@sun.uucp> nowicki%rose@Sun.COM (Bill Nowicki) writes:
> We fed bug fixes back to Berkeley, and the 4.3BSD sendmail is much
> better than the 4.2BSD one.  I am biased, but I
> think the SunOS 3.4 (and SunOS 4.0!) sendmails are even better.
>
>	-- WIN

The SunOS3.4 sendmail does not support MX records. It also does not support
the $[ $] cannonicalization convention. These two things alone make it 
difficult to use the SunOS 3.4 sendmail on an internet Sun that has to
deal with lots of mail. I would hope the SunOS 4.0 sendmail can do this 
stuff.

Stan Barber


-- 
Stan	     uucp:{killer,rice,hoptoad}!academ!sob     Opinions expressed here
Olan         domain:sob@tmc.edu                            are ONLY mine &
Barber       CIS:71565,623   BBS:(713)790-9004               noone else's.

matt@oddjob.UChicago.EDU (Matt Crawford) (07/27/87)

In article <166@academ.UUCP> sob@academ.UUCP (Stan Barber) writes:
) The SunOS3.4 sendmail does not support MX records. It also does not support
) the $[ $] cannonicalization convention. These two things alone make it 
) difficult to use the SunOS 3.4 sendmail on an internet Sun that has to
) deal with lots of mail. I would hope the SunOS 4.0 sendmail can do this 
) stuff.

Having recently converted oddjob from a 780 to a 3/280, I have
the (very few) diffs needed to make the 4.3 sendmail work under
sunos 3.3 (and, I presume, 3.4).  Write if you want them.  Of
course they're no good w/o 4.3 source ...  but source can be had
for sendmail by anonymous ftp from ucbarpa.

		Matt Crawford

sob@academ.UUCP (Stan Barber) (07/28/87)

In article <166@academ.UUCP>,  I write:
# The SunOS3.4 sendmail does not support MX records. It also does not support
# the $[ $] cannonicalization convention. These two things alone make it 
# difficult to use the SunOS 3.4 sendmail on an internet Sun that has to
# deal with lots of mail. I would hope the SunOS 4.0 sendmail can do this 
# stuff.

Following this posting, I received mail from Sun agreeing that 3.4 does
not support MX records. However, the writer called the statement about
$[ $] "an outright lie". And asked that I post a "correction". I replied
that I would explain the statement and so I am. 

---------------------- BEGIN EXPLAINATION -----------------------------
The claim that $[ $] does not work is based on tests I ran. Does the
3.4OS Sendmail use the YP to satisfy this resolution or does it ask
NAMED? If it uses YP, it takes MUCH MUCH longer than asking NAMED 
directly to resolve the address. In fact I could not get it to work, but
I was unwilling to wait more than five or six minutes. If it asks NAMED 
directly, something is seriously wrong since 4.3MXsendmail resolves in
seconds. That is the basis for the "does not support" statement. Perhaps
I should say "It also takes greater than 5 or six minutes in the tests
I ran to do the $[ $] cannonicaliztion. Using the same tests and the
same configuration file, it takes seconds with MX_sendmail." That is not
an "outright lie".
-------------------- END OF EXPLAINATION -------------------------------

He was also good enough to supply me with a new sendmail that does
use MX records. For this, I thank him. If Sun supports all their
customers like this, they are to be congratulated.








-- 
Stan	     uucp:{killer,rice,hoptoad}!academ!sob     Opinions expressed here
Olan         domain:sob@tmc.edu                            are ONLY mine &
Barber       CIS:71565,623   BBS:(713)790-9004               noone else's.

zemon@felix.UUCP (Art Zemon) (07/28/87)

Well, I found and solved my problem so no more suggestions,
please!  The problem turned out to not be the sending
sendmail but the receiving one.  I have two machines here,
Fritz and Felix.  Felix talks to the outside world and
Fritz, hiding Fritz from the outside world.  (Fritz is shy.)
Fritz's sendmail.cf sends all uucp addresses to Felix via an
SMTP connection for resolution.

So my mailing list looks like this:
    a!b!c
    x!y!z
    ...
    q!r!s
and so on for lots more than thirty lines.  Stupid me, I
never tried sending something to my list from Felix.  I
always sent from Fritz.  And Felix barfed after about thirty
addresses and told Fritz to eat the rest of them.

Well, I finally tried sending something from Felix and, lo
and behold, Felix p-r-o-c-e-s-s-e-d the entire list, albiet
s-l-o-w-l-y. So now I know the workaround and can
investigate the cure sometime soon.

Thanks for all the suggestions,
--
	-- Art Zemon
	   FileNet Corporation
	   Costa Mesa, California
	   ...!hplabs!felix!zemon