[comp.mail.mh] Moderating a mailing list digest

scott@sage.uchicago.edu (Scott Deerwester) (08/17/90)

I should have known.  I mean, what right did I have to expect that I
could use :include: in /usr/lib/aliases on a file of 170 names?
Sendmail loses its lunch (I discovered after sending issue *3*!), and
some random subset of the people on the list are actually receiving
it.  I think that something like 130/170 are actually getting the
thing, but I don't really know.  I suspect that the problem (and this
is raw speculation) is that it's messing up lines where multiple
people on a single machine are listed.

A line from sendmail's log:

  Aug 16 14:07:36 localhost:      3331 sendmail: AA03304: to=<polyglot%xanadu.
  uucp@uunet.uu.net>>et>polyglot%visix.uucp@uunet.uu.net>,<polyglot%viar.uucp@
  uunet.uu.net>,polyglot%iii-sed.uucp@uunet.uu.net>,<polyglot%aspect.uucp@uune
  t.uu.net>,<polyglot%edsr.eds.com@uunet.uu.net>,<

This is *not* a happy computer program...

So, is anybody out there using a UNIX machine to moderate a mailing
list digest?  I've been using MH to handle incoming mail.  Submissions
go to one folder, articles for the current issue in another and
requests in a third.  Here's the shell script (written by Keith
Waclena; thanks, Keith!) that I have been using to send the thing out.
It all works just fine, except that sendmail core dumps in the middle
(end? beginning? *shrug*) of sending it.

-------

#!/bin/sh
#
# distdigest

PATH=$PATH:/usr/local/lib/mh

COMPS=/tmp/comps$$

folder +working >/dev/null

cd `mhpath +working`

(
    cat $HOME/Mail/polyglotcomps

    header -f Subject `/bin/ls -1 | sort -n` |
    sed 's/[^:]*://' |
    uniq -c |
    perl -pe 's/^\s+(\d+)\s+(.*)/$2 ($1)/;' -e 's/ \(1\)$//' |
    ff -c -P 1 -h "" -H 1 -F 1
) >$COMPS

forw -digest Polyglot -form $COMPS all

-------

Perhaps I could use forw anyway, to construct the digest (I have a
"drafts" folder where an issue goes while it's being built).  Perhaps
then I could so something like:

--------
(
    cat $HOME/Mail/polyglotcomps

    sed 's/^/Bcc: /' $AddressFile

    header -f Subject `/bin/ls -1 | sort -n` |
    ...
-------

and then invoke "sendmail -t -fPolyglot-dist" to actually send it.  In
any case, what I'm really looking for is help from somebody who's
really doing this successfully.  Anybody out there?

-------
Scott Deerwester            | Internet: scott@tira.uchicago.edu  | ~{P;N,5B~}
Center for Information and  | Phone:    312-702-6948             |
   Language Studies         | 1100 E. 57th, CILS                 |
University of Chicago       | Chicago, IL 60637                  |

marvit@HPLPM.HPL.HP.COM (Peter Marvit) (08/18/90)

I have had great success using MH's own aliasing with a moderated mailing
list of over 900 names.  I found our local sendmail had a magic limit of
around 500 addresses, though that was many revisions ago.  I decided to
take the job up to the User Agent, away from the transport layer.

I suggest you do something like:

	To: my-mailing-list: my-mh-alias;

to do "address hiding" according RFC-822.  Recipients will only see

	To: my-mailing-list:;

which tends also to cut down on bogus replies to everyone.

As an extra precaution, perhaps unnecessary now, I also split my large
list dynamically into smaller chucks of 50-100 addresses when I send out
a Digest.  I'll be glad to send ksh scripts, but they're a little
specialized for my exact setup.

You might talk to your local sendmail guru to see if there are any
hidden/magic limits you might be exceeding at your site, if you don't
want to bother with the MH alias usage.

	-Peter "Neuron Digest Moderator" Marvit

: Peter Marvit   Hewlett-Packard Labs in Palo Alto, CA   (415) 857-6646    :
: Internet: <marvit@hplabs.hpl.hp.com>  uucp: {any backbone}!hplabs!marvit :

jamesp@metolius.wr.tek.COM (James T Perkins) (08/18/90)

Scott, and others,

I've successfully used :include: in the aliases file with sendmail for a
mailing list for a number of years now.  Currently the list is at 156
people.  MH is wonderful for managing folders of messages for archives
and in automatic preparation of digests.

My :include: file consists of lines of addresses, a single address per
line, in the form:

	user@host.subdomain.domain (User's name)

Note that I use a single person per line, and I place the comment (the
stuff in parens) after the address.  I had difficulty at first, and
discovered it was because I was listing comments ahead of addresses.  I
also presort the addresses in this list, with the domain name being the
first key, subdomain the second key, etc.  This makes sendmail minimize
the number of connections it starts up, and keeps the load lower on the
Tek mail server.

For managing my digests, I use an alias which redirects the mail to the
stdin of a shell script.  The shell script invokes a setuid program
which spawns a second shell script, setuid to my personal uid and gid.
This second shell script uses `mhpath new +$incfolder` to determine a
new message file pathname to plop the message in.  As the message is
being deposited, I use awk to append a serial number line to the message
header "Archive-Message-Number: NNN".

Nightly, the digesting at(1) script wakes up.  It uses mhpath again to
find the messages in the incfolder.  I create my digests with awk and
shell scripts, and use mhpath.  In this way I avoid using mh commands
that would change my mh context, plus it gives me very free rein on the
format of the digest.  I make sure that the output is burstable with MH
burst, though.  I use mh's "post -dist" command to send my mail out,
using "Resent-To" fields to send the digest.

If you'd like, I'm willing to share my source code, such as it is.  It's
not terribly well-written, it's overly complex, but it's probably better
than a poke in the eye with a sharp stick.

James