[comp.mail.uucp] uucp mail inefficiency

roy@phri.UUCP (04/02/87)

In article <7326@boring.mcvax.cwi.nl> jack@boring.UUCP (Jack Jansen) writes:
> one of the gross misfeatures of most uucp's is [...] for small files, the
> overhead is enourmous (two files transmitted for each tiny mail message)

	What does uucp have to do with mail?  Uucp is a transport protocol
which only knows how to transmit files and perform remote execution of
commands.  Uucp itself doesn't know from mail at all.  It would be trivial
(or at least straight-forward) to design a batched mail system which ran on
top of uucp -- take all the messages bound for a given site and batch them
into a single data file and a single XQT file for transmission.  You could
even compress the files before sending them (just like news does).  I'm not
sure if it's worth the effort, but it could be done.  Come to think of it,
you could probably use kermit as the mail transport protocol instead of
uucp; modern kermit servers can transfer files and execute remote commands
as well as uucp can.

	My guess is that the single most cost effective way to increase the
efficiency of uucp is to make sure the comm port you use has DMA input and
output.  It doesn't matter much if you're running kermit, zmodem, uucp,
SLIP, or just plain "cat > file"; a steady stream of input, even at 1200
baud, on an interrupt-per-character port is going to swamp most machines.
-- 
Roy Smith, {allegra,cmcl2,philabs}!phri!roy
System Administrator, Public Health Research Institute
455 First Avenue, New York, NY 10016

"you can't spell deoxyribonucleic without unix!"

fair@ucbarpa.Berkeley.EDU.UUCP (04/03/87)

UUCP mail can be sent one copy for multiple recipients to any host
except a 4.1 BSD UNIX (their "rmail" is screwed up). However, in all
cases, you will run up against the size of the command execution
buffer in UUXQT, which will ungracefully core-dump when you exceed it.

If we really wanted to do this right, there is a version of the SMTP
(Simple Mail Transfer Protocol) for batched systems that BITNET uses
called (strangely enough) BSMTP (for Batched SMTP). Someone could
write a new command to be executed on the remote end of a UUCP link
(let's say "bsmtp" with no arguments), and it can read standard input
for the BSMTP protocol and the letter(s), which can be of arbitrary
length, since we're getting it on stdin, rather than in the command
line argument buffers.

I can probably cause the RFC to be posted to mod.sources.doc if enough
people are interested.

	Erik E. Fair	ucbvax!fair	fair@ucbarpa.berkeley.edu

sl@van-bc.UUCP (04/03/87)

In article <2629@phri.UUCP> roy@phri.UUCP (Roy Smith) writes:
>In article <7326@boring.mcvax.cwi.nl> jack@boring.UUCP (Jack Jansen) writes:
>> one of the gross misfeatures of most uucp's is [...] for small files, the
>> overhead is enourmous (two files transmitted for each tiny mail message)
>
>commands.  Uucp itself doesn't know from mail at all.  It would be trivial
>(or at least straight-forward) to design a batched mail system which ran on
>top of uucp -- take all the messages bound for a given site and batch them
>into a single data file and a single XQT file for transmission.  You could

It is. And there's even a standard (sort of). AT&T Mail accepts "bundles" of
mail from PC type machines (via XModem but that's not overly important).
These bundles consist of multiple messages, each message has a header called
"Content-Length:" which has the size in bytes of the content part of the
message. This allows simple unbatching of the mail at the other end because
the receiver knows when to start looking for the next message. The addresses
are carried in the message with "To:" header lines.

This is not strictly 822 compatible, but if AT&T can do it.....

I have two programs just about finished which will: 

	pcmail: create a mail bundle on a PC.
	
	rpcmail: unbundle on a Unix system, feeding the messages to 
			rmail one at a time. Addresses taken from To: lines.

PC: command for sending mail to unix

		uux unix!rpcmail < mailbundle


>even compress the files before sending them (just like news does).  I'm not


PC: command for sending compressed mail bundle to unix

		uux unix!rpcmailz < `zcat mailbundle`

Unix: rpcmailz script for receving compressed bundles

		zcat | rpcmail 

If there is interest I will post these to net when tested (between my Mac
and Callan with uuslave or rz/sz).


Post Script. 

I think that mail transport to / from PC's will be solved in the next
few months thanks to the efforts of John Gilmore and such getting uuslave
whipped into shape. 

What we need now is a nice mail/news reader program.  Assume mail and 
news coming into the PC, but only mail going out. To post news articles 
mail them to a nearby unix host who can set up a "postnews" postbox using
recnews to receive and post the article. 

Does anyone know if there's a new version of IMS (Intelligent Mail
System). This looks like it might be a good starting point for above. Who
wrote IMS? The copy I have contains no information about copyright,
ownership, or authorship.


-- 
Stuart Lynne	ihnp4!alberta!ubc-vision!van-bc!sl     Vancouver,BC,604-937-7532

sl@van-bc.UUCP (04/04/87)

In article <18164@ucbvax.BERKELEY.EDU> fair@ucbarpa.Berkeley.EDU (Erik E. Fair) writes:
>UUCP mail can be sent one copy for multiple recipients to any host
>except a 4.1 BSD UNIX (their "rmail" is screwed up). However, in all

A simple script to copy stdin to a temp file, and then copy to real rmail for
each arguement is a quick fix for this.

>If we really wanted to do this right, there is a version of the SMTP
>(Simple Mail Transfer Protocol) for batched systems that BITNET uses
>called (strangely enough) BSMTP (for Batched SMTP). Someone could
>write a new command to be executed on the remote end of a UUCP link
>(let's say "bsmtp" with no arguments), and it can read standard input
>for the BSMTP protocol and the letter(s), which can be of arbitrary
>length, since we're getting it on stdin, rather than in the command
>line argument buffers.

Actually there is already a "standard" way to do this. According to RFC 976,
section 2.3 Batch SMTP, a conforming host should set up a mail ID called
b-smtp. All mail sent to this ID is interpreted as Batch SMTP commands. The
example given for BSD systems with sendmail:

	b-smtp: "|egrep '^#' | sed 's/^#//' | /usr/lib/sendmail -bs"

Of course we still need the bsmtp program to do the interpretation for non
BSD systems that don't have sendmail. Any volunteers!


-- 
Stuart Lynne	ihnp4!alberta!ubc-vision!van-bc!sl     Vancouver,BC,604-937-7532

david@ukma.UUCP (04/05/87)

As for batching up many pieces of mail into one file, there is *already*
a method for doing exactly that, which requires NO silly additions to
the header, and is known to work really well.

It's called BSMTP.  It's a simple derivation of SMTP which was devised
by Allan Crosswell (the guy that wrote one of the better mail systems
for IBM Mainframes), and is heavily used on BITNET.  Since BITNET is
the same sort of network as UUCP then BSMTP is just as applicable.
("same type" means Store & Forward).

A BSMTP package looks something like:

HELO sender.domain.name
MAIL FROM:<rfc822-address-or-path>
RCPT TO:<rfc822-address-or-path>
   (Repeat the RCPT TO:'s as many times as necessary)
DATA
   "something" goes here ... in principle "something" could be anything,
   but the spec says to put an rfc822 mail message here.  one point of
   order is that any line which begins with a "." has the "." doubled,
   and the receiving end takes away the new ".".
.
   (You can repeat a MAIL FROM:, RCPT TO:, DATA sequences as many times
   as you like)
QUIT

Go and read RFC976 where it's suggested to use BSMTP in the UUCP world,
and the method for doing that.  Another applicable RFC is RFC821 where
SMTP is documented.

In case y'all're interested, I have a BSMTP decoder written which will
feed mail into mmdf, sendmail, or /bin/mail delivery systems (without
recompiling either ... treeky..).  Expect a copy of my program to be
in the next smail distribution ... at least I did give them a copy of
the program along with permission to use it.  I'm using it every
day here to receive mail from BITNET, it's been running w/o problems
since last November.


-- 
-----   David Herron,  cbosgd!ukma!david, david@UKMA.BITNET, david@ms.uky.csnet
-----  (also "postmaster", "news", and the Usenet map maintainer for Kentucky.)
-----
/*EOF

johnc@haddock.UUCP (04/07/87)

>					...  It would be trivial
>(or at least straight-forward) to design a batched mail system which ran on
>top of uucp -- take all the messages bound for a given site and batch them
>into a single data file and a single XQT file for transmission.  You could
>even compress the files before sending them (just like news does).  I'm not
>sure if it's worth the effort, but it could be done.  

It would be rather easy.  This is exactly what news does already.  All you'd
have to do is clone the batch command and make it put a different program
name in the lines in the message that say "#rnews".  Then you clone the rnews
program and give it the new name, and modify it so that it passes its files
to a routing mail reader, which would discover the embedded "To:" lines and
send them on their way.

Alternatively, you could be even lazier and read the installation instructions
for creating the to.<host> newsgroups.  Instead, you create some mail.<host> 
newsgroups, and arrange with your neighbors to install a daemon (a clone of
the uumap package) that automatically reads these groups and sends the mail
on to the next host.  Presumably the daemon would just hand the articles to 
the same routing mailer.  Of course, you want to compile usenet with the 
FASCIST option in usenet, so you can prevent local users from perusing the 
mail.* newsgroups.

Let's see, there's gotta be yet another cute way to do it....

[Hey, anyone wanna pay me to do it?]

-- 
	John Chambers	(617)247-1155 <...!ima!johnc>
[The above opinions are my own; for a small fee, they can be yours, too.]

allbery@ncoast.UUCP (04/08/87)

As quoted from <561@van-bc.UUCP> by sl@van-bc.UUCP:
+---------------
| Does anyone know if there's a new version of IMS (Intelligent Mail
| System). This looks like it might be a good starting point for above. Who
| wrote IMS? The copy I have contains no information about copyright,
| ownership, or authorship.
+---------------

My goodness, that critter's popular.  I think it's got a wider distribution
than UNaXcess Conferencing, for all that I sent it out once with no fanfare.

Yes, a new version of ims(1) is in the works as part of UNaXcess Mail.  I
find it interesting that the first version, which was basically a little
critter I wrote for my own purposes anbd decided to post ``just because'',
is used as THE mailer on a number of local systems, and I get a fair amount of
mail on it still.

++Brando
-- 
 ____   ______________
/    \ / __   __   __ \   Brandon S. Allbery		|  QUOTE OF THE DAY:
 ___  | /__> /  \ /  \    aXcess Company		|
/   \ | |    `--, `--,    6615 Center St. #A1-105	|	  `
|     | \__/ \__/ \__/    Mentor, OH 44060-4101		|
\____/ \______________/   +1 216 974 9210		|
cbatt!cwruecmp!ncoast!allbery ncoast!allbery%case.CSNET@relay.CS.NET BALLBERY
	    (UUCP)                      (CSNET/Internet)             (MCIMail)