[comp.unix.questions] General file transfer by E-mail

booth@vax135.UUCP (04/02/87)

	WANTED: Network Independant File Transfer by Electronic Mail
	------------------------------------------------------------

The problem: how to transfer an arbitrary file from one arbitrary site
to another using existing network connections.

The only common denominator in network access is electronic mail
service; thus it makes sense to use electronic mail as a basis for
general purpose, network-independant file transfer software.

Specifically, the software should meet the following criteria:

1.  The software should be able to send a file: (a) of ANY SIZE; (b) of
ANY CONTENTS; (c) from ANY SITE to any other site that is reachable by
electronic mail.

2.  The software should use error detection and/or correction so that
the received file can be GUARANTEED CORRECT and complete, with a
negligible probability of error (if it arrives at all).  In particular,
the software must get around the various arbitrary constraints imposed
by existing mailers, such as message line and character maximums,
printable characters only, no "." alone on a line, no "from" in the
first column, etc.  In general, files will have to be encoded in a
limited "safe" character set, and mailed in pieces that are small
enough for intervening mailers to handle.  Corresponding software at
the destination site must decode and reassemble the pieces, which will
not necessarily arrive in order.

3.  It should be very EASY TO PORT the software to a new site that does
not already have it, so that no site needs to depend on another site
already having the software.  (For example, by first sending a small
receiving program, and using it to bootstrap the full software.) This
implies criterion 4 . . .

4.  The software should be COPYABLE -- either in the public domain, or
copyrighted but with explicit permission to copy being automatically
granted for appropriate use.

Does anyone know of such software?  Does anyone know of an existing
effort to write it?  Does anyone have comments on how it should be
done?

Electronic mail is already in place, and arbitrary file transfer is
such a fundamental need, it seems crazy (and frustrating!) that we
still must revert to the stone age practice of writing and shipping
magnetic tapes.
			David Booth
	UUCP: {harvard, seismo, ucbvax}!vax135!booth
	Arpanet: vax135!booth@ucbvax.berkeley.edu

edw@ius2.cs.cmu.edu.UUCP (04/05/87)

In article <1824@vax135.UUCP>, booth@vax135.UUCP (David Booth) writes:
> 
> 	WANTED: Network Independant File Transfer by Electronic Mail
> 	------------------------------------------------------------
> 
> The problem: how to transfer an arbitrary file from one arbitrary site
> to another using existing network connections.


	Whats wrong with FTP?  You can get to anywhere else on the 
network using it.

-- 
					Eddie Wyatt

Those who know what's best for us-
Must rise and save us from ourselves
Quick to judge ... Quick to anger ... Slow to understand...
Ignorance and prejudice and fear [all] Walk hand in hand.
					- RUSH 

wagner@iaoobelix.UUCP (04/06/87)

I suggest the package atob/btoa (ascii to binary) which is (tell me if I'm
wrong) in the public domain. This package comes with compress 4.0 and contains
some nice utilities to send arbitrary files (even directory structures) via
email. I have received the files from Jeff Peck (peck@sun.com).

Tarmail works as follows:

You make a somehow large and bulky file in tar format, pipe it into compress,
convert it (by means of btoa) to a simple text file (i.e. a file containing no
control chars) and just mail it over to the people you want. If the entire
file gets too large, I use the slice utility (the one you have to be careful
not to slice the crt) to split up the large file into smaller pieces. Upon
receipt at the remote site, the pieces can be glued together and unpacked
using untarmail.

Although it is very rarely the case that I am mailing binaries (i.e.
precompiled files), I found tarmail very useful to sent out non-text data
files or whole directory trees in a quite compact format. And as far as I can
see these programs are quite portable (I compiled them without changes on a
VAX/BSD4.2, a Sun/OS3.2 and a uVAX/ULTRIX, what is surprising, anyway, if I
think of experiences with nice little incompatibilities among these machines
and *IXes).

Well, that's my experience with tarmail and friends. I hope this is of use
(and doesn't lead to an overflow of Jeff's mailbox :-).

Juergen Wagner,		      USENET: ...seismo!unido!iaoobel!wagner
("Gandalf")			or	  ...!pyramid!iaoobel!wagner

Mail:	Juergen Wagner
	Fraunhofer-Institut IAO
	Rosenbergstr. 28
	D-7000 Stuttgart 1
	Federal Republic of Germany
Phone:	+ 49 711 6648-205

"If you put garbage in a computer nothing comes out but garbage. But this
 garbage, having passed through a very expensive machine, is somehow enobled
 and none dare criticize it."

mark@cogent.UUCP (04/07/87)

In article <1824@vax135.UUCP> booth@vax135.UUCP (David Booth) writes:
>
>	WANTED: Network Independant File Transfer by Electronic Mail
>	------------------------------------------------------------
>
>The problem: how to transfer an arbitrary file from one arbitrary site
>to another using existing network connections.

A possible strategy might be to treat all files as binary rather than
"this is text/that is binary", and then do a binary-to-text conversion.
There is some such program to do this now.  I forget the name.  Then,
the stuff would be mailable, regardless of it's original contents.

>1.  The software should be able to send a file: (a) of ANY SIZE; (b) of
>ANY CONTENTS; (c) from ANY SITE to any other site that is reachable by
>electronic mail.

(1) would be achieved i *think* (not certain), (2) would be achieved for
sure, but it would cost you some efficiency since the total byte count
would be increased, (3) would be achieved as long as each site has the 
program to convert files to a mailable form AND also a program to put it
back to it's original form again upon receipt (my next point, see below).

>2.  The software should use error detection and/or correction so that
>the received file can be GUARANTEED CORRECT and complete, with a
>negligible probability of error (if it arrives at all).  In particular,

UUCP would take care of that.

>the software must get around the various arbitrary constraints imposed
>by existing mailers, such as message line and character maximums,
>printable characters only, no "." alone on a line, no "from" in the
>first column, etc.

The decoding side would have to be "smart" so that it could discard
unwanted "garbage lines" above and below the text to be decoded.
That would make things much easier for the person on the receiving
end.  It would be a "black box" to them.  Little expertise needed.

>3.  It should be very EASY TO PORT the software to a new site that does

In C for portability of course...

>(For example, by first sending a small
>receiving program, and using it to bootstrap the full software.) 

Initial installation would require technical expertise most likely.
It is doubtful that this can be escaped.

>Does anyone know of such software?  Does anyone know of an existing
>effort to write it?  Does anyone have comments on how it should be
>done?

bits and pieces of this exists already in the UUCP world, and would
need to be pieced together under a neat "roof".  I assume that you
don't wish to be using the file xfer facilites that UUCP aleady
offers (or perhaps you cannot for some reason).  If you wish to
extend this outside of the UUCP realm then some portability must be 
sacrificed, but it still might be a workable concept.

>Electronic mail is already in place, and arbitrary file transfer is
>such a fundamental need, it seems crazy (and frustrating!) that we
>still must revert to the stone age practice of writing and shipping
>magnetic tapes.

Agreed.  We use UUCP file transfer extensively.  We send very FEW tapes.
This issue is not new.  Nor are the points I have made.
-- 
+----------------------------------------------------------------------------+
|     Mark Steven Jeghers: the terrorist smuggling CIA weapons to Libya      |
|                                                                            |
|     {ihnp4,cbosgd,lll-lcc,lll-crg}|{dual,ptsfa}!cogent!mark                |
|                                                                            |
|     Standard Disclaimer:  Contents may have settled during shipment.       |
+----------------------------------------------------------------------------+

woods@hao.UUCP (04/15/87)

In article <1824@vax135.UUCP> booth@vax135.UUCP (David Booth) writes:
>1.  The software should be able to send a file: (a) of ANY SIZE; (b) of
>ANY CONTENTS; (c) from ANY SITE to any other site that is reachable by
>electronic mail.

  Anyone sending huge files by mail through our machine will have his mail
thrown away. We have to pay real money to transmit that stuff. If you want
to send files to another site, establish a uucp link to that site and
pay for it yourself. There's no such thing as a free lunch, which is
what you are asking for. In the first place, many sites, including ours,
have a limit on the maximum size message they will pass, which is built
into the mailer. This automatically makes 1(a) above impossible. Electronic
mail was NOT DESIGNED to send files. Please don't try to use it for a purpose
it is not suited for. 

--Greg
-- 
UUCP: {hplabs, seismo, nbires, noao}!hao!woods
CSNET: woods@ncar.csnet  ARPA: woods%ncar@CSNET-RELAY.ARPA
INTERNET: woods@hao.ucar.edu

randy@umn-cs.UUCP (04/17/87)

In article <1084@ius2.cs.cmu.edu> edw@ius2.cs.cmu.edu (Eddie Wyatt) writes:
>In article <1824@vax135.UUCP>, booth@vax135.UUCP (David Booth) writes:
>> 	WANTED: Network Independant File Transfer by Electronic Mail
>> The problem: how to transfer an arbitrary file from one arbitrary site
>> to another using existing network connections.
>
>	Whats wrong with FTP?  You can get to anywhere else on the 
>network using it.
>					Eddie Wyatt

"anywhere else on the network?"  What network are you talking about?  There
are many sites that still don't have an arpanet connection (a perfect
example: nermal here, a Wicat 150 with a whopping 6 serial ports!
(TCP/IP?  never heard of it...) hopefully soon I'll have USENET coming in
on my 1200 baud modem, but for now my main file transfer method is ~%take)

The point is that arpanet isn't everywhere, by a LONG shot.  However,
I do think it would be nice to come up with a mechanism that would
interface uucp with ftp, for file transfer across multiple networks.
Fits in nice with the domain names, too...  uucp ux.acss.umn.edu!test x
regardless of what networks are traversed between here and ux.
Any feasability thoughts?
	-randy
-- 
Randy Orrison, University of Minnesota Math Department
(even though I'm a Computer Science major)
UUCP: {ihnp4, ?}!umn-cs!randy	ARPA: randy@cs.umn.arpa
"Any opinions expressed are not necessarily irrelevant."

coscilr@uhnix1.UUCP (09097) (04/28/87)

In article <1824@vax135.UUCP> booth@vax135.UUCP (David Booth) writes:
>
>	WANTED: Network Independant File Transfer by Electronic Mail
>	------------------------------------------------------------
>
>The problem: how to transfer an arbitrary file from one arbitrary site
>to another using existing network connections.
>

I have one which does file transfer directly on the 3bnet between 3b2s and
3b20s (reliable transfer).  Tell me more about your configuration if you
think this is ok.

Francis  (CSNET: mkkam@houston.edu)