[comp.emacs] RMAIL file ---> UNIX mail file

yvo@cs.utexas.edu (Yvonne van Olphen) (02/02/89)

I accidentally started up RMAIL while in emacs, and it sucked up all
my (30+) messages that were in my spool file.  I don't know RMAIL, and
I want my messages back in a UNIX mail file without having to C-o
*every single* message individually.  Is there some way I can
completely convert the RMAIL file back into a UNIX mail file
painlessly??  HELP!

	-Yvonne

pierson@mist (Dan Pierson) (02/04/89)

In article <156@ai.cs.utexas.edu>, yvo@cs (Yvonne van Olphen) writes:
>
>I accidentally started up RMAIL while in emacs, and it sucked up all
>my (30+) messages that were in my spool file.  I don't know RMAIL, and
>I want my messages back in a UNIX mail file without having to C-o
>*every single* message individually.  Is there some way I can
>completely convert the RMAIL file back into a UNIX mail file
>painlessly??  HELP!
>
>	-Yvonne
Well, I haven't used this in a while, but the following code may help.
In response to an earlier question: one way to get from RMAIL/BABYL
format to MH format is to convert back to UNIX mail format, then use
``inc +mh-mailbox -file unix-mailfile''.

(defun rmail-apply (function &rest args)
  "Apply a function to all messages.
For each message, FUNCTION is applied to the ARGS."
  (message "Applying function to all messages...")
  (let ((msgnum 1)
	(buffer-read-only nil))
    (save-restriction
      (save-excursion
	(widen)
	(goto-char (point-min))
	(while (>= rmail-total-messages msgnum)
	  (rmail-show-message msgnum)
	  (or (null function)
	      (apply function args))
	  (setq msgnum (1+ msgnum))))))
  (message "Applying function to all messages...done"))

(defvar babyl-file-root "~/mail/"
  "Pathname of the directory where BABYL files live")

(defun babyl-to-mail (name)
  (interactive "sMail File: ")
  (let ((in (concat babyl-file-root name))
	(out (concat "~/" name ".mbox")))
    (rmail in)
    (rmail-apply 'rmail-output out)
    (rmail-quit)))
-- 
                                            dan

In real life: Dan Pierson, Encore Computer Corporation, Research
UUCP: {talcott,linus,necis,decvax}!encore!pierson
Internet: pierson@multimax.encore.com

kjones@talos.UUCP (Kyle Jones) (02/06/89)

In article <156@ai.cs.utexas.edu> yvo@cs.utexas.edu (Yvonne van Olphen) writes:
 >I accidentally started up RMAIL while in emacs, and it sucked up all
 >my (30+) messages that were in my spool file.  I don't know RMAIL, and
 >I want my messages back in a UNIX mail file without having to C-o
 >*every single* message individually.  Is there some way I can
 >completely convert the RMAIL file back into a UNIX mail file
 >painlessly??  HELP!

I've heard this complaint before.  I think `rmail' should be a
disabled command.  Even though it's not bound to a key by default, I
still don't like the idea of an editor command gobbling up my mailbox,
converting it to another format, and deleting the original without a
word of warning.

Back in Novemeber Ed Wilkinson (E.Wilkinson@massey.ac.nz) posted a
program called `b2m' which converts BABYL format files into UNIX
mail format.

popovich@gollum.cs.columbia.edu (Steve Popovich) (02/07/89)

I also had a run-in with this once, when I finally decided to change
over to RMAIL after my mbox got TOO too big to handle.  (Actually what I
probably want is more like mh-e, but mh isn't working here.)  I started it
up on my new mail, and SURPRISE -- the default isn't just to suck in NEW
mail, it wants to grab my OLD mail as well!  Crunch, crunch, grind, grind...
(grovel through THOUSANDS of messages)...URK!  Out of memory...(for those
of you who don't remember the original Emacs. :-)  Thought I'd lost my new
mail for sure after that, but eventually found the .newmail file it left
hanging around...

I don't think rmail needs to be a disabled command, but I do think that
having "mbox" as one of its default inputs is a little excessive.  If
somebody WANTS to convert all of their old mail, they certainly can -- but
why check for it every time?  Most of all, why give The Big Gotcha to
first-time users?  Mail, MM, MH...none of them convert all of your old
messages to their format by default.  Why should RMAIL?

kjones@talos.UUCP (Kyle Jones) (02/10/89)

Steve Popovich writes:
>I don't think rmail needs to be a disabled command, but I do think that
>having "mbox" as one of its default inputs is a little excessive.  If
>somebody WANTS to convert all of their old mail, they certainly can -- but
>why check for it every time?  Most of all, why give The Big Gotcha to
>first-time users?  Mail, MM, MH...none of them convert all of your old
>messages to their format by default.  Why should RMAIL?

Indeed, why should RMAIL convert your mail to a different format at
all?  What's so wrong the existing UNIX mail format that RMAIL
couldn't have layered its features on top of it?  I really want to
read my mail under Emacs, but I steadfastly refuse to convert my
mailbox files to an alien format just to please RMAIL.

One of these days the urge to rewrite RMAIL to use the UNIX mail
format will be too great... before that happens, would the RMAIL
implementors (or anyone who knows) care to comment on why the existing
mail format was not used?

pf@csc.ti.com (Paul Fuqua) (02/12/89)

    Date: Friday, February 10, 1989  8:30am (CST)
    From: kjones at talos.UUCP (Kyle Jones)
    Subject: Re: RMAIL file ---> UNIX mail file
    Newsgroups: comp.emacs
    
    Indeed, why should RMAIL convert your mail to a different format at
    all?  What's so wrong the existing UNIX mail format that RMAIL
    couldn't have layered its features on top of it?

1.  Compatibility with Babyl, the ITS/Twenex Emacs mail-reader, which
brings with it compatibility with lispm mail-readers.  (Wasn't Babyl's
ancestor named Rmail?)

2.  Where do you put the labels (keywords, whatever) and attributes
(like Unseen, Answered, etc)?

3.  Where do you cache the reformatted header?

4.  Where do you put file-specific attributes like the inbox list and
others that rmail doesn't use yet?


Paul Fuqua                     pf@csc.ti.com
                               {smu,texsun,cs.utexas.edu,rice}!ti-csl!pf
Texas Instruments Computer Science Center
PO Box 655474 MS 238, Dallas, Texas 75265

mkhaw@teknowledge-vaxc.ARPA (Mike Khaw) (02/14/89)

<69658@ti-csl.csc.ti.com>, by pf@csc.ti.com (Paul Fuqua):
> 2.  Where do you put the labels (keywords, whatever) and attributes
> (like Unseen, Answered, etc)?

What's wrong with using the "Status: " header that other Unix
mail user-agents (including plain old "mail") use?

Mike Khaw
-- 
internet: mkhaw@teknowledge.com
uucp:	  {uunet|sun|ucbvax|decwrl|ames|hplabs}!mkhaw%teknowledge.com
hardcopy: Teknowledge Inc, 1850 Embarcadero Rd, POB 10119, Palo Alto, CA 94303

kjones@talos.UUCP (Kyle Jones) (02/15/89)

In article <69658@ti-csl.csc.ti.com>
  Paul Fuqua gives reasons for RMAIL's use of BABYL files:

> 1.  Compatibility with Babyl, the ITS/Twenex Emacs mail-reader, which
> brings with it compatibility with lispm mail-readers.  (Wasn't Babyl's
> ancestor named Rmail?)

But isn't GNU Emacs primarily targeted at UNIX systems?  Specifically
the BSD-like system that GNU will eventually be?  No UNIX system that
I know of uses BABYL format mail files.

One reason I find BABYL files irritating is because I have to treat
them almost like binaries.  If I want to concatenate them, I can't use
the cat(1) command.  The format contains control characters, so I
can't mail a BABYL file either.  If I move to another system, it
better have GNU Emacs otherwise I and my mail files are out of luck.
(Any other BABYL mail readers running on UNIX systems out there?)

> 2.  Where do you put the labels (keywords, whatever) and attributes
> (like Unseen, Answered, etc)?

Use the `X-' header extensions provided by RFC822.  E.g. X-RMAIL-Labels.
Berkeley mail(1) does this now, except it uses a Status header
instead of the technically correct X-Status.

> 3.  Where do you cache the reformatted header?

What reformatted header?  Do you mean the BABYL header generated by
RMAIL?  If I were writing RMAIL again I wouldn't have it use the
BABYL format at all, even internally.

> 4.  Where do you put file-specific attributes like the inbox list and
> others that rmail doesn't use yet?

I'm not sure what the inbox list is supposed to do.  But the fact that
it is file specific rather than message specific is a problem.  How
about this: A dummy message could be prepended to the mail file that
contains this (and any other file specific) information.  This message
would contain a special header (e.g. X-RMAIL-Data) so that RMAIL would
know not to show this message to the user.  Other UNIX mailers wouldn't
understand this convention but they could still parse the mail file.

			* *

At one point I attempted to rewrite the existing RMAIL to use the
RFC822 message format but was confounded by the fact that the BABYL
file dependencies are scattered throughout.  I now believe rewriting
it from scratch would be the most expedient solution.

pf@csc.ti.com (Paul Fuqua) (02/16/89)

    Date: Wednesday, February 15, 1989  8:39am (CST)
    From: kjones at talos.UUCP (Kyle Jones)
    Subject: Re: RMAIL file ---> UNIX mail file
    Newsgroups: comp.emacs,gnu.emacs

I had a long reply mostly composed, when I realised that I was missing
the point:  Rmail uses a special mail-file format because it is a closer
textual representation of the abstract notion of a mail-file.

A mail file is just a sequence of messages.  The file has some
attributes (inbox, global label list), and each message has some text
and some attributes (labels, unseen/deleted/answered, summary line, old
header).  The Babyl/Rmail choice is to parse incoming messages into that
abstract form, and save them back out in a textual form that is closer
to the abstract than to the original.

I think that many problems would be solved if Rmail's O and ^O listened
to numeric arguments.  Then, to convert a file, you could go to the
first message and type 9999 ^O (Help M in Rmail should mention this one
prominently for the accidental users).  To move all the messages to
another Rmail file, do 9999 O.

Yes, you can stick the labels into a special header line, and keep the
file attributes in a special message, and trim the headers from the
originals at load time.  If your priorities are compatibility with Unix
mailers and manipulation with Unix file tools, you'd probably want to do
so.  You'll pay a price in runtime, though, because of the extra
recalculation and searching.

My priorities are different, but I'm a lispm/Teco hacker, not a Unix
hacker.  If you want to discuss them, or if you want specific answers
to your questions, let's do it by mail.

Paul Fuqua                     pf@csc.ti.com
                               {smu,texsun,cs.utexas.edu,rice}!ti-csl!pf
Texas Instruments Computer Science Center
PO Box 655474 MS 238, Dallas, Texas 75265

jr@bbn.com (John Robinson) (02/16/89)

In article <434@talos.UUCP>, kjones@talos (Kyle Jones) writes:
>In article <69658@ti-csl.csc.ti.com>
>  Paul Fuqua gives reasons for RMAIL's use of BABYL files:
>
>> 1.  Compatibility with Babyl, the ITS/Twenex Emacs mail-reader, which
>> brings with it compatibility with lispm mail-readers.  (Wasn't Babyl's
>> ancestor named Rmail?)
>
>But isn't GNU Emacs primarily targeted at UNIX systems?  Specifically
>the BSD-like system that GNU will eventually be?  No UNIX system that
>I know of uses BABYL format mail files.

Emacs started with no rmail code, but it was easy to steal it from
elsewhere, namely the existing implementation on some random lisp
machine, which happened to be using Babyl format.  This is not an
endorsement of that, or an anti-endorsement of Berkeley mail; it just
happened.  Yes, Babyl's predecessor on ITS/Twenex was named rmail.
The grandaddy of them all was called readmail, and its buddy was
sndmsg.

At a more general level, GNU emacs' target is GNU, alias "GNU's not
Unix".  I suspect the mail "standard" for GNU is not settled yet.  If
a package were available under the FSF terms and conditions, and it
happened to have good functionality, and it happened to use Babyl
format and not Berkeley mail, I have no doubt what GNU Unix would
offer.  Given the tahoe effort, though, Berkeley may already be
available under the right terms, so perhaps FSF would like to change
over.

>	 If I were writing RMAIL again I wouldn't have it use the
>BABYL format at all, even internally.

I doubt anyone else would either, but I suspect Babyl/rmail predates
GNU emacs.

>At one point I attempted to rewrite the existing RMAIL to use the
>RFC822 message format but was confounded by the fact that the BABYL
>file dependencies are scattered throughout.  I now believe rewriting
>it from scratch would be the most expedient solution.

If you step back (again), you would look around for more modern models
of how mail ought to be done (independent of whether it is accessed
from inside emacs, or what machine is hosting the service).  This
would lead you to X.400, an international standard set that provides
for electronic mail.  The implementation model is like what we are
running on machines here at BBN - MMDF is the mail delivery agent, and
MH is the mail user agent.  Separating these groups of functions is
crucial.  In emacs, mh-e is an interface (front-end, really) to the MH
user agent software.  In principle, you could write all the support
directly in elisp, but mh-e doesn't use that approach.

The internet is working out a transition plan for X.400.  At some
point it will supplant RFC822.  There will be gateways between the two
for quite a long time.

I have redirected followups to gnu.emacs.
--
/jr
jr@bbn.com or bbn!jr

rlk@think.com (Robert Krawitz) (02/16/89)

In article <434@talos.UUCP>, kjones@talos (Kyle Jones) writes:
]In article <69658@ti-csl.csc.ti.com>
]  Paul Fuqua gives reasons for RMAIL's use of BABYL files:
]
]> 1.  Compatibility with Babyl, the ITS/Twenex Emacs mail-reader, which
]> brings with it compatibility with lispm mail-readers.  (Wasn't Babyl's
]> ancestor named Rmail?)
]
]But isn't GNU Emacs primarily targeted at UNIX systems?  Specifically
]the BSD-like system that GNU will eventually be?  No UNIX system that
]I know of uses BABYL format mail files.

Unix mail file format is pretty lame, actually.  No provisions for
labels or anything (status is no substitute for full keywords &
attributes), non-transparent delimiters between messages, no per-file
attributes, etc.  Babyl file format has more implementations than most
other formats.

]One reason I find BABYL files irritating is because I have to treat
]them almost like binaries.  If I want to concatenate them, I can't use
]the cat(1) command.  The format contains control characters, so I
]can't mail a BABYL file either.  If I move to another system, it
]better have GNU Emacs otherwise I and my mail files are out of luck.
](Any other BABYL mail readers running on UNIX systems out there?)

It isn't the non-graphic characters that prevent you from cat'ing them
together, it's the file header.  You can use rmail to concatenate
them; merely use 1g to append the second file to the first.  The fact
that the delimiters are control characters makes it that much harder
to confuse a delimiter with a message body.  Besides, the high bit
isn't set; the control-underscore and control-l characters are
certainly mailable.  Unix mail format isn't all that portable either;
the silly From line (not the From: line!) is a pure unixism.

]> 2.  Where do you put the labels (keywords, whatever) and attributes
]> (like Unseen, Answered, etc)?
]
]Use the `X-' header extensions provided by RFC822.  E.g. X-RMAIL-Labels.
]Berkeley mail(1) does this now, except it uses a Status header
]instead of the technically correct X-Status.

I'm unhappy enough with the way the summary line is stored, although I
must admit that it speeds things up (it's stored in the header).

]> 3.  Where do you cache the reformatted header?
]
]What reformatted header?  Do you mean the BABYL header generated by
]RMAIL?  If I were writing RMAIL again I wouldn't have it use the
]BABYL format at all, even internally.

Well, you want somewhere to hide the headers that you aren't showing
to the reader.

]> 4.  Where do you put file-specific attributes like the inbox list and
]> others that rmail doesn't use yet?
]
]I'm not sure what the inbox list is supposed to do.  But the fact that
]it is file specific rather than message specific is a problem.  How
]about this: A dummy message could be prepended to the mail file that
]contains this (and any other file specific) information.  This message
]would contain a special header (e.g. X-RMAIL-Data) so that RMAIL would
]know not to show this message to the user.  Other UNIX mailers wouldn't
]understand this convention but they could still parse the mail file.

The inbox list is supposed to specify the set of mailboxes that you
wish the file to read mail from.  Since I use pmd, I have multiple
mailboxes, and some of my mail files read mail from multiple inboxes.
This hack would work, but would be a mess.

]At one point I attempted to rewrite the existing RMAIL to use the
]RFC822 message format but was confounded by the fact that the BABYL
]file dependencies are scattered throughout.  I now believe rewriting
]it from scratch would be the most expedient solution.

But Unix mail format isn't RFC822, either!  It's closer, but close
doesn't count in computers!
-- 
harvard >>>>>>  |	Robert Krawitz <rlk@think.com>	245 First St.
bloom-beacon >  |think!rlk	(postmaster)		Cambridge, MA  02142
topaz >>>>>>>>  .	Thinking Machines Corp.		(617)876-1111

jr@bbn.com (John Robinson) (02/16/89)

In article <36061@bbn.COM>, I write:
>Emacs started with no rmail code, but it was easy to steal it from
>elsewhere, namely the existing implementation on some random lisp
>machine, which happened to be using Babyl format.

Now that was a pretty lousy way to say that.  I meant that the Babyl
structure for a file of messages had been kicking around for
quite a while, including on lisp machines and others, and it was
certainly a method that the GNU emacs builders were familiar with, so
it was a natural choice to implement for GNU emacs.  I did not mean to
imply anything sinister about the origins of the actual code in
rmail.el.

Given the FSF's public stance on how software ought to be distributed,
I am sure that they are in fact extra scrupulous about what they
choose to call their own.
--
/jr
jr@bbn.com or bbn!jr

rbj@nav.icst.nbs.GOV (Root Boy Jim) (03/11/89)

? From: Robert Krawitz <rlk@think.com>

? ]> 3.  Where do you cache the reformatted header?
? ]
? ]What reformatted header?  Do you mean the BABYL header generated by
? ]RMAIL?  If I were writing RMAIL again I wouldn't have it use the
? ]BABYL format at all, even internally.

? Well, you want somewhere to hide the headers that you aren't showing
? to the reader.

Nowhere. I think `ignored' should mean delete them completely. Do you
realize that Received: lines alone take up something like 10% of the
entire X mailing list headers? Alternetely, one could reorder the
header to put all the ignored header lines first, and the region
displaying the message could start at the first `interesting' header.
I am also against the recaching of interesting headers.

I like the idea of a numeric argument to o and ^o.

? harvard >>>>>>  |	Robert Krawitz <rlk@think.com>	245 First St.
? bloom-beacon >  |think!rlk	(postmaster)		Cambridge, MA  02142
? topaz >>>>>>>>  .	Thinking Machines Corp.		(617)876-1111

	Catman Rshd <rbj@nav.icst.nbs.gov>
	Author of "The Daemonic Versions"