[comp.sys.next] NeXT multi-media message file format

ttl@sti.fi (Timo Lehtinen) (09/28/90)

I'd like to get some information on the file format of the NeXT multimedia 
messages. For example do they conform to RFC822 at all? How is bodypart 
encapsulation done? 

I'd appreciate it if some kind soul would send me info and/or example
messages. Also if the NeXT system has any online documentation on this a 
copy would be much appreciated!

Thanks in advance,

Timo Lehtinen

-- 
       ____/ ___   ___/    /		Kivihaantie 8 C 25
      /           /       /		SF-00310 HELSINKI, Finland
   ____  /       /       /	Phone:	+358 0 573 161, +358 49 424 012
  Stream Technologies Inc.	Fax:	+358 0 571 384

mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) (09/28/90)

In article <1990Sep27.212624.6406@sti.fi> ttl@sti.fi (Timo Lehtinen) writes:
>I'd like to get some information on the file format of the NeXT multimedia 
>messages. For example do they conform to RFC822 at all? How is bodypart 
>encapsulation done? 

The problem with the NeXT multimedia mail format is that it is only
used on NeXT computers.  It does not exist on any other platform.
What's more, my attempts to get a specification of the NeXT multimedia
mail format were rebuffed; apparently NeXT considers it proprietary.

Since open standards are the name of the game in this day and age, I
decided not to reverse-engineer the NeXT format and instead to use an
existing standard for multi-part, multi-structured messages in the
RFC-822 mail world; to wit, RFC-1154.  RFC-1154 specifies a new
"Encoding:" header field.  RFC-1154 is the format which is expected to
be used by other TCP/IP applications for multi-structured mail,
including the Privacy-Enhanced Mail project.

NeXT Mail can only interoperate with other NeXTs running NeXT Mail,
whereas applications that use RFC-1154 will interoperate with a wide
variety of applications and platforms.

My project is a distributed electronic mail system using the IMAP2
protocol of RFC-1176.  I have been distributing IMAP2 client and
server software on a variety of platforms, including the NeXT, for a
couple of years now.  You can get the current distribution version
(July 27) from FTPHOST.CAC.WASHINGTON.EDU, as file imap/imap.tar.Z and
the other files on the imap/ directory.  A new release of my
mailsystem with support for multi-structured mail will be available
sometime in December.

The thrust of my current work in multimedia mail is to support
attachments, e.g. the ability to mail spreadsheets and similar binary
files in a seamless manner.  I consider this to be more important than
mailing voice or pictures, although I am not precluding this.  For
voice communications, a pair of NeXTs and the Internet is a poor and
uneconomical substitute for the telephone and VoiceMail.  I can check
my VoiceMail messages from anyplace there's a phone; I can't do that
with NeXT Mail voice.  As for pictures, I believe that network FAX is
going to become the wave of the future; this would probably be the
functionality that I implement.

 _____   | ____ ___|___   /__ Mark ("Gaijin") Crispin  "Gaijin! Gaijin!"
 _|_|_  -|- ||   __|__   /  / R90/6 pilot, DoD #0105   "Gaijin ha doko?"
|_|_|_|  |\-++-  |===|  /  /  Atheist & Proud          "Niichan ha gaijin."
 --|--  /| ||||  |___|    /\  (206) 842-2385/543-5762  "Chigau.  Gaijin ja nai.
  /|\    | |/\| _______  /  \ MRC@CAC.Washington.EDU    Omae ha gaijin darou"
 / | \   | |__|  /   \  /    \"Iie, boku ha nihonjin." "Souka. Yappari gaijin!"
Hee, dakedo UNIX nanka wo tsukatte, umaku ikanaku temo shiranai yo.

rd0k+@andrew.cmu.edu (Richard Drew Dean) (09/28/90)

NeXT's multi-media mail is RFC-822 compatible and formatted something like:
	take all the stuff which isn't plain ASCII, and tar/compress/uuencode
it into the message body.  The trick here is compress; the filename part
of the uuencoded file does _not_ end in .Z, but the file has been run
through compress. 
	Send a Next message to yourself on a Sun/VAX/generic Un*x box, strip
the headers with your favorite editor, uudecode, uncompress, tar -xvf,
and you'll have the right things....

Drew Dean
rd0k+@andrew
(Certainly not an official opinion of CMU...)

mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) (09/29/90)

In article <cb0qMBW00VpaIIFEd6@andrew.cmu.edu> rd0k+@andrew.cmu.edu (Richard Drew Dean) writes:
>NeXT's multi-media mail is RFC-822 compatible and formatted something like:
>	take all the stuff which isn't plain ASCII, and tar/compress/uuencode
>it into the message body.  The trick here is compress; the filename part
>of the uuencoded file does _not_ end in .Z, but the file has been run
>through compress. 
>	Send a Next message to yourself on a Sun/VAX/generic Un*x box, strip
>the headers with your favorite editor, uudecode, uncompress, tar -xvf,
>and you'll have the right things....

Ahem.  You said "formatted something like".  The's the whole point.
It's pretty easy for a human to take a NeXT Mail message and
unscramble it on some other platform.  It is quite another thing for
an application to do so with all possible forms of NeXT Mail, much
less compose a message that is guaranteed to be compatible with NeXT
Mail, without a *complete* specification.

In any case, RFC-1154 is a standard, and offers a superset of the
apparent NeXT Mail format.

 _____   | ____ ___|___   /__ Mark ("Gaijin") Crispin  "Gaijin! Gaijin!"
 _|_|_  -|- ||   __|__   /  / R90/6 pilot, DoD #0105   "Gaijin ha doko?"
|_|_|_|  |\-++-  |===|  /  /  Atheist & Proud          "Niichan ha gaijin."
 --|--  /| ||||  |___|    /\  (206) 842-2385/543-5762  "Chigau.  Gaijin ja nai.
  /|\    | |/\| _______  /  \ MRC@CAC.Washington.EDU    Omae ha gaijin darou"
 / | \   | |__|  /   \  /    \"Iie, boku ha nihonjin." "Souka. Yappari gaijin!"
Hee, dakedo UNIX nanka wo tsukatte, umaku ikanaku temo shiranai yo.

eps@toaster.SFSU.EDU (Eric P. Scott) (10/01/90)

In article <8263@milton.u.washington.edu>
	mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) writes:
>In any case, RFC-1154 is a standard

Since when?  As far as I can tell, RFC 1140 is still operative,
and lists RFC 1154's state/status as Experimental/Elective.

I would very much like NeXT to document their current format, and
when standards do exist, to adhere to them.

While our future workstation acquisitions will consider heavily
price/performance, we were "forced" to purchase some
SPARCstations ->because the NeXTstation Color wasn't available<-,
and we are committed to supporting them for five years.  We are
very interested in a mail application for those machines that is
capable of handling RTF, and .vox and .tiff attachments, and is
(to the extent possible) fully interoperable with the NeXTs.

					-=EPS=-

wjs@milton.u.washington.edu (William Jon Shipley) (10/16/90)

Mark Crispin writes:
>Ahem.  You said "formatted something like".  The's the whole point.
>It's pretty easy for a human to take a NeXT Mail message and
>unscramble it on some other platform.  It is quite another thing for
>an application to do so with all possible forms of NeXT Mail, much
>less compose a message that is guaranteed to be compatible with NeXT
>Mail, without a *complete* specification.

Mark and I have been going back and forth on this for quite a while now.

Actually, it's not "something like", it's really simple.  You uudecode,
uncompress, and untar.  You then have one or more files, one of which
will be called "index.rtf".  This is a (surprise!) rtf file that supports
a '\attach' extension, which says "insert myfile right here, please".
The other files will have been untarred when index.rtf was.

This is tres handy because it means you don't dictate what file formats
your target machines understand; I could send someone a tiff file, but a
Mac users could send a PICT file to another Mac user, and assumedly a
Mac version of the mailer would understand this as readily as the NeXT
version understands tiff.

The only objection I have to Mail's encoding is that messages aren't
readily readable from non-NeXTs.  This is still a problem in 2.0.
My solution to this problem is to diff (not quite diff, actually)
the rtf message and a text-only version of the message, and start the
message with the text version, and then tack on a ^L, followed by
the diffs to change the text version back into rtf, followed by compressed
and encoded files that may be attached.

I've suggested this scheme to NeXT, they didn't seemed overwhelmingly
thrilled by it.

-william shipley
once more, just a student

mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) (10/16/90)

In article <9304@milton.u.washington.edu> wjs@milton.u.washington.edu (William Jon Shipley) writes:
>Actually, it's not "something like", it's really simple.  You uudecode,
>uncompress, and untar.

Sigh, this is getting tiresome.

If you receive a NeXT Mail message and use some other mailer, all you
need to know to extract its contents is that you extract the uuencode
segment with an editor, uncompress, and untar, then use index.rtf to
get you around the other files.

However, this is not enough information for an implementor to write a
program to decode NeXT mail, or one to transmit NeXT mail.

In particular, there are magic cookies in the message header that have
to be set up that are specific to the NeXT.  In particular, there is
the Next-Attachment header line.  What RFC describes this?  There are
also some interesting semantics regarding what characters you are
allowed to put in the subject of your message.

There is, by the way, a good reason why this is not (and should not
be) public information, and why other implementations should not use
the NeXT format, but it can't be discussed here.

 _____   | ____ ___|___   /__ Mark ("Gaijin") Crispin  "Gaijin! Gaijin!"
 _|_|_  -|- ||   __|__   /  / R90/6 pilot, DoD #0105   "Gaijin ha doko?"
|_|_|_|  |\-++-  |===|  /  /  Atheist & Proud          "Niichan ha gaijin."
 --|--  /| ||||  |___|    /\  (206) 842-2385/543-5762  "Chigau.  Gaijin ja nai.
  /|\    | |/\| _______  /  \ MRC@CAC.Washington.EDU    Omae ha gaijin darou"
 / | \   | |__|  /   \  /    \"Iie, boku ha nihonjin." "Souka. Yappari gaijin!"
Hee, dakedo UNIX nanka wo tsukatte, umaku ikanaku temo shiranai yo.

eps@toaster.SFSU.EDU (Eric P. Scott) (10/17/90)

In article <9314@milton.u.washington.edu>
	mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) writes:
>In particular, there are magic cookies in the message header that have
>to be set up that are specific to the NeXT.  In particular, there is
>the Next-Attachment header line.  What RFC describes this?

The RFC describing Berkeley's Line Printer protocol wasn't issued
until August 1990, but no one seemed to have any trouble
implementing it and making it the de facto standard over the
years.  And vendors' "proprietary" methodologies have a funny way
of becoming standards when groups like OSF come along and pick
the "best" of what's out there.  Why NeXT isn't submitting stuff
like netinfo for consideration is beyond me.

NeXT has a pretty good track record for adherence to standards,
when such standards exist.  In the absence of standards, you
make them up as you go along.  That's an evolutionary process.
Meanwhile, you ship product.

					-=EPS=-

louie@sayshell.umd.edu (Louis A. Mamakos) (10/17/90)

In article <896@toaster.SFSU.EDU> eps@cs.SFSU.EDU (Eric P. Scott) writes:
>The RFC describing Berkeley's Line Printer protocol wasn't issued
>until August 1990, but no one seemed to have any trouble
>implementing it and making it the de facto standard over the
>years.

The fact that the source code for the Berkeley line printer software was
available had quite a bit to do with this, I'm sure.  I'd be surprised if
people implemented the line printer protocol "by observation".

>NeXT has a pretty good track record for adherence to standards,
>when such standards exist.

I have to disagree here.  NeXT visited this NetInfo crock upon us, when a
good distributed computing environment like Athena (with Kerberos, Hesiod,
etc) already existed.  They could have provided significant added value
by providing those services with a sexy NeXT user interface.  What I have
today is NetInfo, turned off as much as possible since I operate in a
hetergenous envrionment for my "interpersonal computing."

Don't get me wrong; I think that they have a wonderful product, and I'm 
buying a NextStation for myself.  I just think that there's a little too
much NIH syndrome here and there that detracts from they truly wonderful
work that they've done.

louie

cnh5730@calvin.tamu.edu (Chuck Herrick) (10/17/90)

In article <9314@milton.u.washington.edu) mrc@Tomobiki-Cho.CAC.Washington.EDU (Mark Crispin) writes:
)In article <9304@milton.u.washington.edu) wjs@milton.u.washington.edu (William Jon Shipley) writes:
)
)Sigh, this is getting tiresome.
)
  Yes, it is. and not just for you, but especially so for those of
us who would like the net to move on to other new and different
subjects. So do this duet by email, please.

-- 
	Chuck Herrick				cnh5730@calvin.tamu.edu

henry@zoo.toronto.edu (Henry Spencer) (10/18/90)

In article <896@toaster.SFSU.EDU> eps@cs.SFSU.EDU (Eric P. Scott) writes:
>The RFC describing Berkeley's Line Printer protocol wasn't issued
>until August 1990, but no one seemed to have any trouble
>implementing it and making it the de facto standard over the
>years...

You are confusing "succeeded, perhaps with great pain and anguish" with
"had no trouble". :-)  Note also that Berkeley made no attempt to keep
the stuff secret, it was just that their documentation (the sources!)
was pretty hard to read.
-- 
"...the i860 is a wonderful source     | Henry Spencer at U of Toronto Zoology
of thesis topics."    --Preston Briggs |  henry@zoo.toronto.edu   utzoo!henry