[comp.mail.misc] X.400 ean mail system

wyle@ethz.UUCP (Mitchell Wyle) (08/24/87)

Why I don't use ean

Mitchell Wyle
wyle@ethz.uucp

The University of British Columbia's X.400 mail system "ean" is the
electronic mail system of choice for the ETH computer science
department.  I have used the system for over a year; it is the system
with which I am most experienced, but I have discovered some features
in the system which have led me to the decision to change mail systems.

Most major decisions of this nature are prompted by unfortunate
accidents.  Such is the case in this situation.  Because ean uses a
bizarre method of storing its data (db5), one must periodically "clean"
the data files ean creates.  Otherwise, they would continue to grow
without bound, until the disk is full.  One day, the  "eanrebuild"
command destroyed all of my ean data.  Upon restoring the data from a
backup tape, ean refused to read the restored data.  A mail system
which  1) needs a dangerous, periodic "clean-up" command, and 2) cannot
read restored data is unacceptable.  The data cannot be moved to tape
and back to disk.  No backup is possible;  therefore ean is
unacceptable.

The eanrebuild command is simply a nuisance which does not belong in a
computer system.  One could live with such an inconvenience if backup
were possible.

Ean's user interface is as poor as most other electronic mail
systems'.  There is no way to specify rules and actions (it is not
programmable).  It has a flat storage method (one level of folders).
It does not take advantage of a video screen terminal.  It has no
information retrieval system for mail messages.

Ean cannot run on all unix systems.  It is specific for the DEC vax.
It's files cannot be used by other applications.  Almost all ean
messages I send arrive with an annoying "Parse error of original
messages in Switzerland" error message, which leads me to assume that
the lower layers of the system are as weak as the user interface and
no-backup features.

The public domain, University of California Berkeley's (UCB) Unix mail
system is not much better, but at least there is no "rebuild" command,
and the files are standard unix text files.  One can use the Unix shell
environment, the "mail-tool" interface on Suns, and the file directory
tree to add structure to mail messages.  UCB Unix mail is programmable
and richer in commands.  It is standard on many different machines, and
identical on Suns and vaxen.  Most importantly, the data are
interchangeable, and can be backed up!

Ean?  No thank you.
-- 
Mitchell F. Wyle           | csnet or arpa:  wyle%ifi.ethz.ch@relay.cs.net
Instituet fuer Informatik  | uucp:           wyle@ethz.uucp
ETH Zentrum / SOT          | Telephone:      011 41 1 256 5237
8092 Zuerich, Switzerland

taylor@hplabsz.HPL.HP.COM (Dave Taylor) (08/25/87)

Mitchell Wyle writes an interesting article about why he doesn't use
the EAN X.400 mail system from the University of British Columbia in
this group.  I'd like to discuss some of the points he raises a bit
further...

First off, I think it is important to note that X.400 and the related
ISO/CCITT standards are indeed an up and coming standard for the 
interchange of electronic mail in the world (certainly in Europe).  In
fact sources from both DEC, HP, and some third party market analysts
tell me that it is almost impossible to sell any large computer boxes
to firms there unless they have available an X.400 mail system...

In an environment of that nature, it is somewhat ironic to read that
a sophisticated, knowledgeable Unix user like Mitchell decides to use
something as simple as the Berkeley Mail system instead of the EAN
package.  My first curious question is how long will Mitchell be able
to use Berkeley Mail to send and receive mail there in Switzerland?  I
can imagine in the very near future his machine suddenly not speaking
the "RFC-822 style" mail that Berkeley Mail expects and generates.  Or,
as an alternative, having the extra overhead of a gateway/translator
for each message sent, even those local to his machine.  

Note that I haven't made any comments about the relative merits of the
X.400 protocol suite/standard nor of the RFC-822 (and related documents)
standard either.  The tough fact of the matter is that it is almost
completely irrelevant whether we like the standard or not - X.400 is
happening now regardless.  There are already countries where, as I said
previously, you *cannot* sell machines unless you offer a complete X.400
system.  (interestingly, the best selling X.400 package, All-In-One from
Digital Equipment Corporation, currently has a number of blatant protocol
violations on its lower-level interactions with other X.400 systems (but
we can be sure that DEC is working like mad to fix that as soon as they
possibly can!))

This is not to make light of the problems of an electronic mail package
user interface!  From dint of the experiences with not only the Msg and
Elm mail packages, but from the research I've put in on various articles
on electronic mail systems (the latest being in the book "The Unix Papers"
from Howard and Sams Publishing...) I've found that perhaps the *most*
important aspect of any system, and especially a system like one for the
creation, transmission and reception of electronic mail, is in fact the
user interface.

So with that preamble, I'd like to highlight the deficient areas of the
EAN package, based *purely* on what Mitchell Wyle posted in his article.

The first problem that is raised is that of the data storage method.  Not
only does EAN use a unique format for the storage of data (which, I would
venture a guess, is related to X.408 (MHS: Encoded Information Type
Conversion Rules) and X.409 (MHS: Presentation Transfer Syntax and Notation)
standards if nothing else..) and that a side effect of this is that 1. the
folder space is one-level deep (e.g. there is no capability for having 
folders within other folders), 2. the utilities that are available for
keeping your dataspace clean aren't totally reliable (e.g. "eanrebuild"),
and 3. EAN didn't like the data that was restored from a backup tape.

One of the stranger aspects of the X.400 suite from our anglocentric
point of view here in the US is that it doesn't store everything in an
English-based format.  In fact, the method by which the message
envelope (as they say) is built up and saved is designed to be as
*in*dependant of host language as possible.  One of the 'headers' of
an X.400 message is in fact the language that it was composed in, so
that the displaying system can easily choose the appropriate character
set.  An obvious side effect of this is that the data storage format
for the messages cannot suddenly be in a different, '822 style, format.
So instead, you're left with the necessity of having to use various
utilities to help maintain your data space.

Given that, I would suspect that the EAN group would be more than just
interested in hearing about problems with the reliability of their 
system!  Certainly if you can fully describe the situation whereby
the "eanrebuild" program destroyed your saved messages it would at least
be a big help in ensuring that they can fix it and make sure it doesn't
happen to other people!   And as to the problem with data off of a backup
tape, I suspect that it is a subtle problem with file modification times
or something, and that, again, if you were to *tell* the EAN people
about the situation they would quickly remedy it.

[general philosophical comment here: I find it unfortunate that so many
 people aren't willing to work with their software vendors in situations
 such as this - software is just like any other complex system and it
 occasionally needs some fine tuning.  But there is *no* way for the 
 intrepid vendor to achieve this if the users have something go wrong
 then simple stop using the product, instead of letting 'em know about
 it!  Let's get on the ball, people!]

Mitchell then goes on to talk about some aspects of the User Interface
to the EAN X.400 mail system.  Specifically, he states that a couple of
the problems he has are that; 1. there is no way to specify rules and
actions and otherwise make it a programmable interface,  2. there is
no way to have multi-level folders,  3. it doesn't exploit the additional
resource of the CRT *screen*,  and 4. it has no intelligent retrieval 
system for archived messages.

I must admit that I'm not sure exactly what Mitchell is talking about
in his first point here - I have on my machine a program which 'screens'
my electronic mail before allowing it to be delivered to me, and it can
reply to mail automatically, store it in folders for later perusal, 
or even instantly display it to the screen.  But that is one of only
three systems that I know of that offer this sort of functionality
and none of them are in widespread use (they are "filter", from the
Elm Mail System, "mh-hook" from the MH package, and MEP from John
Antypas).   I'm very interested in hearing more about what is being
suggested here (and shan't bias it by talking about how *I* think
mail systems should work!)

The problem with multi-level folders, and the related problem of
not having an intelligent retrieval system are actually symptomatic
of a much larger problem with generalized information retrieval
systems.  I believe that the *real* answer to this is to have a
hypertext system available and to obsolete the entire idea of
folders.  I know of a couple of systems in the works that support
just this, and I imagine now that Apple has released HyperCard for
the Macintosh that it's just a matter of time before some bright
cookie exploits that for the same area.  The idea here would be
to specify a list of criteria to base selection of possible 
messages on - sort of like full-text keywording - and then have
the system select messages based on that.  (I think this is really
the wave of the future and I'm looking forward to having it on
MY machine soon!)

As to the EAN interface not exploiting the resources and space of
the CRT screen, well, all I can say is that Andrew Draskoy of the
University of British Columbia and I have talked a couple of times
about full-screen interfaces for the EAN system and that he has
recently requested a copy of the Elm Mail System from me for some
ideas of how to exploit a full screen and still retain the ease of
use of a simple system...I wouldn't be suprised to see them release
an impressive and powerful new interface to their system within a
year.

By the way, Mitchell also refers to the Berkeley Mail package as being
"in the public domain".  This, of course, is not the case at all, and
the package is in fact covered under the same copyright and licensing
restrictions as the rest of the BSD binaries and sources.

Finally, I hope those of you that have managed to read this far have
found this an informative note.  I would like to hear further discussion
on electronic mail user interfaces and shall be forwarding the comments
as appropriate to interested other parties.  (there are also a number
of mailing lists dealing with mail user agents, mail handling systems,
the X.400 protocol suite, and so on - check the "list of lists" that
sporadically appears on the net...)

			From the far corners of the globe,

					-- Dave Taylor
					Hewlett-Packard Labs

daveb@geac.UUCP (Brown) (08/26/87)

Mitchell Wyle comments:
| ....  Because ean uses a
| bizarre method of storing its data (db5), one must periodically "clean"
| the data files ean creates.  Otherwise, they would continue to grow
| without bound, until the disk is full.  One day, the  "eanrebuild"
| command destroyed all of my ean data...

Dave Taylor replies:
| The first problem that is raised is that of the data storage method.  Not
| only does EAN use a unique format for the storage of data ...
| An obvious side effect of this is that the data storage format
| for the messages cannot suddenly be in a different, '822 style, format.
| So instead, you're left with the necessity of having to use various
| utilities to help maintain your data space.

  I think that dave has a misconception here: X.400 defines a format
for the contents of a file which constitutes {message*}.  This format
is very non-ascii and not fully convertable to RFC 822 ascii.  It does
not, as I read the specification (which is not fully convertable to a
human language), require the storage of the messages in a file or file
sequence that is not manipulatable with more-or-less normal system
tools.  The use of db5, if true, is a decisision made by the ean
developers, and appears to have caused a problem...

 --dave (i think i like the format, but i can't read the spec) cb

-- 
 David Collier-Brown.                 {mnetor|yetti|utgpu}!geac!daveb
 Geac Computers International Inc.,   |  Computer Science loses its
 350 Steelcase Road,Markham, Ontario, |  memory (if not its mind)
 CANADA, L3R 1B3 (416) 475-0525 x3279 |  every 6 months.

lamy@utegc.UUCP (08/27/87)

In article <186@bernina.UUCP> wyle@ethz.UUCP (Mitchell Wyle) writes:
>The public domain, University of California Berkeley's (UCB) Unix mail
>system is not much better, but at least there is no "rebuild" command,
>and the files are standard unix text files.  One can use the Unix shell
>environment, the "mail-tool" interface on Suns, and the file directory
>tree to add structure to mail messages.  UCB Unix mail is programmable
>and richer in commands.  It is standard on many different machines, and
>identical on Suns and vaxen.  Most importantly, the data are
>interchangeable, and can be backed up!

UCB mail is not public domain, and it does not really mesh well with the Unix
directories (retrieving an individual message is a pain).  There is no
programmability either.  The mailtool on the Suns does not allow you to chose
which editor you want to edit the messages.  UCB mail is not screen oriented.
In fact, I must admit that I really don't give much credence to your claim
that UCB mail's interface is any richer than EAN's.  Yes, I have used both.

ELM will give you a screen oriented interface. If you want full
configurability, then use MH, from within an editor if you wish.  MH uses the
directory structure for folders and gives you a query facility to retrieve
messages.

In fact, there is no real reason either why you can't have the best of all
worlds.  This message is composed in a MH mode in GNU-emacs (with roughly the
same features as mailtool), will be processed by the standard Unix sendmail,
and may end up being sent by EAN, acting as a message transfer agent (we also
have connection to the Internet, UUCP and Bitnet). Getting EAN to
interface with other mailers can be a pain, but there is hope (this site
is a living proof that an interface with the standard Unix setup can be done,
and the VMS/EAN gatewaying seems to work fine as well),  In other words,
a little patience may be in order.

Wrt your comment about parse errors, note that even relay.cs.net
will complain about (hackish) addresses with a % in them.  I don't see
this as a problem with EAN.

I am not the greatest fan of X.400 mail (especially with respect to 
the proposed organization-oriented directory service), but I don't think
EAN is such a bad deal for a site with a single connection to the outside
world.

Jean-Francois Lamy                      lamy@ai.toronto.edu (CSnet,UUCP,Bitnet)
AI Group, Dept of Computer Science      lamy@ai.toronto.cdn (EAN X.400)
University of Toronto, Canada M5S 1A4   {seismo,watmath}!ai.toronto.edu!lamy

zentrale@rmi.UUCP (08/28/87)

In article <719@hplabsz.HPL.HP.COM> taylor@hplabsz.UUCP (Dave Taylor) writes:
: First off, I think it is important to note that X.400 and the related
: ISO/CCITT standards are indeed an up and coming standard for the 
: interchange of electronic mail in the world (certainly in Europe).  In
: fact sources from both DEC, HP, and some third party market analysts
: tell me that it is almost impossible to sell any large computer boxes
: to firms there unless they have available an X.400 mail system...

Espacially interesting is: X.400 is the prot of the administrations. Those
"official" Mailboxsystems use X.400 and will gateway to others. If a host
wants to paricipate the commercial networks beeing established, it has to
to it by using X.400.

: 
: One of the stranger aspects of the X.400 suite from our anglocentric
: point of view here in the US is that it doesn't store everything in an
: English-based format. [...]
 
What means "stranger aspects? There are countries with users, who don't
speak English . So an language indepent Prot has its merits.

: 
: [general philosophical comment here: I find it unfortunate that so many
:  people aren't willing to work with their software vendors in situations
:  such as this - software is just like any other complex system and it
:  occasionally needs some fine tuning.  But there is *no* way for the 
:  intrepid vendor to achieve this if the users have something go wrong
:  then simple stop using the product, instead of letting 'em know about
:  it!  Let's get on the ball, people!]
: 

Stands for itself, just repeated to be kept in mind!

: 
: 					-- Dave Taylor
: 					Hewlett-Packard Labs

Trying to get hand on X.400 Software for UN*X I was given the address
of Sidney International. Contacting them resulted in the following
information: Source License Pound Sterling 250,000, Binary 5,000! Just
for an mailing system ... :-)

                                Rupert Mohr
                                RMI Aachen

peter@ethz.UUCP (Peter Beadle) (08/31/87)

In article <719@hplabsz.HPL.HP.COM> taylor@hplabsz.UUCP (Dave Taylor) writes:

>First off, I think it is important to note that X.400 and the related
>ISO/CCITT standards are indeed an up and coming standard for the 
>interchange of electronic mail in the world (certainly in Europe).  In
>fact sources from both DEC, HP, and some third party market analysts
>tell me that it is almost impossible to sell any large computer boxes
>to firms there unless they have available an X.400 mail system...

To this I agree, the X.400 protocols are quiet nice. Unfortunately the
group here responsible for maintaining the ean mail system repeatedly
tell me "the EAN implementation of the x.400 protocol package is a heap
of sh*t".

I manage the mail and wide area network for the machine Mitchel Wyle's
mail goes through. This machine has 4 gateways to disparate networks
(uucp, cipon, ean and ACSnet) and sometimes (depending on the weather)
acts as the ean-csnet gateway. ACSnet runs itself and tells me what
it's been doing once a month (it suports remote job execution, binary
transfer (on 7bit, 8bit and x25 lines), file server, remote printing,
and acts as an inter network gateway) all the others require work.

All the machines in a cluster of ~40 run uucp (over ethernet) as the
transport level with the  upas mail system (Bell Labs) doing the
routing and acting as a name server.

Running on top of that we have a modified /bin/mail, ucb/mail, mh and
sun's mailtool interface to /ucb/mail there are also a gagle of emacs
based mail programs and Mail (if anyone remembers that antique).

On the gateway machine we also have ean running. I don't administer ean
and frankly wish it would go away. I have used the system and gave up
after many complaints from correspondants who got tired of "auto
forwarding" as the non-delivery reason in the bounce messages they
received.

It also appears that there is no way in ean to do any of the usefull
mail filtering things that make email usefull in administering
machines.  I say apparently because the only documentation available is
the online help from inside the ean system, there is no machine manual
and no paper manual available to users of the system and I doubt it is
available to the administrators.

>package.  My first curious question is how long will Mitchell be able
>to use Berkeley Mail to send and receive mail there in Switzerland?  I
>can imagine in the very near future his machine suddenly not speaking
>the "RFC-822 style" mail that Berkeley Mail expects and generates.  Or,
>as an alternative, having the extra overhead of a gateway/translator
>for each message sent, even those local to his machine.  

Yes the ean-isation of Switzerland is well underway. It's a pity they
couldnt be bothered to develope their own x400 system instead of
importing a bad one from Canada.  I am also not really worried about
the translation time, I simply want a mail system that works. Most
sites understand either rfc-822 or uucp mail headers, we provide both.
We have no trouble gatewaying into ean with our rfc-822 headers.
Unfortunately the ean system can't even provide a from address to our
mailer so it can correctly set the from fields on mail coming from ean
into uucp/rfc-822. I have also been told that on the next release of
ean they will provide a date field in the header. A great leap
forward.

>standard either.  The tough fact of the matter is that it is almost
>completely irrelevant whether we like the standard or not - X.400 is
>happening now regardless.  There are already countries where, as I said
>previously, you *cannot* sell machines unless you offer a complete X.400

The point is the ean implementation of x.400, not x.400 itself. Could you
please supply some proof for your claim about x400 and computer sales.

>So with that preamble, I'd like to highlight the deficient areas of the
>EAN package, based *purely* on what Mitchell Wyle posted in his article.

Have you ever tried to use the ean package. If not I sugest you try to get
access to an ean system and see how it feels. (give me a call and I will
give you our modem number).

>The first problem that is raised is that of the data storage method.  Not
>only does EAN use a unique format for the storage of data (which, I would
>venture a guess, is related to X.408 (MHS: Encoded Information Type
>Conversion Rules) and X.409 (MHS: Presentation Transfer Syntax and Notation)
>standards if nothing else..) 

No it's because ean seems to use the cheap nasty dbm data base that comes
with 4.2 with the result that the files it creates can't be manipulated
with standard unix comands. Still I supose popple live with the Itallian
postal service (average delivery speed 6 m.p.h. and a nasty habit of
shreading the mail backlog.

>One of the stranger aspects of the X.400 suite from our anglocentric
>point of view here in the US is that it doesn't store everything in an
>English-based format.  In fact, the method by which the message
>envelope (as they say) is built up and saved is designed to be as
>*in*dependant of host language as possible.  One of the 'headers' of
>an X.400 message is in fact the language that it was composed in, so
>that the displaying system can easily choose the appropriate character
>set.  An obvious side effect of this is that the data storage format
>for the messages cannot suddenly be in a different, '822 style, format.
>So instead, you're left with the necessity of having to use various
>utilities to help maintain your data space.

All very interesting, if you could rely on your data base the it might
even work. If you could rely on your communication medium there would be no
need for uuencode/decode, binhex and innumerable other programs that
convert binary into ascii and back and you would be much further towards a
usable "communications" system. I think the reliable communication system
would bring you a lot more than selecting the language style in the mail
header.

>Given that, I would suspect that the EAN group would be more than just
>interested in hearing about problems with the reliability of their 
>system!

They already know of the problems, see my earlier quote about their summary
of the system.

> people aren't willing to work with their software vendors in situations
> such as this 

It's a little hard when the software vendor is a University on the
other side of the Atlantic and the people who developed th code have long
since graduated and moved on. 

I refer you to another article in this news
group about a commercial x400 system, source lisence only 250,000
pounds sterling. Per cpu lisence only 5000 pounds.

Summary:
There is nothing wrong with the x.400 protocols. It would be good if they
become a european standard.

The currently available x-400 implementations stink.

I would love to hear somone from the University of British Columbia defend
their system.

Ean appears to lack:
	. adequate documentation for users,
	. support for mail filtering,
	. methods of turning off stupidities (like auto forwarding as a
		non delivery reason)
	. alternate presentation system (you get the ean presentation
		system and thats all)
	. support for file transfer and remote execution which where there
		in the most primative uucp mail systems

Peter Beadle
uucp		peter@eiger.uucp
ean/csnet	peter@ifi.ethz.ch (try this one for a non 
					delivery bounce message)
uucp		mcvax!cernvax!ethz!eiger!peter
acsnet		peter@bernina.ch

lubich@ethz.UUCP (Hannes Lubich) (08/31/87)

Peter, just one point:
I don't think that I will try to defend the old ean version that we are
running here (got, that, John :-)) but I would wish that everybody would
document software in the way ubc did it for ean. The user documentation
is very good (there is a seperate documentation for VMS managers, Unix-
managers and simple users, there are least two quick reference cards
and we added a whole bunch of additonal information (including a
telephone hotline service and the like). In addition, a large ammount
of documentation is available on an online basis, so there is really no
reason for complaining about the documentation.

We all agree that EAN V1 has some major disadvantages but I think that 
all the people that now tend to flame ean should have
	a) spoken up earlier
	b) spoken up in a more usable fashion
	c) directed their complaints to ubc

	--Hannes

-- 
~ UUCP/Usenet   :   {known world}!seismo!mcvax!cernvax!ethz!lubich
~ CSNET         :   lubich%ifi.ethz.chunet@relay.cs.net
~ ARPA          :   lubich%ifi.ethz.chunet@csnet-relay.arpa                
The usual disclaimer : No, it wasn't me, somebody must have used my account.

taylor@hplabsz.HPL.HP.COM (Dave Taylor) (09/03/87)

Rupert Mohr replies to my comments about EAN and X.400 with some interesting
comments of his own, but at one place he misses my point entirely, alas:

>: One of the stranger aspects of the X.400 suite from our anglocentric
>: point of view here in the US is that it doesn't store everything in an
>: English-based format. [...]
> 
>What means "stranger aspects? There are countries with users, who don't
>speak English . So an language indepent Prot[ocol spec] has its merits.

I think the problem here is with the word 'anglocentric'.  I used it to
mean 'centered on the anglo/english-speaking world'.  The comment was
more to the effect of 'it is unfortunate that here in the US we're so
convinced in the ultimate value of English that we, for the most part,
view any system that is to support other languages by *not* being in
English as suspicious' (althought that isn't much clearer either!!)

In general I think that X.400 is a good thing (are we going to start
talking about the value of X.400 here??) and that since it specifies
a non-ascii-based message envelope it makes the most sense for the
messages to also be stored in that format.  (a strong case can be made
that it should be translated to the local language upon storage, though,
and I'd be interested in hearing peoples ideas on this either way...)

As I've tried to say too, however, the Unix system especially is
very much oriented around having 'flat' ASCII files and there are 
a plethora of useful and interesting tools that you can then use to
wreak havoc on these text files.  Storing X.400 mailboxes as non-ASCII
data unfortunately loses this major Unix advantage, and the X.400
vendor is then put in the position of offering their own tools to
replace and/or supplement the already-built-and-tested Unix ASCII
tools...

		six of one, half-dozen of the other....

						-- Dave Taylor
						   Hewlett-Packard Labs