[comp.mail.elm] mush and mmdf help wanted

pcg@cs.aber.ac.uk (Piercarlo Grandi) (07/26/90)

In article <9007211604.AA17220@Morgan.COM> dpk@Morgan.COM (Doug Kingston) writes:

   I would love to see this facility in MUSH.  In retrospect, we should
   have switched to network by ordering in the file, and perhaps fixed
   lenght ascii readers a long time ago, but now is as good a time as
   any.  Startup would be an order of magnitude faster on large mailboxes.
   Basicly you want all the incore information written out in a fixed
   format for retrival.

MUSH unfortunately currently has a fairly slow way of doing things. I
have just got these numbers:

	folder: 1,094,949 bytes with 348 messages

	MUSH 7.1 PL2: grows to 99 4KB pages, display 348 msgs in 15 screens

			real	user	sys

	read		16.2	5.2	4.0
	browse		34.4	12.1	8.4
	sort -i -s -d	73.5	15.6	17.9 (sorting by subject and date)
	sort -l		17.6	5.9	4.3  (sorting by number of lines)

	ELM 2.3 PL5: grows to 137 4KB pages, displays 348 msgs in 21 screens

	read		15.9	7.9	4.1
	browse		20.1	9.2	6.9
	sort		17.5	9.1	4.5

	Notes: on SysV/386 (ESIX) on a 20 Mhz cached 386, approx. 4
	MIPS; programs fully memory resident, no paging; system
	otherwise inactive; times in seconds; the folder was contiguous
	on disc and optimally laid out; timings have been repeated and
	are consistent; 'read' is just opening the folder and exiting;
	'browse' is opening the folder and paging down all the
	screenfuls; 'sort' is opening the folder and sorting the
	headers; to make real time measures meaningful all the needed
	commands have been typed ahead while the editor was reading the
	folder. All times are therefore the 'best' times.

The colossal difference in favour of ELM for one of the sorts and for
browsing is because ELM keeps in memory the headers of all messages, in
a binary form, when the folder is opened (see the higher startup time),
while MUSH refetches and reparses most header lines on every access to
the message. Were it not for that, MUSH would be as fast as ELM, as the
sort by lines (the line count is kept in memory...) seems to indicate.

If this problem were fixed one of my two main objections to MUSH would
be removed (the other one is that I'd like to *optionally* ask that the
folder be copied to a temp file and back only when I ask for it or quit,
not immediately when I start browsing it).
--
Piercarlo "Peter" Grandi           | ARPA: pcg%cs.aber.ac.uk@nsfnet-relay.ac.uk
Dept of CS, UCW Aberystwyth        | UUCP: ...!mcsun!ukc!aber-cs!pcg
Penglais, Aberystwyth SY23 3BZ, UK | INET: pcg@cs.aber.ac.uk