[comp.mail.mush] **WARNING** Mush 7.0.0 scrambles mailbox

loverso@Xylogics.COM (John Robert LoVerso) (12/15/89)

I waited until 7.0 was released before using it.  I put it up the
day after it was released, and put up 7.0.1 yesterday.  Anyway, I
have about 450 peices (1.1MB) of mail (some unanswered) in my
maildrop.  This morning I was doing to do my monthly cleaning and
see if I can get that number down a bit.  When I went back to
message 1, I found that chunks of recent mail (from last week) had
been overlayed onto the beginning of my maildrop - leaving me with
bits and pieces of those messages.  This is not good!

Luckily, I had made a copy of my maildrop before using 7.0.0, and
so I know it must have be 7.0.0 who did the scrambling.  For now,
I've backed off (for reading) to 6.5.

The host machine here is an Encore Multimax running (mostly) their mutant
4.2BSD.

While I'm at it, here is a list of comments on 7.0.0.  Some of these
are problems and some are suggestions (request-for-features).

1. File completion is too verbose

   In particular, it expands "+" into the pathname of my folder directory,
   "/u6/loverso/Mail.  When you've got over 50 folders in there, this tends
   to output a lot of useless information.

   Actually, to be more specific, typing +^D just says "/u6/loverso/Mail/" -
   I need to type +/^D to get it to list out the individual folders.  This
   is misleading, because +foo^D lists folders that start with +foo - no "/"
   is required.

   Additionally, I have many subfolders (over 250 in several levels of
   subdirectories).  At the minimum, it would be useful to have directories
   listed with a trailing "/" on them.  This way, +^D would list me all my
   folders with info on which are subdirs, and it should fit in 7 lines,
   not take up the whole screen.

   Actually, for subfolders, my shell script "listfolder" handles things
   nicer, in that it has umpteen different listing formats, of which
   my favorite is a recursive listing given full folder names, as in:
	foo/blah  foo/goo  foo/lost/first  foo/lost/second

   I've looked at the completion code and its very simple (and clean!).
   If I get a chance sometime soon, I'll hack in some of the above and
   send along the diffs.

   Lastly, completion doesn't work for "~w" et al in compose mode.

2. Including headers in forwarded mail
   I usually need to do two different things: send an extract of a message
   to someone, or (under Mr.Postmaster hat) forward a bounced message
   or somesuch to some other Mr.Postermaster.  In the former case, I
   just want minimal headers included with ~I; I can do this by settin
   alwaysignore.  However, to get all the headers, I need to unset
   this variable.  This is a little clumsy.

   I tried to make a map! to handle it, but found map!'s can match
   initial newlines (I only want the `command' recognized at BOL).
   I.e., something like:
	map! \nI\n <blah>
   it doesn't work, because the match for the initial newline fails.
   Taking out the initial newline makes it work.  Actually, I was
   hoping for something like:
	map! \n~II\n <blah>

3. From: line handling
   I need to change my `From:' at my discretion at certain times.  I know
   who I am - why do I need to prove it to Mush?  If I fill in a From:
   line that has my address in a different domain, it tosses it out
   (silently).  I don't want Mush changing this at all.

   The current method I use to get around this is a vi macro that does
   the following:
	1G!G/usr/lib/sendmail -t

   For that matter, Mush needn't supply a default From: line nor a default
   Date: at all.  My MTA does it for me.  Perhaps a SMART_MTA define might
   be in order?

   BTW, in the BUGS section of the manaul page, comment about "edit_hdrs"
   and changing the From: line is out of date.

4. Forwarding vs. resending
   I'm butting up against the "resent" mail restriction again.  I.e.,
   if you "mail -f" and never take the message into an editor, the
   message goes out as "resent" mail, in which "resent-" headers are
   generated.  This loses if you have auto_edit set.  Besides, usually
   I like to add an editorial comment at the top of such mail, which
   of course, also loses.

   Actually, it would be nice to have a different switch trigger
   resent- mail and forwarded mail; the later would be marked by:
	----- Forwarded mail from ...

   Note that the "mail" command now has a "-U" switch, it would be nice
   if "mail -f" didn't assume this; i.e., the current "mail -f" behavior
   would be replaced by "mail -fU".

John
-- 
John Robert LoVerso			Xylogics, Inc.  617/272-8140
loverso@Xylogics.COM			Annex Terminal Server Development Group

schaefer@ogicse.ogc.edu (Barton E. Schaefer) (12/16/89)

In article <8912151519.AA05853@xenna.Xylogics.COM> loverso@Xylogics.COM (John Robert LoVerso) writes:
} [I] have about 450 peices (1.1MB) of mail (some unanswered) in my
} maildrop.  This morning I was doing to do my monthly cleaning and
} see if I can get that number down a bit.  When I went back to
} message 1, I found that chunks of recent mail (from last week) had
} been overlayed onto the beginning of my maildrop - leaving me with
} bits and pieces of those messages.  This is not good!

Some more details, please -- is this before or after an update?  (If before,
something in the temp file handling is getting confused; if after, then the
problem is with writing back to the folder and is even more serious.)  Are
you using any kind of NFS mounting of your "maildrop"?  What sorts of things
did you do (sorting, deleting, how many updates, etc.)?  Did new mail arrive
while you were working?

Incidentally, I ran 7.0.1 just now on a 500 message folder amounting to
more than 2MB, randomly deleted about 200 messages with assorted "pick"
commands, read several other messages, and updated, with no problems.

} 1. File completion is too verbose
} 
}    In particular, it expands "+" into the pathname of my folder directory,

I presume you're complaining about file listing, e.g. with ^D, not just
completion.  Listing is verbose because it's much more powerful than (say)
csh ^D listing -- you can list any globbing pattern, not just the files
in a particular directory.  Csh ^D displays only the file names, not the
preceding path; but what confusion there would be if you typed

	*/*/*^D

and got only the "tail" of every path that was generated!  Even removing
common prefixes is insufficient, because you still might chop off something
that was generated by the glob pattern, leaving the user with no idea where
the listed files really are.

I will look into special-casing to csh-like behavior when there is no glob
pattern at all.

}    Actually, to be more specific, typing +^D just says "/u6/loverso/Mail/" -
}    I need to type +/^D to get it to list out the individual folders.  This
}    is misleading, because +foo^D lists folders that start with +foo - no "/"
}    is required.

This in intentional.  In effect, "+foo" expands to "$folder/foo" but "+"
expands only to "$folder".  You have to type ~/^D to list files in your
home directory; that's the analogy on which listing with +/^D is based.
+/foo^D will work, too, so I don't see how the / is "misleading".  

The reason the / appears at the end of "/u6/loverso/Mail/" has to do with
the expansion of "+" by routines that have been around far longer than
globbing and completion, and which were too well entrenched to revise.

}    Additionally, I have many subfolders (over 250 in several levels of
}    subdirectories).  At the minimum, it would be useful to have directories
}    listed with a trailing "/" on them.

This was considered but rejected because it requires a stat() on every
file in the listing.  On many systems, this would cause a very noticable
slowing of the completion routines.

}    I've looked at the completion code and its very simple (and clean!).

Thank you. :-)

}    Lastly, completion doesn't work for "~w" et al in compose mode.

This statement is just plain wrong.  Completion works in compose mode.
In fact, it works ANYWHERE in compose mode, so don't bind your completion
characters to things you habitually type in messages.

There DOES have to be a space between the "~w" and whatever you are 
completing, or mush will try to find a file in your home directory (~)
whose name begins with "w" ....

} 2. Including headers in forwarded mail
}    [ In some cases I ]
}    just want minimal headers included with ~I; I can do this by settin
}    alwaysignore.  However, to get all the headers, I need to unset
}    this variable.  This is a little clumsy.
} 
}    I tried to make a map! to handle it, but found map!'s can match
}    initial newlines (I only want the `command' recognized at BOL).

I handle this sort of thing by mapping a control character (somthing not
usually typed, like ^F) to newline/tilde-command/newline, e.g.

    map! \CF '\n~:unset alwaysignore\n~I\n~:set alwaysignore\n'

You can leave off the leading "\n" if you're confident you'll only be
typing it at the beginning of a line.

} 3. From: line handling
}    I need to change my `From:' at my discretion at certain times.  I know
}    who I am - why do I need to prove it to Mush?  If I fill in a From:
}    line that has my address in a different domain, it tosses it out
}    (silently).  I don't want Mush changing this at all.

Put your other selves in the "alts" list; mush will allow the From: line
to contain any of your alternate selves, as long as you've told it about
them.

}    For that matter, Mush needn't supply a default From: line nor a default
}    Date: at all.  My MTA does it for me.  Perhaps a SMART_MTA define might
}    be in order?

PICKY_MAILER covers this.

}    BTW, in the BUGS section of the manaul page, comment about "edit_hdrs"
}    and changing the From: line is out of date.

It's not in the BUGS section, but I found it and corrected it.  Thanks.

} 4. Forwarding vs. resending
}    I'm butting up against the "resent" mail restriction again.  I.e.,
}    if you "mail -f" and never take the message into an editor, the
}    message goes out as "resent" mail, in which "resent-" headers are
}    generated.  This loses if you have auto_edit set.

It "loses"?  If by this you mean that -f ignores $autoedit, then I guess
that's a loss.  Or do you mean that you have no choice of NOT getting
autoedit behavior with "mail -ef"?  That complaint I could understand,
though an editor is required to add anything above a such a message.

Why not

    cmd resend "mail -f"
    cmd forward "mail -ef"

}    Note that the "mail" command now has a "-U" switch, it would be nice
}    if "mail -f" didn't assume this; i.e., the current "mail -f" behavior
}    would be replaced by "mail -fU".

I'll file the suggestion for discussion with Dan.
-- 
Bart Schaefer     "Miserable miscreant!  Question MY integrity, will you?"
               "I have to see some *evidence* of it before I can question it."
                                                            -- Calvin & Hobbes
schaefer@cse.ogi.edu (used to be cse.ogc.edu)

tneff@bfmny0.UU.NET (Tom Neff) (12/16/89)

In article <6200@ogicse.ogc.edu> schaefer@ogicse.UUCP (Barton E. Schaefer) writes:
>In article <8912151519.AA05853@xenna.Xylogics.COM> loverso@Xylogics.COM (John Robert LoVerso) writes:
>}    Additionally, I have many subfolders (over 250 in several levels of
>}    subdirectories).  At the minimum, it would be useful to have directories
>}    listed with a trailing "/" on them.
>
>This was considered but rejected because it requires a stat() on every
>file in the listing.  On many systems, this would cause a very noticable
>slowing of the completion routines.

Consider adopting the clever hack mentioned recently in another
newsgroup.  Under UNIX if the link count for a directory is 2 or less,
it has no subdirectories and you need not perform the stat(2) on each of
its files.  If it has link count 3 or greater, subdirectories exist, but
you need only keep stat(2)'ing until you have found (link count - 2) of
them, after that everything must be a file.

This is admittedly UNIX dependent but would be worth having optionally
in config.h for those who want it.
-- 
"DO NOT, repeat, DO NOT blow the hatch!"  /)\   Tom Neff
"Roger....hatch blown!"                   \(/   tneff@bfmny0.UU.NET

loverso@Xylogics.COM (John Robert LoVerso) (12/16/89)

Bart,

I was just composing a followup to my earlier mail when your's arrived,
so I'll interleave the two...

> Some more details, please -- is this before or after an update?

I didn't notice it until several (3?) days of use.  There were two splices,
in the low-numbered messages (<30 for both).  This is a straight local
maildrop using flock.  I never sort; I do a few updates a day, tend to
run a single mush all day (quitting when I leave) and then another at
night.  I try and delete old mail (really!  I do!).  New mail *always*
arrives for me.

As an additional data point, I noticed that my 7.0.1 was saying it was
saying it was 7.0.0.  After investigating, I found that the makefile
has no dependencies, so when I recompiled, everything that needed it
didn't get rebuild!  Thus it couldn't been the 7.0.1 patch that caused
the problem.  However, non of the .h's changed in a way that looks
like it would cause havoc, so...  Anyway, you should either give a
default (non /usr/include) list of dependencies in each of the makefiles.
Or include a reminder note in the patches.

As for my large mdrop; this gave me incentive to catch up on those old
messages (actually, that was what I doing in the first place when I
discovered the problem - I shudder to think what would've happened if
I hadn't looked at those old messages at all).  Anyway, my mdrop is
now down to a confortable 19 messages/25KB - and updating now only
takes 1 second, as opposed to the 25 seconds I was used to!

And, I'm also keeping close track of the mail thats in there to 
see if I get corruption again.

> } 1. File completion is too verbose
> }    In particular, it expands "+" into the pathname of my folder directory,
> 
> Listing is verbose because it's much more powerful than ... [it allows]
> 	*/*/*^D

(I didn't even know that)  However, if I say */*^D, I should see things like
	blax/ajax  blax/garbel  blax/zot  qup/rrr  qup/goble
not the absolute pathname in front of those paths.  If I say +blax/^D,
I'm only interested in
	+blax/ajax  +blax/garbel  +blax/zot
not
	/u6/loverso/Mail/blax/ajax  /u6/loverso/Mail/blax/garbel
	/u6/loverso/Mail/blax/zot
Its too much to parse!

> }    Actually, to be more specific, typing +^D just says "/u6/loverso/Mail/" -
> }    I need to type +/^D to get it to list out the individual folders.
> 
> This in intentional.

Ah, but it makes me type an extra character!  If I care'd where my $folder
was, I'd look at the variable.  100% of the time, I'm really just looking
to see which top-level folders I have, and ~^D is 33% faster to type than ~/^D.

> expands only to "$folder".  You have to type ~/^D to list files in your
> home directory; that's the analogy on which listing with +/^D is based.

Ah, but ~^D will list all the valid names that can come after ~ - so why
shouldn't +^D list all the valid folder names? 

> }    At the minimum, it would be useful to have directories
> }    listed with a trailing "/" on them.
> 
> This was considered but rejected because it requires a stat() on every
> file in the listing.  On many systems, this would cause a very noticable
> slowing of the completion routines.

Not on real UNIX machines with inode caching 8-}.  You could #ifdef
the code with "SLOW_and_PUDGY".

> }    Lastly, completion doesn't work for "~w" et al in compose mode.
> 
> This statement is just plain wrong.

This is one of the things I was going to correct myself on.  When I was
testing that, I wasn't running with "set completion", so naturally it
didn't work for me!

> }    I tried to make a map! to handle it, but found map!'s can match
> }    initial newlines (I only want the `command' recognized at BOL).
> 
> I handle this sort of thing by mapping a control character (somthing not
> usually typed, like ^F) to newline/tilde-command/newline, e.g.

I said that wrong; I meant "map!'s can NOT match initial newlines".
What you suggested is what I'm using.

> }    For that matter, Mush needn't supply a default From: line nor a default
> }    Date: at all.  My MTA does it for me.
> 
> PICKY_MAILER covers this.

I thought PICKY_MAILER did something else...  again, my mistake!

> } 4. Forwarding vs. resending

Sorry to rehash this up again - while going though my old mail, I came 
across the discussion you, Dan, and I had about this over the summer.
My only problem is that I really want to (sometimes) add a editorial
comment at the top of resent mail.  Mush then causes such mail to not
be resent, etc.  That's the behavior I want to elide...

Thanks for the quick response!

John

lmb@vicom.com (Larry Blair) (12/16/89)

In article <6200@ogicse.ogc.edu> schaefer@ogicse.UUCP (Barton E. Schaefer) writes:
=In article <8912151519.AA05853@xenna.Xylogics.COM> loverso@Xylogics.COM (John Robert LoVerso) writes:
=}    Actually, to be more specific, typing +^D just says "/u6/loverso/Mail/" -
=}    I need to type +/^D to get it to list out the individual folders.  This
=}    is misleading, because +foo^D lists folders that start with +foo - no "/"
=}    is required.
=
=This in intentional.  In effect, "+foo" expands to "$folder/foo" but "+"
=expands only to "$folder".  You have to type ~/^D to list files in your
=home directory; that's the analogy on which listing with +/^D is based.
=+/foo^D will work, too, so I don't see how the / is "misleading".  

Because the "+" implies the folders, not just the directory they are stored in.
As the person who wrote the program, you are thinking about how you implemented
"+" rather than what it means.
=
=There DOES have to be a space between the "~w" and whatever you are 
=completing, or mush will try to find a file in your home directory (~)
=whose name begins with "w" ....

Shouldn't "~w" mean the home directory of user "w"?
-- 
Larry Blair   ames!vsi1!lmb   lmb@vicom.com