oester@rchland.ibm.com (Bob Oesterlin) (07/17/90)
What does the following message indicate from CUI? Found 2 old hidden messages in/afs/rchland.ibm.com/usr0/bboard/.MESSAGES/advisor/aix-andrew (0 orphanedsnapshots, 0 processing errors). We see this type of mesaage frequently when cui (to epoch) or nns is run(ora variant "check for old hidden messages in ..."). This may be related: every once in a while we'll see messages show up in afolder that are from 2 weeks to 6 months old... what causes this? Bob
nsb@THUMPER.BELLCORE.COM (Nathaniel Borenstein) (07/17/90)
Excerpts from internet.info-andrew: 17-Jul-90 CUI messages Bob Oesterlin@rchland.ib (470+0) > What does the following message indicate from CUI? > Found 2 old hidden messages in > /afs/rchland.ibm.com/usr0/bboard/.MESSAGES/advisor/aix-andrew (0 > orphaned snapshots, 0 processing errors). > We see this type of mesaage frequently when cui (to epoch) or nns is > run(or a variant "check for old hidden messages in ..."). > This may be related: every once in a while we'll see messages show up in > a folder that are from 2 weeks to 6 months old... what causes this? They are indeed related. This is just AMS doing its level best to make sure no messages ever get lost, no matter what the file system happens to do. You remember last week I explained all the things AMS does when it reads in new incoming mail? Well, this is related to one of them. Consider that for each new message, two files must be written: 1. The body file itself. 2. The index file for the folder (.MS_MsgDir). Now, whichever order you choose to write them in, you have the disturbing possibility that the file system will flake out on you in between the two steps. Well, there are lots of possible algorithms for dealing with this. We wanted to achieve two goals: 1. No messages should ever be lost. 2. The body file should be written first, because otherwise you'd have situations where the caption showed up but the body was unavailable. Now, the way we do this is by surrounding the two write operations with the creation and deletion of a mark in the file system that says "there's something being added to this folder." This is the .AMS_DIRMOD file -- it is created before the append operation, and deleted afterwards. If the file system goes down in the middle, this file stays around, and the next time the messageserver opens the directory for writing, it notices and scavenges for "old hidden messages". At that point, the messages, which otherwise would have remained invisible, are indexed in the .MS_MsgDir file and become visible. (You can also force this check to happen by hand with the CUI "scavenge" command.) Actually, it's even more complicated than this. Similar transient error problems can afflict the folder purge operation, and messages can be similarly "orphaned" during that process. Thus, sometimes the "scavenge" operation finds not messages that never showed up, but messages that showed up a long time ago and were incompletely purged. In short, the messages are harmless and can be ignored. In fact, we put them there to monitor how well the mechanism was working when it was first installed; it isn't necessarily clear that it should even bother printing the messages any more... -- Nathaniel
ghoti+@ANDREW.CMU.EDU (Adam Stoller) (07/17/90)
Excerpts from internet.info-andrew: 17-Jul-90 CUI messages Bob Oesterlin@rchland.ib (470+0) [..] > Found 2 old hidden messages in > /afs/rchland.ibm.com/usr0/bboard/.MESSAGES/advisor/aix-andrew (0 > orphaned snapshots, 0 processing errors). [..] > This may be related: every once in a while we'll see messages show up in > a folder that are from 2 weeks to 6 months old... what causes this? Yes - they are related -- (I hope I get this correct, it's been a while...) it's basically a built-in scavenger. At times, it is possible for a message to miss getting indexed in the .MS_MsgDir file (or perhaps lose it's index?) -- I believe this happens mostly in combination with some sort of file server problem (certainly with AFS, probably with other similar filesystems - maybe even plain UNIX). It isn't (wasn't) always a case of a problem with incomming mail - in fact, for a while it was most common with respect to mail/posts that had been deleted, and had never been fully purged from the directory. Every so often (? I don't remember the details here - sorry) when the messageserver is adding a new message to the folder/index - it will notice that something might be amiss, and attempt to do a (hopefully quick) scavenge in the directory to find any messages which are missing from the index. If it finds any, it prints out a warning (upper excerpt above), and appends them to the folder. Note, that is -> append <- -- hence at times you will see posts that could a few minutes to several years old appear as if they had just arrived (lower excerpt above). If it's happening a lot - someone (?) will probably have to try and track down the cause - if it happens occasionally with no real regularity - I wouldn't worry about it too much (unless you want to ;-) --fish (PS: be sure to look for followups from Craig and NSB - which will catch the many mistakes I've probably made in the above *sigh*)