[comp.soft-sys.andrew] Amazing AMS bug

nsb@THUMPER.BELLCORE.COM (Nathaniel Borenstein) (03/02/90)

A first-time AMS user here at Bellcore just stumbled on a core-dumping
AMS bug that has, as far as I can tell lurked undiscovered for several
years in code that is executed many thousands of times each day (the
fact that it has never been stumbled on before is the amazing part). 
The fix is trivial -- it is on line 85 of ams/libs/ms/bldpvmap.c.  The
old version is:

		while (PreorderSubscriptionStrcmp(SubsInPathOrder[subsindex]->key,
name) < 0 && subsindex < NumSubsInUse &&
SubsInPathOrder[subsindex]->pathelt == pathindex) {

The fixed version is:

		while (subsindex < NumSubsInUse &&
PreorderSubscriptionStrcmp(SubsInPathOrder[subsindex]->key, name) < 0 
&& SubsInPathOrder[subsindex]->pathelt == pathindex) {

In other words, all that changes is that the first and second test in
the while loop are exchanged.  This is necessary because if subsindex is
greater than or equal to NumSubsInUse, the current first test will be
indexing out of the bounds of the array; apparently this is usually
harmless, but in our current case it produces core dumps very quickly
(dereferencing a NULL pointer.)

This would probably be a good thing to put in a future patch.  By the
way, the symptom of this bug is that Messages dies on "expose all" and
cui dies on "list".    Cheers.  -- Nathaniel