[comp.sys.next] buggy behaviour with 68040 fcmps

avery@rana.usc.edu (Avery Wang) (05/14/91)

I sent this a few weeks ago, but it didn't seem to get posted:
-------------------------------------------------------
Hi there,

After doing some assembly hacking to track down a weird bug that
suddenly appeared in my program after upgrading to the 68040 system
I think I may have found a bug in the 68040 fcmps instruction.

According the UNIX man pages:
MATH(3M)            UNIX Programmer's Manual             MATH(3M)
...	If x != x then x is
	NaN; every other predicate (x > y, x = y, x < y,
	...) is FALSE if NaN is involved.
...

I was using this feature to detect if my numerical simulations
were generating NaNs.  Here is a test program which captures the
essence of what happens:

main ()
{
	float a,t;
	
//	We all know that 0.2 in binary cannot be represented exactly
//	so we can get weird stuff to happen in the LSBs of the FP registers.

	a=0.5;
	t=0.2;
	t=t+a;
	if (!(t==t)) {
		printf ("not matching\n");
	}
	else {
		printf ("matching\n");
	}
}

The output is:
scruffy> test
not matching
scruffy> 

The gdb disassembled code looks like:
0x3e38 <main>:  linkw fp,#-8
0x3e3c <main+4>:        movel @#0x3ff8 <_fvmlib_init1+24>,-4(fp)
0x3e44 <main+12>:       movel @#0x3ffc <_fvmlib_init1+28>,-8(fp)
0x3e4c <main+20>:       fmoves -8(fp),fp0
0x3e52 <main+26>:       fadds -4(fp),fp0
0x3e58 <main+32>:       fmoves fp0,-8(fp)
0x3e5e <main+38>:       fcmps -8(fp),fp0
0x3e64 <main+44>:       fbeq 0x3e78 <main+64>
0x3e68 <main+48>:       pea @#0x3fe0 <_fvmlib_init1>
0x3e6e <main+54>:       bsr 0x5002ce4 <.branch_table_slot_536>
0x3e74 <main+60>:       addqw #4,sp
0x3e76 <main+62>:       bra 0x3e86 <main+78>
0x3e78 <main+64>:       pea @#0x3fee <_fvmlib_init1+14>
0x3e7e <main+70>:       bsr 0x5002ce4 <.branch_table_slot_536>
0x3e84 <main+76>:       addqw #4,sp
0x3e86 <main+78>:       unlk fp
0x3e88 <main+80>:       rts

By looking at the registers and memory, we see that after <main+32>
we get:
fp0     0.7000000029802322  (raw 0x3ffe0000b333334000000000)

I believe the 4 at the end of the 3's is some sort of a garbage,
because after doing 'fmoves fp0,-8(fp)' followed by 
'fmoves -8(fp),fp0' (not shown directly above), we get 
fp0     0.699999988079071  (raw 0x3ffe0000b333330000000000)

I've done other playing around and have found that the fsglmuls instruction cleans out the lower bits, but that the fsubs and fadds instructions leave garbage in the lower bits.  This is seen when the two float operands to the fadds instruction have different exponents.
The bug could be here.

I think though, that the bug may be in the fcmps instruction, since there should be information in the extended fp format that tells if the operand is to be treated as a single or extended precision object, and that it may be left up to the fcmps instruction to ignore the garbage bits in the single-precision format.  I don't have access to a 68040 or 68882 manual so I can't look up the IEEE 754 format.

Any comments?  I usually stay out of the machine level stuff, but I couldn't figure out why my program suddenly broke after the upgrade.

Thanks,
-Avery Wang

RUB-Institut fuer Neuroinformatik
ND 03, Postfach 10 21 48
W-4630 Bochum 1
Germany