[comp.bugs.sys5] dmdpi floating-point problem

gwyn@smoke.BRL.MIL (Doug Gwyn) (08/01/90)

The nifty "dmdpi" debugger, therefore probably also "pi" on Research UNIX,
will occasionally have the host part terminate abnormally on some systems
such as DEC VAXes.  I tracked this down and offer a quick fix for the bug.

In source dmdpi/c++host/termcore.c (or the corresponding cfront output
dmdpi/chost/termcore.c), add the lines containing "DAG":

...
VERSION(@(#)termcore.c	1.1.1.5	(5/10/87));
...
Cslfd *TermCore.peek(long loc, Cslfd *fail)
{
...
	// on 3B20, assigning certain values to c.dbl will 
	// cause a floating exception
	// DAG -- on VAX, an illegal instruction (reserved operand fault) occurs
	//
	// hadfpe and fpesave must be static variables because
	// catchfpe is not passed "this" pointer
	hadfpe = 0;
	fpesave = c.lng;
	signal(SIGFPE, (SIG_PF)&TermCore::catchfpe);
	signal(SIGILL, (SIG_PF)&TermCore::catchfpe);	// DAG
	c.flt = *(float*)&fpesave;
	c.dbl = *(float*)&fpesave;
	signal(SIGILL, (SIG_PF)(SIG_DFL));		// DAG
	signal(SIGFPE, (SIG_PF)(SIG_DFL));
	if(hadfpe)
...
void TermCore.catchfpe()
{
	signal(SIGFPE, (SIG_PF)&TermCore::catchfpe);
	signal(SIGILL, (SIG_PF)&TermCore::catchfpe);	// DAG
	fpesave = 0; // to stop recursive exceptions on return
	hadfpe = 1;
}
...

I was somewhat amused by the level of debugging involved:  I used a debugger
on the host part of dmdpi, which in turn was being used to debug the
terminal part of the "anim" package, which I was using to view a movie to
debug an application we developed, which we use mostly to debug military
vehicle designs..