donn@CS.UTAH.EDU (Donn Seeley) (10/28/89)
[I tried sending this to bug-gas@prep, but evidently the sendmail on
prep lies when it says that the address exists because the mail
bounced... Sorry to bother GCC fixers with this; maybe the GAS fixers
are reading this too?]
It will be hard to duplicate this yourself, but here's how it happened
to us. We built GAS 1.34 with GCC 1.36 -O -fstrength-reduce on our
68000-based HP 9000/370 running 4.3 BSD, and it began dumping core when
asked to assemble code that contained certain combinations of floating
point constants. The following short example, boiled down from the
Berkeley math library cosh() function, was sufficient to elicit the
problem:
.data
.even
_small.2:
.double 0r1
.text
.even
.globl _cosh
_cosh:
link a6,#0
fcmpd #0r.5,fp2
rts
Investigation with the debugger revealed that strength reduction had
increased the amount of stack allocated by the function m68_ip() in
m68k.c, and something stomped on a pointer stored on the stack,
causing the assembler to get a segmentation fault the next time the
pointer was dereferenced. Tracing showed that the trashing occurred
during a call to get_num(). Eventually we isolated the problem --
flonum_multip() was writing into generic_floating_point_number's
buffer, but the buffer address was on the stack! More specifically,
get_num() wrote over the pointer on the stack with the exponent and
high bits of the mantissa from (appropriately) a NaN. It was very
surprising to me to find that generic_floating_point_number's buffer
was on the stack, but a grep of the source showed that atof_m68k() sets
generic_floating_point_number.low to a stack buffer. This would be
fine if all uses of the buffer were under a call to atof_m68k(), but
operand() in expr.c calls atof_generic() without going through
atof_m68k() and as a result when flonum_multip() is called with
&generic_floating_point_number as a target, a random piece of stack is
blown away.
It seems fairly clear that generic_floating_point_number's buffer
should never be on the stack. The following change to atof-m68k.c
prevents our core dumps:
*** /tmp/,RCSt1000876 Sat Oct 28 01:09:31 1989
--- atof-m68k.c Sat Oct 28 00:58:40 1989
***************
*** 145,151 ****
char what_kind; /* 'd', 'f', 'g', 'h' */
LITTLENUM_TYPE * words; /* Build the binary here. */
{
! LITTLENUM_TYPE bits [MAX_PRECISION + MAX_PRECISION + GUARD];
/* Extra bits for zeroed low-order bits. */
/* The 1st MAX_PRECISION are zeroed, */
/* the last contain flonum bits. */
--- 145,151 ----
char what_kind; /* 'd', 'f', 'g', 'h' */
LITTLENUM_TYPE * words; /* Build the binary here. */
{
! static LITTLENUM_TYPE bits [2 * MAX_PRECISION + GUARD];
/* Extra bits for zeroed low-order bits. */
/* The 1st MAX_PRECISION are zeroed, */
/* the last contain flonum bits. */
Donn Seeley University of Utah CS Dept donn@cs.utah.edu
40 46' 6"N 111 50' 34"W (801) 581-5668 utah-cs!donn
PS -- A note: the 'generic_buffer' hack in expr.c is NOT guaranteed
to work. GCC, unlike PCC, does not always lay out data objects in
the order they are declared... Fortunately, as the above bug shows
:-), the 68k GAS doesn't use the normal buffer.