FILLMORE%EMRCAN.bitnet@ugw.utcs.utoronto.ca (11/24/89)
While developing a Benchmark Modula-2 IEEE module I discovered a bug in the routine IEEEDPFieee in mathieeedoubtrans.library. This routine converts a single-precision IEEE number to double precision. It works correctly for many cases but here is a simple case that fails: The value to be converted is 0.1 decimal. Precision Hex value Decimal value Comments single 3DCCCCCD 0.10000 value from SPTieee double 3FB9999999999999 0.0999999999 value from AmigaBasic double 3FB9DDDDA0000000 0.1010416522 value from IEEEDPFieee As you can see AmigaBasic double precision (also IEEE) gives much more accurate values than the output of IEEEDPFieee. Also, if 0.1 is calculated by dividing 1 by 10 in double precision the answer is correct. Has anybody noticed this before? Is there a fix for this? ________________________ Bob Fillmore, Systems Software & Communications BITNET: FILLMORE@EMRCAN Computer Services Centre, BIX: bfillmore Energy, Mines, & Resources Canada Voice: (613) 992-2832 588 Booth St., Ottawa, Ontario, Canada K1A 0E4 FAX: (613) 996-2953