BRYAN@SU-SIERRA.ARPA (Doug Bryan) (10/19/85)
We have a question for all you model number gurus out there: [3.5.7(6)] The reference manual states that the minimum number of binary digits, B, required after the point in the binary mantissa of a floating point number is... ceiling (D * ln(10)/ln(2) + 1.0) where D is the minimal number of decimal digits in the decimal mantissa. Consider D = 3... ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11 But... is not 10 binary digits sufficient to represent 3 decimal digits? 2**(-10) < 10**(-3) We think the "+ 1.0" in the above expression can be removed. doug bryan and geoff mendal -------
KFL@MIT-MC.ARPA ("Keith F. Lynch") (10/19/85)
Date: Fri 18 Oct 85 19:39:57-PDT
From: Doug Bryan <BRYAN@SU-SIERRA.ARPA>
[3.5.7(6)]
The reference manual states that the minimum number of binary digits, B,
required after the point in the binary mantissa of a floating point
number is...
ceiling (D * ln(10)/ln(2) + 1.0)
where D is the minimal number of decimal digits in the decimal mantissa.
Consider D = 3...
ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11
But... is not 10 binary digits sufficient to represent 3 decimal digits?
2**(-10) < 10**(-3)
We think the "+ 1.0" in the above expression can be removed.
Consider D=2. 2**(-6) > 10**(-2). The "+ 1.0" should stay.
...Keithdik@zuring.UUCP (10/20/85)
In article <8510190414.AA24234@UCB-VAX> BRYAN@SU-SIERRA.ARPA (Doug Bryan) writes: >[3.5.7(6)] >The reference manual states that the minimum number of binary digits, B, >required after the point in the binary mantissa of a floating point >number is... > > ceiling (D * ln(10)/ln(2) + 1.0) > >where D is the minimal number of decimal digits in the decimal mantissa. > >Consider D = 3... > > ceiling (3 * ln(10)/ln(2) + 1.0) = ceiling (10.9657) = 11 > >But... is not 10 binary digits sufficient to represent 3 decimal digits? > 2**(-10) < 10**(-3) >We think the "+ 1.0" in the above expression can be removed. > >doug bryan and geoff mendal > No. The relative precision of the binary representation should not be less than that of the decimal representation. For difits 3 the relative precision of the decimal representation ranges from 1 in 500 to 1 in 999, and with 10 bits for the binary representation from 1 in 512 to 1 in 1023. And indeed we find the following: 819/8192 = .099975 818/8192 = .099853 817/8192 = .099731 So there is no representation for .0998, a valid digits 3 number. -- dik t. winter, cwi, amsterdam, nederland UUCP: {seismo|decvax|philabs}!mcvax!dik