rader@Chicago.ira.uka.de (Klaus Radermacher) (03/03/88)
Setting up a database with VAXDBMS database system on a VAX8700 I encountered
the following problem:
Having a record with only one single item of type FLOATING I could not fetch
an record of this type using the following DML-Statement:
fetch first <record-name> where <item-name> eq 8.05
The system's answer was END-OF-COLLECTION, although the record definitely
existed in the database.
The following statement did find the record:
fetch first <record-name> where <item-name> ge 8.05 and
<item-name> le 8.0505
The system gave out the correct record. I understand that the problem arises
due to the internal representation of the type FLOATING, the stored value
must be a little bit larger than 8.05.
However, DEC claims in the manuals, that FLOATING has a precision of about
7 digits, and the stored value was exactly 8.05!
Has anyone encountered similar problems yet? How did you get around the
problem?
I'm glad to hear from you.
Klaus Radermacher
Institut fuer Programmstrukturen
und Datenorganisation
University of Karlsruhe
FRG
e-mail: rader@ira.uka.de
andy@garfield.columbia.edu (Andy Lowry) (03/08/88)
In article <482@iraun1.ira.uka.de> rader@ira.uka.de (Klaus Radermacher) writes: >However, DEC claims in the manuals, that FLOATING has a precision of about >7 digits, and the stored value was exactly 8.05! It's important, when relying on those advertised precisions, to remember that the mantissa in the floating point representation is stored, in most cases, as a *binary* fraction. Thus any statement as to number of correct decimal digits can be a bit misleading. As an example, in the present case, the fractional part 0.05 cannot be stored exactly, since the binary representation of 0.05 is non-terminating. It starts out as 0.0000110011000100.... In fact, there are very few fractions that can be expressed exactly in seven decimal digits and can be expressed exactly in binary at all (like 0.5). So when you type a number like 8.05 to a program that must use a binary floating point representation, internally you will necessarily get only an approximation, no matter how big the mantissa is. Of course, these comments don't apply to machines that can directly manipulate numbers stored in a decimal representation. -Andy