Knowledge Base

BUG: Incorrect Results When Comparing Large Numbers

Article ID: 189866

Article Last Modified on 12/11/1999

Microsoft Visual FoxPro 5.0 Standard Edition
Microsoft Visual FoxPro 5.0a
Microsoft Visual FoxPro 6.0 Professional Edition

This article was previously published under Q189866

SYMPTOMS

When you compare two numbers for equality, you might receive an erroneous result when the resulting numbers are very large.

NOTE: This behavior does not occur in Visual FoxPro 3.0x.

RESOLUTION

Use the ROUND() function when checking the equality of very large floating- point numbers. For example, the following line of code produces the correct result:

   ?ROUND(1+(1048575+(i/1000)),5)=1048576.001

STATUS

Microsoft has confirmed this to be a bug in the Microsoft products listed at the beginning of this article. We are researching this bug and will post new information here in the Microsoft Knowledge Base as it becomes available.

MORE INFORMATION

Steps to Reproduce Behavior

Type the following commands in the Command window:

   SET DECIMAL to 5
   i=1
   ?(1+(1048575+(i/1000)))=1048576.001
   ?1+(1048575+(1/1000))=1048576.001

Notice that the first calculation, using the variable i, evaluates as false while the calculation using the integer evaluates true.

Additional query words: kbVFp500bug kbVFp500abug kbVFp600bug kbXBase

Keywords: kbbug kbnofix KB189866