Microsoft Knowledge Base Article
This article contents is Microsoft Copyrighted material.
©2005-©2007 Microsoft Corporation. All rights reserved.
Terms
of Use |
Trademarks
Article ID: 131578 - Last Review: September 30, 2003 - Revision: 3.1
FIX: Math Problem with INT() Function
This article was previously published under Q131578
The INT(nExpression) Function evaluates a numeric expression and returns
incorrect values.
To work around this problem, change the SET DECIMAL setting to a larger
more appropriate value such as 10 (the default is 2).
Microsoft has confirmed this to be a problem in the Microsoft products
listed at the beginning of this article. This problem has been fixed in
Visual FoxPro 5.0 for Windows.
Steps to Reproduce Problem
Run the following code in Visual FoxPro:
pnum1 = 1000000
pnum2 = 999999
SET DECI TO 2 && this is default
? int(pnum2/pnum1) && result is 1 it should be 0
SET DECI TO 10
? int(pnum2/pnum1) && result is 0, which is correct
APPLIES TO
- Microsoft Visual FoxPro 3.0 Standard Edition
- Microsoft Visual FoxPro 3.0b Standard Edition
| kbbug kbfix kbvfp500fix KB131578 |
Community Feedback System
Very often, it takes hours to solve a problem. Very often, you've looked high
and low, and have tried a lot of solutions. When you finally found it, chances
are, it was because someone else helped you. Here's your chance to give back.
Use our community feedback tool to let others know what worked for you and what
didn't.
Please also understand that the community feedback system is not warranted to be
correct, it's simply a system that we've built to let people try and help each
other. If something in a feedback response doesn't make sense to you, or you're
not comfortable making changes that the feedback talks about (like registry
edits), please consult a professional.
Thank you for using kbAlertz.com Feedback System.
-- Scott Cate
Be the first to leave feedback, to help others about this knowledge base
article.
(Optional) Name
(Optional)
Public URL Or Email
Comments
No
HTML -- Text Only Please