Strange Rounding
Terry Pickering
pickering@4j.lane.edu
Wed, 09 Oct 2002 11:08:05 -0700
Here is a small example of code that duplicates a problem I've encountered.
Basically, I'm taking a text field that contains a number with a decimal
point (from an Excel PRN file), and converting to a standard integer field.
DEFINE T-AMT CHAR*8 = PARM PROMPT "ENTER AMOUNT "
DEFINE N-AMT INT*8 = NCONVERT(T-AMT) * 100
REPORT &
T-AMT &
N-AMT PIC "^^^,^^^.^^"
If you enter the number "518.30" for T-AMT, it reports 518.29 for N-AMT. At
first thought I assumed it must be rounding issue with the NCONVERT
function, but then I tried "18.30" and "1518.30" and both of them worked ok.
Another number I've encountered with this strange occurrence is "2351.44"
I've tried splitting the define of N-AMT into two separate defines. I've tried
DEFINE X-AMT = NCONVERT(T-AMT)
DEFINE N-AMT INT*8 = X-AMT * 100
The workaround that seems to fix the problem is:
DEFINE N-AMT INT*8 = ROUND(NCONVERT(T-AMT * 100))
I think we have lots of code that uses the first method of conversion. Any
ideas why it is doing what it does? Why does it work for "18.30" but not
"518.30"?
____________________________
Terry Pickering