Rounding question

Pickering, John (NORBORD) John.Pickering@norbord.com
Tue, 3 May 2005 12:06:44 -0400


Guy

I think you'll find that you don't really have a value of .0015 -- but
instead have a value of .00149999999999999999 (and as many more nines as
possible). This is one of the joys of floating point numbers. I'd probably
round it with something like FLOOR(your_float + .0001). You may have to
divide by some power of 10 to get eh decimal in the right place.

Regards,
JWP

-----Original Message-----
From: Guy Werry [mailto:guy.werry@hbms.ca]
Sent: Tuesday, May 03, 2005 11:47 AM
To: PH Mailing List (E-mail)
Subject: Rounding question


Ph 7.33.D3, Ux 10.20

The rounding function is giving me fits.  Specifically, I have a value in a
FLOAT data item and was rounding it like this:
	round(inp_val,3,near)

So I'm trying to round to 3 decimals.  If I enter 0.0015, it rounds DOWN to
0.001!

I have tried various combinations of the "multiply by 1000, round it, divide
by 1000" rigamarole, with no success.

Is there a clean, neat solution to this?  Is there some way of telling the
stupid thing to round UP on that 5?  Note that the "UP" and "DOWN" options
don't give the results that I need.

[The frustration mirrored here is largely due to the fact that COBOL rounds
wonderfully and predictably and I've NEVER found another language that could
round its way out of a wet paper bag!]

Thanks,
Guy L. Werry
Senior Systems Analyst
Hudson Bay Mining & Smelting Co., Limited.