Rounding question
brian_matthewsbrian matthews
brian_matthews_bmw@hotmail.com
Wed, 04 May 2005 22:15:10 +0000
the 4th para could default to the 2nd para +1 where no entry is made
regards Bri,
>From: "brian_matthewsbrian matthews" <brian_matthews_bmw@hotmail.com>
>To: guy.werry@hbms.ca, powerh-l@lists.sowder.com
>Subject: RE: Rounding question
>Date: Wed, 04 May 2005 20:00:38 +0000
>
>I would expect the round function to work as field displays work; namely,
>if you increase the number of decimals for the field you will see more of
>the true digits present, by the same token if you reduce the number of
>decimals you will see fewer digits, and the last digit will be ‘rounded’
>based on the now invisible digits.
>
>If a value of .0001499999… is held on disk, and the value is displayed in a
>field of 5dps the value will appears as .00015 and not as .00014, the round
>function should behave in the same way. In order for this to work, a fourth
>parameter could be added to the round function in order to tell the round
>function to pre-round the value at a certain point ( as must be the case
>with field displays now ) this new value is then used as the input to the
>round function.
>
>I know it seems odd to round before rounding, but it’s better than having
>to code as Guy is having to for every similar scenario.
>
>regards Bri,
>
>
>
>
>>From: Guy Werry <guy.werry@hbms.ca>
>>To: PowerHouse List <powerh-l@lists.sowder.com>
>>Subject: RE: Rounding question
>>Date: Wed, 4 May 2005 11:26:30 -0500
>>
>>Thanks Robert ... you are pretty close. It's a data format issue,
>>relating
>>to the format of FLOAT data on the HP. Any language working with a float
>>would have the same problem.
>>
>>In terms of using a different data type, I did some experimentation there
>>but with no success, because Powerhouse uses float for its arithmetic. No
>>matter what data type I start with, I wind up with FLOAT when it counts,
>>in
>>the actual calculation.
>>
>>So, I used the "add 0.000001" before rounding trick.
>>
>>Does anyone remember the days of HP's Transact language, where if you had
>>multiple calculations on one line it would use the LEAST precise number of
>>decimals for the internal calculations? We once had a percentage
>>calculation that had a rounding error of around $500 because one of the
>>data
>>items involved was an INTEGER, which meant that the whold calculation
>>rounded to ZERO decimals all the way through! That was fun tracking down
>>....
>>
>>Anyways, we've got the problem licked. Thanks all!
>>
>>Guy L. Werry
>>Senior Systems Analyst
>>Hudson Bay Mining & Smelting Co., Limited.
>>
>>-----Original Message-----
>>From: Robert Edis [mailto:robeconsult@sbcglobal.net]
>>Sent: Wednesday, May 04, 2005 11:12 AM
>>To: PowerHouse List
>>Subject: RE: Rounding question
>>
>>
>>G'day Guy
>>
>>This may not be a PH issue but rather a database
>>issue. The database may be storing your floating
>>point number of .0015 not as .001500000000000000
>>(whatever the number of positions is the default for
>>the data type) but as .00149999999999999991. SQL will
>>return the correct value as .0015 but when the ROUND()
>>function is applied PH (or SQL) may think the value is
>>closer to .001 than .002.
>>
>>I have seen this as a problem in SQL Server many
>>times. Have you tried making the data type a DECIMAL
>>instead? Have you tried converting the FLOAT to a
>>DECIMAL and then rounded?
>>
>>Blue
>>
>>
>>--- David Morrison - Corporate
>><dmorrison@mcbrideelectric.com> wrote:
>> > Guy,
>> >
>> > How about adding something like .0000000001 to
>> > inp_val, prior to rounding?
>> >
>> > Thanks.
>> >
>> > David Morrison
>> > McBride Electric
>> >
>> > -----Original Message-----
>> > From: powerh-l-admin@lists.sowder.com
>> > [mailto:powerh-l-admin@lists.sowder.com]On Behalf Of
>> > Guy Werry
>> > Sent: Tuesday, May 03, 2005 8:47 AM
>> > To: PH Mailing List (E-mail)
>> > Subject: Rounding question
>> >
>> >
>> > Ph 7.33.D3, Ux 10.20
>> >
>> > The rounding function is giving me fits.
>> > Specifically, I have a value in a
>> > FLOAT data item and was rounding it like this:
>> > round(inp_val,3,near)
>> >
>> > So I'm trying to round to 3 decimals. If I enter
>> > 0.0015, it rounds DOWN to
>> > 0.001!
>> >
>> > I have tried various combinations of the "multiply
>> > by 1000, round it, divide
>> > by 1000" rigamarole, with no success.
>> >
>> > Is there a clean, neat solution to this? Is there
>> > some way of telling the
>> > stupid thing to round UP on that 5? Note that the
>> > "UP" and "DOWN" options
>> > don't give the results that I need.
>> >
>> > [The frustration mirrored here is largely due to the
>> > fact that COBOL rounds
>> > wonderfully and predictably and I've NEVER found
>> > another language that could
>> > round its way out of a wet paper bag!]
>> >
>> > Thanks,
>> > Guy L. Werry
>> > Senior Systems Analyst
>> > Hudson Bay Mining & Smelting Co., Limited.
>> >
>> >
>> >
>> > --
>> > = = = = = = = = = = = = = = = = = = = = = = = = = =
>> > = =
>> > Mailing list: powerh-l@lists.sowder.com
>> > Subscribe: "subscribe" in message body to
>> > powerh-l-request@lists.sowder.com
>> > Unsubscribe: "unsubscribe <password>" in
>> > message body to powerh-l-request@lists.sowder.com
>> > http://lists.sowder.com/mailman/listinfo/powerh-l
>> > This list is closed, thus to post to the list you
>> > must be a subscriber.
>> >
>> > --
>> > = = = = = = = = = = = = = = = = = = = = = = = = = =
>> > = =
>> > Mailing list: powerh-l@lists.sowder.com
>> > Subscribe: "subscribe" in message body to
>> > powerh-l-request@lists.sowder.com
>> > Unsubscribe: "unsubscribe <password>" in
>> > message body to powerh-l-request@lists.sowder.com
>> > http://lists.sowder.com/mailman/listinfo/powerh-l
>> > This list is closed, thus to post to the list you
>> > must be a subscriber.
>> >
>>--
>>= = = = = = = = = = = = = = = = = = = = = = = = = = = =
>>Mailing list: powerh-l@lists.sowder.com
>>Subscribe: "subscribe" in message body to
>>powerh-l-request@lists.sowder.com
>>Unsubscribe: "unsubscribe <password>" in message body to
>>powerh-l-request@lists.sowder.com
>>http://lists.sowder.com/mailman/listinfo/powerh-l
>>This list is closed, thus to post to the list you must be a subscriber.
>>--
>>= = = = = = = = = = = = = = = = = = = = = = = = = = = =
>>Mailing list: powerh-l@lists.sowder.com
>>Subscribe: "subscribe" in message body to
>>powerh-l-request@lists.sowder.com
>>Unsubscribe: "unsubscribe <password>" in message body to
>>powerh-l-request@lists.sowder.com
>>http://lists.sowder.com/mailman/listinfo/powerh-l
>>This list is closed, thus to post to the list you must be a subscriber.
>
>_________________________________________________________________
>It's fast, it's easy and it's free. Get MSN Messenger 7.0 today!
>http://messenger.msn.co.uk
>
>--
>= = = = = = = = = = = = = = = = = = = = = = = = = = = =
>Mailing list: powerh-l@lists.sowder.com
>Subscribe: "subscribe" in message body to powerh-l-request@lists.sowder.com
>Unsubscribe: "unsubscribe <password>" in message body to
>powerh-l-request@lists.sowder.com
>http://lists.sowder.com/mailman/listinfo/powerh-l
>This list is closed, thus to post to the list you must be a subscriber.
_________________________________________________________________
Be the first to hear what's new at MSN - sign up to our free newsletters!
http://www.msn.co.uk/newsletters