Rounding question

[Deskin, Bob]

Had to read through the thread before I responded.

I view floating point as a function of the platform. Now that may include the hardware, including microcode, operating system, etc. The bottom line is that whatever PowerHouse gets as float, we are stuck with. Yes, I know, assuming we use float at all. Unfortunately, that decision was made over 25 years ago and I don't see it changing.

Look at the FLOAT section in the PowerHouse Rules book, specifically p. 257 in the pdf. You'll see the structure of the float we have available and the significant digits.

As for FIELDVALUE, look at p 248 of the same book. It says that FIELDVALUE takes the internal representation of the corresponding item. So if the entry is float, fieldvalue is float, if the entry is integer, then FIELDVALUE is integer. However, that only serves to hold the appropriate value. As soon as you start a calculation or expression, the value in FIELDVALUE is converted to floating point internally and the processing done on that. So in Guy's example (ROUND(FIELDVALUE + 0.000001,NUM-DECIMALS,NEAR)), as soon as he uses FIELDVALUE in the calculation, it's converted to floating point, regardless of what it was to begin with. However, in this case, for the calculation to make sense, it must have been floating point to begin with.

As for the "internal buffer" versus "server", the "platform" provides the ability to use certain datatypes. It's up to the application what to make use of. The "internal buffers" are simply areas in memory that the application identifies as a particular datatype when it processes. I mean if I tell a floating point calculation that a set of 8 bytes is to be used and that 8 bytes contains character data, the calculation will operate because float will take anything as input.

Bob

-----Original Message-----
From: brian_matthewsbrian matthews
[mailto:brian_matthews_bmw@hotmail.com]
Sent: May 3, 2005 6:09 PM
To: John.Pickering@norbord.com; powerh-l@lists.sowder.com
Cc: Deskin, Bob
Subject: RE: Rounding question


I am not alone after all , I had always assumed that float eight was the 
internal assumption ( clearly I would make no poker player :)

regards Bri,




>From: "Pickering, John (NORBORD)" <John.Pickering@norbord.com>
>To: powerh-l@lists.sowder.com
>CC: "Deskin Bob (E-mail)" <bob.deskin@cognos.com>
>Subject: RE: Rounding question
>Date: Tue, 3 May 2005 17:43:45 -0400
>
>But fieldtext and fieldvalue ARE internal quick buffers!
>
>Question for Mr. Deskin -- if the data item is of date type integer, what
>data type is fieldvalue?
>
>
>-----Original Message-----
>From: brian_matthewsbrian matthews
>[mailto:brian_matthews_bmw@hotmail.com]
>Sent: Tuesday, May 03, 2005 4:57 PM
>To: guy.werry@hbms.ca; powerh-l@lists.sowder.com
>Subject: RE: Rounding question
>
>
>many thanks for the update - I always assumed that fieldtext and fieldvalue
>were internal quick buffers and independent of the server - but this is
>based on ignorant assumption only - so I am always happy to be corrected :)
>
>regards Bri,
>
>
>
>
>--
>= = = = = = = = = = = = = = = = = = = = = = = = = = = =
>Mailing list: powerh-l@lists.sowder.com
>Subscribe: "subscribe" in message body to powerh-l-request@lists.sowder.com
>Unsubscribe: "unsubscribe &lt;password&gt;" in message body to 
>powerh-l-request@lists.sowder.com
>http://lists.sowder.com/mailman/listinfo/powerh-l
>This list is closed, thus to post to the list you must be a subscriber.

_________________________________________________________________
Winks & nudges are here - download MSN Messenger 7.0 today! 
http://messenger.msn.co.uk 
  
       This message may contain privileged and/or confidential information.  If you have received this e-mail in error or are not the intended recipient, you may not use, copy, disseminate or distribute it; do not open any attachments, delete it immediately from your system and notify the sender promptly by e-mail that you have done so.  Thank you.