Re: Floating point error - Mailing list pgsql-general

From John R Pierce
Subject Re: Floating point error
Date
Msg-id 512ACBD9.4030700@hogranch.com
Whole thread Raw
In response to Re: Floating point error  (John R Pierce <pierce@hogranch.com>)
Responses Re: Floating point error
List pgsql-general
On 2/24/2013 6:20 PM, John R Pierce wrote:
> On 2/24/2013 6:13 PM, Tom Duffey wrote:
>> - The Java app on production shows "10.3884573" while the test app
>> shows "10.3885"
>
> 'real' is single precision, which is only about 6 digits of decimal
> accuracy.   if your java variables were double precision, you probably
> should have used double in postgres too.
>
> 0.1 decimal is a repeating fraction in binary.

ooops, missed my third point.    if precise decimal numbers are
important, use a decimal data type, which is NUMERIC in SQL (I'm not
familiar enough with Java to know if it has a BCD or other decimal type).

--
john r pierce                                      37N 122W
somewhere on the middle of the left coast

pgsql-general by date:

Previous
From: John R Pierce
Date:
Subject: Re: Floating point error
Next
From: Edson Richter
Date:
Subject: Re: Floating point error