Postgres Pro
Downloads
Home   >   mailing lists

Re: Add numeric_trim(numeric) - Mailing list pgsql-hackers

From Pavel Stehule
Subject Re: Add numeric_trim(numeric)
Date
Msg-id CAFj8pRBW73THWPN8aX0CpXt_hAzk7yvd_tGdfRaXoLzg0yToGw@mail.gmail.com
Whole thread Raw
In response to Add numeric_trim(numeric)  (Marko Tiikkaja <marko@joh.to>)
Responses Re: Add numeric_trim(numeric)
List pgsql-hackers
Tree view
Hi

2015-11-19 3:58 GMT+01:00 Marko Tiikkaja <marko@joh.to>:
Hi,

Here's a patch for the second function suggested in 5643125E.1030605@joh.to.  This is my first patch trying to do anything with numerics, so please be gentle.  I'm sure it's full of stupid.

January's commit fest, feedback welcome, yada yada..

I am looking on this patch and I don't understand to formula

dscale = (ndigits - arg.weight - 1) * DEC_DIGITS;

the following rule is valid

DEC_DIGITS * ndigits >= dscale + arg.weight + 1

so dscale should be calculated like

dscale <= DEC_DIGITS * ndigits - arg.weight - 1

?

but your formula is correct and working. Can you explain it?

Regards

Pavel




 


.m

pgsql-hackers by date:

Previous
From: Joe Conway
Date:
Subject: Re: exposing pg_controldata and pg_config as functions
Next
From: Jeff Janes
Date:
Subject: Re: 9.5rc1 brin_summarize_new_values