Thread: width_bucket issue
Hi,
The width_bucket function doesn't seem to work the way I'd expect:
postgres=# SELECT width_bucket(4, 0, 12, 3) b1, width_bucket(4 :: NUMERIC, 0, 12, 3) b2;
b1 | b2
----+----
2 | 1
(1 row)
I'd expect b1 = b2 = 2. What am I missing?
This is with 10.4 running as a docker container, if it matters:
postgres=# SELECT version();
version
---------------------------------------------------------------------------------------
PostgreSQL 10.4 (Debian 10.4-2.pgdg90+1) on x86_64-pc-linux-gnu, compiled by gcc (Debi
(1 row)
TIA,
> On Jul 24, 2018, at 13:02, Raphaël Berbain <raphael.berbain@gmail.com> wrote:
> I'd expect b1 = b2 = 2. What am I missing?
The problem appears to be due to rounding during the intermediate calculations. In compute_bucket() in numeric.c:
div_var(&operand_var, &bound1_var, result_var,
select_div_scale(&operand_var, &bound1_var), true);
... produces 0.99999999999999999999 for that particular value, instead of 1, and the subsequent +1 and FLOOR() result
in1 instead of 2.
--
-- Christophe Pettus
xof@thebuild.com