Thread: Random note of encouragement
Hey All,
I've been doing some (futile) work trying to speed up aggregates with a group by in PostgreSQL 9.5.
I installed PostgreSQL 9.6 on the same machine to see if I could get anything running in parallel when using partitioning - which didn't work.
But - I did find this:
With the following setup:
CREATE TABLE base(
view_time TIMESTAMP WITHOUT time ZONE,
view_time_day TIMESTAMP WITHOUT time ZONE,
count_n numeric);
INSERT INTO base
SELECT view_time,
date_trunc('day', view_time),
COUNT::numeric,
FROM
(SELECT
TIMESTAMP '2015-12-01' + random() * interval '30 days' AS view_time,
trunc(random() * 99 + 1) AS COUNT
FROM generate_series(1,30000000)) a;
analyze base;
Now when I run the following SQL (multiple times to allow for getting everything into shared buffers, which is 4GB on my machine):
select sum(count_n) from base group by view_time_day;
I get the following results:
PSQL 9.5 - ~21 secondsPSQL 9.6 devel - ~8.5 seconds
I think that's pretty good!
I know this is a devel release, things may change, blah blah. But still, something has changed for the better here!
I get the same plan on both nodes:
HashAggregate (cost=670590.56..670590.95 rows=31 width=13)Group Key: view_time_day-> Seq Scan on base (cost=0.00..520590.04 rows=30000104 width=13)
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
I've actually just tested this on 9.3 - and I get roughly the same as 9.6devel.
Now going back to make sure my 9.5 environment is sane.
Hopefully this isn't me jumping the gun.
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
On Thu, Feb 25, 2016 at 10:50 AM, Thomas Munro <thomas.munro@enterprisedb.com> wrote:
On Thu, Feb 25, 2016 at 12:26 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Thu, Feb 25, 2016 at 10:06:34AM +1100, James Sewell wrote:
>> Now when I run the following SQL (multiple times to allow for getting
>> everything into shared buffers, which is 4GB on my machine):
>>
>>
>> select sum(count_n) from base group by view_time_day;
>>
>>
>> I get the following results:
>>
>>
>> PSQL 9.5 - ~21 seconds
>> PSQL 9.6 devel - ~8.5 seconds
>>
>>
>> I think that's pretty good!
>>
>> I know this is a devel release, things may change, blah blah. But still,
>> something has changed for the better here!
>
> Wow, that is cool. Can anyone suggest which commit improved this?
Since it sums numerics, maybe integer transition functions from commit
959277a4f579da5243968c750069570a58e92b38 helped?
--
Thomas Munro
http://www.enterprisedb.com
The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.
Argh seems like a false alarm for now.
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
I installed 9.5 from RPM source (the other was one I had installed previously) and the performance matched 9.6
Sorry about that, I must have *something* screwed up on the other one.
Cheers,
James Sewell,
PostgreSQL Team Lead / Solutions Architect
______________________________________
The contents of this email are confidential and may be subject to legal or professional privilege and copyright. No representation is made that this email is free of viruses or other defects. If you have received this communication in error, you may not copy or distribute any part of it or otherwise disclose its contents to anyone. Please advise the sender of your incorrect receipt of this correspondence.