Postgres Pro
Downloads
Home   >   mailing lists

Re: Scaling shared buffer eviction - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Scaling shared buffer eviction
Date
Msg-id CAA4eK1KaUfpkzmc9-pSozkmCaEmJz4VNHCStwU2ZYEkjFW6a=A@mail.gmail.com
Whole thread Raw
In response to Re: Scaling shared buffer eviction  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: Scaling shared buffer eviction
Re: Scaling shared buffer eviction
List pgsql-hackers
Tree view
On Thu, Aug 28, 2014 at 4:41 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> I have yet to collect data under varying loads, however I have
> collected performance data for 8GB shared buffers which shows
> reasonably good performance and scalability.
>
> I think the main part left for this patch is more data for various loads
> which I will share in next few days, however I think patch is ready for
> next round of review, so I will mark it as Needs Review.

I have collected more data with the patch.  I understand that you
have given more review comments due to which patch require
changes, however I think it will not effect the performance data
to a great extent and I have anyway taken the data, so sharing the
same. 

> Performance Data:
> -------------------------------
>
> Configuration and Db Details
> IBM POWER-7 16 cores, 64 hardware threads
> RAM = 64GB
> Database Locale =C
> checkpoint_segments=256
> checkpoint_timeout    =15min
> scale factor = 3000
> Client Count = number of concurrent sessions and threads (ex. -c 8 -j 8)
> Duration of each individual run = 5mins
>
> All the data is in tps and taken using pgbench read-only load

Common configuration remains same as above.

Shared_Buffers = 500MB
Client Count/Patch_Ver8163264128
HEAD562481001121213418112856552
Patch59389112483157034185740166725


Shared_Buffers = 1GB
Client Count/Patch_Ver8163264128
HEAD564011025571216438168657091
Patch59361114813157528188209167752


Shared_Buffers = 14GB
Client Count/Patch_Ver8163264128
HEAD6005911058215205113071897014
Patch61462117377169767225803229083


Shared_Buffers = 15GB
Client Count/Patch_Ver8163264128
HEAD6000511292815365013520336343
Patch6134511556916876722615036985


Observations
---------------------
1. Performance improvement is upto 2~3 times for higher client
counts (64, 128).
2. For lower client count (8), we can see 2~5 % performance
improvement.
3. Overall, this improves the read scalability.
4. For lower number of shared buffers, we see that there is a minor
dip in tps even after patch (it might be that we can improve it by
tuning higher water mark for the number of buffers on freelist, I will
try this by varying high water mark).
5. For larger shared buffers (15GB), we can see that there is still a
dip at large client count, although situation is not bad as compare to
HEAD.  The reason is that at such high shared buffers and client count,
I/O starts happening because all the data can't be contained in RAM.

I will try to take some data for tpc-b load as well.  Kindly let me know
if you want to see data for some other configuration. 


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Xiaoyulei
Date:
Subject: Re: why after increase the hash table partitions, TPMC decrease
Next
From: Amit Kapila
Date:
Subject: Re: Scaling shared buffer eviction