Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning") - Mailing list pgsql-hackers

From Amit Langote
Subject Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning")
Date
Msg-id 80510685-527f-fab7-4cf3-b11b2e0407c6@lab.ntt.co.jp
Whole thread Raw
In response to Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning")  (David Rowley <david.rowley@2ndquadrant.com>)
Responses Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning")
List pgsql-hackers
On 2018/04/04 9:27, David Rowley wrote:
> On 4 April 2018 at 07:46, Justin Pryzby <pryzby@telsasoft.com> wrote:
>> TLDR: even with "faster pruning" patch, pg11dev open()+lseek() every file
>> backing every table being queried, even for those partitions eventually
>> "excluded".
> 
> Yeah, this will no doubt be due to the fact that we still build
> RelOptInfos in the planner for all partitions, so we're still doing
> get_relation_info() and estimate_rel_size() for each of those. It
> might be possible to delay that call until we know we're going to need
> the partition.  I imagine we wouldn't need to know the size of the
> relation until after set_rel_sizes, but I've not checked.

Yeah, one of the earliest patches on the "faster partition pruning" had
tried to solve this issue, but we decided it was better to come to it
after we're done dealing with just making the pruning faster.

Thanks,
Amit



pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning")
Next
From: David Rowley
Date:
Subject: Re: open/lseek overhead with many partitions (including with "fasterpartitioning pruning")