Thread: [HACKERS] Remaining 2017-03 CF entries
Hi, When I started writing this, there were the following reamining CF items, minus bugfix ones which aren't bound by the code freeze. I think it makes sense to go through those and see whether it's realistic to commit any of them. Ready for Committer: Add GUCs for predicate lock promotion thresholds: - claimed by Kevin, should be easy enough initdb configurable wal_segment_size - parts have been committed - significantly derailed by segment naming discussion - possibly committable if we decide to skip the naming bit? But also a bit late given that it touches some quite sensitivecode. Create fdw_outerpath for foreign - haven't really followed discussion - only marked as ready-for-committer 2017-04-04 Vacuum: allow usage of more than 1GB of work mem - hm, maybe? Will take a look. Unique Joins - Tom's discussing things with David, not sure. Push down more UPDATEs/DELETEs in postgres_fdw - claimed by Robert? postgres_fdw: support parameterized foreign joins - think that depends on fdw_outerpath? Waiting on Author: SQL statements statistics counter view (pg_stat_sql) - the code doesn't look quite ready - don't think we quite have design agreement, e.g. I don't like where it's hooked into query execution Write Amplification Reduction Method (WARM) - fair number of people don't think it's ready for v10. - can't move to next fest because it's waiting-on-author, which doesn't allow that. Doesn't strike me as a useful restriction. BRIN optimize memory allocation - I think Alvaro has indicated that he wants to take care of that? Indexes with Included Columns (was Covering + unique indexes) - Don't think concerns about #columns on truncated tuples have been addressed. Should imo be returned-with-feedback. Needs-Review: Better estimate merging for duplicate vars in clausesel.c - has been submitted pretty late (2017-02-24) and discussion is ongoing - I'm inclined to punt on this one to the next release, previous proposal along that line got some pushback new plpgsql extra_checks - Winner of the "most opaque CF title" award - hasn't received a whole lot of review - don't think we're even close to having design agreement Generic type subscripting - still some review back and forth - probably should be punted Any comments? Greetings, Andres
On Fri, Apr 7, 2017 at 1:37 PM, Andres Freund <andres@anarazel.de> wrote: > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > > Ready for Committer: > > Add GUCs for predicate lock promotion thresholds: > - claimed by Kevin, should be easy enough I was planning on pushing this today. -- Kevin Grittner
Andres Freund wrote: > Unique Joins > - Tom's discussing things with David, not sure. This one was already included-and-removed from 9.6, Tom had said he'd give it priority during the current cycle as I recall. It seems unfair that it's still waiting for review on the last day of pg10's last commitfest. > Write Amplification Reduction Method (WARM) > - fair number of people don't think it's ready for v10. I'm going over this one now with Pavan, with the intent of getting it in committable shape. I may be biased, but the claimed performance gains are so large that I can't let it slip through without additional effort. > BRIN optimize memory allocation > - I think Alvaro has indicated that he wants to take care of that? I am happy to see it move to pg11 to give priority to WARM. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund <andres@anarazel.de> wrote: > Write Amplification Reduction Method (WARM) > - fair number of people don't think it's ready for v10. > - can't move to next fest because it's waiting-on-author, which doesn't > allow that. Doesn't strike me as a useful restriction. I agree that that CF app restriction makes little sense. > Indexes with Included Columns (was Covering + unique indexes) > - Don't think concerns about #columns on truncated tuples have been > addressed. Should imo be returned-with-feedback. +1. -- Peter Geoghegan
Andres Freund <andres@anarazel.de> writes:
> I think it makes sense to go through those and see whether it's
> realistic to commit any of them.
> Unique Joins
> - Tom's discussing things with David, not sure.
Working on this one today.
> Generic type subscripting
> - still some review back and forth
> - probably should be punted
Yeah, I do not think we should hustle this in at the last minute.
regards, tom lane
On 2017-04-07 15:45:33 -0300, Alvaro Herrera wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. > > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. > > I may be biased, but the claimed performance gains are so large that I > can't let it slip through without additional effort. I strongly object to pushing it into v10. The potential negative impact of a patch that touches the on-disk representation is also pretty large. I think we'll have to discuss among a few more committers whether we're ok with pushing this one. Andres
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Andres Freund wrote:
>> Write Amplification Reduction Method (WARM)
>> - fair number of people don't think it's ready for v10.
> I'm going over this one now with Pavan, with the intent of getting it in
> committable shape.
I have to agree with Andres that this is not something to push in, on the
last day before feature freeze, when a number of people aren't comfortable
with it. It looks much more like a feature to push at the start of a
development cycle.
regards, tom lane
On Fri, Apr 7, 2017 at 2:53 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Alvaro Herrera <alvherre@2ndquadrant.com> writes: >> Andres Freund wrote: >>> Write Amplification Reduction Method (WARM) >>> - fair number of people don't think it's ready for v10. > >> I'm going over this one now with Pavan, with the intent of getting it in >> committable shape. > > I have to agree with Andres that this is not something to push in, on the > last day before feature freeze, when a number of people aren't comfortable > with it. It looks much more like a feature to push at the start of a > development cycle. I strongly agree. Testing has found some noticeable regressions in some cases as well, even if there were no outright bugs. I'm frankly astonished by the ongoing unwillingness to admit that the objections (by multiple people) to this patch have any real merit. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Peter Geoghegan wrote: > On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund <andres@anarazel.de> wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. Given the number of votes against putting this on pg10, I am going to back off from this patch now, with an eye towards putting it in pg11 as soon as the tree opens. Either I or Pavan are going to post another version of this patch series, within the next couple of weeks, so that others can base their testing, review and suggestions. > > - can't move to next fest because it's waiting-on-author, which doesn't > > allow that. Doesn't strike me as a useful restriction. > > I agree that that CF app restriction makes little sense. What the restriction means is that if a patch is in waiting-on-author, the proper "close" action is to return-with-feedback. There is no point in moving the patch to the next commitfest if there is no further patch version. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > - can't move to next fest because it's waiting-on-author, which doesn't > > > allow that. Doesn't strike me as a useful restriction. > > > > I agree that that CF app restriction makes little sense. > > What the restriction means is that if a patch is in waiting-on-author, > the proper "close" action is to return-with-feedback. There is no point > in moving the patch to the next commitfest if there is no further patch > version. That's true if the patch has been in that state for a while, but if you find some relatively minor issues, and then move it soon after, it seems to make sense to keep it open in the next CF. - Andres
On Fri, Apr 7, 2017 at 3:30 PM, Andres Freund <andres@anarazel.de> wrote: > On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: >> Peter Geoghegan wrote: >> > > - can't move to next fest because it's waiting-on-author, which doesn't >> > > allow that. Doesn't strike me as a useful restriction. >> > >> > I agree that that CF app restriction makes little sense. >> >> What the restriction means is that if a patch is in waiting-on-author, >> the proper "close" action is to return-with-feedback. There is no point >> in moving the patch to the next commitfest if there is no further patch >> version. > > That's true if the patch has been in that state for a while, but if you > find some relatively minor issues, and then move it soon after, it seems > to make sense to keep it open in the next CF. In an ideal world, we wouldn't do that. Of course, we do not live in an ideal world... -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Fri, Apr 7, 2017 at 12:28 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote: > Peter Geoghegan wrote: >> On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund <andres@anarazel.de> wrote: >> > Write Amplification Reduction Method (WARM) >> > - fair number of people don't think it's ready for v10. > > Given the number of votes against putting this on pg10, I am going to > back off from this patch now, with an eye towards putting it in pg11 as > soon as the tree opens. Either I or Pavan are going to post another > version of this patch series, within the next couple of weeks, so that > others can base their testing, review and suggestions. My offer to work with you on amcheck verification of WARM invariants remains open. If nothing else, structuring things so that verification is possible may clarify your design. Formalizing the preconditions, postconditions, and legal states for on-disk structures might just be a useful exercise, even if verification never actually finds a problem. I anticipate that amcheck verification will become my main focus for Postgres 11, in any case. -- Peter Geoghegan VMware vCenter Server https://www.vmware.com/
Peter Geoghegan wrote: > My offer to work with you on amcheck verification of WARM invariants > remains open. If nothing else, structuring things so that verification > is possible may clarify your design. Formalizing the preconditions, > postconditions, and legal states for on-disk structures might just be > a useful exercise, even if verification never actually finds a > problem. Agreed. Thanks much. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Sat, Apr 8, 2017 at 3:37 AM, Andres Freund <andres@anarazel.de> wrote: > Hi, > > When I started writing this, there were the following reamining CF > items, minus bugfix ones which aren't bound by the code freeze. > > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > > Ready for Committer: > > Add GUCs for predicate lock promotion thresholds: > - claimed by Kevin, should be easy enough > > initdb configurable wal_segment_size > - parts have been committed > - significantly derailed by segment naming discussion > - possibly committable if we decide to skip the naming bit? But also a > bit late given that it touches some quite sensitive code. > > Create fdw_outerpath for foreign > - haven't really followed discussion > - only marked as ready-for-committer 2017-04-04 > > Vacuum: allow usage of more than 1GB of work mem > - hm, maybe? Will take a look. > > Unique Joins > - Tom's discussing things with David, not sure. > > Push down more UPDATEs/DELETEs in postgres_fdw > - claimed by Robert? > > postgres_fdw: support parameterized foreign joins > - think that depends on fdw_outerpath? > > > Waiting on Author: > > SQL statements statistics counter view (pg_stat_sql) > - the code doesn't look quite ready > - don't think we quite have design agreement, e.g. I don't like where it's > hooked into query execution > > Write Amplification Reduction Method (WARM) > - fair number of people don't think it's ready for v10. > - can't move to next fest because it's waiting-on-author, which doesn't > allow that. Doesn't strike me as a useful restriction. > > BRIN optimize memory allocation > - I think Alvaro has indicated that he wants to take care of that? > > Indexes with Included Columns (was Covering + unique indexes) > - Don't think concerns about #columns on truncated tuples have been > addressed. Should imo be returned-with-feedback. > > > Needs-Review: > > Better estimate merging for duplicate vars in clausesel.c > - has been submitted pretty late (2017-02-24) and discussion is ongoing > - I'm inclined to punt on this one to the next release, previous > proposal along that line got some pushback > > new plpgsql extra_checks > - Winner of the "most opaque CF title" award > - hasn't received a whole lot of review > - don't think we're even close to having design agreement > > Generic type subscripting > - still some review back and forth > - probably should be punted > > > Any comments? > HI, Could you consider the item 2PC on FDW as well? It is marked as "Move to Next CF" early yesterday but I'm not sure that reason.. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On Sat, Apr 8, 2017 at 1:09 PM, Masahiko Sawada <sawada.mshk@gmail.com> wrote: > On Sat, Apr 8, 2017 at 3:37 AM, Andres Freund <andres@anarazel.de> wrote: >> Hi, >> >> When I started writing this, there were the following reamining CF >> items, minus bugfix ones which aren't bound by the code freeze. >> >> I think it makes sense to go through those and see whether it's >> realistic to commit any of them. >> >> Ready for Committer: >> >> Add GUCs for predicate lock promotion thresholds: >> - claimed by Kevin, should be easy enough >> >> initdb configurable wal_segment_size >> - parts have been committed >> - significantly derailed by segment naming discussion >> - possibly committable if we decide to skip the naming bit? But also a >> bit late given that it touches some quite sensitive code. >> >> Create fdw_outerpath for foreign >> - haven't really followed discussion >> - only marked as ready-for-committer 2017-04-04 >> >> Vacuum: allow usage of more than 1GB of work mem >> - hm, maybe? Will take a look. >> >> Unique Joins >> - Tom's discussing things with David, not sure. >> >> Push down more UPDATEs/DELETEs in postgres_fdw >> - claimed by Robert? >> >> postgres_fdw: support parameterized foreign joins >> - think that depends on fdw_outerpath? >> >> >> Waiting on Author: >> >> SQL statements statistics counter view (pg_stat_sql) >> - the code doesn't look quite ready >> - don't think we quite have design agreement, e.g. I don't like where it's >> hooked into query execution >> >> Write Amplification Reduction Method (WARM) >> - fair number of people don't think it's ready for v10. >> - can't move to next fest because it's waiting-on-author, which doesn't >> allow that. Doesn't strike me as a useful restriction. >> >> BRIN optimize memory allocation >> - I think Alvaro has indicated that he wants to take care of that? >> >> Indexes with Included Columns (was Covering + unique indexes) >> - Don't think concerns about #columns on truncated tuples have been >> addressed. Should imo be returned-with-feedback. >> >> >> Needs-Review: >> >> Better estimate merging for duplicate vars in clausesel.c >> - has been submitted pretty late (2017-02-24) and discussion is ongoing >> - I'm inclined to punt on this one to the next release, previous >> proposal along that line got some pushback >> >> new plpgsql extra_checks >> - Winner of the "most opaque CF title" award >> - hasn't received a whole lot of review >> - don't think we're even close to having design agreement >> >> Generic type subscripting >> - still some review back and forth >> - probably should be punted >> >> >> Any comments? >> > > HI, > > Could you consider the item 2PC on FDW as well? It is marked as "Move > to Next CF" early yesterday but I'm not sure that reason.. > Oops, I meant "Transactions involving multiple postgres foreign servers"[1]. [1] https://commitfest.postgresql.org/13/928/ Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Hi, On 2017-04-08 13:09:13 +0900, Masahiko Sawada wrote: > Could you consider the item 2PC on FDW as well? It is marked as "Move > to Next CF" early yesterday but I'm not sure that reason.. I've not moved it, but given that it was moved just before the feature freeze, it doesn't seem wrong to me - it's too large a patchset to just have been merged yesterday afternoon. I also suspect that some more design discussion will be needed next cycle... Greetings, Andres Freund
On Sat, Apr 8, 2017 at 11:24 PM, Andres Freund <andres@anarazel.de> wrote: > Hi, > > On 2017-04-08 13:09:13 +0900, Masahiko Sawada wrote: >> Could you consider the item 2PC on FDW as well? It is marked as "Move >> to Next CF" early yesterday but I'm not sure that reason.. > > I've not moved it, but given that it was moved just before the feature > freeze, it doesn't seem wrong to me - it's too large a patchset to just > have been merged yesterday afternoon. I also suspect that some more > design discussion will be needed next cycle... > Thank you for comment. Yeah, I agree to push this patch out to next cycle. It needs more design discussion and feedback. Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center