== PostgreSQL Weekly News - October 7, 2018 == - Mailing list pgsql-announce
| From | David Fetter |
|---|---|
| Subject | == PostgreSQL Weekly News - October 7, 2018 == |
| Date | |
| Msg-id | 20181007230816.GA10656@fetter.org Whole thread Raw |
| List | pgsql-announce |
== PostgreSQL Weekly News - October 7, 2018 == PGConf APAC 2019 will be held in Singapore March 19-21, 2019. The CfP is open at http://2019.pgconfapac.org/cfp through November 16, 2018. http://2019.pgconfapac.org/ == PostgreSQL Product News == oracle_fdw 2.1.0 released. https://laurenz.github.io/oracle_fdw pgAdmin4 3.4, a web- and native GUI control center for PostgreSQL, released. https://www.pgadmin.org/docs/pgadmin4/dev/release_notes_3_4.html == PostgreSQL Jobs for October == http://archives.postgresql.org/pgsql-jobs/2018-10/ == PostgreSQL Local == PostgresConf South Africa 2018 will take place in Johannesburg on October 9, 2018 https://postgresconf.org/conferences/SouthAfrica2018 PostgreSQL Conference Europe 2018 will be held on October 23-26, 2018 at the Lisbon Marriott Hotel in Lisbon, Portugal. https://2018.pgconf.eu/ 2Q PGConf will be on December 4-5, 2018 in Chicago, IL. http://www.2qpgconf.com/ PGConf.ASIA 2018 will take place on December 10-12, 2018 in Akihabara, Tokyo, Japan. http://www.pgconf.asia/EN/2018/ pgDay Paris 2019 will be held in Paris, France on March 12, 2019 at 199bis rue Saint-Martin. The CfP is open until November 30, 2018. http://2019.pgday.paris/callforpapers/ PGDay.IT 2019 will take place May 16th and May 17th in Bologna, Italy. The CfP is open at https://2019.pgday.it/en/blog/cfp and the Call for Workshops is at https://2019.pgday.it/en/blog/cfw until January 15, 2019. https://2019.pgday.it/en/ == PostgreSQL in the News == Planet PostgreSQL: http://planet.postgresql.org/ PostgreSQL Weekly News is brought to you this week by David Fetter Submit news and announcements by Sunday at 3:00pm PST8PDT to david@fetter.org. == Applied Patches == Peter Eisentraut pushed: - doc: Clarify CREATE TABLESPACE documentation. Be more specific about when and how to create the directory and what permissions it should have. Discussion: https://www.postgresql.org/message-id/flat/5ca60e1a-26f9-89fd-e912-021dd2b8afe2%40gmail.com https://git.postgresql.org/pg/commitdiff/a6949ca34d3aca018870815cf6cb690024aeea04 - Change PROCEDURE to FUNCTION in CREATE EVENT TRIGGER syntax. This was claimed to have been done in 0a63f996e018ac508c858e87fa39cc254a5db49f, but that actually only changed the documentation and not the grammar. (That commit did fully change it for CREATE TRIGGER.) https://git.postgresql.org/pg/commitdiff/cf3dfea45b13662b3c23d7c481a9f77d67e77c45 - Remove redundant allocation. Author: Nikita Glukhov <n.gluhov@postgrespro.ru> https://git.postgresql.org/pg/commitdiff/b5f03dc7657e4cc62b975f3bc07b217916324a54 Tom Lane pushed: - Fix ALTER COLUMN TYPE to not open a relation without any lock. If the column being modified is referenced by a foreign key constraint of another table, ALTER TABLE would open the other table (to re-parse the constraint's definition) without having first obtained a lock on it. This was evidently intentional, but that doesn't mean it's really safe. It's especially not safe in 9.3, which pre-dates use of MVCC scans for catalog reads, but even in current releases it doesn't seem like a good idea. We know we'll need AccessExclusiveLock shortly to drop the obsoleted constraint, so just get that a little sooner to close the hole. Per testing with a patch that complains if we open a relation without holding any lock on it. I don't plan to back-patch that patch, but we should close the holes it identifies in all supported branches. Discussion: https://postgr.es/m/2038.1538335244@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/e27453bd839f3d0f55f94afa554be7066a841ab3 - Fix tuple_data_split() to not open a relation without any lock. contrib/pageinspect's tuple_data_split() function thought it could get away with opening the referenced relation with NoLock. In practice there's no guarantee that the current session holds any lock on that rel (even if we just read a page from it), so that this is unsafe. Switch to using AccessShareLock. Also, postpone closing the relation, so that we needn't copy its tupdesc. Also, fix unsafe use of att_isnull() for attributes past the end of the tuple. Per testing with a patch that complains if we open a relation without holding any lock on it. I don't plan to back-patch that patch, but we should close the holes it identifies in all supported branches. Discussion: https://postgr.es/m/2038.1538335244@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/b66827ca7c5a4c9e31b1a1eced677f8677efc0cf - Add assertions that we hold some relevant lock during relation open. Opening a relation with no lock at all is unsafe; there's no guarantee that we'll see a consistent state of the relevant catalog entries. While use of MVCC scans to read the catalogs partially addresses that complaint, it's still possible to switch to a new catalog snapshot partway through loading the relcache entry. Moreover, whether or not you trust the reasoning behind sometimes using less than AccessExclusiveLock for ALTER TABLE, that reasoning is certainly not valid if concurrent users of the table don't hold a lock corresponding to the operation they want to perform. Hence, add some assertion-build-only checks that require any caller of relation_open(x, NoLock) to hold at least AccessShareLock. This isn't a full solution, since we can't verify that the lock level is semantically appropriate for the action --- but it's definitely of some use, because it's already caught two bugs. We can also assert that callers of addRangeTableEntryForRelation() hold at least the lock level specified for the new RTE. Amit Langote and Tom Lane Discussion: https://postgr.es/m/16565.1538327894@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/b04aeb0a053e7cf7faad89f7d47844d8ba0dc839 - Fix corner-case failures in has_foo_privilege() family of functions. The variants of these functions that take numeric inputs (OIDs or column numbers) are supposed to return NULL rather than failing on bad input; this rule reduces problems with snapshot skew when queries apply the functions to all rows of a catalog. has_column_privilege() had careless handling of the case where the table OID didn't exist. You might get something like this: select has_column_privilege(9999,'nosuchcol','select'); ERROR: column "nosuchcol" of relation "(null)" does not exist or you might get a crash, depending on the platform's printf's response to a null string pointer. In addition, while applying the column-number variant to a dropped column returned NULL as desired, applying the column-name variant did not: select has_column_privilege('mytable','........pg.dropped.2........','select'); ERROR: column "........pg.dropped.2........" of relation "mytable" does not exist It seems better to make this case return NULL as well. Also, the OID-accepting variants of has_foreign_data_wrapper_privilege, has_server_privilege, and has_tablespace_privilege didn't follow the principle of returning NULL for nonexistent OIDs. Superusers got TRUE, everybody else got an error. Per investigation of Jaime Casanova's report of a new crash in HEAD. These behaviors have been like this for a long time, so back-patch to all supported branches. Patch by me; thanks to Stephen Frost for discussion and review Discussion: https://postgr.es/m/CAJGNTeP=-6Gyqq5TN9OvYEydi7Fv1oGyYj650LGTnW44oAzYCg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/3d0f68dd30612db8d8cf6529455a369ad121c521 - Set snprintf.c's maximum number of NL arguments to be 31. Previously, we used the platform's NL_ARGMAX if any, otherwise 16. The trouble with this is that the platform value is hugely variable, ranging from the POSIX-minimum 9 to as much as 64K on recent FreeBSD. Values of more than a dozen or two have no practical use and slow down the initialization of the argtypes array. Worse, they cause snprintf.c to consume far more stack space than was the design intention, possibly resulting in stack-overflow crashes. Standardize on 31, which is comfortably more than we need (it looks like no existing translatable message has more than about 10 parameters). I chose that, not 32, to make the array sizes powers of 2, for some possible small gain in speed of the memset. The lack of reported crashes suggests that the set of platforms we use snprintf.c on (in released branches) may have no overlap with the set where NL_ARGMAX has unreasonably large values. But that's not entirely clear, so back-patch to all supported branches. Per report from Mateusz Guzik (via Thomas Munro). Discussion: https://postgr.es/m/CAEepm=3VF=PUp2f8gU8fgZB22yPE_KBS0+e1AHAtQ=09schTHg@mail.gmail.com https://git.postgresql.org/pg/commitdiff/625b38ea0e98cb596b393c70e5eaba67c6f4279e - Change rewriter/planner/executor/plancache to depend on RTE rellockmode. Instead of recomputing the required lock levels in all these places, just use what commit fdba460a2 made the parser store in the RTE fields. This already simplifies the code measurably in these places, and follow-on changes will remove a bunch of no-longer-needed infrastructure. In a few cases, this change causes us to acquire a higher lock level than we did before. This is OK primarily because said higher lock level should've been acquired already at query parse time; thus, we're saving a useless extra trip through the shared lock manager to acquire a lesser lock alongside the original lock. The only known exception to this is that re-execution of a previously planned SELECT FOR UPDATE/SHARE query, for a table that uses ROW_MARK_REFERENCE or ROW_MARK_COPY methods, might have gotten only AccessShareLock before. Now it will get RowShareLock like the first execution did, which seems fine. While there's more to do, push it in this state anyway, to let the buildfarm help verify that nothing bad happened. Amit Langote, reviewed by David Rowley and Jesper Pedersen, and whacked around a bit more by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/6e35939febf83069430fedda6f89ab1fbe0f9e10 - Make assorted performance improvements in snprintf.c. In combination, these changes make our version of snprintf as fast or faster than most platforms' native snprintf, except for cases involving floating-point conversion (which we still delegate to the native sprintf). The speed penalty for a float conversion is down to around 10% though, much better than before. Notable changes: * Rather than always parsing the format twice to see if it contains instances of %n$, do the extra scan only if we actually find a $. This obviously wins for non-localized formats, and even when there is use of %n$, we can avoid scanning text before the first % twice. * Use strchrnul() if available to find the next %, and emit the literal text between % escapes as strings rather than char-by-char. * Create a bespoke function (dopr_outchmulti) for the common case of emitting N copies of the same character, in place of writing loops around dopr_outch. * Simplify construction of the format string for invocations of sprintf for floats. * Const-ify some internal functions, and avoid unnecessary use of pass-by-reference arguments. Patch by me, reviewed by Andres Freund Discussion: https://postgr.es/m/11787.1534530779@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/abd9ca377d669a6e0560e854d7e987438d0e612e - Provide fast path in snprintf.c for conversion specs that are just "%s". This case occurs often enough (around 45% of conversion specs executed in our regression tests are just "%s") that it's worth an extra test per conversion spec to allow skipping all the logic associated with field widths and padding when it happens. Discussion: https://postgr.es/m/26193.1538582367@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/6d842be6c11887930a0f4128fd6aa4de427cfd2a - Rationalize snprintf.c's handling of "ll" formats. Although all known platforms define "long long" as 64 bits, it still feels a bit shaky to be using "va_arg(args, int64)" to pull out an argument that the caller thought was declared "long long". The reason it was coded like this, way back in commit 3311c7669, was to work around the possibility that the compiler had no type named "long long" --- and, at the time, that it maybe didn't have 64-bit ints at all. Now that we're requiring compilers to support C99, those concerns are moot. Let's make the code clearer and more bulletproof by writing "long long" where we mean "long long". This does introduce a hazard that we'd inefficiently use 128-bit arithmetic to convert plain old integers. The way to tackle that would be to provide two versions of fmtint(), one for "long long" and one for narrower types. Since, as of today, no platforms require that, we won't bother with the extra code for now. Discussion: https://postgr.es/m/1680.1538587115@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/595a0eab7f425e3484639fae1f7e221fe9c2651a - Change executor to just Assert that table locks were already obtained. Instead of locking tables during executor startup, just Assert that suitable locks were obtained already during the parse/plan pipeline (or re-obtained by the plan cache). This must be so, else we have a hazard that concurrent DDL has invalidated the plan. This is pretty inefficient as well as undercommented, but it's all going to go away shortly, so I didn't try hard. This commit is just another attempt to use the buildfarm to see if we've missed anything in the plan to simplify the executor's table management. Note that the change needed here in relation_open() exposes that parallel workers now really are accessing tables without holding any lock of their own, whereas they were not doing that before this commit. This does not give me a warm fuzzy feeling about that aspect of parallel query; it does not seem like a good design, and we now know that it's had exactly no actual testing. I think that we should modify parallel query so that that change can be reverted. Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/9a3cebeaa7fdc1b0485475eb18121eb06968dc5d - Centralize executor's opening/closing of Relations for rangetable entries. Create an array estate->es_relations[] paralleling the es_range_table, and store references to Relations (relcache entries) there, so that any given RT entry is opened and closed just once per executor run. Scan nodes typically still call ExecOpenScanRelation, but ExecCloseScanRelation is no more; relation closing is now done centrally in ExecEndPlan. This is slightly more complex than one would expect because of the interactions with relcache references held in ResultRelInfo nodes. The general convention is now that ResultRelInfo->ri_RelationDesc does not represent a separate relcache reference and so does not need to be explicitly closed; but there is an exception for ResultRelInfos in the es_trig_target_relations list, which are manufactured by ExecGetTriggerResultRel and have to be cleaned up by ExecCleanUpTriggerState. (That much was true all along, but these ResultRelInfos are now more different from others than they used to be.) To allow the partition pruning logic to make use of es_relations[] rather than having its own relcache references, adjust PartitionedRelPruneInfo to store an RT index rather than a relation OID. Amit Langote, reviewed by David Rowley and Jesper Pedersen, some mods by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/9ddef36278a9f676c07d0b4d9f33fa22e48ce3b5 - In the executor, use an array of pointers to access the rangetable. Instead of doing a lot of list_nth() accesses to es_range_table, create a flattened pointer array during executor startup and index into that to get at individual RangeTblEntrys. This eliminates one source of O(N^2) behavior with lots of partitions. (I'm not exactly convinced that it's the most important source, but it's an easy one to fix.) Amit Langote and David Rowley Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/d73f4c74dd34b19c19839f7ae09fb96442728509 - Ensure that PLPGSQL_DTYPE_ROW variables have valid refname fields. Without this, the syntax-tree-dumping functions in pl_funcs.c crash, and there are other places that might be at risk too. Per report from Pavel Stehule. Looks like I broke this in commit f9263006d, so back-patch to v11. Discussion: https://postgr.es/m/CAFj8pRA+3f5n4642q2g8BXCKjbTd7yU9JMYAgDyHgozk6cQ-VA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/113a659914daa8e8c9ced55ca82bfbd5bdcedbd4 - Allow btree comparison functions to return INT_MIN. Historically we forbade datatype-specific comparison functions from returning INT_MIN, so that it would be safe to invert the sort order just by negating the comparison result. However, this was never really safe for comparison functions that directly return the result of memcmp(), strcmp(), etc, as POSIX doesn't place any such restriction on those library functions. Buildfarm results show that at least on recent Linux on s390x, memcmp() actually does return INT_MIN sometimes, causing sort failures. The agreed-on answer is to remove this restriction and fix relevant call sites to not make such an assumption; code such as "res = -res" should be replaced by "INVERT_COMPARE_RESULT(res)". The same is needed in a few places that just directly negated the result of memcmp or strcmp. To help find places having this problem, I've also added a compile option to nbtcompare.c that causes some of the commonly used comparators to return INT_MIN/INT_MAX instead of their usual -1/+1. It'd likely be a good idea to have at least one buildfarm member running with "-DSTRESS_SORT_INT_MIN". That's far from a complete test of course, but it should help to prevent fresh introductions of such bugs. This is a longstanding portability hazard, so back-patch to all supported branches. Discussion: https://postgr.es/m/20180928185215.ffoq2xrq5d3pafna@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/c87cb5f7a67962f4bdfb897139fff91b7d8e0d90 - Propagate xactStartTimestamp and stmtStartTimestamp to parallel workers. Previously, a worker process would establish values for these based on its own start time. In v10 and up, this can trivially be shown to cause misbehavior of transaction_timestamp(), timestamp_in(), and related functions which are (perhaps unwisely?) marked parallel-safe. It seems likely that other behaviors might diverge from what happens in the parent as well. It's not as trivial to demonstrate problems in 9.6 or 9.5, but I'm sure it's still possible, so back-patch to all branches containing parallel worker infrastructure. In HEAD only, mark now() and statement_timestamp() as parallel-safe (other affected functions already were). While in theory we could still squeeze that change into v11, it doesn't seem important enough to force a last-minute catversion bump. Konstantin Knizhnik, whacked around a bit by me Discussion: https://postgr.es/m/6406dbd2-5d37-4cb6-6eb2-9c44172c7e7c@postgrespro.ru https://git.postgresql.org/pg/commitdiff/07ee62ce9e507c14632c0517aeeae4e60b0d1997 - Don't use is_infinite() where isinf() will do. Places that aren't testing for sign should not use the more expensive function; it's just wasteful, not to mention being a cognitive load for readers who may know what isinf() is but not is_infinite(). As things stand, we actually don't need is_infinite() anyplace except float4out/float8out, which means it could potentially go away altogether after the changes I proposed in <13178.1538794717@sss.pgh.pa.us>. https://git.postgresql.org/pg/commitdiff/0209f0285d9b1c60bf74cc9f5f0133d7bdd887c3 - Remove more redundant relation locking during executor startup. We already have appropriate locks on every relation listed in the query's rangetable before we reach the executor. Take the next step in exploiting that knowledge by removing code that worries about taking locks on non-leaf result relations in a partitioned table. In particular, get rid of ExecLockNonLeafAppendTables and a stanza in InitPlan that asserts we already have locks on certain such tables. In passing, clean up some now-obsolete comments in InitPlan. Amit Langote, reviewed by David Rowley and Jesper Pedersen, and whacked around a bit more by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/f2343653f5b2aecfc759f36dbb3fd2a61f36853e - Restore sane locking behavior during parallel query. Commit 9a3cebeaa changed things so that parallel workers didn't obtain any lock of their own on tables they access. That was clearly a bad idea, but I'd mistakenly supposed that it was the intended end result of the series of patches for simplifying the executor's lock management. Undo that change in relation_open(), and adjust ExecOpenScanRelation() so that it gets the correct lock if inside a parallel worker. In passing, clean up some more obsolete comments about when locks are acquired. Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/29ef2b310da9892fda075ff9ee12da7f92d5da6e - Remove some unnecessary fields from Plan trees. In the wake of commit f2343653f, we no longer need some fields that were used before to control executor lock acquisitions: * PlannedStmt.nonleafResultRelations can go away entirely. * partitioned_rels can go away from Append, MergeAppend, and ModifyTable. However, ModifyTable still needs to know the RT index of the partition root table if any, which was formerly kept in the first entry of that list. Add a new field "rootRelation" to remember that. rootRelation is partly redundant with nominalRelation, in that if it's set it will have the same value as nominalRelation. However, the latter field has a different purpose so it seems best to keep them distinct. Amit Langote, reviewed by David Rowley and Jesper Pedersen, and whacked around a bit more by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp https://git.postgresql.org/pg/commitdiff/52ed730d511b7b1147f2851a7295ef1fb5273776 Michaël Paquier pushed: - Refactor relation opening for VACUUM and ANALYZE. VACUUM and ANALYZE share similar logic when it comes to opening a relation to work on in terms of how the relation is opened, in which order locks are tried and how logs should be generated when something does not work as expected. This commit refactors things so as both use the same code path to handle the way a relation is opened, so as the integration of new options becomes easier. Author: Michael Paquier Reviewed-by: Nathan Bossart Discussion: https://postgr.es/m/20180927075152.GT1659@paquier.xyz https://git.postgresql.org/pg/commitdiff/e3a25ab9ea56ac540dc683cdf6f6a4b923bd22be - Fix documentation of pgrowlocks using "lock_type" instead of "modes". The example used in the documentation is outdated as well. This is an oversight from 0ac5ad5, which bumped up pgrowlocks but forgot some bits of the documentation. Reported-by: Chris Wilson Discussion: https://postgr.es/m/153838692816.2950.12001142346234155699@wrigleys.postgresql.org Backpatch-through: 9.3 https://git.postgresql.org/pg/commitdiff/80810ca629c159191387f677b368a0551bbf2351 - Add option SKIP_LOCKED to VACUUM and ANALYZE When specified, this option allows VACUUM to skip the work on a relation if there is a conflicting lock on it when trying to open it at the beginning of its processing. Similarly to autovacuum, this comes with a couple of limitations while the relation is processed which can cause the process to still block: - when opening the relation indexes. - when acquiring row samples for table inheritance trees, partition trees. or certain types of foreign tables, and that a lock is taken on some leaves of such trees. Author: Nathan Bossart Reviewed-by: Michael Paquier, Andres Freund, Masahiko Sawada Discussion: https://postgr.es/m/9EF7EBE4-720D-4CF1-9D0E-4403D7E92990@amazon.com Discussion: https://postgr.es/m/20171201160907.27110.74730@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/803b1301e8c9aac478abeec62824a5d09664ffff - Refactor user-facing SQL functions signalling backends. This moves the system administration functions for signalling backends from backend/utils/adt/misc.c into a separate file dedicated to backend signalling. No new functionality is introduced in this commit. Author: Daniel Gustafsson Reviewed-by: Michael Paquier, Álvaro Herrera Discussion: https://postgr.es/m/C2C7C3EC-CC5F-44B6-9C78-637C88BD7D14@yesql.se https://git.postgresql.org/pg/commitdiff/09921f397b84bdae8406f3dc6f473ead730a6dd3 - Add pg_ls_tmpdir function. This lists the contents of a temporary directory associated to a given tablespace, useful to get information about on-disk consumption caused by temporary files used by a session query. By default, pg_default is scanned, and a tablespace can be specified as argument. This function is intended to be used by monitoring tools, and, unlike pg_ls_dir(), access to them can be granted to non-superusers so that those monitoring tools can observe the principle of least privilege. Access is also given by default to members of pg_monitor. Author: Nathan Bossart Reviewed-by: Laurenz Albe Discussion: https://postgr.es/m/92F458A2-6459-44B8-A7F2-2ADD3225046A@amazon.com https://git.postgresql.org/pg/commitdiff/9cd92d1a33699f86aa53d44ab04cc3eb50c18d11 - Assign constraint name when cloning FK definition for partitions. This is for example used when attaching a partition to a partitioned table which includes foreign keys, and in this case the constraint name has been missing in the data cloned. This could lead to hard crashes, as when validating the foreign key constraint, the constraint name is always expected. Particularly, when using log_min_messages >= DEBUG1, a log message would be generated with this unassigned constraint name, leading to an assertion failure on HEAD. While on it, rename a variable in ATExecAttachPartition which was declared twice with the same name. Author: Michael Paquier Reviewed-by: Álvaro Herrera Discussion: https://postgr.es/m/20181005042236.GG1629@paquier.xyz Backpatch-through: 11 https://git.postgresql.org/pg/commitdiff/38921d1416c62bb743f6cc5439d0462efefdb286 - Add regression test for ATTACH PARTITION. This test case uses a SQL function as partitioning operator, whose evaluation results in the table's relcache being rebuilt partway through the execution of an ATTACH PARTITION command. It is extracted from 39808e8, which fixed a bug where this test case failed on master and REL_11_STABLE, but passed on REL_10_STABLE. The partitioning code has changed a lot during v11 development, so this makes sure that any patch applied to REL_10_STABLE fixing a partition-related bug does not break it again. Author: Amit Langote Reviewed-by: Michaël Paquier, Álvaro Herrera Discussion: https://postgr.es/m/CAKcux6=nTz9KSfTr_6Z2mpzLJ_09JN-rK6=dWic6gGyTSWueyQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/afe9b9e68afb93b6831a939a7a18973ee5286d68 Amit Kapila pushed: - Test passing expanded-value representations to workers. Currently, we don't have an explicit test to pass expanded-value representations to workers, so we don't know whether it works on all kind of platforms. We suspect that the current code won't work on alignment-sensitive hardware. This commit will test that aspect and can lead to failure on some of the buildfarm machines which we will fix in the later commit. Author: Tom Lane and Amit Kapila Discussion: https://postgr.es/m/11629.1536550032@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/0fd6a8a7d0ce6dcf0edb8330fe7e6d6fde8f40db - MAXALIGN the target address where we store flattened value. The API (EOH_flatten_into) that flattens the expanded value representation expects the target address to be maxaligned. All it's usage adhere to that principle except when serializing datums for parallel query. Fix that usage. Diagnosed-by: Tom Lane Author: Tom Lane and Amit Kapila Backpatch-through: 9.6 Discussion: https://postgr.es/m/11629.1536550032@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/9bc9f72b28fe4d2c22244f3443af8f1b98b56474 Andres Freund pushed: - Use slots more widely in tuple mapping code and make naming more consistent. It's inefficient to use a single slot for mapping between tuple descriptors for multiple tuples, as previously done when using ConvertPartitionTupleSlot(), as that means the slot's tuple descriptors change for every tuple. Previously we also, via ConvertPartitionTupleSlot(), built new tuples after the mapping even in cases where we, immediately afterwards, access individual columns again. Refactor the code so one slot, on demand, is used for each partition. That avoids having to change the descriptor (and allows to use the more efficient "fixed" tuple slots). Then use slot->slot mapping, to avoid unnecessarily forming a tuple. As the naming between the tuple and slot mapping functions wasn't consistent, rename them to execute_attr_map_{tuple,slot}. It's likely that we'll also rename convert_tuples_by_* to denote that these functions "only" build a map, but that's left for later. Author: Amit Khandekar and Amit Langote, editorialized by me Reviewed-By: Amit Langote, Amit Khandekar, Andres Freund Discussion: https://postgr.es/m/CAJ3gD9fR0wRNeAE8VqffNTyONS_UfFPRpqxhnD9Q42vZB+Jvpg@mail.gmail.com https://postgr.es/m/e4f9d743-cd4b-efb0-7574-da21d86a7f36%40lab.ntt.co.jp Backpatch: - https://git.postgresql.org/pg/commitdiff/cc2905e963e950d01cd2cb6c860638ce9506c63d - Fix issues around EXPLAIN with JIT. I (Andres) was more than a bit hasty in committing 33001fd7a7072d48327 after last minute changes, leading to a number of problems (jit output was only shown for JIT in parallel workers, and just EXPLAIN without ANALYZE didn't work). Lukas luckily found these issues quickly. Instead of combining instrumentation in in standard_ExecutorEnd(), do so on demand in the new ExplainPrintJITSummary(). Also update a documentation example of the JIT output, changed in 52050ad8ebec8d831. Author: Lukas Fittl, with minor changes by me Discussion: https://postgr.es/m/CAP53PkxmgJht69pabxBXJBM+0oc6kf3KHMborLP7H2ouJ0CCtQ@mail.gmail.com Backpatch: 11, where JIT compilation was introduced https://git.postgresql.org/pg/commitdiff/c03c1449c0925637d382bd16197796e6c5cab31d - Ensure that snprintf.c's fmtint() doesn't overflow when printing INT64_MIN. This isn't actually a live bug, as the output happens to be the same. But it upsets tools like UBSan, which makes it worthwhile to fix. As it's an issue without practical consequences, don't backpatch. Author: Andres Freund Discussion: https://postgr.es/m/20180928001121.hhx5n6dsygqxr5wu@alap3.anarazel.de https://git.postgresql.org/pg/commitdiff/4868e4468590bc32f9c3afed4ec795d6a7732c9d - Replace uint64 use introduced in 4868e446859 in light of 595a0eab7f42. Reported-By: Tom Lane Discussion: https://postgr.es/m/527.1538598263@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/d173652797cf0cf456ac2c38eca898159a7c19fc Andrew Dunstan pushed: - Don't build static libraries on Cygwin. Cygwin has been building and linking against static libraries. Although a bug this has been relatively harmless until now, when this has caused errors due to changes in the way we build certain libraries. So this patch makes things work the way we always intended, namely that we would link against the dynamic libraries (cygpq.dll etc.) and just not build the static libraries. The downstream packagers have been doing this for some time, so this just aligns with their practice. Extracted from a patch by Marco Atzeri, with a suggestion from Tom Lane. Discussion: https://postgr.es/m/1056.1538235347@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/a33245a8537f8ecbd2cd50c67b2b234f65e8fe14 Álvaro Herrera pushed: - Fix duplicate primary keys in partitions. When using the CREATE TABLE .. PARTITION OF syntax, it's possible to cause a partition to get two primary keys if the parent already has one. Tighten the check to disallow that. Reported-by: Rajkumar Raghuwanshi Author: Amul Sul Discussion: https://postgr.es/m/CAKcux6=OnSV3-qd8Gb6W=KPPwcCz6Fe_O_MQYjTa24__Xn8XxA@mail.gmail.com https://git.postgresql.org/pg/commitdiff/fb9e93a2c5cf99e59a7e9362afd462f29e71bc1e - Fix event triggers for partitioned tables. Index DDL cascading on partitioned tables introduced a way for ALTER TABLE to be called reentrantly. This caused an an important deficiency in event trigger support to be exposed: on exiting the reentrant call, the alter table state object was clobbered, causing a crash when the outer alter table tries to finalize its processing. Fix the crash by creating a stack of event trigger state objects. There are still ways to cause things to misbehave (and probably other crashers) with more elaborate tricks, but at least it now doesn't crash in the obvious scenario. Backpatch to 9.5, where DDL deparsing of event triggers was introduced. Reported-by: Marco Slot Authors: Michaël Paquier, Álvaro Herrera Discussion: https://postgr.es/m/CANNhMLCpi+HQ7M36uPfGbJZEQLyTy7XvX=5EFkpR-b1bo0uJew@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ad08006ba03797fed431af87de6e66c6c0985b7a - Fix catalog insertion order for ATTACH PARTITION. Commit 2fbdf1b38bc changed the order in which we inserted catalog rows when creating partitions, so that we could remove an unsightly hack required for untimely relcache invalidations. However, that commit only changed the ordering for CREATE TABLE PARTITION OF, and left ALTER TABLE ATTACH PARTITION unchanged, so the latter can be affected when catalog invalidations occur, for instance when the partition key involves an SQL function. Reported-by: Rajkumar Raghuwanshi Author: Amit Langote Reviewed-by: Michaël Paquier Discussion: https://postgr.es/m/CAKcux6=nTz9KSfTr_6Z2mpzLJ_09JN-rK6=dWic6gGyTSWueyQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/39808e8868c8fac383b33aa103ab57539b0e2a69 Bruce Momjian pushed: - doc: update PG 11 release notes. Discussion: https://postgr.es/m/1f5b2e66-7ba8-98ec-c06a-aee9ff33f050@postgresql.org Author: Jonathan S. Katz Backpatch-through: 11 https://git.postgresql.org/pg/commitdiff/6eb612fea9d080f2ae77ecb7091e73dc9f298c97 Dean Rasheed pushed: - Improve the accuracy of floating point statistical aggregates. When computing statistical aggregates like variance, the common schoolbook algorithm which computes the sum of the squares of the values and subtracts the square of the mean can lead to a large loss of precision when using floating point arithmetic, because the difference between the two terms is often very small relative to the terms themselves. To avoid this, re-work these aggregates to use the Youngs-Cramer algorithm, which is a proven, numerically stable algorithm that directly aggregates the sum of the squares of the differences of the values from the mean in a single pass over the data. While at it, improve the test coverage to test the aggregate combine functions used during parallel aggregation. Per report and suggested algorithm from Erich Schubert. Patch by me, reviewed by Madeleine Thompson. Discussion: https://postgr.es/m/153313051300.1397.9594490737341194671@wrigleys.postgresql.org https://git.postgresql.org/pg/commitdiff/e954a727f0c8872bf5203186ad0f5312f6183746 == Pending Patches == Fabien COELHO sent in another revision of a patch to pgbench to add a pseudo-random permutation function. Petr Jelínek sent in a patch to fix the return value of subscription-relation mapping manipulation functions. Chris Travers sent in a patch to clarify the documentation of signal event handling. Peter Eisentraut sent in a patch to add settings to control the SSL/TLS protocol version. Masahiko Sawada sent in a patch to add a FREEZE_ONLY option to VACUUM. Amit Kapila and Tom Lane traded patches to fix datum serialization. Daniel Gustafsson and Michaël Paquier traded patches to add an optional message when terminating/cancelling a backend. Thomas Munro sent in another revision of a patch to enable parallel query with SERIALIZABLE isolation and enable the read-only SERIALIZABLE optimization for parallel query. Thomas Munro sent in another revision of a patch to keep file descriptors open to avoid losing errors and add an fsync request pipe for Windows. Fabien COELHO sent in another revision of a patch to pgbench to enable storing select results into variables. John Naylor sent in another revision of a patch to add a pg_language lookup and replace /ad hoc/ format for conversion functions in the BKI infrastructure. John Naylor sent in a patch to refactor the core scanner to make it easier to share with ECPG and psql. Robbie Harwood sent in another revision of a patch to refactor GSSAPI encryption and add support for it in libpq. Masahiko Sawada sent in another revision of a patch to support atomic commit among multiple foreign servers. Kyotaro HORIGUCHI sent in two more revisions of a patch to implement a shared-memory-based stats collector. Etsuro Fujita sent in two more revisions of a patch to fix some misbehavior in foreign modify. Julien Demoor sent in another revision of a patch to make NOTIFY/pg_notify() be able to allow collapse multiple identical NOTIFIES in a transaction into a single one. Andreas Karlsson sent in another revision of a patch to inline CTEs with a new option to preserve the previous must-materialize behavior. Tom Lane and Andrew Gierth traded patches to improve performance in snprintf.c. Takeshi Ideriha sent in another revision of a patch to add a shared_catcache_mem GUC. Masahiko Sawada and Michaël Paquier traded patches to ensure that anti-wraparound VACUUMs are always aggressive. Michaël Paquier and Amit Langote traded patches to add a pg_partition_tree() function to display information about partitions. Thomas Munro sent in four more revisions of a patch to relax transactional restrictions on ALTER TYPE ... ADD VALUE. Peter Geoghegan sent in another revision of a patch to make nbtree indexes have unique keys in tuples. Konstantin Knizhnik sent in another revision of a patch to implement libpq compression. Laurenz Albe sent in a patch to add a pg_promote() function for promoting standby servers. Amit Khandekar sent in another revision of a patch to implement the TupleTableSlot abstraction. Amit Langote sent in another revision of a patch to prune the PlanRowMark of relations that are pruned from a plan. Konstantin Knizhnik and David Rowley traded patches to skip redundant partition quals. Peter Eisentraut sent in another revision of a patch to lower the lock level for renaming indexes. Peter Eisentraut sent in another revision of a patch to advance the transaction timestamp in intra-procedure transactions. Peter Eisentraut sent in a patch to track procedure calls in pg_stat_user_functions. David Rowley sent in another revision of a patch to speed up INSERT and UPDATE on partitioned tables. Peter Eisentraut sent in a patch to test that event triggers work in functions and procedures and slightly correct the context check for event triggers. Michaël Paquier sent in two revisions of a patch to add TAP tests for pg_verify_checksums. John Naylor sent in two revisions of a patch to avoid creation of the free space map for small tables. Pavel Stěhule sent in two more revisions of a patch to implement schema variables. Laurenz Albe and Michaël Paquier traded patches to clarify the error message produced for some cases where foreign keys are being created on partitioned tables. Andrew Dunstan sent in a patch to support msys2.
pgsql-announce by date: