Correctifs appliqués

Peter Eisentraut pushed:

Tom Lane pushed:

  • Skip setting up shared instrumentation for Hash node if not needed. We don't need to set up the shared space for hash join instrumentation data if instrumentation hasn't been requested. Let's follow the example of the similar Sort node code and save a few cycles by skipping that when we can. This reverts commit d59ff4ab3 and instead allows us to use the safer choice of passing noError = false to shm_toc_lookup in ExecHashInitializeWorker, since if we reach that call there should be a TOC entry to be found. Thomas Munro Discussion:
  • Ensure that all temp files made during pg_upgrade are non-world-readable. pg_upgrade has always attempted to ensure that the transient dump files it creates are inaccessible except to the owner. However, refactoring in commit 76a7650c4 broke that for the file containing "pg_dumpall -g" output; since then, that file was protected according to the process's default umask. Since that file may contain role passwords (hopefully encrypted, but passwords nonetheless), this is a particularly unfortunate oversight. Prudent users of pg_upgrade on multiuser systems would probably run it under a umask tight enough that the issue is moot, but perhaps some users are depending only on pg_upgrade's umask changes to protect their data. To fix this in a future-proof way, let's just tighten the umask at process start. There are no files pg_upgrade needs to write at a weaker security level; and if there were, transiently relaxing the umask around where they're created would be a safer approach. Report and patch by Tom Lane; the idea for the fix is due to Noah Misch. Back-patch to all supported branches. Security: CVE-2018-1053
  • Doc: move info for btree opclass implementors into main documentation. Up to now, useful info for writing a new btree opclass has been buried in the backend's nbtree/README file. Let's move it into the SGML docs, in preparation for extending it with info about "in_range" functions in the upcoming window RANGE patch. To do this, I chose to create a new chapter for btree indexes in Part VII (Internals), parallel to the chapters that exist for the newer index AMs. This is a pretty short chapter as-is. At some point somebody might care to flesh it out with more detail about btree internals, but that is beyond the scope of my ambition for today. Discussion:
  • Support all SQL:2011 options for window frame clauses. This patch adds the ability to use "RANGE offset PRECEDING/FOLLOWING" frame boundaries in window functions. We'd punted on that back in the original patch to add window functions, because it was not clear how to do it in a reasonably data-type-extensible fashion. That problem is resolved here by adding the ability for btree operator classes to provide an "in_range" support function that defines how to add or subtract the RANGE offset value. Factoring it this way also allows the operator class to avoid overflow problems near the ends of the datatype's range, if it wishes to expend effort on that. (In the committed patch, the integer opclasses handle that issue, but it did not seem worth the trouble to avoid overflow failures for datetime types.) The patch includes in_range support for the integer_ops opfamily (int2/int4/int8) as well as the standard datetime types. Support for other numeric types has been requested, but that seems like suitable material for a follow-on patch. In addition, the patch adds GROUPS mode which counts the offset in ORDER-BY peer groups rather than rows, and it adds the frame_exclusion options specified by SQL:2011. As far as I can see, we are now fully up to spec on window framing options. Existing behaviors remain unchanged, except that I changed the errcode for a couple of existing error reports to meet the SQL spec's expectation that negative "offset" values should be reported as SQLSTATE 22013. Internally and in relevant parts of the documentation, we now consistently use the terminology "offset PRECEDING/FOLLOWING" rather than "value PRECEDING/FOLLOWING", since the term "value" is confusingly vague. Oliver Ford, reviewed and whacked around some by me Discussion:
  • Fix RelationBuildPartitionKey's processing of partition key expressions. Failure to advance the list pointer while reading partition expressions from a list results in invoking an input function with inappropriate data, possibly leading to crashes or, with carefully crafted input, disclosure of arbitrary backend memory. Bug discovered independently by Álvaro Herrera and David Rowley. This patch is by Álvaro but owes something to David's proposed fix. Back-patch to v10 where the issue was introduced. Security: CVE-2018-1052
  • Last-minute updates for release notes. Security: CVE-2018-1052, CVE-2018-1053
  • Fix oversight in CALL argument handling, and do some minor cleanup. CALL statements cannot support sub-SELECTs in the arguments of the called procedure, since they just use ExecEvalExpr to evaluate such arguments. Teach transformSubLink() to reject the case, as it already does for other contexts in which subqueries are not supported. In passing, s/EXPR_KIND_CALL/EXPR_KIND_CALL_ARGUMENT/ to make that enum symbol line up more closely with the phrasing of the error messages it is associated with. And fix someone's weak grasp of English grammar in the preceding EXPR_KIND_PARTITION_EXPRESSION addition. Also update an incorrect comment in resolve_unique_index_expr (possibly it was correct when written, but nowadays transformExpr definitely does reject SRFs here). Per report from Pavel Stehule --- but this resolves only one of the bugs he mentions. Discussion:
  • Avoid premature free of pass-by-reference CALL arguments. Prematurely freeing the EState used to evaluate CALL arguments led, in some cases, to passing dangling pointers to the procedure. This was masked in trivial cases because the argument pointers would point to Const nodes in the original expression tree, and in some other cases because the result value would end up in the standalone ExprContext rather than in memory belonging to the EState --- but that wasn't exactly high quality programming either, because the standalone ExprContext was never explicitly freed, breaking assorted API contracts. In addition, using a separate EState for each argument was just silly. So let's use just one EState, and one ExprContext, and make the latter belong to the former rather than be standalone, and clean up the EState (and hence the ExprContext) post-call. While at it, improve the function's commentary a bit. Discussion:
  • Fix assorted errors in pg_dump's handling of extended statistics objects. pg_dump supposed that a stats object necessarily shares the same schema as its underlying table, and that it doesn't have a separate owner. These things may have been true during early development of the feature, but they are not true as of v10 release. Failure to track the object's schema separately turns out to have only limited consequences, because pg_get_statisticsobjdef() always schema- qualifies the target object name in the generated CREATE STATISTICS command (a decision out of step with the rest of ruleutils.c, but I digress). Therefore the restored object would be in the right schema, so that the only problem is that the TOC entry would be mislabeled as to schema. That could lead to wrong decisions for schema-selective restores, for example. The ownership issue is a bit more serious: not only was the TOC entry potentially mislabeled as to owner, but pg_dump didn't bother to issue an ALTER OWNER command at all, so that after restore the stats object would continue to be owned by the restoring superuser. A final point is that decisions as to whether to dump a stats object or not were driven by whether the underlying table was dumped or not. While that's not wrong on its face, it won't scale nicely to the planned future extension to cross-table statistics. Moreover, that design decision comes out of the view of stats objects as being auxiliary to a particular table, like a rule or trigger, which is exactly where the above problems came from. Since we're now treating stats objects more like independent objects in their own right, they ought to behave like standalone objects for this purpose too. So change to using the generic selectDumpableObject() logic for them (which presently amounts to "dump if containing schema is to be dumped"). Along the way to fixing this, restructure so that getExtendedStatistics collects the identity info (only) for all extended stats objects in one query, and then for each object actually being dumped, we retrieve the definition in dumpStatisticsExt. This is necessary to ensure that schema-qualification in the generated CREATE STATISTICS command happens with respect to the search path that pg_dump will now be using at restore time (ie, the schema the stats object is in, not that of the underlying table). It's probably also significantly faster in the typical scenario where only a minority of tables have extended stats. Back-patch to v10 where extended stats were introduced. Discussion:

Robert Haas pushed:

Magnus Hagander pushed:

  • Change default git repo URL to https. Since we now support the server side handler for git over https (so we're no longer using the "dumb protocol"), make https the primary choice for cloning the repository, and the git protocol the secondary choice. In passing, also change the links to from http to https. Reviewed by Stefan Kaltenbrunner and David G. Johnston

Álvaro Herrera pushed:

Correctifs en attente

Mark Rofail sent in two more revisions of a patch to implement foreign key arrays.

Takayuki Tsunakawa sent in another revision of a patch to fix an ECPG bug where freeing memory for pgtypes crashes on Windows.

Michaël Paquier sent in a patch to use base backup exclusion filters to reduce data transferred with pg_rewind.

Michaël Paquier sent in a patch to fix a typo in a comment in pg_multixact/offset in multixact.c.

Kyotaro HORIGUCHI sent in another revision of a patch to allow booleans to be partition bounds.

Tomas Vondra sent in another revision of a patch to implement BRIN multi-range indexes.

Takayuki Tsunakawa sent in a patch to reset temp schema on connect.

Amit Langote sent in two more revisions of a patch to refactor the partition tuple conversion maps handling code and initialize per-partition objects lazily during tuple-routing.

Pierre Ducroquet sent in another revision of a patch atop the JIT patch to support LLVM 9.1.

Ildus Kurbangaliev sent in another revision of a patch to implement custom compression methods.

Nikhil Sontakke sent in another revision of a patch to fix some faulty Logical Decoding and HeapTupleSatisfiesVacuum assumptions.

Thomas Munro sent in another revision of a patch to fix a comment in src/backend/access/transam/xlog.c.

Peter Geoghegan sent in a patch to mark logtape.c buffer's tail as defined to Valgrind.

Claudio Freire sent in four more revisions of a patch to enable VACUUM to use more than 1GB of work mem.

Nikhil Sontakke sent in another revision of a patch to implement logical decoding of two-phase transactions.

Pavel Stěhule sent in another revision of a patch to implement schema variables.

Artur Zakirov sent in another revision of a patch to implement shared Ispell dictionaries.

Nathan Bossart sent in a patch to allow users to change the WAL segment size of a cluster with pg_resetwal.

Peter Eisentraut sent in a patch to fix some confusing SSL test names.

Ashutosh Bapat sent in two more revisions of a patch to do better partition matching for partition-wise joins.

Pavan Deolasee sent in another revision of a patch to implement MERGE.

Pierre Ducroquet sent in another revision of a patch atop the JIT patch to support JIT compiling with LLVM v10.0.

Jeevan Chalke sent in another revision of a patch to implement partition-wise aggregation/grouping.

Ildus Kurbangaliev sent in a patch to add 'autovacuum_table_priority' to the current list of automatic vacuuming settings.

Peter Geoghegan sent in another revision of a patch to add a Bloom filter data structure implementation and use same to add amcheck verification of indexes against the heap.

Claudio Freire sent in another revision of a patch to vacuum the FSM more frequently.

Masahiko Sawada sent in another revision of a patch to keep track of writing on non-temporary relations, support atomic commits involving multiple foreign servers, and add postgres_fdw support for atomic distributed transaction commit.

Michaël Paquier sent in a patch to disable src/test/[ssl|ldap] when not building with SSL/LDAP support.

Kyotaro HORIGUCHI sent in a patch to let plan_create_index_workers honor_dsm_none.

Kyotaro HORIGUCHI sent in a patch to increase the minimum allowable value of max_connections from 10 to 20.

Konstantin Knizhnik sent in a patch for PL/pgsql which reports an error in the case of an empty attribute list for SELECT INTO.

Amit Langote sent in another revision of a patch to implement faster partition pruning.

Konstantin Knizhnik sent in another revision of a patch to implement a built-in connection pooler.

Peter Eisentraut sent in a patch to add ldapi support.

Andrew Dunstan sent in another revision of a patch to implement a faster ALTER TABLE ... ADD COLUMN ... DEFAULT ...

Etsuro Fujita sent in a patch to add some regression tests for the PostgreSQL FDW.

Andrey Borodin sent in a patch to enable using ICU as the default collation provider.

Thomas Munro sent in a patch to register and document LWTRANCHE_PARALLEL_HASH_JOIN.