Correctifs appliqués

Simon Riggs pushed:

Robert Haas pushed:

  • Pass eflags down to parallel workers. Currently, there are no known consequences of this oversight, so no back-patch. Several of the EXEC_FLAG_* constants aren't usable in parallel mode anyway, and potential problems related to the presence or absence of OIDs (see EXEC_FLAG_WITH_OIDS, EXEC_FLAG_WITHOUT_OIDS) seem at present to be masked by the unconditional projection step performed by Gather and Gather Merge. In general, however, it seems important that all participants agree on the values of these flags, which modify executor behavior globally, and a pending patch to skip projection in Gather (Merge) would be outright broken in certain cases without this fix. Patch by me, based on investigation of a test case provided by Amit Kapila. This patch was also reviewed by Amit Kapila. Discussion:
  • Tweak use of ExecContextForcesOids by Gather (Merge). Specifically, pass the outer plan's PlanState instead of our own PlanState. At present, ExecContextForcesOids doesn't actually care which PlanState we pass; it just looks through to the underlying EState to find the result relation or top-level eflags. However, in the future it might care. If that happens, and if our goal is to get a tuple descriptor that matches that of the outer plan, then I think what we care about is whether the outer plan's context forces OIDs, rather than whether our own context forces OIDs, just as we use the outer node's target list rather than our own. Patch by me, reviewed by Amit Kapila. Discussion:
  • Fix multiple problems with satisfies_hash_partition. Fix the function header comment to describe the actual behavior. Check that table OID, modulus, and remainder arguments are not NULL before accessing them. Check that the modulus and remainder are sensible. If the table OID doesn't exist, return NULL instead of emitting an internal error, similar to what we do elsewhere. Check that the actual argument types match, or at least are binary coercible to, the expected argument types. Correctly handle invocation of this function using the VARIADIC syntax. Add regression tests. Robert Haas and Amul Sul, per a report by Andreas Seltenreich and subsequent followup investigation. Discussion:
  • Provide for forward compatibility with future minor protocol versions. Previously, any attempt to request a 3.x protocol version other than 3.0 would lead to a hard connection failure, which made the minor protocol version really no different from the major protocol version and precluded gentle protocol version breaks. Instead, when the client requests a 3.x protocol version where x is greater than 0, send the new NegotiateProtocolVersion message to convey that we support only 3.0. This makes it possible to introduce new minor protocol versions without requiring a connection retry when the server is older. In addition, if the startup packet includes name/value pairs where the name starts with "_pq_.", assume that those are protocol options, not GUCs. Include those we don't support (i.e. all of them, at present) in the NegotiateProtocolVersion message so that the client knows they were not understood. This makes it possible for the client to request previously-unsupported features without bumping the protocol version at all; the client can tell from the server's response whether the option was understood. It will take some time before servers that support these new facilities become common in the wild; to speed things up and make things easier for a future 3.1 protocol version, back-patch to all supported releases. Robert Haas and Badrul Chowdhury Discussion: Discussion:
  • Avoid projecting tuples unnecessarily in Gather and Gather Merge. It's most often the case that the target list for the Gather (Merge) node matches the target list supplied by the underlying plan node; when this is so, we can avoid the overhead of projecting. This depends on commit f455e1125e2588d4cd4fc663c6a10da4e003a3b5 for proper functioning. Idea by Andres Freund. Patch by me. Review by Amit Kapila. Discussion:

Tom Lane pushed:

  • Use out-of-line M68K spinlock code for OpenBSD as well as NetBSD. David Carlier (from a patch being carried by OpenBSD packagers) Discussion:
  • Add support for Motorola 88K to s_lock.h. Apparently there are still people out there who care about this old architecture. They probably care about dusty versions of Postgres too, so back-patch to all supported branches. David Carlier (from a patch being carried by OpenBSD packagers) Discussion:
  • Support index-only scans in contrib/cube and contrib/seg GiST indexes. To do this, we only have to remove the compress and decompress support functions, which have never done anything more than detoasting. In the wake of commit d3a4f89d8, this results in automatically enabling index-only scans, since the core code will now know that the stored representation is the same as the original data (up to detoasting). The only exciting part of this is that ALTER OPERATOR FAMILY lacks a way to drop a support function that was declared as being part of an opclass rather than being loose in the family. For the moment, we'll hack our way to a solution with a manual update of the pg_depend entry type, which is what distinguishes the two cases. Perhaps someday it'll be worth providing a cleaner way to do that, but for now it seems like a very niche problem. Note that the underlying C functions remain, to support use of the shared libraries with older versions of the modules' SQL declarations. Someday we may be able to remove them, but not soon. Andrey Borodin, reviewed by me Discussion:
  • Doc: fix broken markup.
  • pgbench: fix stats reporting when some transactions are skipped. pgbench can skip some transactions when both -R and -L options are used. Previously, this resulted in slightly silly statistics both in progress reports and final output, because the skipped transactions were counted as executed for TPS and related stats. Discount skipped xacts in TPS numbers, and also when figuring the percentage of xacts exceeding the latency limit. Also, don't print per-script skipped-transaction counts when there is only one script. That's redundant with the overall count, and it's inconsistent with the fact that we don't print other per-script stats when there's only one script. Clean up some unnecessary interactions between what should be independent options that were due to that decision. While at it, avoid division-by-zero in cases where no transactions were executed. While on modern platforms this would generally result in printing "NaN" rather than a crash, that isn't spelled consistently across platforms and it would confuse many people. Skip the relevant output entirely when practical, else print zeroes. Fabien Coelho, reviewed by Steve Singer, additional hacking by me Discussion:
  • Ensure sizeof(GenerationChunk) is maxaligned. Per buildfarm. Also improve some comments.
  • Fix unstable regression test added by commits 59b71c6fe et al. The query didn't really have a preferred index, leading to platform- specific choices of which one to use. Adjust it to make sure tenk1_hundred is always chosen. Per buildfarm.
  • Fix bug in generation.c's valgrind support. This doesn't look like the last such bug, but it's one that the test_decoding regression test is tripping over. Per buildfarm. Tomas Vondra Discussion:
  • Mostly-cosmetic improvements in memory chunk header alignment coding. Add commentary about what we're doing and why. Apply the method used for padding in GenerationChunk to AllocChunkData, replacing the rather ad-hoc solution used in commit 7e3aa03b4. Reorder fields in GenerationChunk so that the padding calculation will work even if sizeof(size_t) is different from sizeof(void *) --- likely that will never happen, but we don't need the assumption if we do it like this. Improve static assertions about alignment. In passing, fix a couple of oversights in the "large chunk" path in GenerationAlloc(). Discussion:
  • Improve valgrind logic in aset.c, and fix multiple issues in generation.c. Revise aset.c so that all the "private" fields of chunk headers are marked NOACCESS when outside the module, improving on the previous coding which protected only requested_size. Fix a couple of corner case bugs, such as failing to re-protect the header during a failure exit from AllocSetRealloc, and wrong padding-size calculation for an oversize allocation request. Apply the same design to generation.c, and also fix several bugs therein that I found by dint of hacking the code to use generation.c as the standard allocator and then running the core regression tests with it. Notably, we have to track the actual size of each block, else the wipe_mem call in GenerationReset clears the wrong amount of memory for an oversize-chunk block; and GenerationCheck needs a way of identifying freed chunks that isn't fooled by palloc(0). I chose to fix the latter by resetting the context pointer to NULL in a freed chunk, roughly like what happens in a freed aset.c chunk. Discussion:
  • Improve planner's handling of set-returning functions in grouping columns. Improve query_is_distinct_for() to accept SRFs in the targetlist when we can prove distinctness from a DISTINCT clause. In that case the de-duplication will surely happen after SRF expansion, so the proof still works. Continue to punt in the case where we'd try to prove distinctness from GROUP BY (or, in the future, source relations). To do that, we'd have to determine whether the SRFs were in the grouping columns or elsewhere in the tlist, and it still doesn't seem worth the trouble. But this trivial change allows us to recognize that "SELECT DISTINCT unnest(foo) FROM ..." produces unique-ified output, which seems worth having. Also, fix estimate_num_groups() to consider the possibility of SRFs in the grouping columns. Its failure to do so was masked before v10 because grouping_planner() scaled up plan rowcount estimates by the estimated SRF multiplier after performing grouping. That doesn't happen anymore, which is more correct, but it means we need an adjustment in the estimate for the number of groups. Failure to do this leads to an underestimate for the number of output rows of subqueries like "SELECT DISTINCT unnest(foo)" compared to what 9.6 and earlier estimated, thus breaking plan choices in some cases. Per report from Dmitry Shalashov. Back-patch to v10 to avoid degraded plan choices compared to previous releases. Discussion:
  • Update buffile.h/.c comments for removal of non-temp option. Commit 11e264517 removed BufFile's isTemp flag, thereby eliminating the possibility of resurrecting BufFileCreate(). But it left that function in place, as well as a bunch of comments describing how things worked for the non-temp-file case. At best, that's now a source of confusion. So remove the long-since-commented-out function and change relevant comments. I (tgl) wanted to rename BufFileCreateTemp() to BufFileCreate(), but that seems not to be the consensus position, so leave it as-is. In passing, fix commit f0828b2fc's failure to update BufFileSeek's comment to match the change of its argument type from long to off_t. (I think that might actually have been intentional at the time, but now that 64-bit off_t is nearly universal, it looks anachronistic.) Thomas Munro and Tom Lane Discussion:
  • Repair failure with SubPlans in multi-row VALUES lists. When nodeValuesscan.c was written, it was impossible to have a SubPlan in VALUES --- any sub-SELECT there would have to be uncorrelated and thereby would produce an InitPlan instead. We therefore took a shortcut in the logic that throws away a ValuesScan's per-row expression evaluation data structures. This was broken by the introduction of LATERAL however; a sub-SELECT containing a lateral reference produces a correlated SubPlan. The cleanest fix for this would be to give up the optimization of discarding the expression eval state. But that still seems pretty unappetizing for long VALUES lists. It seems to work to just prevent the subexpressions from hooking into the ValuesScan node's subPlan list, so let's do that and see how well it works. (If this breaks, due to additional connections between the subexpressions and the outer query structures, we might consider compromises like throwing away data only for VALUES rows not containing SubPlans.) Per bug #14924 from Christian Duta. Back-patch to 9.3 where LATERAL was introduced. Discussion:
  • Avoid formally-undefined use of memcpy() in hstoreUniquePairs(). hstoreUniquePairs() often called memcpy with equal source and destination pointers. Although this is almost surely harmless in practice, it's undefined according to the letter of the C standard. Some versions of valgrind will complain about it, and some versions of libc as well (cf. commit ad520ec4a). Tweak the code to avoid doing that. Noted by Tomas Vondra. Back-patch to all supported versions because of the hazard of libc assertions. Discussion:
  • Replace raw timezone source data with IANA's new compact format. Traditionally IANA has distributed their timezone data in pure source form, replete with extensive historical comments. As of release 2017c, they've added a compact single-file format that omits comments and abbreviates command keywords. This form is way shorter than the pure source, even before considering its allegedly better compressibility. Hence, let's distribute the data in that form rather than pure source. I'm pushing this now, rather than at the next timezone database update, so that it's easy to confirm that this data file produces compiled zic output that's identical to what we were getting before. Discussion:
  • Update MSVC build process for new timezone data. Missed this dependency in commits 7cce222c9 et al.
  • Pad XLogReaderState's main_data buffer more aggressively. Originally, we palloc'd this buffer just barely big enough to hold the largest xlog record seen so far. It turns out that that can result in valgrind complaints, because some compilers will emit code that assumes it can safely fetch padding bytes at the end of a struct, and those padding bytes were unallocated so far as aset.c was concerned. We can fix that by MAXALIGN'ing the palloc request size, ensuring that it is big enough to include any possible padding that might've been omitted from the on-disk record. An additional objection to the original coding is that it could result in many repeated palloc cycles, in the worst case where we see a series of gradually larger xlog records. We can ameliorate that cheaply by imposing a minimum buffer size that's large enough for most xlog records. BLCKSZ/2 was chosen after a bit of discussion. In passing, remove an obsolete comment in struct xl_heap_new_cid that the combocid field is free due to alignment considerations. Perhaps that was true at some point, but it's not now. Back-patch to 9.5 where this code came in. Discussion:

Magnus Hagander pushed:

Noah Misch pushed:

Fujii Masao pushed:

Peter Eisentraut pushed:

  • Convert documentation to DocBook XML. Since some preparation work had already been done, the only source changes left were changing empty-element tags like <xref linkend="foo"> to <xref linkend="foo"/>, and changing the DOCTYPE. The source files are still named *.sgml, but they are actually XML files now. Renaming could be considered later. In the build system, the intermediate step to convert from SGML to XML is removed. Everything is build straight from the source files again. The OpenSP (or the old SP) package is no longer needed. The documentation toolchain instructions are updated and are much simpler now. Peter Eisentraut, Alexander Lakhin, Jürgen Purtz

Andres Freund pushed:

  • Fix handling of NULLs returned by aggregate combine functions. When strict aggregate combine functions, used in multi-stage/parallel aggregation, returned NULL, we didn't check for that, invoking the combine function with NULL the next round, despite it being strict. The equivalent code invoking normal transition functions has a check for that situation, which did not get copied in a7de3dc5c346. Fix the bug by adding the equivalent check. Based on a quick look I could not find any strict combine functions in core actually returning NULL, and it doesn't seem very likely external users have done so. So this isn't likely to have caused issues in practice. Add tests verifying transition / combine functions returning NULL is tested. Reported-By: Andres Freund Author: Andres Freund Discussion: Backpatch: 9.6, where parallel aggregation was introduced

Dean Rasheed pushed:

Joe Conway pushed:

Correctifs en attente

Anthony Bykov sent in another revision of a patch to implement a Jsonb TRANSFORM for PL/PythonU.

Alexander Korotkov sent in another revision of a patch to fix KNN and negative coordinate handling for contrib/cube.

Brian Cloutier sent in another revision of a patch to add PGDLLIMPORT lines to some variables.

Martín Marqués sent in a patch to quiet MSVC's warnings about isatty().

Masahiko Sawada sent in three more revisions of a patch to fix an assertion failure when a non-exclusive pg_stop_backup is aborted.

Martín Marqués sent in two more revisions of a patch to make pg_basebackup use isatty().

Ildar Musin sent in a patch to remove the redundant IndexTupleDSize macro.

Masahiko Sawada sent in another revision of a patch to move relation extension locks out of the heavyweight lock manager.

Alik Khilazhev and Fabien COELHO traded patches to add a Zipfian distribution to pgbench.

Michaël Paquier and Peter Eisentraut traded patches to allow SSL connection from a v11 client to a v10 server with SCRAM channel binding.

Torikoshi Atsushi sent in four revisions of a patch to ensure that old ReorderBuffer spilled files are deleted.

Tomas Vondra sent in two revisions of a patch to implement a custom lz4 compression method atop Ildus Kurbangaliev's patch which enables such customizations.

高增琦 (GaoZengqi) sent in two more revisions of a patch to add a dependency between client executables and static libraries.

Huong Dangminh sent in another revision of a patch to fix the slot_store error callback.

Sophie Herold sent in a patch to add a pg_to_typemod() function.

Masahiko Sawada sent in another revision of a patch to correctly establish the replication status with logical replication.

Alexander Korotkov sent in another revision of a patch to implement incremental sort.

Amit Khandekar and Amit Langote traded patches to make UPDATE that changes a partition key to another partition Just Work.

Amit Khandekar and Amul Sul traded patches to implement parallel Append.

Ildus Kurbangaliev sent in two more revisions of a patch to implement custom compression methods.

Michaël Paquier sent in two more revisions of a patch to add a connection parameter "saslchannelbinding" and implement channel binding tls-server-end-point for SCRAM.

Kyotaro HORIGUCHI sent in another revision of a patch to show index scans of the last vacuum in pg_stat_all_tables, and add vacuum_required, vacuum execution status and truncation information to pg_stat_all_tables.

Jing Wang sent in another revision of a patch to implement COMMENT ON DATABASE CURRENT_DATABASE.

Thomas Munro sent in another revision of a patch to implement parallel hash.

Artur Zakirov sent in another revision of a patch to fix a bug in how to_timestamp() worked in the case of an FX prefix.

Rushabh Lathia and Amit Langote traded patches to fix an issue where range partition didn't handle NULL partition keys.

Amit Langote sent in another revision of a patch to make partition pruning faster.

Rushabh Lathia sent in a patch to fix a typo in the ExecBuildSlotPartitionKeyDescription prologue.

Jeevan Chalke sent in another revision of a patch to implement partition-wise aggregation/grouping.

Nikhil Sontakke sent in another revision of a patch to implement logical decoding of two-phase transactions.

Amul Sul sent in another revision of a patch to restrict concurrent update/delete with UPDATE of partition key.

Amit Kapila and Rafia Sabih traded patches to speed up processing at Gather nodes.

Thomas Munro sent in another revision of a patch to support huge pages on Windows.

Amit Langote sent in a patch to allow ON CONFLICT DO NOTHING on partitioned tables.

Etsuro Fujita sent in another revision of a patch to support tuple routing for foreign partitions.

Antonin Houska sent in another revision of a patch to enable separate logging per extension.

Pavel Stěhule sent in another revision of a patch to add default namespaces for XPath expressions.

Oliver Ford sent in a patch to add RANGE with values and exclusions clauses to the Window Functions.

Michaël Paquier sent in another revision of a patch to refactor syscache routines to get attribute name, extend lookup routines for FDW and foreign server with NULL handling, refactor routines for subscription and publication lookups, and eliminate user-visible cache lookup errors for objaddr SQL functions.

Fabien COELHO sent in a patch to add \if to pgbench.

Tomas Vondra sent in a patch to fix an issue in multivariate histograms and MCV lists.

Mark Dilger sent in two revisions of a patch to clean up some code in extended_stats.c.

Hadi Moshayedi sent in a patch to fix a crash in int8_avg_combine().

Yura Sokolov sent in another revision of a patch to fix performance degradation of contended LWLock on NUMA.