Correctifs appliqués

Stephen Frost pushed:

Teodor Sigaev pushed:

Heikki Linnakangas pushed:

�lvaro Herrera pushed:

Magnus Hagander pushed:

Tom Lane pushed:

  • Further cleanup of client dependencies on src/include/catalog headers. In commit 9c0a0de4c, I'd failed to notice that catalog/catalog.h should also be considered a frontend-unsafe header, because it includes (and needs) the full form of pg_class.h, not to mention relcache.h. However, various frontend code was depending on it to get TABLESPACE_VERSION_DIRECTORY, so refactoring of some sort is called for. The cleanest answer seems to be to move TABLESPACE_VERSION_DIRECTORY, as well as the OIDCHARS symbol, to common/relpath.h. Do that, and mop up inclusions as necessary. (I found that quite a few current users of catalog/catalog.h don't seem to need it at all anymore, apparently as a result of the refactorings that created common/relpath.[hc]. And initdb.c needed it only as a route to pg_class_d.h.) Discussion:
  • Reduce chattiness of and Make these scripts emit just one log message when they run, not one per output file. The latter is way too verbose in the wake of commit 372728b0d. The specific wording used is what already existed in the MSVC scripts. John Naylor Discussion:
  • Make preserve all blank lines. In its original form, smashed consecutive blank lines to a single blank line, which was helpful for mopping up excess whitespace during the bootstrap data format conversion. But going forward, there seems little reason to do that; if developers want to put in multiple blank lines, let 'em. This makes it conform to the documentation I (tgl) wrote, too. In passing, clean up some sloppy markup choices in bki.sgml. John Naylor Discussion:
  • Fix partial-build problems introduced by having more generated headers. Commit 372728b0d created some problems for usages like building a subdirectory without having first done "make all" at the top level, or for proceeding directly to "make install" without "make all". The only reasonably clean way to fix this seems to be to force the submake-generated-headers rule to fire in *any* "make all" or "make install" command anywhere in the tree. To avoid lots of redundant work, as well as parallel make jobs possibly clobbering each others' output, we still need to be sure that the rule fires only once in a recursive build. For that, adopt the same MAKELEVEL hack previously used for "temp-install". But try to document it a bit better. The submake-errcodes mechanism previously used in src/port/ and src/common/ is subsumed by this, so we can get rid of those special cases. It was inadequate for src/common/ anyway after the aforesaid commit, and it always risked parallel attempts to build errcodes.h. Discussion:
  • Fix to not try to build generated headers in external builds. Per Julien Rouhaud and the buildfarm. This is not quite Julien's patch: there's no need to lobotomize this build rule when building contrib modules in-tree, so set NO_GENERATED_HEADERS only if PGXS. In passing, also set NO_TEMP_INSTALL in external builds. This doesn't seem to be fixing any live bug, because "make check" in an external build just produces the expected error message without first trying to make a temp install ... but it's far from obvious why it doesn't, so this change seems like good future-proofing. Julien Rouhaud and Tom Lane Discussion:
  • Put back parallel-safety guards in plpython and src/test/regress/. I'd hoped that commit 3b8f6e75f was sufficient to ensure parallel safety even when a build started in a subdirectory requires rebuilding of generated headers. This isn't so, because making submake-generated-headers a prerequisite of "all" isn't enough to ensure it's completed before starting on "all"'s other prerequisites. The explicit dependencies we put on the recursive make targets ensure safe ordering before we recurse into child directories, but they don't protect targets to be made in the current directory. Hence, put back some ordering dependencies in directories that we've traditionally expected to be starting points for "standalone" builds, to wit src/pl/plpython and src/test/regress. (The former needs this in order to minimize the work involved in building for both python 2 and python 3; the latter to support packagings that make the regression tests available for out-of-build-tree execution.) Adjust some other dependencies so that these two cases work correctly even at high -j settings. I'm not terribly happy with this partial solution, but I don't see a way to do better without massive makefile restructuring, which we surely aren't doing at this point in the development cycle. In any case, it's little if any worse than what we had in prior releases. Discussion:
  • Fix incorrect close() call in dsm_impl_mmap(). One improbable error-exit path in this function used close() where it should have used CloseTransientFile(). This is unlikely to be hit in the field, and I think the consequences wouldn't be awful (just an elog(LOG) bleat later). But a bug is a bug, so back-patch to 9.4 where this code came in. Pan Bian Discussion:
  • Invoke submake-generated-headers during "make check", too. The MAKELEVEL hack to prevent submake-generated-headers from doing anything in child make runs means that we have to explicitly invoke it at top level for "make check", too, in case somebody proceeds directly to that without an explicit "make all". (I think this usage had parallel-make hazards even before the addition of more generated headers; but it was totally broken as of 3b8f6e75f.) Out of paranoia, force the submake-libpq target to depend on submake-generated-headers, too. This seems to not be absolutely necessary today, but it's not really saving us anything to omit the ordering dependency, and it'll likely break someday without it. Discussion:
  • Do not select new object OIDs that match recently-dead entries. When selecting a new OID, we take care to avoid picking one that's already in use in the target table, so as not to create duplicates after the OID counter has wrapped around. However, up to now we used SnapshotDirty when scanning for pre-existing entries. That ignores committed-dead rows, so that we could select an OID matching a deleted-but-not-yet-vacuumed row. While that mostly worked, it has two problems: * If recently deleted, the dead row might still be visible to MVCC snapshots, creating a risk for duplicate OIDs when examining the catalogs within our own transaction. Such duplication couldn't be visible outside the object-creating transaction, though, and we've heard few if any field reports corresponding to such a symptom. * When selecting a TOAST OID, deleted toast rows definitely *are* visible to SnapshotToast, and will remain so until vacuumed away. This leads to a conflict that will manifest in errors like "unexpected chunk number 0 (expected 1) for toast value nnnnn". We've been seeing reports of such errors from the field for years, but the cause was unclear before. The fix is simple: just use SnapshotAny to search for conflicting rows. This results in a slightly longer window before object OIDs can be recycled, but that seems unlikely to create any large problems. Pavan Deolasee Discussion:
  • Ignore nextOid when replaying an ONLINE checkpoint. The nextOid value is from the start of the checkpoint and may well be stale compared to values from more recent XLOG_NEXTOID records. Previously, we adopted it anyway, allowing the OID counter to go backwards during a crash. While this should be harmless, it contributed to the severity of the bug fixed in commit 0408e1ed5, by allowing duplicate TOAST OIDs to be assigned immediately following a crash. Without this error, that issue would only have arisen when TOAST objects just younger than a multiple of 2^32 OIDs were deleted and then not vacuumed in time to avoid a conflict. Pavan Deolasee Discussion:
  • Fix YA parallel-make hazard, this one in "make check" in plpython. We have to ensure that submake-generated-headers is finished before the topmost make run launches any child makes. Discussion:
  • YA attempt to stabilize the results of the postgres_fdw regression test. We've made multiple attempts to stabilize the plans shown by commit 1bc0100d2, with little success so far. The reason for the remaining instability seems to be that if a transaction (such as auto-analyze) is running concurrently with the test, then get_actual_variable_range may return a maximum value for "T 1"."C 1" that's far away from the actual max, as a result of our having transiently inserted such a value earlier in the test. Because we use a non-MVCC snapshot to fetch the value (for performance reasons), the presence of other transactions can cause that function to return entries that are actually dead. To fix, use a less extreme value in the earlier transient insertion, so that whether it is visible or not won't affect the selectivity estimate. The use of 9999 there seems to have been picked with the aid of a dartboard anyway, rather than having a specific reason. Discussion:
  • Fix bogus affix-merging code. NISortAffixes() compared successive compound affixes incorrectly, thus possibly failing to merge identical affixes, or (less likely) merging ones that shouldn't be merged. The user-visible effects of this are unclear, to me anyway. Per bug #15150 from Alexander Lakhin. It's been broken for a long time, so back-patch to all supported branches. Arthur Zakirov Discussion:
  • In libpq, free any partial query result before collecting a server error. We'd throw away the partial result anyway after parsing the error message. Throwing it away beforehand costs nothing and reduces the risk of out-of-memory failure. Also, at least in systems that behave like glibc/Linux, if the partial result was very large then the error PGresult would get allocated at high heap addresses, preventing the heap storage used by the partial result from being released to the OS until the error PGresult is freed. In psql >= 9.6, we hold onto the error PGresult until another error is received (for \errverbose), so that this behavior causes a seeming memory leak to persist for awhile, as in a recent complaint from Darafei Praliaskouski. This is a potential performance regression from older versions, justifying back-patching at least that far. But similar behavior may occur in other client applications, so it seems worth just back-patching to all supported branches. Discussion:
  • Improve regression test coverage for src/backend/tsearch/spell.c. In passing, throw an error if the AF count is too small, rather than just silently discarding extra affix entries. Note that the new regression test cases require installing the updated src/backend/tsearch/dicts files. Arthur Zakirov Discussion:
  • Add commentary explaining why MaxIndexTuplesPerPage calculation is safe. MaxIndexTuplesPerPage ignores the fact that btree indexes sometimes store tuples with no data payload. But it also ignores the possibility of "special space" on index pages, which offsets that, so that the result isn't an underestimate. This all seems worth documenting, though. In passing, remove #define MinIndexTupleSize, which was added by commit 2c03216d8 but not used in that commit nor later ones. Comment text by me; issue noticed by Peter Geoghegan. Discussion:
  • Fix enforcement of SELECT FOR UPDATE permissions with nested views. SELECT FOR UPDATE on a view should require UPDATE (as well as SELECT) permissions on the view, and then the view's owner needs those same permissions against the relations it references, and so on all the way down to base tables. But ApplyRetrieveRule did things in the wrong order, resulting in failure to mark intermediate view levels as needing UPDATE permission. Thus for example, if user A creates a table T and an updatable view V1 on T, then grants only SELECT permissions on V1 to user B, B could create a second view V2 on V1 and then would be allowed to perform SELECT FOR UPDATE via V2 (since V1 wouldn't be checked for UPDATE permissions). To fix, just switch the order of expanding sub-views and marking referenced objects as needing UPDATE permission. I think additional simplifications are now possible, but that's distinct from the bug fix proper. This is certainly a security issue, but the consequences are pretty minor (just the ability to lock rows that shouldn't be lockable). Against that we have a small risk of breaking applications that are working as-desired, since nested views have behaved this way since such cases worked at all. On balance I'm inclined not to back-patch. Per report from Alexander Lakhin. Discussion:
  • Improve regression test coverage of expand_tuple(). I was dissatisfied with the code coverage report for expand_tuple() in the wake of commit 7c44c46de: while better than no coverage at all, it was still not exercising the core function of inserting out-of-line default values, nor was the HeapTuple-output path covered. So far as I can find, the only code path that reaches the latter at present is EvalPlanQual fetches for non-locked tables. Hence, extend eval-plan-qual.spec to test cases where out-of-line defaults must be inserted into a tuple fetched from a non-locked table. Discussion:
  • Simplify view-expansion code in rewriteHandler.c. In the wake of commit 50c6bb022, it's not necessary for ApplyRetrieveRule to have a forUpdatePushedDown parameter. By the time control gets here for any given view-referencing RTE, we should already have pushed down the effects of any FOR UPDATE/SHARE clauses affecting the view from outer query levels. Hence if we don't find a RowMarkClause at the current query level, that's sufficient proof that there is no outer one either. This in turn means we need no forUpdatePushedDown parameter for fireRIRrules. I wonder whether we oughtn't also revert commit cba2d2717, since it now seems likely that that was band-aiding around the bad effects of doing FOR UPDATE pushdown and view expansion in the wrong order. However, in the absence of evidence that the current coding of markQueryForLocking is actually buggy (i.e. missing RTEs it ought to mark), it seems best to leave it alone. Discussion:
  • Clean up callers of JsonbIteratorNext(). Coverity complained about the lack of a check on the return value in parse_jsonb_index_flags' last call of JsonbIteratorNext. Seems like a reasonable gripe to me, especially since the code is depending on that being WJB_DONE to not leak memory, so add a check. In passing, improve a couple other places where the result was being ignored, either by adding an assert or at least a cast to void. Also, don't spell "WJB_DONE" as "0". That's horrid coding style, and it wasn't consistent either.
  • Fix potentially-unportable code in contrib/adminpack. Spelling access(2)'s second argument as "2" is just horrid. POSIX makes no promises as to the numeric values of W_OK and related macros. Even if it accidentally works as intended on every supported platform, it's still unreadable and inconsistent with adjacent code. In passing, don't spell "NULL" as "0" either. Yes, that's legal C; no, it's not project style. Back-patch, just in case the unportability is real and not theoretical. (Most likely, even if a platform had different bit assignments for access()'s modes, there'd not be an observable behavior difference here; but I'm being paranoid today.)

Andrew Dunstan pushed:

Peter Eisentraut pushed:

Simon Riggs pushed:

  • Revert MERGE patch. This reverts commits d204ef63776b8a00ca220adec23979091564e465, 83454e3c2b28141c0db01c7d2027e01040df5249 and a few more commits thereafter (complete list at the end) related to MERGE feature. While the feature was fully functional, with sufficient test coverage and necessary documentation, it was felt that some parts of the executor and parse-analyzer can use a different design and it wasn't possible to do that in the available time. So it was decided to revert the patch for PG11 and retry again in the future. Thanks again to all reviewers and bug reporters. List of commits reverted, in reverse chronological order: f1464c5380 Improve parse representation for MERGE ddb4158579 MERGE syntax diagram correction 530e69e59b Allow cpluspluscheck to pass by renaming variable 01b88b4df5 MERGE minor errata 3af7b2b0d4 MERGE fix variable warning in non-assert builds a5d86181ec MERGE INSERT allows only one VALUES clause 4b2d44031f MERGE post-commit review 4923550c20 Tab completion for MERGE aa3faa3c7a WITH support in MERGE 83454e3c2b New files for MERGE d204ef6377 MERGE SQL Command following SQL:2016 Author: Pavan Deolasee Reviewed-by: Michael Paquier

Correctifs en attente

Aleksandr Parfenov sent in another revision of a patch to add a more flexible configuration for full-text search.

Amit Langote sent in two revisions of a patch to fix the partition-pruning code to determine the comparison function correctly.

Heikki Linnakangas sent in a patch to clean up the inclusion of GID in WAL records for 2PC in logical replication.

Kyotaro HORIGUCHI sent in four more revisions of a patch to fix a race condition in full-page-writes between config reload on checkpointer and StartupXLOG.

Sergei Kornilov sent in three more revisions of a patch to allow skipping a full-table scan when doing ALTER TABLE ... ALTER COLUMN ... SET NOT NULL if the column already has a CHECK constraint that would prevent it from being NULL in the first place.

Konstantin Knizhnik sent in another revision of a patch to make range queries faster.

Peter Eisentraut sent in a patch to change pg_dump to use current_database() instead of PQdb().

Craig Ringer sent in a patch to PANIC when we detect a possible fsync I/O error instead of retrying fsync.

Alexander Kuzmenkov sent in two more revisions of a patch to enable sending a signal to the logging collector process. This in turn makes it rotate logs.

Kyotaro HORIGUCHI and Jonathan Katz traded patches to allow setting any immutable expression as a partition bound.

Mart�n Marqu�s sent in two revisions of a patch to document the autovacuum_vacuum_cost_limit GUC better.

�lvaro Herrera and Amit Langote traded patches to fix runtime partition pruning.

Amit Langote sent in a patch to fix a memory context bug in RelationBuildPartitionKey.

Thomas Munro sent in another revision of a patch to add kqueue support to WaitEventSet.

Micha�l Paquier sent in a patch to fix some typos in the covering index patch.

Micha�l Paquier sent in a patch to fix the documentation of covering indexes for CREATE and ALTER TABLE, to wit, it adds a mention of same to the ALTER TABLE documentation, and adds mentions of the fact that exclusion constraints can use INCLUDE clauses.

Thomas Munro sent in another revision of a patch to replace PostmasterIsAlive() calls with WL_POSTMASTER_DEATH checks.

Jeevan Chalke sent in a patch to add partial path only when rel's consider_parallel is true.

Andrey Borodin sent in two revisions of a patch to implement covering GiST indexes.

Micha�l Paquier sent in a patch to make description of heap records more talkative for flags.

Amul Sul sent in a patch to fix an installcheck failure due to MovedPartitions.

Andrey Borodin sent in a patch to ensure that the REINDEX test for covering indexes is actually testing REINDEX.

Julian Markwort sent in another revision of a patch to implement clientcert=verify-full as a pg_hba option.

Konstantin Knizhnik sent in another revision of a patch to fix an issue where PostgreSQL could get stuck in deadlock detection.

Jeff Davis sent in a patch to make it possible to set rpath on