Correctifs appliqués

Robert Haas pushed:

Tom Lane pushed:

  • Support boolean columns in functional-dependency statistics. There's no good reason that the multicolumn stats stuff shouldn't work on booleans. But it looked only for "Var = pseudoconstant" clauses, and it will seldom find those for boolean Vars, since earlier phases of planning will fold "boolvar = true" or "boolvar = false" to just "boolvar" or "NOT boolvar" respectively. Improve dependencies_clauselist_selectivity() to recognize such clauses as equivalent to equality restrictions. This fixes a failure of the extended stats mechanism to apply in a case reported by Vitaliy Garnashevich. It's not a complete solution to his problem because the bitmap-scan costing code isn't consulting extended stats where it should, but that's surely an independent issue. In passing, improve some comments, get rid of a NumRelids() test that's redundant with the preceding bms_membership() test, and fix dependencies_clauselist_selectivity() so that estimatedclauses actually is a pure output argument as stated by its API contract. Back-patch to v10 where this code was introduced. Discussion:
  • Clean up assorted messiness around AllocateDir() usage. This patch fixes a couple of low-probability bugs that could lead to reporting an irrelevant errno value (and hence possibly a wrong SQLSTATE) concerning directory-open or file-open failures. It also fixes places where we took shortcuts in reporting such errors, either by using elog instead of ereport or by using ereport but forgetting to specify an errcode. And it eliminates a lot of just plain redundant error-handling code. In service of all this, export fd.c's formerly-static function ReadDirExtended, so that external callers can make use of the coding pattern dir = AllocateDir(path); while ((de = ReadDirExtended(dir, path, LOG)) != NULL) if they'd like to treat directory-open failures as mere LOG conditions rather than errors. Also fix FreeDir to be a no-op if we reach it with dir == NULL, as such a coding pattern would cause. Then, remove code at many call sites that was throwing an error or log message for AllocateDir failure, as ReadDir or ReadDirExtended can handle that job just fine. Aside from being a net code savings, this gets rid of a lot of not-quite-up-to-snuff reports, as mentioned above. (In some places these changes result in replacing a custom error message such as "could not open tablespace directory" with more generic wording "could not open directory", but it was agreed that the custom wording buys little as long as we report the directory name.) In some other call sites where we can't just remove code, change the error reports to be fully project-style-compliant. Also reorder code in restoreTwoPhaseData that was acquiring a lock between AllocateDir and ReadDir; in the unlikely but surely not impossible case that LWLockAcquire changes errno, AllocateDir failures would be misreported. There is no great value in opening the directory before acquiring TwoPhaseStateLock, so just do it in the other order. Also fix CheckXLogRemoved to guarantee that it preserves errno, as quite a number of call sites are implicitly assuming. (Again, it's unlikely but I think not impossible that errno could change during a SpinLockAcquire. If so, this function was broken for its own purposes as well as breaking callers.) And change a few places that were using not-per-project-style messages, such as "could not read directory" when "could not open directory" is more correct. Back-patch the exporting of ReadDirExtended, in case we have occasion to back-patch some fix that makes use of it; it's not needed right now but surely making it global is pretty harmless. Also back-patch the restoreTwoPhaseData and CheckXLogRemoved fixes. The rest of this is essentially cosmetic and need not get back-patched. Michael Paquier, with a bit of additional work by me Discussion:
  • Improve error handling in RemovePgTempFiles(). Modify this function and its subsidiaries so that syscall failures are reported via ereport(LOG), rather than silently ignored as before. We don't want to throw a hard ERROR, as that would prevent database startup, and getting rid of leftover temporary files is not important enough for that. On the other hand, not reporting trouble at all seems like an odd choice not in line with current project norms, especially since any failure here is quite unexpected. On the same reasoning, adjust these functions' AllocateDir/ReadDir calls so that failure to scan a directory results in LOG not ERROR. I also removed the previous practice of silently ignoring ENOENT failures during directory opens --- there are some corner cases where that could happen given a previous database crash, but that seems like a bad excuse for ignoring a condition that isn't expected in most cases. A LOG message during postmaster start seems OK in such situations, and better than no output at all. In passing, make RemovePgTempRelationFiles' test for "is the file name all digits" look more like the way it's done elsewhere. Discussion:
  • Simplify do_pg_start_backup's API by opening pg_tblspc internally. do_pg_start_backup() expects its callers to pass in an open DIR pointer for the pg_tblspc directory, but there's no apparent advantage in that. It complicates the callers without adding any flexibility, and there's no robustness advantage, since we surely have to be prepared for errors during the scan of pg_tblspc anyway. In fact, by holding an extra kernel resource during operations like the preliminary checkpoint, we might be making things a fraction more failure-prone not less. Hence, remove that argument and open the directory just for the duration of the actual scan. Discussion:
  • Treat directory open failures as hard errors in ResetUnloggedRelations(). Previously, this code just reported such problems at LOG level and kept going. The problem with this approach is that transient failures (e.g., ENFILE) could prevent us from resetting unlogged relations to empty, yet allow recovery to appear to complete successfully. That seems like a data corruption hazard large enough to treat such problems as reasons to fail startup. For the same reason, treat unlink failures for unlogged files as hard errors not just LOG messages. It's a little odd that we did it like that when file-level errors in other steps (copy_file, fsync_fname) are ERRORs. The sole case that I left alone is that ENOENT failure on a tablespace (not database) directory is not an error, though it will now be logged rather than just silently ignored. This is to cover the scenario where a previous DROP TABLESPACE removed the tablespace directory but failed before removing the pg_tblspc symlink. I'm not sure that that's very likely in practice, but that seems like the only real excuse for the old behavior here, so let's allow for it. (As coded, this will also allow ENOENT on $PGDATA/base/. But since we'll fail soon enough if that's gone, I don't think we need to complicate this code by distinguishing that from a true tablespace case.) Discussion:
  • Fix broken markup.
  • Adjust regression test cases added by commit ab7271677. I suppose it is a copy-and-paste error that this test doesn't actually test the "Parallel Append with both partial and non-partial subplans" case (EXPLAIN alone surely doesn't qualify as a test of executor behavior). Fix that. Also, add cosmetic aliases to make it possible to tell apart these otherwise-identical test cases in log_statement output.
  • In plpgsql, unify duplicate variables for record and row cases. plpgsql's function exec_move_row() handles assignment of a composite source value to either a PLpgSQL_rec or PLpgSQL_row target variable. Oddly, rather than taking a single target argument which it could do run-time type detection on, it was coded to take two separate arguments (only one of which is allowed to be non-NULL). This choice had then back-propagated into storing two separate target variables in various plpgsql statement nodes, with lots of duplicative coding and awkward interface logic to support that. Simplify matters by folding those pairs down to single variables, distinguishing the two cases only where we must ... which turns out to be only in exec_move_row itself. This is purely refactoring and should not change any behavior. In passing, remove unused field PLpgSQL_stmt_open.returntype. Discussion:
  • Fix plpgsql to reinitialize record variables at block re-entry. If one exits and re-enters a DECLARE ... BEGIN ... END block within a single execution of a plpgsql function, perhaps due to a surrounding loop, the declared variables are supposed to get re-initialized to null (or whatever their initializer is). But this failed to happen for variables of type "record", because while exec_stmt_block() expected such variables to be included in the block's initvarnos list, plpgsql_add_initdatums() only adds DTYPE_VAR variables to that list. This bug appears to have been there since the aboriginal addition of plpgsql to our tree. Fix by teaching plpgsql_add_initdatums() to include DTYPE_REC variables as well. (We don't need to consider other DTYPEs because they don't represent separately-stored values.) I failed to resist the temptation to make some nearby cosmetic adjustments, too. No back-patch, because there have not been field complaints, and it seems possible that somewhere out there someone has code depending on the incorrect behavior. In any case this change would have no impact on correctly-written code. Discussion:

Peter Eisentraut pushed:

Andres Freund pushed:

  • Fix EXPLAIN ANALYZE of hash join when the leader doesn't participate. If a hash join appears in a parallel query, there may be no hash table available for explain.c to inspect even though a hash table may have been built in other processes. This could happen either because parallel_leader_participation was set to off or because the leader happened to hit the end of the outer relation immediately (even though the complete relation is not empty) and decided not to build the hash table. Commit bf11e7ee introduced a way for workers to exchange instrumentation via the DSM segment for Sort nodes even though they are not parallel-aware. This commit does the same for Hash nodes, so that explain.c has a way to find instrumentation data from an arbitrary participant that actually built the hash table. Author: Thomas Munro Reviewed-By: Andres Freund Discussion:

Noah Misch pushed:

  • MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T. Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl distributions like Strawberry Perl and modern ActivePerl. Perl has no robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so test this. Back-patch to 9.3 (all supported versions). The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when $Config{gccversion} is nonempty. That banks on every gcc-built Perl using the same ABI. gcc could change its default ABI the way MSVC once did, and one could build Perl with gcc and the non-default ABI. The GNU make build system could benefit from a similar test, without which it does not support MSVC-built Perl. For now, just add a comment. Most users taking the special step of building Perl with MSVC probably build PostgreSQL with MSVC. Discussion:
  • MSVC 2012+: Permit linking to 32-bit, MinGW-built libraries. Notably, this permits linking to the 32-bit Perl binaries advertised on, namely Strawberry Perl and ActivePerl. This has a side effect of permitting linking to binaries built with obsolete MSVC versions. By default, MSVC 2012 and later require a "safe exception handler table" in each binary. MinGW-built, 32-bit DLLs lack the relevant exception handler metadata, so linking to them failed with error LNK2026. Restore the semantics of MSVC 2010, which omits the table from a given binary if some linker input lacks metadata. This has no effect on 64-bit builds or on MSVC 2010 and earlier. Back-patch to 9.3 (all supported versions). Reported by Victor Wagner. Discussion:

Magnus Hagander pushed:

Correctifs en attente

Yura Sokolov sent in another revision of a patch to improve compactify_tuples.

Dmitry Dolgov sent in another revision of a patch to implement generic type subscripting.

Jing Wang sent in another revision of a patch to implement COMMENT ON DATABASE CURRENT_DATABASE.

John Naylor sent in a patch to make bootstrap data simpler and faster.

Sergei Kornilov sent in another revision of a patch to implement ALTER TABLE ... SET ... NOT NULL by constraints only.

Oliver Ford sent in a patch to add the GROUPS option to window functions.

Fabien COELHO sent in another revision of a patch to add more functions and operators to pgbench.

Konstantin Knizhnik sent in another revision of a patch to add a projection function to CREATE INDEX.

Konstantin Knizhnik sent in another revision of a patch to implement auto-prepare.

Alexander Kuzmenkov sent in another revision of a patch to implement full merge join on comparison clauses.

Konstantin Knizhnik sent in two more revisions of a patch to optimize secondary indexes.

Robert Haas sent in another revision of a patch to improve processing at GATHER nodes.

Raúl Marín Rodríguez sent in another revision of a patch to add pow() support for pgbench.

Nikhil Sontakke and Peter Eisentraut traded patches to implement logical decoding of two-phase transactions.

Alexander Kuzmenkov sent in another revision of a patch to implement CSN-based snapshots.

Dang Minh Huong and Masahiko Sawada traded patches to fix an issue with user-defined types in logical replication.

Craig Ringer sent in a patch to fix an issue where standby disconnect could corrupt serialized reorder buffers.

Rajkumar Raghuwanshi sent in two revisions of a patch to implement partition-wise join with default partitions.

Thomas Munro sent in another revision of a patch to support kqueue.

Nathan Bossart sent in a patch to fix some crash bugs in VACUUM.

Thomas Munro sent in another revision of a patch to fix es_query_dsa.

Nikita Glukhov and Alexander Korotkov traded patches to add compress methods for SP-GiST.

Ashutosh Bapat sent in two revisions of a patch to mention ordered datums in PartitionBoundInfoData comment.

Amit Kapila and Thomas Munro traded patches to fix the EXPLAIN ANALYZE output for the parallel worker case.

Andres Freund sent in a patch to allow tupleslots to have a fixed tupledesc and use same in executor nodes.

Andrew Dunstan sent in another revision of a patch to make ALTER TABLE ADD COLUMN fast with DEFAULT.

Petr Jelínek sent in another revision of a patch to fix walsender timeouts when decoding a large transaction.

Nikita Glukhov sent in two more revisions of a patch to implement compress method for SP-GiST.

Álvaro Herrera sent in two more revisions of a patch to fix freezing of a dead HOT-updated tuple.

Peter Eisentraut sent in another revision of a patch to implement transaction control for procedures in the built-in PLs.

Anthony Bykov sent in another revision of a patch to implement a JSONB transform for PL/PythonU.

David Rowley sent in another revision of a patch to remove Merge]Append nodes which contain a single subpath.

Anthony Bykov sent in another revision of a patch to implement a JSONB transform for PL/Perl.

Peter Geoghegan sent in another revision of a patch to add a Bloom filter data structure implementation and use same to add amcheck verification of indexes against heap.

Peter Eisentraut sent in a patch to start a separate test suite for PL/pgsql.

Alexander Korotkov sent in a patch to fix some pg_trgm word_similarity inconsistencies.

Beena Emerson sent in another revision of a patch to implement runtime partition pruning.

David Rowley sent in a patch to fix an out of date comment in cached_plan_cost.

Ian Barwick sent in a patch to add %r as part of the psql prompt format to show recovery status.

Alexander Korotkov sent in another revision of a patch to fix contrib/cube's KNN.

Rushabh Lathia sent in two more revisions of a patch to implement parallel tuplesort for parallel B-tree index creation.

Thomas Munro sent in a patch to add defenses against pre-crash files to BufFileOpenShared().

Thomas Munro sent in another revision of a patch to make SERIALIZABLE isolation work with parallel query.

Etsuro Fujita sent in a patch to fix incorrect debug info printed in generate_partition_wise_join_paths.

Alexander Korotkov sent in another revision of a patch to implement incremental sort.

Masahiko Sawada sent in another revision of a patch to move relation extension locks out of the heavyweight lock manager.

Masahiko Sawada sent in another revision of a patch to fix an assertion failure when the non-exclusive pg_stop_backup aborted.

Amit Langote sent in a patch to teach operator_predicate_proof() to strip RelabelType.

Andres Freund sent in a patch to provide overflow safe integer math inline functions and hand code string to int32 conversion for performance.

Thomas Munro sent in a patch to add an extra barrier function to enable changing the party size of a static barrier.

Peter Geoghegan sent in a patch to remove a leftover reference to replacement selection 1 run case.

Tom Lane sent in a patch to refactor memory context creation.

Emre Hasegeli sent in a patch to fix a potential endless loop in polygon @> polygon.