Correctifs appliqués

Tom Lane pushed:

  • Fix unportable disregard of alignment requirements in RADIUS code. The compiler is entitled to store a char[] local variable with no particular alignment requirement. Our RADIUS code cavalierly took such a local variable and cast its address to a struct type that does have alignment requirements. On an alignment-picky machine this would lead to bus errors. To fix, declare the local variable honestly, and then cast its address to char * for use in the I/O calls. Given the lack of field complaints, there must be very few if any people affected; but nonetheless this is a clear portability issue, so back-patch to all supported branches. Noted while looking at a Coverity complaint in the same code.
  • Fix some minor resource leaks in PerformRadiusTransaction(). Failure to free serveraddrs pointed out by Coverity, failure to close socket noted by code-reading. These bugs seem to be quite old, but given the low probability of taking these error-exit paths and the minimal consequences of the leaks (since the process would presumably exit shortly anyway), it doesn't seem worth back-patching. Michael Paquier and Tom Lane
  • Fix typos in logical replication support for initial data copy. Fix an incorrect assert condition (noted by Coverity), and spell the new name of the function correctly. Typos introduced in commit 7c4f52409. Michael Paquier
  • Use ExecPrepareExpr in place of ExecPrepareCheck where appropriate. Change one more place where ExecInitCheck/ExecPrepareCheck's insistence on getting implicit-AND-format quals wasn't really helpful, because the caller had to do make_ands_implicit() for no reason that it cared about. Using ExecPrepareExpr directly simplifies the code and saves cycles. The only remaining use of these functions is to process resultRelInfo->ri_PartitionCheck quals. However, implicit-AND format does seem to be what we want for that, so leave it alone.
  • Improve performance of ExecEvalWholeRowVar. In commit b8d7f053c, we needed to fix ExecEvalWholeRowVar to not change the state of the slot it's copying. The initial quick hack at that required two rounds of tuple construction, which is not very nice. To fix, add another primitive to tuptoaster.c that does precisely what we need. (I initially tried to do this by refactoring one of the existing functions into two pieces; but it looked like that might hurt performance for the existing case, and the amount of code that could be shared is not very large, so I gave up on that.) Discussion:
  • Show ignored constants as "$N" rather than "?" in pg_stat_statements. The trouble with the original choice here is that "?" is a valid (and indeed used) operator name, so that you could end up with ambiguous statement texts like "SELECT ? ? ?". With this patch, you instead see "SELECT $1 ? $2", which seems significantly more readable. The numbers used for this purpose begin after the last actual $N parameter in the particular query. The conflict with external parameters has its own potential for confusion of course, but it was agreed to be an improvement over the previous behavior. Lukas Fittl Discussion:
  • Suppress implicit-conversion warnings seen with newer clang versions. We were assigning values near 255 through "char *" pointers. On machines where char is signed, that's not entirely kosher, and it's reasonable for compilers to warn about it. A better solution would be to change the pointer type to "unsigned char *", but that would be vastly more invasive. For the moment, let's just apply this simple backpatchable solution. Aleksander Alekseev Discussion: Discussion:
  • Make new expression eval code reject references to dropped columns. Formerly, a Var referencing an already-dropped column was allowed and would always produce a NULL value. However, that behavior was implemented in slot_getattr which the new expression code doesn't use; thus there is now a risk of returning theoretically-deleted data. We had regression test cases that purported to exercise this, but they failed to expose any problem, apparently because plpgsql filters the dropped column and produces an output tuple that has a NULL there already. Ideally the DROP or ALTER attempt in these test cases would get rejected due to dependency checks; but until that happens, let's modify the behavior so that we fail the query during executor start. This was already true for the related case of a column having changed type underneath us, and there's no obvious reason why we need to be laxer for dropped columns. In passing, adjust the error messages in CheckVarSlotCompatibility to include the composite type name. In the cases shown in the regression tests this is always just "record", but it should be more useful in actual stale-plan cases, where the slot tupdesc would be a table's tupdesc directly. Discussion:
  • Support \if ... \elif ... \else ... \endif in psql scripting. This patch adds nestable conditional blocks to psql. The control structure feature per se is complete, but the boolean expressions understood by \if and \elif are pretty primitive; basically, after variable substitution and backtick expansion, the result has to be "true" or "false" or one of the other standard spellings of a boolean value. But that's enough for many purposes, since you can always do the heavy lifting on the server side; and we can extend it later. Along the way, pay down some of the technical debt that had built up around psql/command.c: * Refactor exec_command() into a function per command, instead of being a 1500-line monstrosity. This makes the file noticeably longer because of repetitive function header/trailer overhead, but it seems much more readable. * Teach psql_get_variable() and psqlscanslash.l to suppress variable substitution and backtick expansion on the basis of the conditional stack state, thereby allowing removal of the OT_NO_EVAL kluge. * Fix the no-doubt-once-expedient hack of sometimes silently substituting mainloop.c's previous_buf for query_buf when calling HandleSlashCmds. (It's a bit remarkable that commands like \r worked at all with that.) Recall of a previous query is now done explicitly in the slash commands where that should happen. Corey Huinker, reviewed by Fabien Coelho, further hacking by me Discussion:
  • Fix broken markup. Per buildfarm.
  • For foreign keys, check REFERENCES privilege only on the referenced table. We were requiring that the user have REFERENCES permission on both the referenced and referencing tables --- but this doesn't seem to have any support in the SQL standard, which says only that you need REFERENCES permission on the referenced table. And ALTER TABLE ADD FOREIGN KEY has already checked that you own the referencing table, so the check could only fail if a table owner has revoked his own REFERENCES permission. Moreover, the symmetric interpretation of this permission is unintuitive and confusing, as per complaint from Paul Jungwirth. So let's drop the referencing-side check. In passing, do a bit of wordsmithing on the GRANT reference page so that all the privilege types are described in similar fashion. Discussion:
  • Fix unstable regression test result. Commit e306df7f9 added a test case that depends on "the" being a stop word, which it is not in non-English locales. Since the point of the test is to check stopword behavior, fix by forcibly selecting the 'english' configuration. Per buildfarm.
  • Fix unstable regression test result. Whoops, missed that same test was made for json as well as jsonb.
  • Allow psql variable substitution to occur in backtick command strings. Previously, text between backquotes in a psql metacommand's arguments was always passed to the shell literally. That considerably hobbles the usefulness of the feature for scripting, so we'd foreseen for a long time that we'd someday want to allow substitution of psql variables into the shell command. IMO the addition of \if metacommands has brought us to that point, since \if can greatly benefit from some sort of client-side expression evaluation capability, and psql itself is not going to grow any such thing in time for v10. Hence, this patch. It allows :VARIABLE to be replaced by the exact contents of the named variable, while :'VARIABLE' is replaced by the variable's contents suitably quoted to become a single shell-command argument. (The quoting rules for that are different from those for SQL literals, so this is a bit of an abuse of the :'VARIABLE' notation, but I doubt anyone will be confused.) As with other situations in psql, no substitution occurs if the word following a colon is not a known variable name. That limits the risk of compatibility problems for existing psql scripts; but the risk isn't zero, so this needs to be called out in the v10 release notes. Discussion:
  • Fix behavior of psql's \p to agree with \g, \w, etc. In commit e984ef586 I (tgl) simplified the behavior of \p to just print the current query buffer; but Daniel Vérité points out that this made it inconsistent with the behavior of \g and \w. It should print the same thing \g would execute. Fix that, and improve related comments. Daniel Vérité Discussion:

Peter Eisentraut pushed:

Robert Haas pushed:

Andrew Gierth pushed:

Teodor Sigaev pushed:

Ãlvaro Herrera pushed:

  • Rework the stats_ext test. As suggested by Tom Lane, avoid printing specific estimated cost values, because they vary across architectures; instead, verify plan shapes (in this case, HashAggregate vs. GroupAggregate), as we do in other planner tests. We can now remove expected/stats_ext_1.out. Author: Tomas Vondra
  • Fix a couple of problems in pg_get_statisticsextdef. There was a thinko whereby we tested the wrong tuple after fetching it from cache; avoid that by using generate_relation_name instead, which is simpler. Also, the statistics name was not qualified, so add that. (It could be argued that qualification should be conditional on the schema not being on search path. We can add that later, but at least this form is correct.) Author: David Rowley, Ãlvaro Herrera Discussion:
  • Fix thinko in estimate_num_groups. The code for the reworked n-distinct estimation on commit 7b504eb282 was written differently in a previous version of the patch, prior to commit; on rewriting it, we missed updating an initializer. This caused the code to (mistakenly) apply a fudge factor even in the case where a single value is applied, leading to incorrect results. This means that the 'relvarcount' variable name is now wrong. Add a comment to try and make the situation clearer, and remove an incorrect comment I added. Problem noticed, and code patch, by Tomas Vondra. Additional commentary by Ãlvaro.
  • Fix uninitialized memory propagation mistakes. Valgrind complains that some uninitialized bytes are being passed around by the extended statistics code since commit 7b504eb282ca2f, as reported by Andres Freund. Silence it. Tomas Vondra submitted a patch which he verified to fix the complaints in his machine; however I messed with it a bit before pushing, so any remaining problems are likely my (Ãlvaro's) fault. Author: Tomas Vondra Discussion:
  • Remove direct uses of ItemPointer.{ip_blkid,ip_posid}. There are no functional changes here; this simply encapsulates knowledge of the ItemPointerData struct so that a future patch can change things without more breakage. All direct users of ip_blkid and ip_posid are changed to use existing macros ItemPointerGetBlockNumber and ItemPointerGetOffsetNumber respectively. For callers where that's inappropriate (because they Assert that the itempointer is is valid-looking), add ItemPointerGetBlockNumberNoCheck and ItemPointerGetOffsetNumberNoCheck, which lack the assertion but are otherwise identical. Author: Pavan Deolasee Discussion:
  • Allow DSM segments to be created as pinned. dsm_create and dsm_attach assumed that a current resource owner was always in place. Exploration with the API show that this is inconvenient: sometimes one must create a dummy resowner, create/attach the DSM, only to pin the mapping later, which is wasteful. Change create/attach so that if there is no current resowner, the dsm is effectively pinned right from the start. Discussion: Reviewed by Thomas Munro.
  • Simplify check of modified attributes in heap_update. The old coding was getting more complicated as new things were added, and it would be barely tolerable with upcoming WARM updates and other future features such as indirect indexes. The new coding incurs a small performance cost in synthetic benchmark cases, and is barely measurable in normal cases. A much larger benefit is expected from WARM, which could actually bolt its needs on top of the existing coding, but it is much uglier and bug-prone than doing it on this new code. Additional optimization can be applied on top of this, if need be. Reviewed-by: Pavan Deolasee, Amit Kapila, Mithun CY Discussion:
  • Fix expected output. Previous commit had a thinko in the expected output for new tests. Per buildfarm
  • BRIN de-summarization. When the BRIN summary tuple for a page range becomes too "wide" for the values actually stored in the table (because the tuples that were present originally are no longer present due to updates or deletes), it can be useful to remove the outdated summary tuple, so that a future summarization can install a tighter summary. This commit introduces a SQL-callable interface to do so. Author: Ãlvaro Herrera Reviewed-by: Eiji Seki Discussion:
  • BRIN auto-summarization. Previously, only VACUUM would cause a page range to get initially summarized by BRIN indexes, which for some use cases takes too much time since the inserts occur. To avoid the delay, have brininsert request a summarization run for the previous range as soon as the first tuple is inserted into the first page of the next range. Autovacuum is in charge of processing these requests, after doing all the regular vacuuming/ analyzing work on tables. This doesn't impose any new tasks on autovacuum, because autovacuum was already in charge of doing summarizations. The only actual effect is to change the timing, i.e. that it occurs earlier. For this reason, we don't go any great lengths to record these requests very robustly; if they are lost because of a server crash or restart, they will happen at a later time anyway. Most of the new code here is in autovacuum, which can now be told about "work items" to process. This can be used for other things such as GIN pending list cleaning, perhaps visibility map bit setting, both of which are currently invoked during vacuum, but do not really depend on vacuum taking place. The requests are at the page range level, a granularity for which we did not have SQL-level access; we only had index-level summarization requests via brin_summarize_new_values(). It seems reasonable to add SQL-level access to range-level summarization too, so add a function brin_summarize_range() to do that. Authors: Ãlvaro Herrera, based on sketch from Simon Riggs. Reviewed-by: Thomas Munro. Discussion:

Simon Riggs pushed:

Andres Freund pushed:

Fujii Masao pushed:

  • Simplify the example of VACUUM in documentation. Previously a detailed activity report by VACUUM VERBOSE ANALYZE was described as an example of VACUUM in docs. But it had been obsolete for a long time. For example, commit feb4f44d296b88b7f0723f4a4f3945a371276e0b updated the content of that activity report in 2003, but we had forgotten to update the example. So basically we need to update the example. But since no one cared about the details of VACUUM output and complained about that mistake for such long time, per discussion on hackers, we decided to get rid of the detailed activity report from the example and simplify it. Back-patch to all supported versions. Reported by Masahiko Sawada, patch by me. Discussion:

Magnus Hagander pushed:

Andrew Dunstan pushed:

Kevin Grittner pushed:

  • Add transition table support to plpgsql. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
  • Add infrastructure to support EphemeralNamedRelation references. A QueryEnvironment concept is added, which allows new types of objects to be passed into queries from parsing on through execution. At this point, the only thing implemented is a collection of EphemeralNamedRelation objects -- relations which can be referenced by name in queries, but do not exist in the catalogs. The only type of ENR implemented is NamedTuplestore, but provision is made to add more types fairly easily. An ENR can carry its own TupleDesc or reference a relation in the catalogs by relid. Although these features can be used without SPI, convenience functions are added to SPI so that ENRs can easily be used by code run through SPI. The initial use of all this is going to be transition tables in AFTER triggers, but that will be added to each PL as a separate commit. An incidental effect of this patch is to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
  • Try to fix breakage of sepgsql hooks by ENR patch. Turned up by buildfarm animal rhinoceros. Fixing blind. Will have to wait for next run by rhinoceros to know whether it worked.
  • Fix two undocumented parameters to functions from ENR patch. On ProcessUtility document the parameter, to match others. On CreateCachedPlan drop the queryEnv parameter. It was not referenced within the function, and had been added on the assumption that with some unknown future usage of QueryEnvironment it might be useful to do something there. We have avoided other "just in case" implementation of unused paramters, so drop it here. Per gripe from Tom Lane

Correctifs en attente

Kyotaro HORIGUCHI sent in a patch to add encoding_dumper and map_dumper.

Michaël Paquier sent in two more revisions of a patch to change detection of corrupted 2PC files as FATAL and minimize window between history file and end-of-recovery.

Nikhil Sontakke sent in another revision of a patch to speed up two-phase transactions.

Stas Kelvich and Craig Ringer traded patches to implement logical decoding of two-phase transactions.

Haribabu Kommi sent in three more revisions of a patch to implement a pg_stat_wal_write statistics view.

Ashutosh Bapat sent in two more revisions of a patch to implement partition-wise join for declaratively partitioned tables.

Kyotaro HORIGUCHI sent in a patch to fix a bug in physical replication slots.

Beena Emerson sent in another revision of a patch to allow increasing the default WAL segment size.

Amit Khandekar sent in two more revisions of a patch to enable UPDATE of partition key for declaratively partitioned tables.

Vaishnavi Prabakaran sent in two more revisions of a patch to implement batch/pipelining support for libpq.

Pavan Deolasee sent in six more revisions of a patch to implement WARM.

Mithun Cy sent in eight more revisions of a patch to expand hash indexes more efficiently.

Ashutosh Sharma sent in two more revisions of a patch to fix a failed assertion found by sqlsmith in _hash_kill_items/MarkBufferDirtyHint.

Pavel Stěhule and Surafel Temesgen traded patches to add CORRESPONDING.

Dmitry Dolgov sent in three more revisions of a patch to implement generic type subscripting.

Ashutosh Sharma sent in three more revisions of a patch to rewrite hash index scans to work a page at a time, remove redundant function _hash_step() and some of the unused members of HashScanOpaqueData, and improve the locking strategy during VACUUM in hash index.

Jan Michálek sent in four more revisions of a patch to add markdown and rst to psql's output formats.

Pavel Stěhule sent in a patch to add an xmltable doc fix and example for XMLNAMESPACES.

Pavel Stěhule and Petr Jelínek traded patches to add a plan_cache control PRAGMA to pl/pgsql.

Jeff Janes sent in a patch to fix an infelicity between free space map and visibility map.

Claudio Freire sent in another revision of a patch for VACUUM which frees the dead tuples array as early as possible.

Jim Nasby sent in a patch to fix a missing increment of vacrelstats->pinskipped_pages.

Ashutosh Bapat and Amit Langote traded patches to fix a comment in src/backend/optimizer/path/allpaths.c.

Haribabu Kommi sent in another revision of a patch to refactor handling of database attributes between pg_dump and pg_dumpall.

Kyotaro HORIGUCHI sent in a patch to show vacuum aggressiveness in autovacuum logs.

Alexander Law sent in a patch to remove trailing spaces in strings in the source code.

Daniel Gustafsson sent in a patch to use American English spelling in pg_waldump error message.

Nikhil Sontakke sent in another revision of a patch to move all am-related reloption code into src/backend/access/[am-name] and get rid of relopt_kind for custom AM.

Alexander Korotkov sent in another revision of a patch to implement incremental sort.

Masahiko Sawada sent in another revision of a patch to implement support for transactions involving multiple postgres foreign servers.

Daniel Gustafsson sent in a patch to error out more informatively when multiple TO VERSION arguments are supplied to ALTER EXTENSION.

Takeshi Ideriha sent in another revision of a patch to implement DECLARE STATEMENT setting up a connection in ECPG.

Peter Eisentraut sent in a patch to adjust min/max values when changing sequence type.

Peter Eisentraut sent in another revision of a patch to implement IDENTITY columns.

Michaël Paquier sent in three more revisions of a patch to allow interrupts on waiting standby.

Vinayak Pokale sent in another revision of a patch to implement an ANALYZE command progress checker.

Tomas Vondra sent in two more revisions of a patch to add page_checksum and bt_page_items(bytea) to page_inspect.

Ashutosh Sharma sent in a patch to fix an issue that manifested as an inconsistent page found on STANDBY server.

Takayuki Tsunakawa sent in another revision of a patch to implement huge pages on Windows.

Kuntal Ghosh sent in a patch to fix some strange parallel query behavior after OOM crashes.

David Rowley sent in two more revisions of a patch to fix functional dependencies in materialized views in light of the extended statistics recently committed.

Thomas Munro sent in another revision of a patch to implement [[Parallel] Shared] Hash.

Anastasia Lubennikova and Teodor Sigaev traded patches to add covering + unique indexes.

Michaël Paquier sent in another revision of a patch to implement SASLprep aka NFKC for SCRAM authentication.

Aleksander Alekseev sent in a patch to remove an unused argument in btree_xlog_split.

Etsuro Fujita sent in another revision of a patch to add epqpath support for foreign joins.

Masahiko Sawada sent in a patch to remove some dead code from the table sync worker.

Fabien COELHO sent in a patch to refactor psql's ef/ev and sf/sv handling functions so they're not copypasta.

Feike Steenbergen sent in a patch to fix an issue where passwords in user mappings were leaked by psql's \deu+ command.

Alexander Korotkov sent in another revision of a patch to implement a LWLock optimization for multicore Power machines.

Takayuki Tsunakawa sent in a patch to fix an issue where savepoint-related statements in a multi-command query terminates the connection unexpectedly.

Mike Palmiotto sent in two revisions of a patch to silence some sepgsql compiler warnings and add partitioned table support to sepgsql.

Petr Jelínek sent in a patch to use weaker locks when updating subscription relation state.

Dilip Kumar sent in a patch to ensure that parallel bitmapscan is exercised in regression tests.

David Rowley sent in another revision of a patch to improve performance for outer joins where the outer side is unique.