Correctifs appliqués

Robert Haas pushed:

Peter Eisentraut pushed:

Álvaro Herrera pushed:

Tom Lane pushed:

  • Improve regression test coverage of regress.c. It's a bit silly to have test functions that aren't tested, so test them. In passing, rename int44in/int44out to city_budget_in/_out so that they match how the regression tests use them. Also, fix city_budget_out so that it emits the format city_budget_in expects to read; otherwise we'd have dump/reload failures when testing pg_dump against the regression database. (We avoided that in the past only because no data of type city_budget was actually stored anywhere.) Discussion:
  • Remove unused functions in regress.c. This patch removes five functions that presumably were once used in the regression tests, but haven't been so used in many years. Nonetheless we've been wasting maintenance effort on them (e.g., by converting them to V1 function protocol). I see no reason to think that reviving them would add any useful test coverage, so drop 'em. In passing, mark regress_lseg_construct static, since it's not called from outside this file. Discussion:
  • Prevent dangling-pointer access when update trigger returns old tuple. A before-update row trigger may choose to return the "new" or "old" tuple unmodified. ExecBRUpdateTriggers failed to consider the second possibility, and would proceed to free the "old" tuple even if it was the one returned, leading to subsequent access to already-deallocated memory. In debug builds this reliably leads to an "invalid memory alloc request size" failure; in production builds it might accidentally work, but data corruption is also possible. This is a very old bug. There are probably a couple of reasons it hasn't been noticed up to now. It would be more usual to return NULL if one wanted to suppress the update action; returning "old" is significantly less efficient since the update will occur anyway. Also, none of the standard PLs would ever cause this because they all returned freshly-manufactured tuples even if they were just copying "old". But commit 4b93f5799 changed that for plpgsql, making it possible to see the bug with a plpgsql trigger. Still, this is certainly legal behavior for a trigger function, so it's ExecBRUpdateTriggers's fault not plpgsql's. It seems worth creating a test case that exercises returning "old" directly with a C-language trigger; testing this through plpgsql seems unreliable because its behavior might change again. Report and fix by Rushabh Lathia; regression test case by me. Back-patch to all supported branches. Discussion:
  • Revert renaming of int44in/int44out. This seemed like a good idea in commit be42eb9d6, but it causes more trouble than it's worth for cross-branch upgrade testing. Discussion:
  • Use the correct tuplestore read pointer in a NamedTuplestoreScan. Tom Kazimiers reported that transition tables don't work correctly when they are scanned by more than one executor node. That's because commit 18ce3a4ab allocated separate read pointers for each executor node, as it must, but failed to make them active at the appropriate times. Repair. Thomas Munro Discussion:
  • Fix up ecpg's configuration so it handles "long long int" in MSVC builds. Although configure-based builds correctly define HAVE_LONG_LONG_INT when appropriate (in both pg_config.h and ecpg_config.h), builds using the MSVC scripts failed to do so. This currently has no impact on the backend, since it uses that symbol nowhere; but it does prevent ecpg from supporting "long long int". Fix that. Also, adjust so that in the constructed ecpg_config.h file, the "#if (_MSC_VER > 1200)" covers only the LONG_LONG_INT-related #defines, not the whole file. AFAICS this was a thinko on somebody's part: ENABLE_THREAD_SAFETY should always be defined in Windows builds, and in branches using USE_INTEGER_DATETIMES, the setting of that shouldn't depend on the compiler version either. If I'm wrong, I imagine the buildfarm will say so. Per bug #15080 from Jonathan Allen; issue diagnosed by Michael Meskes and Andrew Gierth. Back-patch to all supported branches. Discussion:
  • Remove restriction on SQL block length in isolationtester scanner. specscanner.l had a fixed limit of 1024 bytes on the length of individual SQL stanzas in an isolation test script. People are starting to run into that, so fix it by making the buffer resizable. Once we allow this in HEAD, it seems inevitable that somebody will try to back-patch a test that exceeds the old limit, so back-patch this change as a preventive measure. Daniel Gustafsson Discussion:
  • Rename base64 routines to avoid conflict with Solaris built-in functions. Solaris 11.4 has built-in functions named b64_encode and b64_decode. Rename ours to something else to avoid the conflict (fortunately, ours are static so the impact is limited). One could wish for less duplication of code in this area, but that would be a larger patch and not very suitable for back-patching. Since this is a portability fix, we want to put it into all supported branches. Report and initial patch by Rainer Orth, reviewed and adjusted a bit by Michael Paquier Discussion:
  • Doc: remove duplicate poly_ops row from SP-GiST opclass table. Commit ff963b393 added two identical copies of this row. Dagfinn Ilmari Mannsåker Discussion:
  • Remove redundant IndexTupleDSize macro. Use IndexTupleSize everywhere, instead. Also, remove IndexTupleSize's internal typecast, as that's not really needed and might mask coding errors. Change some pointer variable datatypes in the call sites to compensate for that and make it clearer what we're assuming. Ildar Musin, Robert Haas, Stephen Frost Discussion:
  • Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
  • Last-minute updates for release notes. Security: CVE-2018-1058
  • Schema-qualify references in test_ddl_deparse test script. This omission seems to be what is causing buildfarm failures on crake. Security: CVE-2018-1058
  • Fix format_type() to restore its old behavior. Commit a26116c6c accidentally changed the behavior of the SQL format_type() function while refactoring. For the reasons explained in that function's comment, a NULL typemod argument should behave differently from a -1 argument. Since we've managed to break this, add a regression test memorializing the intended behavior. In passing, be consistent about the type of the "flags" parameter. Noted by Rushabh Lathia, though I revised the patch some more. Discussion:
  • Remove out-of-date comment about formrdesc(). formrdesc's comment listed the specific catalogs it is called for, but the list was out of date. Rather than jumping back onto that maintenance treadmill, let's just remove the list. It tells the reader nothing that can't be learned quickly and more reliably by searching relcache.c for callers of formrdesc(). Oversight noted by Kyotaro Horiguchi. Discussion:
  • Fix IOS planning when only some index columns can return an attribute. Since 9.5, it's possible that some but not all columns of an index support returning the indexed value for index-only scans. If the same indexed column appears in index columns that behave both ways, check_index_only() supposed that it'd be OK to do an index-only scan testing that column; but that fails if we have to recheck the indexed condition on one of the columns that doesn't support this. In principle we could make this work by remapping the recheck expressions to pull the value from a column that does support returning the indexed value. But such cases are so weird and rare that, at least for now, it doesn't seem worth the trouble. Instead, just teach check_index_only that a value is returnable only if all the index columns containing it are returnable, rather than any of them. Per report from David Pereiro Lagares. Back-patch to 9.5 where the possibility of this situation appeared. Kyotaro Horiguchi Discussion:
  • Use ereport not elog for some corrupt-HOT-chain reports. These errors have been seen in the field in corrupted-data situations. It seems worthwhile to report them with ERRCODE_DATA_CORRUPTED, rather than the generic ERRCODE_INTERNAL_ERROR, for the benefit of log monitoring and tools like amcheck. However, use errmsg_internal so that the text strings still aren't translated; it seems unlikely to be worth translators' time to do so. Back-patch to 9.3, like the predecessor commit d70cf811f that introduced these elog calls originally (replacing Asserts). Peter Geoghegan Discussion:
  • Make gistvacuumcleanup() count the actual number of index tuples. Previously, it just returned the heap tuple count, which might be only an estimate, and would be completely the wrong thing if the index is partial. Since this function scans every index page anyway to find free pages, it's practically free to count the surviving index tuples. Let's do that and return an accurate count. This is easily visible as a wrong reltuples value for a partial GiST index following VACUUM, so back-patch to all supported branches. Andrey Borodin, reviewed by Michail Nikolaev Discussion:
  • Fix pgbench TAP test to work in VPATH builds. Previously, it'd try to create log files under the source directory not the build directory. This fell over if the source isn't writable by the building user. Fabien Coelho Discussion:
  • Fix VM buffer pin management in heap_lock_updated_tuple_rec(). Sloppy coding in this function could lead to leaking a VM buffer pin, or to attempting to free the same pin twice. Repair. While at it, reduce the code's tendency to free and reacquire the same page pin. Back-patch to 9.6; before that, this routine did not concern itself with VM pages. Amit Kapila and Tom Lane Discussion:
  • Minor cleanup in Separate out the pg_attribute logic of into its own function. Drop unnecessary "defined $catalog->{data}" check. This both narrows and shortens the data writing loop of the script. There is no functional change (the emitted files are the same as before). John Naylor Discussion:
  • Trivial adjustments in preparation for bootstrap data conversion. Rationalize a couple of macro names: * In catalog/pg_init_privs.h, rename Anum_pg_init_privs_privs to Anum_pg_init_privs_initprivs to match the column's actual name. * In ecpg, rename ZPBITOID to BITOID to match catalog/pg_type.h. This reduces reader confusion, and will allow us to generate these macros automatically in future. In catalog/pg_tablespace.h, fix the ordering of related DATA and #define lines to agree with how it's done elsewhere. This has no impact today, but simplifies life for the bootstrap data conversion scripts. John Naylor Discussion:
  • Fix assorted issues in convert_to_scalar(). If convert_to_scalar is passed a pair of datatypes it can't cope with, its former behavior was just to elog(ERROR). While this is OK so far as the core code is concerned, there's extension code that would like to use scalarltsel/scalargtsel/etc as selectivity estimators for operators that work on non-core datatypes, and this behavior is a show-stopper for that use-case. If we simply allow convert_to_scalar to return FALSE instead of outright failing, then the main logic of scalarltsel/scalargtsel will work fine for any operator that behaves like a scalar inequality comparison. The lack of conversion capability will mean that we can't estimate to better than histogram-bin-width precision, since the code will effectively assume that the comparison constant falls at the middle of its bin. But that's still a lot better than nothing. (Someday we should provide a way for extension code to supply a custom version of convert_to_scalar, but today is not that day.) While poking at this issue, we noted that the existing code for handling type bytea in convert_to_scalar is several bricks shy of a load. It assumes without checking that if the comparison value is type bytea, the bounds values are too; in the worst case this could lead to a crash. It also fails to detoast the input values, so that the comparison result is complete garbage if any input is toasted out-of-line, compressed, or even just short-header. I'm not sure how often such cases actually occur --- the bounds values, at least, are probably safe since they are elements of an array and hence can't be toasted. But that doesn't make this code OK. Back-patch to all supported branches, partly because author requested that, but mostly because of the bytea bugs. The change in API for the exposed routine convert_network_to_scalar() is theoretically a back-patch hazard, but it seems pretty unlikely that any third-party code is calling that function directly. Tomas Vondra, with some adjustments by me Discussion:

Andres Freund pushed:

Noah Misch pushed:

  • Document security implications of search_path and the public schema. The ability to create like-named objects in different schemas opens up the potential for users to change the behavior of other users' queries, maliciously or accidentally. When you connect to a PostgreSQL server, you should remove from your search_path any schema for which a user other than yourself or superusers holds the CREATE privilege. If you do not, other users holding CREATE privilege can redefine the behavior of your commands, causing them to perform arbitrary SQL statements under your identity. "SET search_path = ..." and "SELECT pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one can use either as the first command of a session. As special exceptions, the following client applications behave as documented regardless of search_path settings and schema privileges: clusterdb createdb createlang createuser dropdb droplang dropuser ecpg (not programs it generates) initdb oid2name pg_archivecleanup pg_basebackup pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb vacuumlo. Not included are core client programs that run user-specified SQL commands, namely psql and pgbench. PostgreSQL encourages non-core client applications to do likewise. Document this in the context of libpq connections, psql connections, dblink connections, ECPG connections, extension packaging, and schema usage patterns. The principal defense for applications is "SELECT pg_catalog.set_config('search_path', '', false)", and the principal defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC". Either one is sufficient to prevent attack. After a REVOKE, consider auditing the public schema for objects named like pg_catalog objects. Authors of SECURITY DEFINER functions use some of the same defenses, and the CREATE FUNCTION reference page already covered them thoroughly. This is a good opportunity to audit SECURITY DEFINER functions for robust security practice. Back-patch to 9.3 (all supported versions). Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy Sharoglazov. Security: CVE-2018-1058
  • Empty search_path in Autovacuum and non-psql/pgbench clients. This makes the client programs behave as documented regardless of the connect-time search_path and regardless of user-created objects. Today, a malicious user with CREATE permission on a search_path schema can take control of certain of these clients' queries and invoke arbitrary SQL functions under the client identity, often a superuser. This is exploitable in the default configuration, where all users have CREATE privilege on schema "public". This changes behavior of user-defined code stored in the database, like pg_index.indexprs and pg_extension_config_dump(). If they reach code bearing unqualified names, "does not exist" or "no schema has been selected to create in" errors might appear. Users may fix such errors by schema-qualifying affected names. After upgrading, consider watching server logs for these errors. The --table arguments of src/bin/scripts clients have been lax; for example, "vacuumdb -Zt pg_am\;CHECKPOINT" performed a checkpoint. That now fails, but for now, "vacuumdb -Zt 'pg_am(amname);CHECKPOINT'" still performs a checkpoint. Back-patch to 9.3 (all supported versions). Reviewed by Tom Lane, though this fix strategy was not his first choice. Reported by Arseniy Sharoglazov. Security: CVE-2018-1058

Fujii Masao pushed:

Magnus Hagander pushed:

Correctifs en attente

Ashutosh Bapat sent in another revision of a patch to improve the partition matching algorithm for partition-wise JOINs.

Ashutosh Bapat sent in a patch to optimize nested ConvertRowtypeExpr execution.

Ildus Kurbangaliev sent in another revision of a patch to implement custom compression methods.

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON functions.

Nikita Glukhov sent in another revision of a patch to implement SQL/JSON JSON_TABLE.

Peter Eisentraut sent in another revision of a patch to add an ssl_passphrase_command setting, which enables specifying an external command for prompting for or otherwise obtaining passphrases for SSL key files.

Chapman Flack sent in a patch to add a regression test to numeric.sql that bits aren't lost casting from float[48] to numeric and updates float4_numeric and float8_numeric in a way that makes the test pass.

Takayuki Tsunakawa sent in two more revisions of a patch to fix a bug where pg_rewind takes a long time because it mistakenly copies data files.

Thomas Munro sent in another revision of a patch to enable parallel query with SERIALIZABLE isolation and enable the read-only SERIALIZABLE optimization for parallel query.

Nikita Glukhov sent in another revision of a patch to implement JSONPATH.

Claudio Freire sent in five more revisions of a patch to update the FSM more frequently during VACUUM.

Stephen Frost sent in a patch to rewrite the pg_dump TAP tests.

Jeevan Chalke and Rafia Sabih traded patches to implement partition-wise aggregation/grouping.

Amit Langote and Robert Haas traded patches to prune partitions faster.

Atsushi Torikoshi sent in a patch to fix a typo in walsender.c.

Etsuro Fujita sent in another revision of a patch to support tuple routing to foreign partitions.

David Steele sent in two more revisions of a patch to exclude unlogged tables from base backups.

Victor Wagner sent in two revisions of a patch to fix make variable processing in

Andrew Dunstan sent in two more revisions of a patch to speed up ALTER TABLE ... ADD COLUMN ... DEFAULT.

Pavan Deolasee sent in another revision of a patch to implement MERGE.

Amit Langote and David Rowley traded patches to prune partitions at runtime.

David Steele sent in another revision of a patch to make it possible to allow group filesystem access.

Nikita Glukhov sent in a patch to implement opclass parameters.

Alexander Korotkov sent in a patch to add a GUC for "cleanup indexes" threshold.

Ivan Kartyshov sent in another revision of a patch to fix some issues that crop up in long transactions on hot standby feedback replica.

Nikita Zhuchkov sent in a patch to create direct casts from numeric types to bool.

Antonin Houska sent in another revision of a patch to implement aggregate pushdown.

Peter Eisentraut sent in another revision of a patch to handle heap rewrites even better in logical decoding.

Peter Eisentraut sent in a patch to PL/pgSQL: to enable nested CALL with transactions.

Peter Eisentraut sent in a patch to support SET TRANSACTION to PL/pgsql.

Peter Eisentraut sent in a patch to support INOUT parameters in procedures in PL/pgsql.

Tom Lane sent in a patch to fix an issue where VPATH build from a tarball fails with some gmake versions.

Amit Langote sent in a patch to fix an issue where inserts into partitioned table may cause a crash.

Anastasia Lubennikova and Nikita Glukhov traded patches to add casts from JSONB to numeric and boolean types.

Anastasia Lubennikova sent in a patch to reduce amount of WAL generated by CREATE INDEX for GiST, GIN and SP-GiST.

Michael Banck sent in a patch to enable sending parallel dump to /dev/null.

Nikhil Sontakke sent in another revision of a patch to decode two-phase transactions.

David Steele sent in a patch to add re-initialization tests for unlogged tables.

Shubham Barai sent in another revision of a patch to implement predicate locking in GIN indexes.

Nikita Glukhov sent in another revision of a patch to support kNN for SP-GiST.

Masahiko Sawada sent in a patch to change the autovacuum launcher scheduling to an "oldest table first" algorithm.

Amit Langote and Álvaro Herrera traded patches to implement ON CONFLICT DO UPDATE for partitioned tables.

Robert Haas sent in another revision of a patch to speed up processing at Gather nodes.

Etsuro Fujita sent in another revision of a patch to fix an oddity in handling of WCO constraints in postgres_fdw.

Magnus Hagander and Daniel Gustafsson traded patches to enable online enabling of checksums.

Anastasia Lubennikova sent in a patch to add a function to track shmem reinit time.

Tomas Vondra sent in two revisions of a patch to fix an issue where user-defined numeric data types were triggering ERROR: unsupported type.

Michael Banck sent in a patch to verify checksums during basebackups.

Euler Taveira de Oliveira sent in a patch to remove unused #includes from test_decoding.

Peter Eisentraut sent in a patch to enable faster testing using symlinks.

Peter Eisentraut sent in a patch to fix more format truncation issues.

Takayuki Tsunakawa sent in another revision of a patch to produce a crash dump before main() on Windows.

Takayuki Tsunakawa sent in a patch to fix a bug where pg_rewind creates corrupt WAL files, making it so that the standby cannot catch up to the primary.

Amit Langote sent in a patch to avoid RelabelType in some cases in get_partition_operator().

Arseny Sher sent in a patch to use a single snapshot for replay.

Alexander Korotkov sent in another revision of a patch to implement incremental sort.

Laurenz Albe sent in a patch to fix a bug where SHOW ALL does not honor pg_read_all_settings membership.

Peter Eisentraut sent in a patch to make CALL optional in PL/pgsql.

Daniel Vérité sent in another revision of a patch to implement a CSV output format for psql.

Nikita Glukhov sent in another revision of a patch to add an enum reloption type.

Fabien COELHO sent in another revision of a patch to add \if to pgbench.

Curt Tilmes sent in two more revisions of a patch to find additional connection service files in the pg_service.conf.d directory.

Euler Taveira de Oliveira sent in a patch to enable row filtering in logical replication.

Peter Eisentraut sent in a patch to help manage transaction isolation in procedures by implementing the SQL standard "chained transactions" feature.

Thomas Munro sent in another revision of a patch to implement synchronous replay.

David Rowley sent in three more revisions of a patch to make it possible to parallelize string_agg and array_agg.

Masahiko Sawada sent in another revision of a patch to report autovac_workitem request failure.

David Rowley sent in a patch to fix an issue where pg_dump outputs invalid syntax for partitions with bool partkeys.

Thomas Munro sent in another revision of a patch to fix costing of parallel hash joins.

Emre Hasegeli sent in another revision of a patch to improve geometric types' use of floating point types.

Alexander Kuzmenkov sent in another revision of a patch to implement full merge join on comparison clause.

Andrey Borodin sent in another revision of a patch to fix GiST stats for partial indexes.

Etsuro Fujita sent in another revision of a patch to fix a too-sensitive regression test for the PostgreSQL FDW.

Amit Kapila sent in a patch to increase the MAX_PAGE_TRANS_INFO_SLOTS constant in anticipation of zHeap.

Magnus Hagander and Tom Lane traded patches to fix some Perl code in the code base per perltidy.

Pavel Stěhule and Tomas Vondra traded patches to add additional extra checks for PL/pgsql.

David Rowley sent in another revision of a patch to add a STATISTICS option to CREATE TABLE ... LIKE ... INCLUDING.

Noriyoshi Shinoda sent in a patch to update the documentation for logical replication security.

Tomas Vondra sent in two more revisions of a patch to add a logical_work_mem to deal with logical streaming of large in-progress transactions.

Tomas Vondra sent in another revision of a patch to implement multivariate histograms and MCV lists.

Andres Freund sent in another revision of a patch to add parenthesized options syntax for ANALYZE and add a NOWAIT option to VACUUM and ANALYZE.

Tomas Vondra sent in another revision of a patch to implement BRIN multi-range indexes.

David Rowley sent in another revision of a patch to remove useless DISTINCT clauses.

Tomas Vondra sent in another revision of a patch to add support for uuid, bool, name, bpchar and anyrange types to btree_gin.

Thomas Munro sent in a patch to fix a select_parallel test failure where gather sometimes loses tuples.

David Rowley sent in another revision of a patch to disallow LEFT JOIN removal when join or base quals have volatile functions and allow LEFT JOINs to be removed in more cases.

Tom Lane sent in a patch to create an infrastructure for version-dependent tab completion in psql.