Correctifs appliqués

Tom Lane pushed:

  • Merge coding of return/exit/continue cases in plpgsql's loop statements. plpgsql's five different loop control statements contained three distinct implementations of the same (or what ought to be the same, at least) logic for handling return/exit/continue result codes from their child statements. At best, that's trouble waiting to happen, and there seems no very good reason for the coding to be so different. Refactor so that all the common logic is expressed in a single macro. Discussion: https://postgr.es/m/26314.1514670401@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/3e724aac74e8325fe48dac8a30c2a7974eff7a14
  • Improve regression tests' code coverage for plpgsql control structures. I noticed that our code coverage report showed considerable deficiency in test coverage for PL/pgSQL control statements. Notably, both exec_stmt_block and most of the loop control statements had very poor coverage of handling of return/exit/continue result codes from their child statements; and exec_stmt_fori was seriously lacking in feature coverage, having no test that exercised its BY or REVERSE features, nor verification that its overflow defenses work. Now that we have some infrastructure for plpgsql-specific test scripts, the natural thing to do is make a new script rather than further extend plpgsql.sql. So I created a new script plpgsql_control.sql with the charter to test plpgsql control structures, and moved a few existing tests there because they fell entirely under that charter. I then added new test cases that exercise the bits of code complained of above. Of the five kinds of loop statements, only exec_stmt_while's result code handling is fully exercised by these tests. That would be a deficiency as things stand, but a follow-on commit will merge the loop statements' result code handling into one implementation. So testing each usage of that implementation separately seems redundant. In passing, also add a couple test cases to plpgsql.sql to more fully exercise plpgsql's code related to expanded arrays --- I had thought that area was sufficiently covered already, but the coverage report showed a couple of un-executed code paths. Discussion: https://postgr.es/m/26314.1514670401@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/dd2243f2ade43bcad8e615e6cf4286be250e374a
  • Ensure proper alignment of tuples in HashMemoryChunkData buffers. The previous coding relied (without any documentation) on the data[] member of HashMemoryChunkData being at a MAXALIGN'ed offset. If it was not, the tuples would not be maxaligned either, leading to failures on alignment-picky machines. While there seems to be no live bug on any platform we support, this is clearly pretty fragile: any addition to or rearrangement of the fields in HashMemoryChunkData could break it. Let's remove the hazard by getting rid of the data[] member and instead using pointer arithmetic with an explicitly maxalign'ed offset. Discussion: https://postgr.es/m/14483.1514938129@sss.pgh.pa.us https://git.postgresql.org/pg/commitdiff/5dc692f78d3bee1e86d095a9e8d9242b44f78b01
  • Teach eval_const_expressions() to handle some more cases. Add some infrastructure (mostly macros) to make it easier to write typical cases for constant-expression simplification. Add simplification processing for ArrayRef, RowExpr, and ScalarArrayOpExpr node types, which formerly went unsimplified even if all their inputs were constants. Also teach it to simplify FieldSelect from a composite constant. Make use of the new infrastructure to reduce the amount of code needed for the existing ArrayExpr and ArrayCoerceExpr cases. One existing test case changes output as a result of the fact that RowExpr can now be folded to a constant. All the new code is exercised by existing test cases according to gcov, so I feel no need to add additional tests. Tom Lane, reviewed by Dmitry Dolgov Discussion: https://postgr.es/m/3be3b82c-e29c-b674-2163-bf47d98817b1@iki.fi https://git.postgresql.org/pg/commitdiff/3decd150a2d5a8f8d43010dd0c207746ba946303
  • Fix some minor errors in new PHJ code. Correct ExecParallelHashTuplePrealloc's estimate of whether the space_allowed limit is exceeded. Be more consistent about tuples that are exactly HASH_CHUNK_THRESHOLD in size (they're "small", not "large"). Neither of these things explain the current buildfarm unhappiness, but they're still bugs. Thomas Munro, per gripe by me Discussion: https://postgr.es/m/CAEepm=34PDuR69kfYVhmZPgMdy8pSA-MYbpesEN1SR+2oj3Y+w@mail.gmail.com https://git.postgresql.org/pg/commitdiff/6fcde24063047c1195d023dfa08309302987cdcf
  • Clean up tupdesc.c for recent changes. TupleDescCopy needs to have the same effects as CreateTupleDescCopy in that, since it doesn't copy constraints, it should clear the per-attribute fields associated with them. Oversight in commit cc5f81366. Since TupleDescCopy has already established the presumption that it can just flat-copy the entire attribute array in one go, propagate that approach into CreateTupleDescCopy and CreateTupleDescCopyConstr. (I'm suspicious that this would lead to valgrind complaints if we had any trailing padding in the struct, but we do not, and anyway fixing that seems like a job for a separate commit.) Add some better comments. Thomas Munro, reviewed by Vik Fearing, some additional hacking by me Discussion: https://postgr.es/m/CAEepm=0NvOGZ8B6GbQyQe2C_c2m3LKJ9w=8OMBaYRLgZ_Gw6Nw@mail.gmail.com https://git.postgresql.org/pg/commitdiff/47c6772eb7222dbfa200db4bbeba8002b96b7976
  • Tweak parallel hash join test case in hopes of improving stability. This seems to make things better on gaur, let's see what the rest of the buildfarm thinks. Thomas Munro Discussion: https://postgr.es/m/CAEepm=1uuT8iJxMEsR=jL+3zEi87DB2v0+0H9o_rUXXCZPZT3A@mail.gmail.com https://git.postgresql.org/pg/commitdiff/934c7986f4a0a6a3b606301d84b784a27c0c324b
  • Fix incorrect computations of length of null bitmap in pageinspect. Instead of using our standard macro for this calculation, this code did it itself ... and got it wrong, leading to incorrect display of the null bitmap in some cases. Noted and fixed by Maksim Milyutin. In passing, remove a uselessly duplicative error check. Errors were introduced in commit d6061f83a; back-patch to 9.6 where that came in. Maksim Milyutin, reviewed by Andrey Borodin Discussion: https://postgr.es/m/ec295792-a69f-350f-6287-25a20e8f31d5@gmail.com https://git.postgresql.org/pg/commitdiff/39cfe86195f0b5cbc5fbe8d4e3aa6e2b0e322d0b
  • Fix new test case to not be endian-dependent. Per buildfarm. Discussion: https://postgr.es/m/ec295792-a69f-350f-6287-25a20e8f31d5@gmail.com https://git.postgresql.org/pg/commitdiff/18869e202b74f36d504c5c3c7d9db9c186039eba
  • Rewrite ConditionVariableBroadcast() to avoid live-lock. The original implementation of ConditionVariableBroadcast was, per its self-description, "the dumbest way possible". Thomas Munro found out it was a bit too dumb. An awakened process may immediately re-queue itself, if the specific condition it's waiting for is not yet satisfied. If this happens before ConditionVariableBroadcast is able to see the wait queue as empty, then ConditionVariableBroadcast will re-awaken the same process, repeating the cycle. Given unlucky timing this back-and-forth can repeat indefinitely; loops lasting thousands of seconds have been seen in testing. To fix, add our own process to the end of the wait queue to serve as a sentinel, and exit the broadcast loop once our process is not there anymore. There are various special considerations described in the comments, the principal disadvantage being that wakers can no longer be sure whether they awakened a real waiter or just a sentinel. But in practice nobody pays attention to the result of ConditionVariableSignal or ConditionVariableBroadcast anyway, so that problem seems hypothetical. Back-patch to v10 where condition_variable.c was introduced. Tom Lane and Thomas Munro Discussion: https://postgr.es/m/CAEepm=0NWKehYw7NDoUSf8juuKOPRnCyY3vuaSvhrEWsOTAa3w@mail.gmail.com https://git.postgresql.org/pg/commitdiff/aced5a92bf46532466417ab485bc94006cf60d91
  • Reorder steps in ConditionVariablePrepareToSleep for more safety. In the admittedly-very-unlikely case that AddWaitEventToSet fails, ConditionVariablePrepareToSleep would error out after already having set cv_sleep_target, which is probably bad, and after having already set cv_wait_event_set, which is very bad. Transaction abort might or might not clean up cv_sleep_target properly; but there is nothing that would be aware that the WaitEventSet wasn't fully constructed, so that all future condition variable sleeps would be broken. We can easily guard against these hazards with slight restructuring. Back-patch to v10 where condition_variable.c was introduced. Discussion: https://postgr.es/m/CAEepm=0NWKehYw7NDoUSf8juuKOPRnCyY3vuaSvhrEWsOTAa3w@mail.gmail.com https://git.postgresql.org/pg/commitdiff/3cac0ec85992829c160bdd8a370dd4676d42f58c
  • Remove return values of ConditionVariableSignal/Broadcast. In the wake of commit aced5a92b, the semantics of these results are a bit squishy: we can tell whether we signaled some other process(es), but we do not know which ones were real waiters versus mere sentinels for ConditionVariableBroadcast operations. It does not help much that ConditionVariableBroadcast will attempt to pass on the signal to the next real waiter, because (a) there might not be one, and (b) that will only happen awhile later, anyway. So these results could overstate how much effect the calls really had. However, no existing caller of either function pays any attention to its result value, so it seems reasonable to just define that as a required property of a correct algorithm. To encourage correctness and save some tiny number of cycles, change both functions to return void. Patch by me, per an observation by Thomas Munro. No back-patch, since if any third parties happen to be using these functions, they might not appreciate an API break in a minor release. Discussion: https://postgr.es/m/CAEepm=0NWKehYw7NDoUSf8juuKOPRnCyY3vuaSvhrEWsOTAa3w@mail.gmail.com https://git.postgresql.org/pg/commitdiff/ccf312a4488ab8bb38dfd87168bf8915045d1a82

Noah Misch pushed:

Andres Freund pushed:

Peter Eisentraut pushed:

  • Don't cast between GinNullCategory and bool. The original idea was that we could use an isNull-style bool array directly as a GinNullCategory array. However, the existing code already acknowledges that that doesn't really work, because of the possibility that bool as currently defined can have arbitrary bit patterns for true values. So it has to loop through the nullFlags array to set each bool value to an acceptable value. But if we are looping through the whole array anyway, we might as well build a proper GinNullCategory array instead and abandon the type casting. That makes the code much safer in case bool is ever changed to something else. Reviewed-by: Michael Paquier <michael.paquier@gmail.com> https://git.postgresql.org/pg/commitdiff/438036264a3b71eaf39b2d2eeca67237ed38ca51
  • Define LDAPS_PORT if it's missing and disable implicit LDAPS on Windows. Some versions of Windows don't define LDAPS_PORT. Also, Windows' ldap_sslinit() is documented to use LDAPS even if you said secure=0 when the port number happens to be 636 or 3269. Let's avoid using the port number to imply that you want LDAPS, so that connection strings have the same meaning on Windows and Unix. Author: Thomas Munro Discussion: https://postgr.es/m/CAEepm%3D23B7GV4AUz3MYH1TKpTv030VHxD2Sn%2BLYWDv8d-qWxww%40mail.gmail.com https://git.postgresql.org/pg/commitdiff/3ad2afc2e98fc85d5cf9529d84265b70acc0b13d
  • Allow ldaps when using ldap authentication. While ldaptls=1 provides an RFC 4513 conforming way to do LDAP authentication with TLS encryption, there was an earlier de facto standard way to do LDAP over SSL called LDAPS. Even though it's not enshrined in a standard, it's still widely used and sometimes required by organizations' network policies. There seems to be no reason not to support it when available in the client library. Therefore, add support when using OpenLDAP 2.4+ or Windows. It can be configured with ldapscheme=ldaps or ldapurl=ldaps://... Add tests for both ways of requesting LDAPS and a test for the pre-existing ldaptls=1. Modify the 001_auth.pl test for "diagnostic messages", which was previously relying on the server rejecting ldaptls=1. Author: Thomas Munro Reviewed-By: Peter Eisentraut Discussion: https://postgr.es/m/CAEepm=1s+pA-LZUjQ-9GQz0Z4rX_eK=DFXAF1nBQ+ROPimuOYQ@mail.gmail.com https://git.postgresql.org/pg/commitdiff/35c0754fadca8010955f6b10cb47af00bdbe1286
  • Refactor channel binding code to fetch cbind_data only when necessary. As things stand now, channel binding data is fetched from OpenSSL and saved into the SCRAM exchange context for any SSL connection attempted for a SCRAM authentication, resulting in data fetched but not used if no channel binding is used or if a different channel binding type is used than what the data is here for. Refactor the code in such a way that binding data is fetched from the SSL stack only when a specific channel binding is used for both the frontend and the backend. In order to achieve that, save the libpq connection context directly in the SCRAM exchange state, and add a dependency to SSL in the low-level SCRAM routines. This makes the interface in charge of initializing the SCRAM context cleaner as all its data comes from either PGconn* (for frontend) or Port* (for the backend). Author: Michael Paquier <michael.paquier@gmail.com> https://git.postgresql.org/pg/commitdiff/f3049a603a7950f313b33ab214f11563c66dc069
  • Implement channel binding tls-server-end-point for SCRAM. This adds a second standard channel binding type for SCRAM. It is mainly intended for third-party clients that cannot implement tls-unique, for example JDBC. Author: Michael Paquier <michael.paquier@gmail.com> https://git.postgresql.org/pg/commitdiff/d3fb72ea6de58d285e278459bca9d7cdf7f6a38b
  • Fix build with older OpenSSL versions. Apparently, X509_get_signature_nid() is only in fairly new OpenSSL versions, so use the lower-level interface it is built on instead. https://git.postgresql.org/pg/commitdiff/ac3ff8b1d8f98da38c53a701e6397931080a39cf
  • Add missing includes. <openssl/x509.h> is necessary to look into the X509 struct, used by ac3ff8b1d8f98da38c53a701e6397931080a39cf. https://git.postgresql.org/pg/commitdiff/1834c1e432d22f9e186950c7dd8598958776e016
  • Another attempt at fixing build with various OpenSSL versions. It seems we can't easily work around the lack of X509_get_signature_nid(), so revert the previous attempts and just disable the tls-server-end-point feature if we don't have it. https://git.postgresql.org/pg/commitdiff/054e8c6cdb7f4261869e49d3ed7705cca475182e

Bruce Momjian pushed:

Álvaro Herrera pushed:

Andrew Dunstan pushed:

  • Fix use of config-specific libraries for Windows OpenSSL. Commit 614350a3 allowed for an different builds of OpenSSL libraries on Windows, but ignored the fact that the alternative builds don't have config-specific libraries. This patch fixes the Solution file to ask for the correct libraries. per offline discussions with Leonardo Cecchi and Marco Nenciarini, Backpatch to all live branches. https://git.postgresql.org/pg/commitdiff/99d5a3ffb9fe61a5a8b01a4759d93c627f018923

Robert Haas pushed:

Simon Riggs pushed:

Correctifs en attente

Jeevan Chalke sent in another revision of a patch to implement partition-wise aggregation/grouping.

Fabien COELHO sent in two more revisions of a patch to add a --random-seed option to pgbench.

Amit Kapila sent in another revision of a patch to ensure that parallel paths include tlist cost.

Shubham Barai sent in another revision of a patch to implement predicate locking in GIN indexes.

Marco Nenciarini sent in another revision of a patch to ensure that session_replication_role = replica with TRUNCATE.

David Steele sent in another revision of a patch to implement a configurable file mode mask.

Amit Khandekar sent in three more revisions of a patch to ensure that UPDATEs to partitioned tables that would have the effect of moving tuples from one partition to another actually do so.

Rushabh Lathia and Peter Geoghegan traded patches to add parallel tuplesort for parallel B-Tree index creation.

Jesper Pedersen sent in a patch documenting how indexes automatically created on partitioned tables are named.

Nikita Glukhov sent in two more revisions of a patch to implement SQL/JSON.

Tomas Vondra sent in another revision of a patch to implement controls on the resources logical replication can consume.

Dagfinn Ilmari Mannsåker sent in a patch to remove a duplicate poly_ops row from the SP-GiST opclass table.

Remi Colinet sent in another revision of a patch to make block and file size for WAL and relations defined at cluster creation.

Haribabu Kommi sent in three more revisions of a patch to make pg_stat_wal_receiver display connected hosts.

Haribabu Kommi sent in another revision of a patch to implement pluggable storage.

Andrew Dunstan sent in two revisions of a patch to implement TZH and TZM format specifiers in to_timestamp and friends.

Tomas Vondra sent in another revision of a patch to implement multivariate histograms and MCV lists.

Fabien COELHO sent in another revision of a patch to add more functions and operators to pgbench.

Thomas Munro sent in a patch to fix warnings about pg_attribute_always_inline.

Fabien COELHO sent in another revision of a patch to add \if support to pgbench.

Emre Hasegeli sent in another revision of a patch to improve the native geometric types.

Dmitry Dolgov sent in another revision of a patch to implement generic type subscripting and use same in arrays and JSONB.

Artur Zakirov sent in another revision of a patch to create a more flexible configuration for full-text search.

Alexander Korotkov sent in another revision of a patch to fix some pg_trgm word_similarity inconsistencies.

Tom Lane sent in another revision of a patch to improve OR conditions on joined columns, which commonly occur in queries on star schemas.

Jing Wang sent in another revision of a patch to implement a 'prefer-read' feature in libpq.

Alexander Korotkov sent in two more revisions of a patch to implement incremental sort.

Álvaro Herrera sent in two more revisions of a patch to implement local indexes for partitioned tables.

Vaishnavi Prabakaran sent in another revision of a patch to implement pipelining batch support for libpq.

Haribabu Kommi sent in another revision of a patch to refactor database attributes between pg_dump and pg_dumpall.

Peter Eisentraut sent in another revision of a patch to implement transaction control in procedures.

Peter Eisentraut sent in a patch to fix ssl tests for when tls-server-end-point is not supported.

Bruce Momjian sent in a patch to fix an invalid pg_upgrade error message during live check.

Álvaro Herrera sent in another revision of a patch to allow indexes on partitioned tables to be unique.

Marco Nenciarini sent in three more revisions of a patch to implement logical decoding of TRUNCATE.

David Rowley sent in three more revisions of a patch to implement runtime partition pruning.

David Rowley sent in another revision of a patch to remove useless DISTINCT clauses where they're easily identified as such.

David Rowley sent in another revision of a patch to remove [Merge]Append nodes which contain a single subpath.

Michaël Paquier sent in another revision of a patch to create backup history files for backups taken from standbys.

Tom Lane sent in another revision of a patch to fix an Oracle-compatible instr function in the documentation.

Pavel Stěhule sent in another revision of a patch to add some new checks to PL/pgsql.

Simon Riggs sent in another revision of a patch to surface replication status for logical replication.

Tomas Vondra sent in another revision of a patch to implement BRIN multi-range indexes.

Artur Zakirov sent in another revision of a patch to enable sharing Ispell dictionaries among backends.

Andrew Dunstan sent in another revision of a patch to implement JSONPATH.