Correctifs appliqués

Heikki Linnakangas pushed:

Magnus Hagander pushed:

Michael Meskes pushed:

Fujii Masao pushed:

  • Various fixes to "ALTER ... SET/RESET" tab completions. Add: ALTER SYSTEM SET/RESET ... -> GUC variables, ALTER TABLE ... SET WITH -> OIDS, ALTER DATABASE/FUNCTION/ROLE/USER ... SET/RESET -> GUC variables, ALTER DATABASE/FUNCTION/ROLE/USER ... SET ... -> FROM CURRENT/TO, ALTER DATABASE/FUNCTION/ROLE/USER ... SET ... TO/= -> possible values, Author: Fujii Masao. Reviewed-by: Michael Paquier, Masahiko Sawada
  • Make GIN regression test stable. Commit 7f46eaf added the regression test which checks that gin_clean_pending_list() cleans up the GIN pending list and returns >0. This usually works fine. But if autovacuum comes along and cleans the list before gin_clean_pending_list() starts, the function may return 0, and then the regression test may fail. To fix the problem, this commit disables autovacuum on the target index of gin_clean_pending_list() by setting autovacuum_enabled reloption to off when creating the table. Also this commit sets gin_pending_list_limit reloption to 4MB on the target index. Otherwise when running "make installcheck" with small gin_pending_list_limit GUC, insertions of data may trigger the cleanup of pending list before gin_clean_pending_list() starts and the function may return 0. This could cause the regression test to fail. Per buildfarm member spoonbill. Reported-By: Tom Lane

Robert Haas pushed:

  • pgbench: Install guards against obscure overflow conditions. Dividing INT_MIN by -1 or taking INT_MIN modulo -1 can sometimes cause floating-point exceptions or otherwise misbehave. Fabien Coelho and Michael Paquier
  • Make all built-in lwlock tranche IDs fixed. This makes the values more stable, which seems like a good thing for anybody who needs to look at at them. Alexander Korotkov and Amit Kapila
  • postgres_fdw: Allow fetch_size to be set per-table or per-server. The default fetch size of 100 rows might not be right in every environment, so allow users to configure it. Corey Huinker, reviewed by Kyotaro Horiguchi, Andres Freund, and me.
  • Remove CustomPath's TextOutCustomPath method. You can't really do anything useful with this in the form it currently exists; among other problems, there's no way to reread whatever information might be produced when the path is output. Work is underway to replace this with a more useful and more general system of extensible nodes, but let's start by getting rid of this bit. Extracted from a larger patch by KaiGai Kohei.
  • Code review for commit dc203dc3ac40a4b02b92fb827848a547d2957153. Remove duplicate assignment. This part by Ashutosh Bapat. Remove now-obsolete comment. This part by me, although the pending join pushdown patch does something similar, and for the same reason: there's no reason to keep two lists of the things in the fdw_private structure that have to be kept in sync with each other.
  • Allow parallel custom and foreign scans. This patch doesn't put the new infrastructure to use anywhere, and indeed it's not clear how it could ever be used for something like postgres_fdw which has to send an SQL query and wait for a reply, but there might be FDWs or custom scan providers that are CPU-bound, so let's give them a way to join club parallel. KaiGai Kohei, reviewed by me.
  • Extend sortsupport for text to more opclasses. Have varlena.c expose an interface that allows the char(n), bytea, and bpchar types to piggyback on a now-generalized SortSupport for text. This pushes a little more knowledge of the bpchar/char(n) type into varlena.c than might be preferred, but that seems like the approach that creates least friction. Also speed things up for index builds that use text_pattern_ops or varchar_pattern_ops. This patch does quite a bit of renaming, but it seems likely to be worth it, so as to avoid future confusion about the fact that this code is now more generally used than the old names might have suggested. Peter Geoghegan, reviewed by Ãlvaro Herrera and Andreas Karlsson, with small tweaks by me.
  • Change the way that LWLocks for extensions are allocated. The previous RequestAddinLWLocks() method had several disadvantages. First, the locks would be in the main tranche; we've recently decided that it's useful for LWLocks used for separate purposes to have separate tranche IDs. Second, there wasn't any correlation between what code called RequestAddinLWLocks() and what code called LWLockAssign(); when multiple modules are in use, it could become quite difficult to troubleshoot problems where LWLockAssign() ran out of locks. To fix, create a concept of named LWLock tranches which can be used either by extension or by core code. Amit Kapila and Robert Haas
  • Add some additional core functions to support join pushdown for FDWs. GetExistingLocalJoinPath() is useful for handling EvalPlanQual rechecks properly, and GetUserMappingById() is needed to make sure you're using the right credentials. Shigeru Hanada, Etsuro Fujita, Ashutosh Bapat, Robert Haas
  • When modifying a foreign table, initialize tableoid field properly. Failure to do this can cause AFTER ROW triggers or RETURNING expressions that reference this field to misbehave. Etsuro Fujita, reviewed by Thom Brown
  • postgres_fdw: Avoid possible misbehavior when RETURNING tableoid column only. deparseReturningList ended up adding up RETURNING NULL to the code, but code elsewhere saw an empty list of attributes and concluded that it should not expect tuples from the remote side. Etsuro Fujita and Robert Haas, reviewed by Thom Brown
  • postgres_fdw: pgindent run. In preparation for upcoming commits.
  • Fix typo. Amit Kapila
  • Fix small goof in comment. Peter Geoghegan
  • Remove parallel-safety check from GetExistingLocalJoinPath. Commit a104a017fc5f67ff5d9c374cd831ac3948a874c2 has this check because I added it to the submitted patch before commit, but that was entirely wrongheaded, as explained to me by Ashutosh Bapat, who also wrote this patch.
  • Fix typo in comment. Michael Paquier
  • Introduce group locking to prevent parallel processes from deadlocking. For locking purposes, we now regard heavyweight locks as mutually non-conflicting between cooperating parallel processes. There are some possible pitfalls to this approach that are not to be taken lightly, but it works OK for now and can be changed later if we find a better approach. Without this, it's very easy for parallel queries to silently self-deadlock if the user backend holds strong relation locks. Robert Haas, with help from Amit Kapila. Thanks to Noah Misch and Andres Freund for extensive discussion of possible issues with this approach.
  • Introduce a new GUC force_parallel_mode for testing purposes. When force_parallel_mode = true, we enable the parallel mode restrictions for all queries for which this is believed to be safe. For the subset of those queries believed to be safe to run entirely within a worker, we spin up a worker and run the query there instead of running it in the original process. When force_parallel_mode = regress, make additional changes to allow the regression tests to run cleanly even though parallel workers have been injected under the hood. Taken together, this facilitates both better user testing and better regression testing of the parallelism code. Robert Haas, with help from Amit Kapila and Rushabh Lathia.
  • Fix parallel-safety markings for pg_upgrade functions. These establish backend-local state which will not be copied to parallel workers, so they must be marked parallel-restricted, not parallel-safe.
  • postgres_fdw: Push down joins to remote servers. If we've got a relatively straightforward join between two tables, this pushes that join down to the remote server instead of fetching the rows for each table and performing the join locally. Some cases are not handled yet, such as SEMI and ANTI joins. Also, we don't yet attempt to create presorted join paths or parameterized join paths even though these options do get tried for a base relation scan. Nevertheless, this seems likely to be a very significant win in many practical cases. Shigeru Hanada and Ashutosh Bapat, reviewed by Robert Haas, with additional review at various points by Tom Lane, Etsuro Fujita, KaiGai Kohei, and Jeevan Chalke.
  • postgres_fdw: Remove unstable regression test. Per Tom Lane and the buildfarm.
  • postgres_fdw: Remove unnecessary variable. It causes warnings in non-Assert-enabled builds. Per report from Jeff Janes.
  • Code cleanup in the wake of recent LWLock refactoring. As of commit c1772ad9225641c921545b35c84ee478c326b95e, there's no longer any way of requesting additional LWLocks in the main tranche, so we don't need NumLWLocks() or LWLockAssign() any more. Also, some of the allocation counters that we had previously aren't needed any more either. Amit Kapila
  • Specify permutations for isolation tests with "invalid" permutations. This is a necessary prerequisite for forthcoming changes to allow deadlock scenarios to be tested by the isolation tester. It is also a good idea on general principle, since these scenarios add no useful test coverage not provided by other scenarios, but do to take time to execute.
  • Modify the isolation tester so that multiple sessions can wait. This allows testing of deadlock scenarios. Scenarios that would previously have been considered invalid are now simply taken as a scenario in which more than one backend will wait.
  • Rename PGPROC fields related to group XID clearing again. Commit 0e141c0fbb211bdd23783afa731e3eef95c9ad7a introduced a new facility to reduce ProcArrayLock contention by clearing several XIDs from the ProcArray under a single lock acquisition. The names initially chosen were deemed not to be very good choices, so commit 4aec49899e5782247e134f94ce1c6ee926f88e1c renamed them. But now it seems like we still didn't get it right. A pending patch wants to add similar infrastructure for batching CLOG updates, so the names need to be clear enough to allow a new set of structure members with a related purpose. Amit Kapila
  • Add some isolation tests for deadlock detection and resolution. Previously, we had no test coverage for the deadlock detector.
  • Use separate lwlock tranches for buffer, lock, and predicate lock managers. This finishes the work - spread across many commits over the last several months - of putting each type of lock other than the named individual locks into a separate tranche. Amit Kapila
  • Make builtin lwlock tranche names consistent. Previously, we had a mix of styles. Amit Kapila
  • Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.

Ãlvaro Herrera pushed:

Teodor Sigaev pushed:

  • Fix lossy KNN GiST when ordering operator returns non-float8 value. KNN GiST with recheck flag should return to executor the same type as ordering operator, GiST detects this type by looking to return type of function which implements ordering operator. But occasionally detecting code works after replacing ordering operator function to distance support function. Distance support function always returns float8, so, detecting code get float8 instead of actual return type of ordering operator. Built-in opclasses don't have ordering operator which doesn't return non-float8 value, so, tests are impossible here, at least now. Backpatch to 9.5 where lozzy KNN was introduced. Author: Alexander Korotkov Report by: Artur Zakirov
  • Improve error reporting in format() Clarify invalid format conversion type error message and add hint. Author: Jim Nasby

Tom Lane pushed:

  • Fix pg_description entries for jsonb_to_record() and jsonb_to_recordset(). All the other jsonb function descriptions refer to the arguments as being "jsonb", but these two said "json". Make it consistent. Per bug #13905 from Petru Florin Mihancea. No catversion bump --- we can't force one in the back branches, and this isn't very critical anyway.
  • Remove unnecessary "implementation of FOO operator" DESCR() entries. Apparently at least one committer hasn't gotten the word that these do not need to be maintained by hand, since initdb will create them automatically. Noted while fixing bug #13905. No catversion bump since the post-initdb state is exactly the same either way. I don't see a need for back-patch, either.
  • Remove printQueryOpt.quote field. This field was included in the original definition of the printQueryOpt struct in commit a45195a191eec367, but it was not used anywhere in that commit, nor since then. Spotted by Dickson S. Guedes.
  • Fix IsValidJsonNumber() to notice trailing non-alphanumeric garbage. Commit e09996ff8dee3f70 was one brick shy of a load: it didn't insist that the detected JSON number be the whole of the supplied string. This allowed inputs such as "2016-01-01" to be misdetected as valid JSON numbers. Per bug #13906 from Dmitry Ryabov. In passing, be more wary of zero-length input (I'm not sure this can happen given current callers, but better safe than sorry), and do some minor cosmetic cleanup.
  • Make hstore_to_jsonb_loose match hstore_to_json_loose on what's a number. Commit e09996ff8dee3f70 removed some ad-hoc code in hstore_to_json_loose that determined whether an hstore value string looked like a number, in favor of calling the JSON parser's is-it-a-number code. However, it neglected the fact that the exact same code appeared in hstore_to_jsonb_loose. This is not a bug, exactly, because the requirements on the two functions are not the same: hstore_to_json_loose must accept only syntactically legal JSON numbers as numbers, or it will produce invalid JSON output, as per bug #12070 which spawned the prior commit. But hstore_to_jsonb_loose could accept anything that numeric_in will eat, other than Inf and NaN. Nonetheless it seems surprising and arbitrary that the two functions don't use the same rules for what is a number versus what is a string; especially since they did use the same rules before the aforesaid commit. For one thing, that means that doing hstore_to_json_loose and then casting to jsonb can produce results different from doing just hstore_to_jsonb_loose. Hence, change hstore_to_jsonb_loose's logic to match hstore_to_json_loose, ie, hstore values are treated as numbers when they match the JSON syntax for numbers. No back-patch, since this is more in the nature of a definitional change than a bug fix.
  • Add hstore_to_jsonb() and hstore_to_jsonb_loose() to hstore documentation. These were never documented anywhere user-visible. Tut tut.
  • In pg_dump, ensure that view triggers are processed after view rules. If a view is split into CREATE TABLE + CREATE RULE to break a circular dependency, then any triggers on the view must be dumped/reloaded after the CREATE RULE; else the backend may reject the CREATE TRIGGER because it's the wrong type of trigger for a plain table. This works all right in plain dump/restore because of pg_dump's sorting heuristic that places triggers after rules. However, when using parallel restore, the ordering must be enforced by a dependency --- and we didn't have one. Fixing this is a mere matter of adding an addObjectDependency() call, except that we need to be able to find all the triggers belonging to the view relation, and there was no easy way to do that. Add fields to pg_dump's TableInfo struct to remember where the associated TriggerInfo struct(s) are. Per bug report from Dennis Kögel. The failure can be exhibited at least as far back as 9.1, so back-patch to all supported branches.
  • Simplify syntax diagram for REINDEX. Since there currently is only one possible parenthesized option, namely VERBOSE, it's a bit pointless to show it with "{ } [, ... ]". The curly braces are useless and therefore confusing, as seen in a recent question from Karsten Hilbert. Remove the extra decoration for the time being; we can put it back when and if REINDEX grows some more options.
  • Add num_nulls() and num_nonnulls() to count NULL arguments. An example use-case is "CHECK(num_nonnulls(a,b,c) = 1)" to assert that exactly one of a,b,c isn't NULL. The functions are variadic, so they can also be pressed into service to count the number of null or nonnull elements in an array. Marko Tiikkaja, reviewed by Pavel Stehule
  • Update time zone data files to tzdata release 2016a. DST law changes in Cayman Islands, Metlakatla, Trans-Baikal Territory (Zabaykalsky Krai). Historical corrections for Pakistan.
  • First-draft release notes for 9.4.6. As usual, the release notes for other branches will be made by cutting these down, but put them up for community review first.
  • Add missing "static" qualifier. Per buildfarm member pademelon.
  • Improve HJDEBUG code a bit. Commit 30d7ae3c76d2de144232ae6ab328ca86b70e72c3 introduced an HJDEBUG stanza that probably didn't compile at the time, and definitely doesn't compile now, because it refers to a nonexistent variable. It doesn't seem terribly useful anyway, so just get rid of it. While I'm fooling with it, use %z modifier instead of the obsolete hack of casting size_t to unsigned long, and include the HashJoinTable's address in each printout so that it's possible to distinguish the activities of multiple hashjoins occurring in one query. Noted while trying to use HJDEBUG to investigate bug #13908. Back-patch to 9.5, because code that doesn't compile is certainly not very helpful.
  • Fix comment block trashed by pgindent. Looks like I put the protective dashes in the wrong place in f4e4b32743.
  • Improve speed of timestamp/time/date output functions. It seems that sprintf(), at least in glibc's version, is unreasonably slow compared to hand-rolled code for printing integers. Replacing most uses of sprintf() in the datetime.c output functions with special-purpose code turns out to give more than a 2X speedup in COPY of a table with a single timestamp column; which is pretty impressive considering all the other logic in that code path. David Rowley and Andres Freund, reviewed by Peter Geoghegan and myself
  • ExecHashRemoveNextSkewBucket must physically copy tuples to main hashtable. Commit 45f6240a8fa9d355 added an assumption in ExecHashIncreaseNumBatches and ExecHashIncreaseNumBuckets that they could find all tuples in the main hash table by iterating over the "dense storage" introduced by that patch. However, ExecHashRemoveNextSkewBucket continued its old practice of simply re-linking deleted skew tuples into the main table's hashchains. Hence, such tuples got lost during any subsequent increase in nbatch or nbuckets, and would never get joined, as reported in bug #13908 from Seth P. I (tgl) think that the aforesaid commit has got multiple design issues and should be reworked rather completely; but there is no time for that right now, so band-aid the problem by making ExecHashRemoveNextSkewBucket physically copy deleted skew tuples into the "dense storage" arena. The added test case is able to exhibit the problem by means of fooling the planner with a WHERE condition that it will underestimate the selectivity of, causing the initial nbatch estimate to be too small. Tomas Vondra and Tom Lane. Thanks to David Johnston for initial investigation into the bug report.
  • Release notes for 9.5.1, 9.4.6, 9.3.11, 9.2.15, 9.1.20.
  • Fix deparsing of ON CONFLICT arbiter WHERE clauses. The parser doesn't allow qualification of column names appearing in these clauses, but ruleutils.c would sometimes qualify them, leading to dump/reload failures. Per bug #13891 from Onder Kalaci. (In passing, make stanzas in ruleutils.c that save/restore varprefix more consistent.) Peter Geoghegan
  • Improve documentation about PRIMARY KEY constraints. Get rid of the false implication that PRIMARY KEY is exactly equivalent to UNIQUE + NOT NULL. That was more-or-less true at one time in our implementation, but the standard doesn't say that, and we've grown various features (many of them required by spec) that treat a pkey differently from less-formal constraints. Per recent discussion on pgsql-general. I failed to resist the temptation to do some other wordsmithing in the same area.
  • Use %u not %d to print OIDs. Oversight in commit 96198d94c. Etsuro Fujita
  • Re-pgindent varlena.c. Just to make sure previous commit worked ...
  • Rename typedef "string" to "VarString". Since pgindent treats typedef names as global, the original coding of b47b4dbf683f13e6 would have had rather nasty effects on the formatting of other files in which "string" is used as a variable or field name. Use a less generic name for this typedef, and rename some other identifiers to match. Peter Geoghegan, per gripe from me
  • Temporarily make pg_ctl and server shutdown a whole lot chattier. This is a quick hack, due to be reverted when its purpose has been served, to try to gather information about why some of the buildfarm critters regularly fail with "postmaster does not shut down" complaints. Maybe they are just really overloaded, but maybe something else is going on. Hence, instrument pg_ctl to print the current time when it starts waiting for postmaster shutdown and when it gives up, and add a lot of logging of the current time in the server's checkpoint and shutdown code paths. No attempt has been made to make this pretty. I'm not even totally sure if it will build on Windows, but we'll soon find out.
  • Fix some regex issues with out-of-range characters and large char ranges. Previously, our regex code defined CHR_MAX as 0xfffffffe, which is a bad choice because it is outside the range of type "celt" (int32). Characters approaching that limit could lead to infinite loops in logic such as "for (c = a; c <= b; c++)" where c is of type celt but the range bounds are chr. Such loops will work safely only if CHR_MAX+1 is representable in celt, since c must advance to beyond b before the loop will exit. Fortunately, there seems no reason not to restrict CHR_MAX to 0x7ffffffe. It's highly unlikely that Unicode will ever assign codes that high, and none of our other backend encodings need characters beyond that either. In addition to modifying the macro, we have to explicitly enforce character range restrictions on the values of \u, \U, and \x escape sequences, else the limit is trivially bypassed. Also, the code for expanding case-independent character ranges in bracket expressions had a potential integer overflow in its calculation of the number of characters it could generate, which could lead to allocating too small a character vector and then overwriting memory. An attacker with the ability to supply arbitrary regex patterns could easily cause transient DOS via server crashes, and the possibility for privilege escalation has not been ruled out. Quite aside from the integer-overflow problem, the range expansion code was unnecessarily inefficient in that it always produced a result consisting of individual characters, abandoning the knowledge that we had a range to start with. If the input range is large, this requires excessive memory. Change it so that the original range is reported as-is, and then we add on any case-equivalent characters that are outside that range. With this approach, we can bound the number of individual characters allowed without sacrificing much. This patch allows at most 100000 individual characters, which I believe to be more than the number of case pairs existing in Unicode, so that the restriction will never be hit in practice. It's still possible for range() to take awhile given a large character code range, so also add statement-cancel detection to its loop. The downstream function dovec() also lacked cancel detection, and could take a long time given a large output from range(). Per fuzz testing by Greg Stark. Back-patch to all supported branches. Security: CVE-2016-0773
  • Last-minute updates for release notes. Security: CVE-2016-0773
  • Add more chattiness in server shutdown. Early returns from the buildfarm show that there's a bit of a gap in the logging I added in 3971f64843b02e4a: the portion of CreateCheckPoint() after CheckPointGuts() can take a fair amount of time. Add a few more log messages in that section of code. This too shall be reverted later.
  • Add still more chattiness in server shutdown. Further investigation says that there may be some slow operations after we've finished ShutdownXLOG(), so add some more log messages to try to isolate that. This is all temporary code too.
  • Revert "Temporarily make pg_ctl and server shutdown a whole lot chattier." This reverts commit 3971f64843b02e4a55d854156bd53e46a0588e45 and a couple of followon debugging commits; I think we've learned what we can from them.
  • Avoid use of sscanf() to parse ispell dictionary files. It turns out that on FreeBSD-derived platforms (including OS X), the *scanf() family of functions is pretty much brain-dead about multibyte characters. In particular it will apply isspace() to individual bytes of input even when those bytes are part of a multibyte character, thus allowing false recognition of a field-terminating space. We appear to have little alternative other than instituting a coding rule that *scanf() is not to be used if the input string might contain multibyte characters. (There was some discussion of relying on "%ls", but that probably just moves the portability problem somewhere else, and besides it doesn't fully prevent BSD *scanf() from using isspace().) This patch is a down payment on that: it gets rid of use of sscanf() to parse ispell dictionary files, which are certainly at great risk of having a problem. The code is cleaner this way anyway, though a bit longer. In passing, improve a few comments. Report and patch by Artur Zakirov, reviewed and somewhat tweaked by me. Back-patch to all supported branches.
  • Code review for isolationtester changes. Fix a few oversights in 38f8bdcac4982215beb9f65a19debecaf22fd470: don't leak memory in run_permutation(), remember when we've issued a cancel rather than issuing another one every 10ms, fix some typos in comments.
  • Make new deadlock isolation test more reproducible. The original formulation of 4c9864b9b4d87d02f07f40bb27976da737afdcab was extremely timing-sensitive, because it arranged for the deadlock detector to be running (and possibly unblocking the current query) at almost exactly the same time as isolationtester would be probing to see if the query is blocked. The committed expected-file assumed that the deadlock detection would finish first, but we see the opposite on both fast and slow buildfarm animals. Adjust the deadlock timeout settings to make it predictable that isolationtester *will* see the query as waiting before deadlock detection unblocks it. I used a 5s timeout for the same reasons mentioned in a7921f71a3c747141344d8604f6a6d7b4cddb2a9.
  • Shift the responsibility for emitting "database system is shut down". Historically this message has been emitted at the end of ShutdownXLOG(). That's not an insane place for it in a standalone backend, but in the postmaster environment we've grown a fair amount of stuff that happens later, including archiver/walsender shutdown, stats collector shutdown, etc. Recent buildfarm experimentation showed that on slower machines there could be many seconds' delay between finishing ShutdownXLOG() and actual postmaster exit. That's fairly confusing, both for testing purposes and for DBAs. Hence, move the code that prints this message into UnlinkLockFiles(), so that it comes out just after we remove the postmaster's pidfile. That is a more appropriate definition of "is shut down" from the point of view of "pg_ctl stop", for example. In general, removing the pidfile should be the last externally-visible action of either a postmaster or a standalone backend; compare commit d73d14c271653dff10c349738df79ea03b85236c for instance. So this seems like a reasonably future-proof approach.
  • Fix typo in comment.
  • Move pg_constraint.h function declarations to new file pg_constraint_fn.h. A pending patch requires exporting a function returning Bitmapset from catalog/pg_constraint.c. As things stand, that would mean including nodes/bitmapset.h in pg_constraint.h, which might be hazardous for the client-side includability of that header. It's not entirely clear whether any client-side code needs to include pg_constraint.h, but it seems prudent to assume that there is some such code somewhere. Therefore, split off the function definitions into a new file pg_constraint_fn.h, similarly to what we've done for some other catalog header files.
  • Remove GROUP BY columns that are functionally dependent on other columns. If a GROUP BY clause includes all columns of a non-deferred primary key, as well as other columns of the same relation, those other columns are redundant and can be dropped from the grouping; the pkey is enough to ensure that each row of the table corresponds to a separate group. Getting rid of the excess columns will reduce the cost of the sorting or hashing needed to implement GROUP BY, and can indeed remove the need for a sort step altogether. This seems worth testing for since many query authors are not aware of the GROUP-BY-primary-key exception to the rule about queries not being allowed to reference non-grouped-by columns in their targetlists or HAVING clauses. Thus, redundant GROUP BY items are not uncommon. Also, we can make the test pretty cheap in most queries where it won't help by not looking up a rel's primary key until we've found that at least two of its columns are in GROUP BY. David Rowley, reviewed by Julien Rouhaud
  • Refactor check_functional_grouping() to use get_primary_key_attnos(). If we ever get around to allowing functional dependency to be proven from other things besides simple primary keys, this code will need to be rethought, but that was true anyway. In the meantime, we might as well not have two very-similar routines for scanning pg_constraint. David Rowley, reviewed by Julien Rouhaud
  • Further tweaking of deadlock isolation tests. The new deadlock-soft-2 test has a timing dependency too: it supposes that isolationtester will detect step s1b as waiting before the deadlock detector runs and grants it the lock. Adjust deadlock_timeout to ensure that that's true even in CLOBBER_CACHE_ALWAYS builds, where the wait detection query is quite slow. Per buildfarm member jaguarundi.
  • Re-pgindent isolationtester.c. Need to do some more hacking on this, and got annoyed that it's not indent clean.
  • Still further tweaking of deadlock isolation tests. It turns out that there is a second race condition in the new deadlock-hard test: once the deadlock detector fires, it's uncertain whether step s7a8 or step s8a1 will report first, because killing s8's transaction unblocks s7. So far, s7 has only been seen to report first in CLOBBER_CACHE_ALWAYS builds, but it's pretty reproducible there, and in theory it should sometimes occur in normal builds too. If s7 were a bit slower than usual, that could also break the test, since the existing expected-file assumes that we'll see s7a8 report the first time we check it after s8a1 completes. To fix, add a post-lock delay to s7a8.
  • isolationtester: don't repeat the is-it-waiting query when retrying a step. If we're retrying a step, then we already decided it was blocked on a lock, and there's no need to recheck that. The original coding of commit 38f8bdcac4982215beb9f65a19debecaf22fd470 resulted in a large number of is-it-waiting queries when dealing with multiple concurrently-blocked sessions, which is fairly pointless and also results in test failures in CLOBBER_CACHE_ALWAYS builds, where the is-it-waiting query is quite slow. This definition also permits appending pg_sleep() calls to steps where it's needed to control the order of finish of concurrent steps. Before, that did not work nicely because we'd decide that a step performing a sleep was not blocked and hang up waiting for it to finish, rather than noticing the completion of the concurrent step we're supposed to notice first. In passing, revise handling of removal of completed waiting steps to make it a bit less messy.
  • Revert "isolationtester: don't repeat the is-it-waiting query when retrying a step." This mostly reverts commit 9c9782f066e0ce5424b8706df2cce147cb78170f. I left in the parts that rearranged removal of completed waiting steps; but the idea of not rechecking a step's blocked-ness isn't working.
  • Revert "Still further tweaking of deadlock isolation tests." This reverts commit d03130d378b5fb071d231a7822784ad87268583a. That was dependent on an isolationtester.c change that now proves to be broken; we will need to find another solution.
  • Increase deadlock_timeout some more in the deadlock-hard isolation test. The previous value of 5s is inadequate for the buildfarm's CLOBBER_CACHE_ALWAYS animals: they take long enough to do the is-it-waiting queries that the timeout expires, allowing the database state to change, before isolationtester is done looking. Perhaps 10s will be enough. (If it isn't, I'm inclined to reduce the number of sessions involved.)
  • Add missing "static" qualifier. Per buildfarm member pademelon.
  • Make GetLockStatusData's header comment resemble reality. The API spec for this function was changed completely (and for the better) by commit 3cba8999b343648c4c528432ab3d51400194e93b, but it didn't bother with anything as mundane as updating the comments.

Peter Eisentraut pushed:

Noah Misch pushed:

Andres Freund pushed:

  • Fix overeager pushdown of HAVING clauses when grouping sets are used. In 61444bfb we started to allow HAVING clauses to be fully pushed down into WHERE, even when grouping sets are in use. That turns out not to work correctly, because grouping sets can "produce" NULLs, meaning that filtering in WHERE and HAVING can have different results, even when no aggregates or volatile functions are involved. Instead only allow pushdown of empty grouping sets. It'd be nice to do better, but the exact mechanics of deciding which cases are safe are still being debated. It's important to give correct results till we find a good solution, and such a solution might not be appropriate for backpatching anyway. Bug: #13863 Reported-By: 'wrb' Diagnosed-By: Dean Rasheed Author: Andrew Gierth Reviewed-By: Dean Rasheed and Andres Freund Discussion: Backpatch: 9.5, where grouping sets were introduced

Joe Conway pushed:

  • Change delimiter used for display of NextXID NextXID has been rendered in the form of a pg_lsn even though it really is not. This can cause confusion, so change the format from %u/%u to %u:%u, per discussion on hackers. Complaint by me, patch by me and Bruce, reviewed by Michael Paquier and Alvaro. Applied to HEAD only. Author: Joe Conway, Bruce Momjian Reviewed-by: Michael Paquier, Alvaro Herrera Backpatch-through: master

Bruce Momjian pushed:

Correctifs rejetés (à ce jour)

No one was disappointed this week :-)

Correctifs en attente

Andreas 'ads' Scherbaum sent in two revisions of a patch to change the 32-bit counter in PL/pgsql's GET DIAGNOSTICS ... ROWCOUNT to 64-bit, allowing larger result sets.

Michaël Paquier sent in a patch to fix psql's tab completion for ALTER FUNCTION.

Alexander Korotkov sent in another revision of a patch to move PinBuffer and UnpinBuffer to atomics.

Artur Zakirov sent in two more revisions of a patch to add fuzzy substring searching with the pg_trgm extension.

Dmitry Ivanov sent in a patch to add phrase search to textsearch.

David Steele sent in a patch to allow hiding messages below ERROR from the client.

Fabien COELHO and Michaël Paquier traded patches to extend pgbench expressions with functions.

Etsuro Fujita sent in a patch to fix a copy-pasto in the ExecForeignDelete documentation.

David Steele sent in another revision of a patch to add a pgaudit extension.

Fabien COELHO and Ãlvaro Herrera traded patches to extend pgbench stats, etc.

Daniel Verité sent in another revision of a patch to create a crosstab view in psql.

Iacob Catalin and Pavel Stěhule traded patches to add an ereport function to PL/PythonU.

Teodor Sigaev sent in another revision of a patch to add tsvector editing functions.

Corey Huinker sent in another revision of a patch to add generate_series(date, date[, integer]).

Kyotaro HORIGUCHI and Fujii Masao traded patches to fix an incorrect formula for SysV IPC parameters.

SAWADA Masahiko and Michaël Paquier traded patches to allow N > 1 synchronous standby servers.

Amit Langote sent in two revisions of a patch to fix a typo in syncrep.c.

Thomas Munro sent in another revision of a patch to add causal reads.

Thomas Munro sent in another revision of a patch to detect SSI conflicts before reporting constraint violations.

Konstantin Knizhnik sent in a patch to allow batch updating of indexes.

Etsuro Fujita and Rushabh Lathia traded patches to speed up updating foreign tables through the PostgreSQL FDW.

Michaël Paquier sent in four more revisions of a patch to fix hot standby checkpoints.

SAWADA Masahiko sent in two more revisions of a patch to add a "frozen" bit to the visibility map.

Heikki Linnakangas sent in a patch to fix the optimization to skip WAL-logging on table created in same xact.

Haribabu Kommi sent in another revision of a patch to create a pg_hba_lookup function to get all matching pg_hba.conf entries.

Kyotaro HORIGUCHI sent in a patch to help improve in-core regression tests.

Michaël Paquier sent in two more revisions of a patch to help fix silent data loss on ext4 filesystems.

Kyotaro HORIGUCHI sent in a patch to add IF (NOT) EXISTS support to tab completion in psql.

Peter Geoghegan sent in another revision of a patch to fix an OpenSSL error queue bug.

Vinayak Pokale sent in another revision of a patch to implement a vacuum progress checker.

Filip Rembiałkowski sent in three revisions of a patch to make NOTIFY list de-duplication optional.

Fabien COELHO sent in another revision of a patch to fix pgbench so it doesn't run much longer than the run length under certain pathological conditions.

Robert Haas sent in a patch to push target-list evaluation down to the parallel worker whenever possible.

Vitaly Burovoy sent in two revisions of a patch to make NOT NULL constraints follow SQL-2011.

Peter Eisentraut sent in another revision of a patch to remove the WAL level "archive."

Thomas Munro sent in another revision of a patch to add a new log line prefix for cluster name.

Fujii Masao sent in another revision of a patch to check for a suitable index when refreshing a materialized view.

Jim Nasby sent in a patch to convert pltcl from strings to objects.

Jeff Janes sent in a patch to add s2k-count to pgcrypto.

Jeff Janes sent in two revisions of a patch to fix a bug in StartupSUBTRANS.

Pavel Stěhule and Jim Nasby traded patches to implement parse an ident to a text array.

Konstantin Knizhnik sent in another revision of a patch to create an extensible transaction manager API.

Stephen Frost sent in a patch to improve docs wrt catalog object ACLs.

Magnus Hagander sent in a patch to update to the backup APIs to support non-exclusive backups.

Vitaly Burovoy sent in another revision of a patch to add pg_size_bytes().

Aleksander Alekseev and Robert Haas traded patches to optimize dynahashes.

Robbie Harwood sent in another revision of a patch to overhaul support for GSSAPI encryption.

Artur Zakirov sent in another revision of a patch to add fuzzy substring searching to the pg_trgm extension.

Fabien COELHO sent in four more revisions of a patch to extend pgbench with functions.

Andreas 'ads' Scherbaum sent in another revision of a patch to allow PL/pgsql's GET DIAGNOSTICS to work with numbers of rows that overflow int32.

Anastasia Lubennikova sent in another revision of a patch to implement covering + unique indexes.

Magnus Hagander sent in a patch to make all the functions PL/pgsql exports are actually prefixed with plpgsql_.

Andres Freund sent in a patch to make SetHintBit() a bit more aggressive, in order to fixes all the potential regressions created by another part of the patch which fixes the overaggressive flushing by the wal writer by only flushing every wal_writer_delay ms or wal_writer_flush_after bytes.

Michaël Paquier sent in another revision of a patch to ensure that GinPageIs* actually return a boolean.

Eugene Kazakov and Michaël Paquier traded patches to fix the TAP tests.

Christian Ullrich sent in another revision of a patch to fix a crash with old Windows on a new CPU.

Magnus Hagander sent in a patch to refactor receivelog.c to be less intertwined with itself.