Les nouveautés des produits dérivés

Offres d'emplois autour de PostgreSQL en mai

PostgreSQL Local

  • FOSS4G NA (Free and Open Source Software for Geospatial - North America) se tiendra à Raleigh, en Caroline du Nord, du 2 au 5 mai 2016 : https://2016.foss4g-na.org/
  • La PGCon 2016 se tiendra du 17 au 21 mai 2016 à Ottawa : http://www.pgcon.org/
  • Le PGDay suisse sera, cette année, tenue à l'Université des Sciences Appliquées (HSR) de Rapperswil le 24 juin 2016 : http://www.pgday.ch/
  • "5432 ... Meet us!" aura lieu à Milan (Italie) les 28 & 29 juin 2016. Les inscriptions sont ouvertes : http://5432meet.us/
  • Le PG Day UK aura lieu le 5 juillet 2016 : http://www.pgconf.uk/
  • Le PostgresOpen 2016 aura lieu à Dallas (Texas, USA) du 13 au 16 septembre. L'appel à conférenciers à été lancé : https://2016.postgresopen.org/callforpapers/
  • Session PostgreSQL le 22 septembre 2016 à Lyon (France). La date limite de candidature est le 20 mai : envoyez vos propositions à call-for-paper AT postgresql-sessions DOT org.
  • La PgConf Silicon Valley 2016 aura lieu du 14 au 16 novembre 2016 : http://www.pgconfsv.com/

PostgreSQL dans les média

PostgreSQL Weekly News / les nouvelles hebdomadaires vous sont offertes cette semaine par David Fetter. Traduction par l'équipe PostgreSQLFr sous licence CC BY-NC-SA. La version originale se trouve à l'adresse suivante : http://www.postgresql.org/message-id/20160502054808.GA24944@fetter.org

Proposez vos articles ou annonces avant dimanche 15:00 (heure du Pacifique). Merci de les envoyer en anglais à david (a) fetter.org, en allemand à pwn (a) pgug.de, en italien à pwn (a) itpug.org et en espagnol à pwn (a) arpug.com.ar.

Correctifs appliqués

Peter Eisentraut pushed:

Tom Lane pushed:

  • Try harder to detect a port conflict in PostgresNode.pm. Commit fab84c7787f25756 tried to get away without doing an actual bind(), but buildfarm results show that that doesn't get the job done. So we must really bind to the target port --- and at least on my Linux box, we need a listen() as well, or conflicts won't be detected. We rely on SO_REUSEADDR to prevent problems from starting a postmaster on the socket immediately after we've bound to it in the test code. (There may be platforms where that doesn't work too well. But fortunately, we only really care whether this works on Windows, and there the default behavior should be OK.) http://git.postgresql.org/pg/commitdiff/40e89e2ab89cb2801f6bc02f08dcc24d547530fc
  • New method for preventing compile-time calculation of degree constants. Commit 65abaab547a5758b tried to prevent the scaling constants used in the degree-based trig functions from being precomputed at compile time, because some compilers do that with functions that don't yield results identical-to-the-last-bit to what you get at runtime. A report from Peter Eisentraut suggests that some recent compilers are smart enough to see through that trick, though. Instead, let's put the inputs to these calculations into non-const global variables, which should be a more reliable way of convincing the compiler that it can't assume that they are compile-time constants. (If we really get desperate, we could mark these variables "volatile", but I do not believe we should have to.) http://git.postgresql.org/pg/commitdiff/6b1a213bbd6599228b2b67f7552ff7cc378797bf
  • Yet more portability hacking for degree-based trig functions. The true explanation for Peter Eisentraut's report of inexact asind results seems to be that (a) he's compiling into x87 instruction set, which uses wider-than-double float registers, plus (b) the library function asin() on his platform returns a result that is wider than double and is not rounded to double width. To fix, we have to force the function's result to be rounded comparably to what happened to the scaling constant asin_0_5. Experimentation suggests that storing it into a volatile local variable is the least ugly way of making that happen. Although only asin() is known to exhibit an observable inexact result, we'd better do this in all the places where we're hoping to get an exact result by scaling. http://git.postgresql.org/pg/commitdiff/82311bcdd76904b2cee7567e14e9fb0cf6c6178c
  • Fix order of shutdown cleanup operations in PostgresNode.pm. Previously, database clusters created by a TAP test were shut down by DESTROY methods attached to the PostgresNode objects representing them. The trouble with that is that if the objects survive into the final global destruction phase (which they do), Perl executes the DESTROY methods in an unspecified order. Thus, the order of shutdown of multiple clusters was indeterminate, which might lead to not-very-reproducible errors getting logged (eg from a slave whose master might or might not get killed first). Worse, the File::Temp objects representing the temporary PGDATA directories might get destroyed before the PostgresNode objects, resulting in attempts to delete PGDATA directories that still have live servers in them. On Windows, this would lead to directory deletion failures; on Unix, it usually had no effects worse than erratic "could not open temporary statistics file "pg_stat/global.tmp": No such file or directory" log messages. While none of this would affect the reported result of the TAP test, which is already determined, it could be very confusing when one is trying to understand from the logs what went wrong with a failed test. To fix, do the postmaster shutdowns in an END block rather than at object destruction time. The END block will execute at a well-defined (and reasonable) time during script termination, and it will stop the postmasters in order of PostgresNode object creation. (Perhaps we should change that to be reverse order of creation, but the main point here is that we now have control which we did not before.) Use "pg_ctl stop", not an asynchronous kill(SIGQUIT), so that we wait for the postmasters to shut down before proceeding with directory deletion. Deletion of temporary directories still happens in an unspecified order during global destruction, but I can see no reason to care about that once the postmasters are stopped. http://git.postgresql.org/pg/commitdiff/08af9219060a9526c69f5408b80eee0aa8f707e9
  • Add a --brief option to git_changelog. In commit c0b050192, Andres introduced the idea of including one-line commit references in our major release notes. Teach git_changelog to emit a (lightly adapted) version of that format, so that we don't have to laboriously add it to the notes after the fact. The default output isn't changed, since I anticipate still using that for minor release notes. http://git.postgresql.org/pg/commitdiff/8067c8f86b5f4516ee204a119a750329f7d126ee
  • Clean up parsing of synchronous_standby_names GUC variable. Commit 989be0810dffd08b added a flex/bison lexer/parser to interpret synchronous_standby_names. It was done in a pretty crufty way, though, making assorted end-use sites responsible for calling the parser at the right times. That was not only vulnerable to errors of omission, but made it possible that lexer/parser errors occur at very undesirable times, and created memory leakages even if there was no error. Instead, perform the parsing once during check_synchronous_standby_names and let guc.c manage the resulting data. To do that, we have to flatten the parsed representation into a single hunk of malloc'd memory, but that is not very hard. While at it, work a little harder on making useful error reports for parsing problems; the previous code felt that "synchronous_standby_names parser returned 1" was an appropriate user-facing error message. (To be fair, it did also log a syntax error message, but separately from the GUC problem report, which is at best confusing.) It had some outright bugs in the face of invalid input, too. I (tgl) also concluded that we need to restrict unquoted names in synchronous_standby_names to be just SQL identifiers. The previous coding would accept darn near anything, which (1) makes the quoting convention both nearly-unnecessary and formally ambiguous, (2) makes it very hard to understand what is a syntax error and what is a creative interpretation of the input as a standby name, and (3) makes it impossible to further extend the syntax in future without a compatibility break. I presume that we're intending future extensions of the syntax, else this parsing infrastructure is massive overkill, so (3) is an important objection. Since we've taken a compatibility hit for non-identifier names with this change anyway, we might as well lock things down now and insist that users use double quotes for standby names that aren't identifiers. Kyotaro Horiguchi and Tom Lane http://git.postgresql.org/pg/commitdiff/4c804fbdfb472cf71db33609258b8e1aaad81943
  • Use memmove() not memcpy() to slide some pointers down. The previous coding here was formally undefined, though it seems to accidentally work on most platforms in the buildfarm. Caught by some OpenBSD platforms in which libc contains an assertion check for overlapping areas passed to memcpy(). Thomas Munro http://git.postgresql.org/pg/commitdiff/ad520ec4acb8f0cdb143b63519be95a9549fa826
  • Revert "Convert contrib/seg's bool-returning SQL functions to V1 call convention." This reverts commit c8e81afc60093b199a128ccdfbb692ced8e0c9cd. That turns out to have been based on a faulty diagnosis of why the VS2015 build was misbehaving. Instead, we need to fix DatumGetBool(). http://git.postgresql.org/pg/commitdiff/f050423052bc9265d4cd27555058435edd4bef87
  • Adjust DatumGetBool macro, this time for sure. Commit 23a41573c attempted to fix the DatumGetBool macro to ignore bits in a Datum that are to the left of the actual bool value. But it did that by casting the Datum to bool; and on compilers that use C99 semantics for bool, that ends up being a whole-word test, not a 1-byte test. This seems to be the true explanation for contrib/seg failing in VS2015. To fix, use GET_1_BYTE() explicitly. I think in the previous patch, I'd had some idea of not having to commit to bool being exactly 1 byte wide, but regardless of what the compiler's bool is, boolean columns and Datums are certainly 1 byte wide. The previous fix was (eventually) back-patched into all active versions, so do likewise with this one. http://git.postgresql.org/pg/commitdiff/23b09e15b9f40baeff527ca4dbc40afc823dd962
  • Fix mishandling of equivalence-class tests in parameterized plans. Given a three-or-more-way equivalence class, such as X.Y = Y.Y = Z.Z, it was possible for the planner to omit one of the quals needed to enforce that all members of the equivalence class are actually equal. This only happened in the case of a parameterized join node for two of the relations, that is a plan tree like Nested Loop -> Scan X -> Nested Loop -> Scan Y -> Scan Z Filter: Z.Z = X.X The eclass machinery normally expects to apply X.X = Y.Y when those two relations are joined, but in this shape of plan tree they aren't joined until the top node --- and, if the lower nested loop is marked as parameterized by X, the top node will assume that the relevant eclass condition(s) got pushed down into the lower node. On the other hand, the scan of Z assumes that it's only responsible for constraining Z.Z to match any one of the other eclass members. So one or another of the required quals sometimes fell between the cracks, depending on whether consideration of the eclass in get_joinrel_parampathinfo() for the lower nested loop chanced to generate X.X = Y.Y or X.X = Z.Z as the appropriate constraint there. If it generated the latter, it'd erroneously suppose that the Z scan would take care of matters. To fix, force X.X = Y.Y to be generated and applied at that join node when this case occurs. This is *extremely* hard to hit in practice, because various planner behaviors conspire to mask the problem; starting with the fact that the planner doesn't really like to generate a parameterized plan of the above shape. (It might have been impossible to hit it before we tweaked things to allow this plan shape for star-schema cases.) Many thanks to Alexander Kirkouski for submitting a reproducible test case. The bug can be demonstrated in all branches back to 9.2 where parameterized paths were introduced, so back-patch that far. http://git.postgresql.org/pg/commitdiff/207d5a656e2ecc98a1db5bdc22ea306f7f7c8d62
  • Remove warning about num_sync being too large in synchronous_standby_names. If we're not going to reject such setups entirely, throwing a WARNING in check_synchronous_standby_names() is unhelpful, because it will cause the warning to be logged again every time the postmaster receives SIGHUP. Per discussion, just remove the warning. In passing, improve the documentation for synchronous_commit, which had not gotten the word that now there can be more than one synchronous standby. http://git.postgresql.org/pg/commitdiff/17d5db352c1780f4721664f67bc3a3f3b1cf933c
  • Fix planner crash from pfree'ing a partial path that a GatherPath uses. We mustn't run generate_gather_paths() during add_paths_to_joinrel(), because that function can be invoked multiple times for the same target joinrel. Not only is it wasteful to build GatherPaths repeatedly, but a later add_partial_path() could delete the partial path that a previously created GatherPath depends on. Instead establish the convention that we do generate_gather_paths() for a rel only just before set_cheapest(). The code was accidentally not broken for baserels, because as of today there never is more than one partial path for a baserel. But that assumption obviously has a pretty short half-life, so move the generate_gather_paths() calls for those cases as well. Also add some generic comments explaining how and why this all works. Per fuzz testing by Andreas Seltenreich. Report: <871t5pgwdt.fsf@credativ.de> http://git.postgresql.org/pg/commitdiff/c45bf5751b6338488bd79ce777210285531da373
  • Small improvements to OPTIMIZER_DEBUG code. Now that Paths have their own rows field, print that rather than the parent relation's rowcount. Show the relid sets associated with Paths using table names rather than numbers; since this code is able to print simple Var references using table names, it seems a bit silly that print_relids can't. Print the cheapest_parameterized_paths list for a RelOptInfo, and include information about a parameterized path's required_outer rels. Noted while trying to use this feature to debug Alexander Kirkouski's recent bug report. http://git.postgresql.org/pg/commitdiff/2a2435e6995133c9d872ef9cb51432f0b678b978
  • Update contrib/unaccent documentation about its unaccent.rules file. Commit 1bbd52cb9a4aa61a didn't bother with such niceties. http://git.postgresql.org/pg/commitdiff/6376a16ba24a5a097fc739b9c79ab555be4d9f1a
  • Add a --non-master-only option to git_changelog. This has the inverse effect of --master-only. It's needed to help find cases where a commit should not be described in major release notes because it was back-patched into older branches, though not at the same time as the HEAD commit. http://git.postgresql.org/pg/commitdiff/8473b7f95fbe8ef25dccd23ff94a4e363797bd90

Kevin Grittner pushed:

  • Fix C comment typo and redundant test http://git.postgresql.org/pg/commitdiff/e65953be4f540dce31f17db2934ee58365077272
  • Add a few entries to the tail of time mapping, to see old values. Without a few entries beyond old_snapshot_threshold, the lookup would often fail, resulting in the more aggressive pruning or vacuum being skipped often enough to matter. This was very clearly shown by a python test script posted by Ants Aasma, and was likely a factor in an earlier but somewhat less clear-cut test case posted by Jeff Janes. This patch makes no change to the logic, per se -- it just makes the array of mapping entries big enough to make lookup misses based on timing much less likely. An occasional miss is still possible if a thread stalls for more than 10 minutes, but that does not create any problem with correctness of behavior. Besides, if things are so busy that a thread is stalling for more than 10 minutes, it is probably OK to skip the more aggressive cleanup at that particular point in time. http://git.postgresql.org/pg/commitdiff/7c3e8039f450eb99b3a73272d0a1661195747d1b

Magnus Hagander pushed:

Robert Haas pushed:

Teodor Sigaev pushed:

  • Fix tsearch docs. Remove mention of setweight(tsquery) which wasn't included in 9.6. Also replace old forgotten phrase operator to new one. Dmitry Ivanov http://git.postgresql.org/pg/commitdiff/f1e3c76066f0066a8a9bb09b80cd97f11e4b2dc4
  • Prevent multiple cleanup process for pending list in GIN. Previously, ginInsertCleanup could exit early if it detects that someone else is cleaning up the pending list, without waiting for that someone else to finish the job. But in this case vacuum could miss tuples to be deleted. Cleanup process now locks metapage with a help of heavyweight LockPage(ExclusiveLock), and it guarantees that there is no another cleanup process at the same time. Lock is taken differently depending on caller of cleanup process: any vacuums and gin_clean_pending_list() will be blocked until lock becomes available, ordinary insert uses conditional lock to prevent indefinite waiting on lock. Insert into pending list doesn't use this lock, so insertion isn't blocked. Also, patch adds stopping of cleanup process when at-start-cleanup-tail is reached in order to prevent infinite cleanup in case of massive insertion. But it will stop only for automatic maintenance tasks like autovacuum. Patch introduces choice of limit of memory to use: autovacuum_work_mem, maintenance_work_mem or work_mem depending on call path. Patch for previous releases should be reworked due to changes between 9.6 and previous ones in this area. Discover and diagnostics by Jeff Janes and Tomas Vondra Patch by me with some ideas of Jeff Janes http://git.postgresql.org/pg/commitdiff/e2c79e14d998cd31f860854bc9210b37b457bb01
  • Prevent to use magic constants. Use macroses for definition amstrategies/amsupport fields instead of hardcoded values. Author: Nikolay Shaplov with addition for contrib/bloom http://git.postgresql.org/pg/commitdiff/f8467f7da8685dbc47187864e5afe130d9c63fff

Noah Misch pushed:

  • Impose a full barrier in generic-xlc.h atomics functions. pg_atomic_compare_exchange_*_impl() were providing only the semantics of an acquire barrier. Buildfarm members hornet and mandrill revealed this deficit beginning with commit 008608b9d51061b1f598c197477b3dc7be9c4a64. While we have no report of symptoms in 9.5, we can't rule out the possibility of certain compilers, hardware, or extension code relying on these functions' specified barrier semantics. Back-patch to 9.5, where commit b64d92f1a5602c55ee8b27a7ac474f03b7aee340 introduced atomics. Reviewed by Andres Freund. http://git.postgresql.org/pg/commitdiff/213c7df0337278c71c98e90605dc83023db1a80e

Andres Freund pushed:

  • Don't open formally non-existent segments in _mdfd_getseg(). Before this commit _mdfd_getseg(), in contrast to mdnblocks(), did not verify whether all segments leading up to the to-be-opened one, were RELSEG_SIZE sized. That is e.g. not the case after truncating a relation, because later segments just get truncated to zero length, not removed. Once a "non-existent" segment has been opened in a session, mdnblocks() will return wrong results, causing errors like "could not read block %u in file" when accessing blocks. Closing the session, or the later arrival of relevant invalidation messages, would "fix" the problem. That, so far, was mostly harmless, because most segment accesses are only done after an mdnblocks() call. But since 428b1d6b29ca we try to open segments that might have been deleted, to trigger kernel writeback from a backend's queue of recent writes. To fix check segment sizes in _mdfd_getseg() when opening previously unopened segments. In practice this shouldn't imply a lot of additional lseek() calls, because mdnblocks() will most of the time already have opened all relevant segments. This commit also fixes a second problem, namely that _mdfd_getseg( EXTENSION_RETURN_NULL) extends files during recovery, which is not desirable for the mdwriteback() case. Add EXTENSION_REALLY_RETURN_NULL, which does not behave that way, and use it. Reported-By: Thom Brown Author: Andres Freund, Abhijit Menon-Sen Reviewd-By: Robert Haas, Fabien Coehlo Discussion: CAA-aLv6Dp_ZsV-44QA-2zgkqWKQq=GedBX2dRSrWpxqovXK=Pg@mail.gmail.com Fixes: 428b1d6b29ca599c5700d4bc4f4ce4c5880369bf http://git.postgresql.org/pg/commitdiff/72a98a639574d2e25ed94652848555900c81a799
  • Emit invalidations to standby for transactions without xid. So far, when a transaction with pending invalidations, but without an assigned xid, committed, we simply ignored those invalidation messages. That's problematic, because those are actually sent for a reason. Known symptoms of this include that existing sessions on a hot-standby replica sometimes fail to notice new concurrently built indexes and visibility map updates. The solution is to WAL log such invalidations in transactions without an xid. We considered to alternatively force-assign an xid, but that'd be problematic for vacuum, which might be run in systems with few xids. Important: This adds a new WAL record, but as the patch has to be back-patched, we can't bump the WAL page magic. This means that standbys have to be updated before primaries; otherwise "PANIC: standby_redo: unknown op code 32" errors can be encountered. XXX: Reported-By: ВаÑильев Дмитрий, Masahiko Sawada Discussion: CAB-SwXY6oH=9twBkXJtgR4UC1NqT-vpYAtxCseME62ADwyK5OA@mail.gmail.com CAD21AoDpZ6Xjg=gFrGPnSn4oTRRcwK1EBrWCq9OqOHuAcMMC=w@mail.gmail.com http://git.postgresql.org/pg/commitdiff/c6ff84b06a68b71719aa1aaa5f6704d8db1b51f8
  • Remember asking for feedback during walsender shutdown. Since 5a991ef8 we're explicitly asking for feedback from the receiving side when shutting down walsender, if there's not yet replicated data. Unfortunately we didn't remember (i.e. set waiting_for_ping_response to true) having asked for feedback, leading to scenarios in which replies were requested at a high frequency. I can't reproduce this problem on my laptop, I think that's because the problem requires a significant TCP window to manifest due to the !pq_is_send_pending() condition. But since this clearly is a bug, let's fix it. There's quite possibly more wrong than just this though. While fiddling with WalSndDone(), I rewrote a hard to understand comment about looking at the flush vs. the write position. Reported-By: Nick Cleaton, Magnus Hagander Author: Nick Cleaton Discussion: CAFgz3kus=rC_avEgBV=+hRK5HYJ8vXskJRh8yEAbahJGTzF2VQ@mail.gmail.com CABUevExsjROqDcD0A2rnJ6HK6FuKGyewJr3PL12pw85BHFGS2Q@mail.gmail.com Backpatch: 9.4, were 5a991ef8 introduced the use of feedback messages during shutdown. http://git.postgresql.org/pg/commitdiff/59455018a8120bb3c02680b0f9764492c5283d99

Andrew Dunstan pushed:

Correctifs rejetés (à ce jour)

No one was disappointed this week :-)

Correctifs en attente

Stephen Frost sent in another revision of a patch to fix pg_dump for catalog ACLs.

Sehrope Sarkuni sent in a patch to add jsonb_compact(...) for whitespace-free jsonb to text.

Peter Eisentraut sent in another revision of a patch to fix an OpenSSL error queue bug.

Kyotaro HORIGUCHI sent in two more revisions of a patch to fix synchronous replication update configuration.

Ashutosh Sharma sent in a patch to allow pg_basebackup to create pg_stat_tmp and pg_replslot as an empty directories if they are symbolic links.

Etsuro Fujita sent in a patch to fix subtransaction callbacks in the PostgreSQL FDW.

Christian Ullrich sent in three revisions of a patch to enable parallel builds with MSVC.

Dean Rasheed sent in a patch to fix an issue in psql's \ev feature which adds support for missing VIEW options like WITH CHECK OPTION.

Daniel Gustafsson sent in a patch to remove some unused macros from contrib.

Julien Rouhaud sent in two revisions of a patch to fix an issue with FK joins that could cause a segfault.

Craig Ringer sent in a patch to allow a stop LSN to be specified to pg_recvlogical.

Andreas Karlsson sent in a patch to fix the parallel safety markings for some built-in functions that got missed in the shuffle.

Ian Lawrence Barwick sent in a patch to add tab completion to psql's \l command.

Amit Kapila sent in a patch to fix an issue sqlsmith uncovered in apply_projection_to_path.