Appendix A: Release Notes for pglogical3 v3.7

pglogical 3.7.12

This is a maintenance release for pglogical 3.7 which includes minor improvements as well as fixes for issues identified previously.

Improvements

  • Add replication status monitoring (BDR-865)
    Track the connection establishment and drops, the number of transactions committed as well as the restart LSN to detect repeated restarts and diagnose stalled replication.

  • Improve performance when used in combination with synchronous replication (BDR-1398)
    Override synchronous_commit to local' for all PGLogical processes performing only bookkeeping transactions which do not need to be replicated. Only the PGLogical writer processes applying user transactions need to follow the synchronous_commit mode configured for synchronous replication.

  • Internal improvements and additional hooks to better support BDR

Resolved Issues

  • Performance improvements for Decoding Worker (BDR-1311, BDR-1357) Speed up lookups of the WAL decoder worker, reduce delays after reaching up to the LSN proviously known to be decoded by the Decoding Worker, and reduce number of system calls when writing one LCR chunk to an LCR segment file.

  • Improve compatibility with Postgres 13 (BDR-1396)
    Adjust to an API change in ReplicationSlotAcquire that may have led to unintended blocking when non-blocking was requestend and vice versa. This version of PGLogical eliminates this potential problem, which has not been observed on production systems so far.

Upgrades

This release supports upgrading from following versions of pglogical:

  • 3.7.9 and higher
  • 3.6.27
  • 2.4.0

pglogical 3.7.11

This is a maintenance release for pglogical 3.7 which includes minor improvements as well as fixes for issues identified previously.

Resolved Issues

  • Add protection against malformed parameter values in pgl output plugin
    This fixes potential crash when some parameters sent to the output plugin were malformed.

  • Get copy of slot tuple when logging conflict (BDR-734)
    Otherwise we could materialize the row early causing wrong update in presence of additional columns on the downstream.

  • Use a separate memory context for processing LCRs (BDR-1237, RT72165)
    This fixes memory leak when using the decoding worker feature of BDR.

  • Truncate LCR segment file after recovery (BDR-1236, BDR-1259)
    This fixes memory errors reported by the decoding worker after crash.

  • Ensure pg_read_and_filter_lcr will exit when postmaster dies (BDR-1226, BDR-1209, RT72083)
    Solves issues with hanging decoder worker on shutdown.

  • Fix memory leak in the pglogical COPY handler (BDR-1219, RT72091)
    This fixes memory leak when synchronizing large tables.

  • Allow binary and internal protocol on more hardware combinations. This currently only affects internal testing.

Upgrades

This release supports upgrading from following versions of pglogical:

  • 3.7.9 and higher
  • 3.6.27
  • 2.4.0

pglogical 3.7.10

This is a maintenance release for pglogical 3.7 which includes minor improvements as well as fixes for issues identified previously.

Improvements

  • Windows support improvements (BDR-976, BDR-1083)

Resolved Issues

  • Fix potential crash during cleanup of bulk copy replication (BDR-1168)

  • Fix issues in generic WAL message handling when the WAL message was produced by something other than pglogical (BDR-670)

  • Redefine werr_age of pglogical.worker_error_summary to report correct age

  • Only use key attributes of covering unique index when used as replica identity This only affects what is being sent over network, no logic change.

pglogical 3.7.9

This is a maintenance release for pglogical 3.7 which includes minor improvements as well as fixes for issues identified previously.

Improvements

  • Support two-phase commit transaction with Decoding Worker (BDR-811)

    A two-phase commit transaction is completely decoded and sent downstream when processing its PREPARE WAL record. COMMIT/ROLLBACK PREPARED is replicated separately when processing the corresponding WAL record.

  • Reduce writes to pg_replication_origin when using parallel apply (RT71077)
    Previously, especially on EPAS pglogical could produce thousands of dead rows in pg_replication_origin system catalog if it had connection problems to upstream.

Resolved Issues

  • Fix flush queue truncation (BDR-890)
    During queue compaction flush to correct LSN instead of always truncating whole flush queue.

  • Fix the pglogical.worker_error columns werr_worker_index and werr_time
    Used to report wrong values.

  • Fix snapshot handling in internal executor usage (BDR-904)
    Row filtering didn't correctly pushed snapshot in some situations.

    Caught thanks to upcoming change in PostgreSQL that double checks for this.

  • Fix handling of domains over arrays and composite types (BDR-29)
    The replication protocol previously could not always handle columns using domains over arrays and domains over composite types.

pglogical 3.7.8

This is a first stable release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Upgrades are supported from pglogical 3.6.25 and 3.7.7 in this release.

Improvements

  • Allow parallel apply on EDB Advanced Server (EE)

Resolved Issues

  • Fix divergence after physical failover (BDR-365, RT68894 and RM19886)
    Make sure that we don't report back LSN on subscriber that hasn't been received by named standbys (pglogical.standby_slot_names).

    This will ensure provider side keeps slot position behind far enough so that if subscriber is replaced by one of said named standbys, the standby will be able to fetch any missing replication stream from the original provider.

  • Fix crash when ERROR happened during fast shutdown of pglogical writer

  • Don't re-enter worker error handling loop recursively (BDR-667)
    This should help make what happens clearer in any cases where we do encounter errors during error processing.

  • Assign collation to the index scan key (BDR-561)
    When doing lookups for INSERT/UPDATE/DELETE, either to find conflicts or key for the operation to be applied, we should use correct collation.

    This fixes index lookups for index on textual fields on Postgres 12+.

  • Use name data type for known fixed length fields (BDR-561)
    This solves potential index collation issues with pglogical catalogs.

  • Progress WAL sender's slot based on WAL decoder input (BDR-567)
    Without slot progression the server will eventually stop working.

  • Fix begin of transaction write when LCR file does not have enough space (BDR-606)

  • Restart decoding a transaction that was not completed in single decoding worker (BDR-247)
    If we crashed during a long transaction that spawns on more then one lcr file we start decoding again from the lsn of the beginning of the transaction and find the last lcr file where we can write into.

  • Generate temp slot with correct length in subscription sync
    Otherwise the name of the slot might be shortened by Postgres leading to confusion on the slot cleanup.

  • Improve detection of mixing temporary and nontemporary objects in DDLs (BDR-485)
    These can break replication so it's important to not allow them.

  • Fix pre-commit message handling in Eager replication (BDR-492)

  • Override GUCs in all pglogical workers, not just in writers.

pglogical 3.7.7

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Upgrades are supported from pglogical 3.6.25 and 3.7.6 in this release.

Improvements

  • Use parallel apply during initial sync during logical join

  • Add worker process index to the worker_error catalog (BDR-229)
    The column werr_worker_index in worker_error table keeps track of the writers for the same subscription.

  • Various improvements for WAL decoder/sender coordination (BDR-232, BDR-335)

  • Name LCR segment similar to XLOG segments (BDR-236, BDR-253, BDR-321, BDR-322)
    An LCR segment file is identified by five 8-digit hex numbers. Timeline first group, XLOG the next two groups and file number the last two groups.

Resolved Issues

  • Restrict adding queue table to the replication set. (EDBAS, EBD-45)

  • Fix deadlock between receiver and writer during queue flush (BDR-483)

  • Force and wait for all writers to exit when one writer dies (BDR-229)

  • Name LCR directory after the replication slot (BDR-60)
    Logical output plugin may be used by multiple replication slots. Store the LCRs from a given replication slot in a directory named after that replication slot to avoid mixing LCRs for different slots.

  • Fix EXPLAIN...INTO TABLE replication issue (EBC-46)

pglogical 3.7.6

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Upgrades are supported from pglogical 3.6.25 in this release.

Improvements

  • Enable parallel apply for CAMO and Eager (RM17858)

  • Improve error handling in Eager/CAMO, especially with parallel apply (BDR-106)

  • Reduce severity of Eager/CAMO feedback errors

  • Add infrastructure necessary for allowing separation of WAL decoding from WalSender process in BDR (RM18868, BDR-51, BDR-60)

Resolved Issues

  • Improve relcache invalidation handling in heap writer
    This should solve missed invalidations after opening table for DML apply.

  • Wait for the writer that has XID assigned rather then one that doesn't (BDR-137)
    This fixes deadlock in parallel apply when there is a mix of empty and non-empty transactions where the non-empty ones conflict.

  • Correct writer state tracking for Eager/CAMO (BDR-107)

  • Correct and improve CAMO misconfiguration handling (BDR-105)
    Properly abort the transaction in case of sync CAMO, so the internal state of the PGL writer is cleared

  • Fix transaction state tracking for CAMO/Eager
    Do not abort the transaction at pre-commit time for non-CAMO nodes, but keep it open until the final commit. Adjust the transaction state tracking accordingly

  • Fix MERGE handling in 2ndQPostgres 12 and 13
    We'd before allow the MERGE command on replicated tables without appropriate REPLICA IDENTITY which could result in broken replication.

  • Fix CAMO feedback sending (RM17858)
    Fixes stalls in CAMO feedback, improving performance compared to previous 3.7 beta releases. This is especially visible with parallel apply enabled.

pglogical 3.7.5

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Improvements

  • Optimize utility command processing (RT69617) For commands that won't affect any DB objects and don't affect pglogical we can skip the processing early without reading any pglogical or system catalogs or calling to DDL replication plugin interfaces. This is optimization for systems with large number of such utility command calls (that is primarily applications that do explicit transaction management).

  • Add upgrade path from pglogical 2.

Resolved Issues

  • Ensure that pglogical.standby_slot_names takes effect when pglogical.standby_slots_min_confirmed is at the default value of -1.

    On 3.6.21 and older pglogical.standby_slot_names was ignored if pglogical.standby_slot_names is set to zero (RM19042).

    Clusters satisfying the following conditions may experience inter-node data consistency issues after a provider failover:

    • Running pglogical 3.0.0 through to 3.6.21 inclusive;
    • Using pglogical subscriptions/or providers directly (BDR3-managed subscriptions between pairs of BDR3 nodes are unaffected);
    • Have a physical standby (streaming replica) of a pglogical provider intended as a failover candidate;
    • Have pglogical.standby_slot_names on the provider configured to list that physical standby;
    • Have left pglogical.standby_slots_min_confirmed unconfigured or set it explicitly to zero;

    This issue can cause inconsistencies between pglogical provider and subscriber and/or between multiple subscribers when a provider is replaced using physical replication based failover. It's possible for the subscriber(s) to receive transactions committed to the pre-promotion original provider that will not exist on the post-promotion replacement provider. This causes provider/subscriber divergence. If multiple subscribers are connected to the provider, each subscriber could also receive a different subset of transactions from the pre-promotion provider, leading to inter-subscriber divergence as well.

    The pglogical.standby_slots_min_confirmed now defaults to the newly permitted value -1, meaning "all slots listed in pglogical.standby_slot_names". The default of 0 on previous releases was intended to have that effect, but instead effectively disabled physical-before-logical replication.

    To work around the issue on older versions the operator is advised to set pglogical.standby_slots_min_confirmed = 100 in postgresql.conf. This has no effect unless pglogical.standby_slot_names is also set.

    No action is generally required for this issue on BDR3 clusters. BDR3 has its own separate protections to ensure consistency during promotion of replicas.

  • Fix pglogical_create_subscriber when "-v" is passed. It will make pg_ctl emit meaningful information, making it easier to debug issues where pg_ctl fails

pglogical 3.7.4

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Improvements

  • Support PostgreSQL 13

  • Add pglogical.replication_origin_status view (RM17074)
    Same as pg_replication_origin_status but does not require superuser permissions to access it.

  • Beta support of upgrades from 3.6 (currently from 3.6.22)

  • Improved SystemTAP support

Resolved Issues

  • Fix race condition in replication table filtering which could cause crash (RM18839)
    The cached info about table might get invalidated while used which would crash the backend during one of the following operations:

    • reading the pglogical.tables view
    • new subcription creation
    • table resynchronization
  • Don't do transparent DDL replication on commands that affect temporary objects (RM19491, RT69170)
    These are likely to not exist on the subscription.

  • Only run pgl specific code in deadlock detector when inside writer (RM18402)
    It's not relevant in user backends and would cause ERRORs there.

pglogical 3.7.3

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

Improvements

  • Parallel Apply (RM6503) Allows configuring number of writers per subscriptions. By default this is still 1, which mean parallel apply is off by default but can be enabled both globally (pglogical.writers_per_subscription) and per subscription (num_writers option in pglogical.create_subscription() and pglogical.alter_subscription_num_writers()).

  • Split "replicating" subscription status into two statuses
    One is still "replicating" and is reported only if something was actually replicated by the subcription since last worker start. The other is "started" and just means that the worker for the subscription is running at the time the status query was executed.
    This should reduce confusion where subscription would report "replicating" status but worker is in restart loop due to an apply error.

  • Substantial test and testing framework improvements

  • Improve 2ndQPostgres and BDR version dependency handling (RM17024)

  • Add PostgreSQL 12 support to pglogical_create_subscriber

  • Rework table resynchronization to use separate receiver process from the main one
    This improves performance of both main apply (primarily latency) and the resynchronization itself.
    It also fixes potential issue where table could be considered synchronized before the catchup finished completely.

Resolved Issues

  • Fix crash on resynchronization of large partitioned tables (RM18154, RM15733, RT68455, RT68352)
    The resync process would keep crashing due to cache invalidation race condition if the COPY took very long or if there was DDL activity on the copied table during the COPY.

  • Prohibit MERGE and UPSERT on a table without replica identity (RM17323, RT68146) These commands can end up doing UPDATE which will break replication if the table has no replica identity as the downstream has no way to find the matching row for updating.

  • Resolve relcache reference leak reports (RM16956) Close the relation correctly in pglogical.show_repset_tables_info()

  • Resolve rare crash in HeapWriter row cleanup code (RM16956)

  • Resolve rare crash on worker exit (RM11686)
    If a pglogical worker exited before it finished initialization it could crash instead of exiting cleanly.

  • Fix apply errors parallel index rebuild afrer TRUNCATE (RM17602)

pglogical 3.7.2

This is a beta release of the pglogical 3.7.

pglogical 3.7 is a major release of pglogical. This release includes major new features as well as smaller enhancements.

pglogical 3.7 introduces several major new features as well as architectural changes some of which affect backward compatibility with existing applications.

Important Notes

  • Beta software is not supported in production - for application test only

  • Upgrade from 3.6 is not supported in this release, yet.

Improvements

  • Add support for Postgres 12, deprecate support for older versions
    pglogical 3.7 now requires at least Postgres 10 and supports up to Postgres 12.

Resolved Issues

  • Keep open the connection until pglogical_create_subscriber finishes (RM13649)
    Set idle_in_transaction_session_timeout to 0 so we avoid any user setting that could close the connection and invalidate the snapshot.

pglogical 3.6.19

This is a security and maintenance release for pglogical 3.6 which includes minor features as well as fixes for issues identified previously.

Resolved Issues

  • SECURITY: Set search_path to empty for internal PGLogical SQL statements (RM15373)
    Also, fully qualify all operators used internally. PGLogical is now protected from attack risks identified in CVE-2018-1058, when the user application avoids the insecure coding practices identified there.

  • Correct parsing of direct WAL messages (RT67762)
    Custom WAL messages emitted by PGLogical (or plugins building on top of it) can be broadcast or direct types. Decoding of the latter was incorrect and could in rare cases (depending on the node name) lead to "insufficient data left in message" or memory allocation errors. Decoding of such direct WAL messages has been corrected.

  • Add pglogical.sync_failover_slots() function (RM14318)
    Signal the supervisor process to restart the mechanism to synchronize the failover slots specifyed in the "pglogical.synchronize_failover_slot_name".

  • Fix the --extra-basebackup-args argument passed to pg_basebackup (RM14808)
    Corrects how the pglogical_create_subscriber tool passes on such extra arguments to pg_backbackup.

Improvements

  • Add more diagnostic information to pglogical.queue message (RM15292)
    A new key info has been added to pglogical.queue providing additional information about a queued DDL operation.

pglogical 3.6.18

This is a maintenance release for pglogical 3.6 which includes minor features as well as fixes for issues identified previously.

Improvements

  • Warn about failover issues if standby_slot_names is not set (RT66767, RM12973) If pglogical.standby_slot_names is not set and a physical standby is configured; failover to this standby will have data consistency issues as per our documentation. However, the replica could just be a simple read replica. In any case, we now warn on the replica about the potential data corruption/divergence that could result if failover is desired to such a standby.

  • Check repsets in create_subscription for pgl2 upstreams also.

  • Various improvements to systemtap integration.

Resolved Issues

  • Prevent a hang in case of an early error in the PGL writer (RT67433, RM14678)

  • Allow postgres to start with pglogical library loaded but activity suspended
    Add start_workers commandline-only GUC to facilitate this.

pglogical 3.6.17

This is a maintenance release for pglogical 3.6 which includes minor features as well as fixes for issues identified previously.

Improvements

  • Make the slot synchronization to standby more configurable (RM13111)
    Added several new configuration parameters which tune the behavior of the synchronization of logical replication slots from a primary to a standby PostgreSQL servers. This allows for better filtering, inclusion of non-pglogical replication sets and also using different connection string than physical replication uses (useful when different user or database should be used to collect information about slots).

Resolved Issues

  • Fix issue with UPDATEs on partitions with different physical row representation than partition root (RM13539, RT67045)
    The partitions must have same logical row as partition root they can have different physical representation (primarily due to dropped columns). UPDATEs on such partitions need to do special handling to remap everything correctly otherwise constraints and not-updated TOAST columns will refer to wrong incoming data.

  • Fix truncation of \_tmp slot names in sync slots
    Long slot names could previously cause the temporary slot to be suffixed by \_tm rather than the expected \_tmp suffix.

Support, Diagnostic and Logging Changes

These changes don't directly change existing behaviour or add new user-facing features. They are primarily of interest to technical support operations and for advanced diagnostic analysis.

  • Expand non-invasive tracing (SystemTap, linux-perf, DTrace, etc) support to cover inspection of the pglogical receiver's input protocol stream, walsender output plugin protocol stream, and other useful events. (RM13517)

  • Add a test and debug utility that decodes captured pglogical protocol streams into human-readable form (RM13538)

  • Improve error context logging in the pglogical writer to show more information about the transaction being applied and its origin.

  • Fix incorrectly reported commit lsn in errcontext messages from the pglogical heap writer (RM13796). This fix only affects logging output. The writer would report the lsn of the original forwarded transaction not the lsn of the immediate source transaction.

  • Add subscription, local node and peer node names to heap writer errcontext log output.

pglogical 3.6.16

This is the sixteenth minor release of the Pglogical 3.6 series. This release includes mostly just enables BDR 3.6.16 without any significant changes to pglogical.

pglogical 3.6.15

This is the fifteenth minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Resolved Issues

  • Fix backwards-compatibility to PGLogical 2 (RM13333, RT66919)
    Recent releases performed additional checks during create_subscription, which are fine against other PGLogical 3 installations, but not backwards-compatible. This release corrects the check to account for backwards-compatibility.

  • Correct a warning about GUC nest level not being reset (EE) (RM13375)
    The addition of the lock_timeout in 3.6.14 led to a warning being issued for CAMO and Eager All Node transaction ("GUC nest level = 1 at transaction start"). With this release, GUC nest levels are properly managed and the warning no longer occurs.

Improvements

  • Add a new pglogical.worker_tasks view that tracks and records pglogical's background worker use. The view exposes information about the number of times a given type of worker has been restarted, how long it has been running, whether it accomplished any useful work, and more. This offers administrators more insight into pglogical's internal activity when diagnosing problems, especially when joined against the pglogical.worker_error table.

  • Add support for rate-limiting pglogical background worker (re)launches. The new pglogical.min_worker_backoff_delay configuration option sets a minimum delay between launches of all types of pglogical background workers so that rapid respawning of workers cannot fill the log files and or excessive load on the system that affects other operations.

    For example, if configured with pglogical.min_worker_backoff_delay = '500ms', pglogical will not retry any given background worker startup more often than twice per second (1000/500 = 2).

    A simple fixed-rate factor was deemed to be the most predictable and production-safe initial approach. Future enhancements may add a heuristic delay factor based on worker type, time from start to exit, number of recent launches, etc.

    The launch backoff delay defaults to 0 (off) to prevent surprises for upgrading users.

    A setting of pglogical.min_worker_backoff_delay = '5s' or similar is a reasonable starting point, and may become the default in a future release.

Upgrades

The PostgreSQL Global Development Group has phased out support for PostgreSQL 9.4 on all Debian based distributions. Following that, this release covers only PostgreSQL 9.5 and newer. We advise to upgrade to a newer version.

For RedHat based distributions, this release is still available for PostgreSQL 9.4.

pglogical 3.6.14

This is the fourteenth minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Resolved Issues

  • Resolve deadlocked CAMO or Eager transactions (RM12903, RM12910)
    Add a lock_timeout as well as an abort feedback to the origin node to resolve distributed deadlocking due to conflicting primary key updates. This also prevents frequent restarts and retries of the PGL writer process for Eager All Node and sync CAMO transactions.

pglogical 3.6.12

This is the twelveth minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Improvements

  • Add infrastructure for check_full_row in DELETE operations used by BDR 3.6.12 (RT66493)

  • Validate requested replication sets at subscribe time (RM12020, RT66310)
    pglogical.create_subscription() now checks that all requested replication sets actually exist on the provider node before returning. If any are missing it will raise an ERROR like: ERROR: replication set(s) "nonexistent_repset" requested by subscription are missing on provider with a DETAIL message listing the full sets requested, etc. On prior releases subscriptions with missing repsets would fail after pglogical.create_subscription(...) returned, during initial sync. The failure would only be visible in the logs where it is much less obvious to the user. Or if schema sync was not enable they could appear to succeed but not populate the initial table contents.

Resolved Issues

  • Fix a potential deadlock at CAMO partner startup. (RM12187)
    After a restart, the CAMO partner resends all confirmations for recent CAMO protected transactions. In case these fill the internal queue between the receiver and writer processes, a deadlock was possible. This release ensures the receiver consumes pending feedback messages allowing the writer to make progress.

pglogical 3.6.11

This is the eleventh minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Improvements

  • Implement remote_commit_flush for CAMO. (RM11564)
    Additional level of robustness for CAMO, only replying when xact is known committed and flushed on partner node.

  • Make receiver-writer shared queues of configurable size. (RM11779)
    Two new GUCs are introduced: pglogical.writer_input_queue_size (default 1MB) pglogical.writer_output_queue_size (default 1MB)

  • Add a warning when user tries to set update_origin_change to skip

  • Add callback to request replay progress update. (RM6747)

Resolved Issues

  • Send TimeZone GUC when replicating DDL (RT66019)
    To ensure that timezone dependent expressions in DDL get evaluated to same value on all nodes.

  • Only use isvalid indexes when searching for conflicts (RT66036)
    Indexes currently being created or failed index creations will be ignored, to prevent concurrency issues with change apply and CREATE INDEX CONCURRENTLY.

  • Fix crash when replication invalidations arrive outside a transaction (RM11159)

  • Make the receiver apply the queue before shutting down (RM11778)
    Upon smart shutdown, the PGL writer no longer terminates immediately, requiring queued transactions to be resent, but applies already received transactions prior to shutting down.

pglogical 3.6.10

This is the tenth minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Improvements

  • Add support for a CAMO remote_write mode (RM6749)

Resolved Issues

  • COMMIT after initial sync of a table. This avoids treating the first catchup xact as if it was part of the initial COPY, which could lead to strange errors or false conflicts. (RM11284).

  • Remove the 4 billion row limit during the initial subscription synchronization (RT66050).

  • Cleanup table replication cache when replication set configuration changes.
    Previously we could use stale cache on multiple calls for table replication info on same connection if user changed the configuration in meantime. This could result in initial sync missing replicated table if the configuration was changed while the subscription was being created.

  • Remember repsets when caching table replication info.
    If the client calls the table replication info with different parameters, we need to remember them otherwise we might return cached value for wrong replication sets. This could result in initial sync copying data from table which were not supposed to be replicated.

pglogical 3.6.9

This is the ninth minor release of the Pglogical 3.6 series. This release includes minor improvements.

Improvements

  • Add support for local, remote_apply and remote_write. (RM11069, RT65801)
    We now accept the use of all the values that PostgreSQL accepts when configuring the "pglogical.synchronous_commit".

  • Immediately forward all messages from the PGL receiver back to origin (BDR CAMO)
    Confirmations for CAMO protected transactions flow from the PGL writer applying the transaction back to origin node via the PGL receiver. This process used to consume only one confirmation message per iteration. It now consumes all pending confirmations from the PGL writer and immediately sends them back to the origin. It also decreases latency for BDR CAMO transactions in case confirmations queue up.

pglogical 3.6.8

This is the eigth minor release of the Pglogical 3.6 series. This release includes fixes for issues identified previously.

Resolved Issues

  • Use RelationGetIndexAttrBitmap to get pkey columns. (RT65676, RT65797)
    No need to try to fetch pkey columns from index itself, we have relcache interface that does exactly what we need and does so in more performant way.

pglogical 3.6.7.1

This is a hot-fix release on top of 3.6.7.

Resolved Issues

  • Fix a protocol violation after removal of an origin. (RT65671, RM10605)
    Removal of a replication subscription may lead to a walsender trying to forward data for unknown origins. Prevent emission of an invalid message in that case.

pglogical 3.6.7

pglogical 3.6.7 is the seventh minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified earlier.

Improvements

  • Replicate TRUNCATE on a partition if only parent table is published in replication set (RT65335)
    Previously, we'd skip such TRUNCATE unless the partition was also published.
  • Generate target_table_missing for TRUNCATE which is executed against non-existing table (RT10291)
    Allows for user-configured decision if it should be a replication-stopping issue or not.
  • Improve performance of repeated UPDATEs and DELETEs executed on origin node by caching the replication configuration of tables in a user session.
  • Reduce CPU usage of receiver worker when writer queue is full (RM10370).

Resolved Issues

  • Fix partition replication set membership detection for multi-level partitioned tables
    Replicate changes correctly for multi-level partitioned tables, where only the intermediate partition is part of replication set (not root or leaf partitions).
  • Support replication TRUNCATE CASCADE on tables referenced by FOREIGN KEY (RT65518)
    Previously this would throw error on the subscriber. This will only work if all tables on subscriber which have FOREIGN KEY on the table being TRUNCATEd are replicated tables. Also it's only supported on PostgreSQL 11 and higher.
  • Flush writer between data copy and constraint restore
    Otherwise there could in some rare cases still be unapplied changes when creating constraints during initial synchronization of a subscription, potentially causing deadlocks.
  • Fix potential writer queue corruption on very wide (1000+ columns) tables

Upgrades

This release supports upgrading from following versions of pglogical:

  • 2.2.0
  • 2.2.1
  • 2.2.2
  • 3.2.0 and higher

pglogical 3.6.6

pglogical 3.6.6 is the sixth minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified earlier.

Improvements

  • New conflict type update_pkey_exists (RM9976)
    Allows resolving conflicts when a PRIMARY KEY was updated to one which already exists on the node which is applying the change.

  • Add pglogical.apply_log_summary (RM6596)
    View over pglogical.apply_log which shows the human-readable conflict type and resolver string instead of internal id.

  • Improve logging during both the initial data synchronization of a subscription and the individual table resynchronization.

Resolved Issues

  • Make sure writer flushes changes after initial data copy (RT65185)
    Otherwise depending on timing and I/O load the subscription might not update positioning info and get data both via initial copy and replication stream catchup that follows.

Upgrades

This release supports upgrading from following versions of pglogical:

  • 2.2.0
  • 2.2.1
  • 2.2.2
  • 3.2.0 and higher

pglogical 3.6.5

pglogical 3.6.5 is the fifth minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified in 3.6.4.

Improvements

  • Improve tuple lock waits during apply for deletes (RM9569)
    This should improve performance of replication of deletes and updates in contentious situation.

Resolved Issues

  • Use consistent table list in initial data copy (RM9651/RT64809) To prevent issues during initial data copy and concurrent table drop.
  • Cleanup worker_dsm_handle on worker detach (internal)
    Otherwise we could leave dangling DSM segment handle for a worker after a crash, which could confuse plugins using this API.
  • Fix handling of empty eager transactions (RM9550)
    In case no relevant change remains to be applied on a replica node, the prepare of such an empty transaction now works just fine.
  • Fix the replication sets output in pglogical.pglogical_node_info()
    Previously it could be garbled.
  • Reduce log level for messages when resolving ERRCODE_T_R_SERIALIZATION_FAILUREs (RM9439)

Upgrades

This release supports upgrading from following versions of pglogical:

  • 2.2.0
  • 2.2.1
  • 2.2.2
  • 3.2.0 and higher

Note that upgrades from 2.2.x are only supported on systems with pglogical.conflict_resolution set to last_update_wins.

pglogical 3.6.4

pglogical 3.6.4 is the fourth minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified in 3.6.3.

New Features

  • Apply statistics tracking (RM9063)
    We now track statistics about replication and resource use for individual subscriptions and relations and make them available in pglogical.stat_subscription and pglogical.stat_relation views. The tracking can be configured via pglogical.stat_track_subscription and pglogical.stat_track_relation configuration parameters.
  • The replicate_inserts option now affects initial COPY
    We now do initial copy of data only if the table replicates inserts.

Resolved Issues

  • Fix initial data copy of multi-level partitioned tables (RT64809)
    The initial data copy used to support only single level partitioning, multiple levels of partitioning are now supported.
  • Don't try to copy initial data twice for partitions in some situations (RT64809)
    The initial data copy used to try to copy data from all tables that are in replication sets without proper regard to partitioning. This could result in partition data to be copied twice if both root partition and individual partitions were published via replication set. This is now solved, we only do the initial copy on the root partition if it's published.
  • Fix handling of indexes when replicating INSERT to a partition (RT64809)
    Close the indexes correctly in all situations.
  • Improve partitioning test coverage (RM9311)
    In light of the partitioning related issues, increase the amount of automated testing done against partitioned tables.
  • Fix a leak in usage of the relation cache (RT64935)
  • Fix a potential queue deadlock between writer and receiver (RT64935, RT64714)

pglogical 3.6.3

pglogical 3.6.3 is the third minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified in 3.6.2.

New Features

  • Support DoNotReplicateId special origin
    This allows correct handling of "do not replicate" origin which allows skipping replication of some changes. Primarily needed internally for other features.
  • Persist the last_xact_replay_timestamp (RT63881)
    So that it's visible even if the subscription connection is down.
  • Rework documentation build procedure for better consistency between HTML and PDF documentation
    This mainly changes the way docs are structured into chapters so that there is single source of chapter list and ordering for both PDF and HTML docs.

Resolved Issues

  • Invalidate local cache when adding new invalidation
    Fixes visibility of changes in the catalog cache view of the transaction which did those changes. Not triggered yet by any code but will be in the future releases.
  • Open indexes after partition routing
    Otherwise we might be opening indexes of the root table rather than the partition, causing issues with handling conflicts for INSERT operation replication.

pglogical 3.6.2

pglogical 3.6.2 is the second minor release of the pglogical 3.6 series. This release includes minor new features as well as fixes for issues identified in 3.6.1.

New Features

  • Support DEFERRED UNIQUE indexes
    They used to work only in limited cases before this release.
  • Support covering UNIQUE indexes (RT64650)
    The use of covering UNIQUE indexes could result in ambiguous error messages in some cases before.
  • Add --log-file option to pglogical_create_subscriber (RT64129) So that log can be saved somewhere other than the current working directory

Resolved Issues

  • Fix error message when the database name in the connection string in pglogical_create_subscriber is missing (RT64129) The previous message was ambiguous.
  • Raise error when unknown parameter was specified for pglogical_create_subscriber (RT64129)
    Otherwise mistakes in command line arguments could be silently ignored.
  • Solve timing issue with workers exiting while another one tries to start using same worker slot
    Before, we could corrupt the worker information causing the newly starting worker to crash (and having to start again later), this will no longer happen.
  • Set statement time on start of every transaction in pglogical workers (RT64572)
    Fixes reporting of xact_start in pg_stat_activity

pglogical 3.6.1

pglogical 3.6.1 is the first minor release of the pglogical 3.6 series. This release includes minor new features and fixes including all the fixes from 3.6.0.1.

New Features

  • Add slot failover documentation
  • Add pglogical.get_sub_progress_timestamp for retrieving origin timestamp of the last committed change by the subscription

Resolved Issues

  • Stop retrying subscription synchronization after unrecoverable error (RT64463)
    If the schema synchronization failed (which is an unrecoverable error) don't keep retrying forever. Instead mark the subscription synchronization as failed and disable the subscription.
  • Improve handling and messaging with missing replication sets in output plugin (RT64451)
    Report all missing and found sets and make sure the sets are looked up using current snapshot.

pglogical 3.6.0.1

The pglogical 3.6.0.1 is the first bug-fix release in the pglogical 3.6 series.

Resolved Issues

  • Improve synchronous remote_write replication performance (RT64397)
  • Re-add support for binary protocol

pglogical 3.6.0

The version 3.6 of pglogical is a major update which brings performance improvements, better conflict handling, bug fixes and infrastructure necessary for BDR 3.6.

New Features

  • Significant replication performance improvement
    • Cache table synchronization state
    • Only send keepalives when necessary
    • Only do flush when necessary
    • Serialize transactions in fewer cases in walsender (2ndQPostgres)
  • Improved replication position reporting which is more in line with how physical streaming replication reports it
  • Conflict detection and resolution improvements
    • Add new types of conflicts (like target_table_missing)
    • Add new types of conflict resolvers
    • Make conflict resolution configurable by subscription and conflict type
    • Improve conflict detection for updates

Resolved Issues

  • Don't try to replicate REINDEX on temporary indexes

Other Improvements

  • Fix potential message parsing error for two-phase commits
  • Make initial COPY of data interruptible