pgsql: Add sequence synchronization for logical replication. - Mailing list pgsql-committers

Add sequence synchronization for logical replication.

This patch introduces sequence synchronization. Sequences that are synced
will have 2 states:
   - INIT (needs [re]synchronizing)
   - READY (is already synchronized)

A new sequencesync worker is launched as needed to synchronize sequences.
A single sequencesync worker is responsible for synchronizing all
sequences. It begins by retrieving the list of sequences that are flagged
for synchronization, i.e., those in the INIT state. These sequences are
then processed in batches, allowing multiple entries to be synchronized
within a single transaction. The worker fetches the current sequence
values and page LSNs from the remote publisher, updates the corresponding
sequences on the local subscriber, and finally marks each sequence as
READY upon successful synchronization.

Sequence synchronization occurs in 3 places:
1) CREATE SUBSCRIPTION
    - The command syntax remains unchanged.
    - The subscriber retrieves sequences associated with publications.
    - Published sequences are added to pg_subscription_rel with INIT
      state.
    - Initiate the sequencesync worker to synchronize all sequences.

2) ALTER SUBSCRIPTION ... REFRESH PUBLICATION
    - The command syntax remains unchanged.
    - Dropped published sequences are removed from pg_subscription_rel.
    - Newly published sequences are added to pg_subscription_rel with INIT
      state.
    - Initiate the sequencesync worker to synchronize only newly added
      sequences.

3) ALTER SUBSCRIPTION ... REFRESH SEQUENCES
    - A new command introduced for PG19 by f0b3573c3a.
    - All sequences in pg_subscription_rel are reset to INIT state.
    - Initiate the sequencesync worker to synchronize all sequences.
    - Unlike "ALTER SUBSCRIPTION ... REFRESH PUBLICATION" command,
      addition and removal of missing sequences will not be done in this
      case.

Author: Vignesh C <vignesh21@gmail.com>
Reviewed-by: shveta malik <shveta.malik@gmail.com>
Reviewed-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Nisha Moond <nisha.moond412@gmail.com>
Reviewed-by: Shlok Kyal <shlok.kyal.oss@gmail.com>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAA4eK1LC+KJiAkSrpE_NwvNdidw9F2os7GERUeSxSKv71gXysQ@mail.gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/5509055d6956745532e65ab218e15b99d87d66ce

Modified Files
--------------
src/backend/catalog/pg_subscription.c          |   2 +-
src/backend/commands/sequence.c                |  18 +-
src/backend/postmaster/bgworker.c              |   5 +-
src/backend/replication/logical/Makefile       |   1 +
src/backend/replication/logical/launcher.c     |  57 +-
src/backend/replication/logical/meson.build    |   1 +
src/backend/replication/logical/sequencesync.c | 745 +++++++++++++++++++++++++
src/backend/replication/logical/syncutils.c    | 139 ++++-
src/backend/replication/logical/tablesync.c    |  77 +--
src/backend/replication/logical/worker.c       |  76 ++-
src/backend/utils/misc/guc_parameters.dat      |   2 +-
src/include/catalog/catversion.h               |   2 +-
src/include/catalog/pg_proc.dat                |   2 +-
src/include/catalog/pg_subscription_rel.h      |  23 +
src/include/commands/sequence.h                |   1 +
src/include/replication/logicalworker.h        |   3 +-
src/include/replication/worker_internal.h      |  22 +-
src/test/subscription/t/036_sequences.pl       | 177 +++++-
src/tools/pgindent/typedefs.list               |   2 +
19 files changed, 1229 insertions(+), 126 deletions(-)