From 8ea3d757b925ab2b3c6e06f0a72155e788430e4e Mon Sep 17 00:00:00 2001 From: Hayato Kuroda Date: Tue, 13 Feb 2024 10:59:47 +0000 Subject: [PATCH v23 02/13] Update documentation --- doc/src/sgml/ref/pg_createsubscriber.sgml | 205 +++++++++++++++------- 1 file changed, 142 insertions(+), 63 deletions(-) diff --git a/doc/src/sgml/ref/pg_createsubscriber.sgml b/doc/src/sgml/ref/pg_createsubscriber.sgml index f5238771b7..7cdd047d67 100644 --- a/doc/src/sgml/ref/pg_createsubscriber.sgml +++ b/doc/src/sgml/ref/pg_createsubscriber.sgml @@ -48,19 +48,99 @@ PostgreSQL documentation - + Description - pg_createsubscriber creates a new logical - replica from a physical standby server. + The pg_createsubscriber creates a new subscriber from a physical + standby server. - The pg_createsubscriber should be run at the target - server. The source server (known as publisher server) should accept logical - replication connections from the target server (known as subscriber server). - The target server should accept local logical replication connection. + The pg_createsubscriber must be run at the target + server. The source server (known as publisher server) must accept both + normal and logical replication connections from the target server (known as + subscriber server). The target server must accept normal local connections. + + + There are some prerequisites for both the source and target instance. If + these are not met an error will be reported. + + + + + + The given target data directory must have the same system identifier than the + source data directory. + + + + + The target instance must be used as a physical standby. + + + + + The given database user for the target instance must have privileges for + creating subscriptions and using functions for replication origin. + + + + + The target instance must have + max_replication_slots + and max_logical_replication_workers + configured to a value greater than or equal to the number of target + databases. + + + + + The target instance must have + max_worker_processes + configured to a value greater than the number of target databases. + + + + + The source instance must have + wal_level as + logical. + + + + + The target instance must have + max_replication_slots + configured to a value greater than or equal to the number of target + databases and replication slots. + + + + + The target instance must have + max_wal_senders + configured to a value greater than or equal to the number of target + databases and walsenders. + + + + + + + After the successful conversion, a physical replication slot configured as + primary_slot_name + would be removed from a primary instance. + + + + The pg_createsubscriber focuses on large-scale + systems that contain more data than 1GB. For smaller systems, initial data + synchronization of logical + replication is recommended. + + @@ -191,7 +271,7 @@ PostgreSQL documentation - Notes + How It Works The transformation proceeds in the following steps: @@ -200,97 +280,89 @@ PostgreSQL documentation - pg_createsubscriber checks if the given target data - directory has the same system identifier than the source data directory. - Since it uses the recovery process as one of the steps, it starts the - target server as a replica from the source server. If the system - identifier is not the same, pg_createsubscriber will - terminate with an error. + Checks the target can be converted. In particular, things listed in + above section would be + checked. If these are not met pg_createsubscriber + will terminate with an error. - pg_createsubscriber checks if the target data - directory is used by a physical replica. Stop the physical replica if it is - running. One of the next steps is to add some recovery parameters that - requires a server start. This step avoids an error. + Creates a publication and a logical replication slot for each specified + database on the source instance. These publications and logical replication + slots have generated names: + pg_createsubscriber_%u (parameters: + Database oid) for publications, + pg_createsubscriber_%u_%d (parameters: + Database oid, Pid int) for + replication slots. - - pg_createsubscriber creates one replication slot for - each specified database on the source server. The replication slot name - contains a pg_createsubscriber prefix. These replication - slots will be used by the subscriptions in a future step. A temporary - replication slot is used to get a consistent start location. This - consistent LSN will be used as a stopping point in the parameter and by the - subscriptions as a replication starting point. It guarantees that no - transaction will be lost. + Stops the target instance. This is needed to add some recovery parameters + during the conversion. - - pg_createsubscriber writes recovery parameters into - the target data directory and start the target server. It specifies a LSN - (consistent LSN that was obtained in the previous step) of write-ahead - log location up to which recovery will proceed. It also specifies - promote as the action that the server should take once - the recovery target is reached. This step finishes once the server ends - standby mode and is accepting read-write operations. + Creates a temporary replication slot to get a consistent start location. + The slot has generated names: + pg_createsubscriber_%d_startpoint + (parameters: Pid int). Got consistent LSN will be + used as a stopping point in the + parameter and by the subscriptions as a replication starting point. It + guarantees that no transaction will be lost. + + + + + Writes recovery parameters into the target data directory and starts the + target instance. It specifies a LSN (consistent LSN that was obtained in + the previous step) of write-ahead log location up to which recovery will + proceed. It also specifies promote as the action that + the server should take once the recovery target is reached. This step + finishes once the server ends standby mode and is accepting read-write + operations. - Next, pg_createsubscriber creates one publication - for each specified database on the source server. Each publication - replicates changes for all tables in the database. The publication name - contains a pg_createsubscriber prefix. These publication - will be used by a corresponding subscription in a next step. + Creates a subscription for each specified database on the target instance. + These subscriptions have generated name: + pg_createsubscriber_%u_%d (parameters: + Database oid, Pid int). + These subscription have same subscription options: + create_slot = false, copy_data = false, enabled = false. - pg_createsubscriber creates one subscription for - each specified database on the target server. Each subscription name - contains a pg_createsubscriber prefix. The replication slot - name is identical to the subscription name. It does not copy existing data - from the source server. It does not create a replication slot. Instead, it - uses the replication slot that was created in a previous step. The - subscription is created but it is not enabled yet. The reason is the - replication progress must be set to the consistent LSN but replication - origin name contains the subscription oid in its name. Hence, the - subscription will be enabled in a separate step. + Sets replication progress to the consistent LSN that was obtained in a + previous step. This is the exact LSN to be used as a initial location for + each subscription. - pg_createsubscriber sets the replication progress to - the consistent LSN that was obtained in a previous step. When the target - server started the recovery process, it caught up to the consistent LSN. - This is the exact LSN to be used as a initial location for each - subscription. + Enables the subscription for each specified database on the target server. + The subscription starts streaming from the consistent LSN. - Finally, pg_createsubscriber enables the subscription - for each specified database on the target server. The subscription starts - streaming from the consistent LSN. + Stops the standby server. - pg_createsubscriber stops the target server to change - its system identifier. + Updates a system identifier on the target server. @@ -300,8 +372,15 @@ PostgreSQL documentation Examples - To create a logical replica for databases hr and - finance from a physical replica at foo: + Here is an example of using pg_createsubscriber. + Before running the command, please make sure target server is stopped. + +$ pg_ctl -D /usr/local/pgsql/data stop + + + Then run pg_createsubscriber. Below tries to + create subscriptions for databases hr and + finance from a physical standby: $ pg_createsubscriber -D /usr/local/pgsql/data -P "host=foo" -S "host=localhost" -d hr -d finance -- 2.41.0.windows.3