Thread: Stale description for pg_basebackup
Hello. It seems to me that there's a stale description in the documentation of pg_basebackup. https://www.postgresql.org/docs/13/app-pgbasebackup.html > Note that there are some limitations in taking a backup from a standby: ... > If you are using -X none, there is no guarantee that all WAL files > required for the backup are archived at the end of backup. Actually, pg_basebackup waits for the all required files to be archived, which is an established behavior by commit 52f8a59dd9@PG10. However, the same commit seems to have forgot to change the doc for pg_basebackup. (The current description is introduced by 9a4d51077c@PG10) The attached is a proposal to rewrite it as the following. + If you are using -X none, pg_basebackup may wait for a long time for + all the required WAL files to be archived. In that case, You may need + to call pg_switch_wal() on the primary to complete it sooner. regards. -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index a3e292d44a..73d5cbeaf7 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -83,8 +83,7 @@ PostgreSQL documentation </listitem> <listitem> <para> - If you are using <literal>-X none</literal>, there is no guarantee that all - WAL files required for the backup are archived at the end of backup. + If you are using <literal>-X none</literal>, pg_basebackup may wait for a long time for all the required WAL filesto be archived. In that case, You may need to call <function>pg_switch_wal()</function> on the primary to completeit sooner. </para> </listitem> <listitem>
At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in > Hello. > > It seems to me that there's a stale description in the documentation > of pg_basebackup. > > https://www.postgresql.org/docs/13/app-pgbasebackup.html > > > Note that there are some limitations in taking a backup from a standby: > ... > > If you are using -X none, there is no guarantee that all WAL files > > required for the backup are archived at the end of backup. > > Actually, pg_basebackup waits for the all required files to be > archived, which is an established behavior by commit > 52f8a59dd9@PG10. However, the same commit seems to have forgot to > change the doc for pg_basebackup. (The current description is > introduced by 9a4d51077c@PG10) > > The attached is a proposal to rewrite it as the following. > > + If you are using -X none, pg_basebackup may wait for a long time for > + all the required WAL files to be archived. In that case, You may need > + to call pg_switch_wal() on the primary to complete it sooner. I forgot to preserve the description about *primary*. It should be as the following instead. + If you are using -X none, there is no guarantee on the primary that + all WAL files required for the backup are archived at the end of + backup. When the standby is configured as archive_mode=always, + pg_basebackup may wait for a long time for all the required WAL files + to be archived. In that case, You may need to call pg_switch_wal() on + the primary to complete it sooner. Attached. regards. -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index a3e292d44a..35d7fb605f 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -83,8 +83,13 @@ PostgreSQL documentation </listitem> <listitem> <para> - If you are using <literal>-X none</literal>, there is no guarantee that all - WAL files required for the backup are archived at the end of backup. + If you are using <literal>-X none</literal>, there is no guarantee on + the primary that all WAL files required for the backup are archived at + the end of backup. When the standby is configured as + archive_mode=always, pg_basebackup may wait for a long time for all the + required WAL files to be archived. In that case, You may need to + call <function>pg_switch_wal()</function> on the primary to complete it + sooner. </para> </listitem> <listitem>
At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in > At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in > > Hello. > > > > It seems to me that there's a stale description in the documentation > > of pg_basebackup. > > > > https://www.postgresql.org/docs/13/app-pgbasebackup.html > > > > > Note that there are some limitations in taking a backup from a standby: > > ... > > > If you are using -X none, there is no guarantee that all WAL files > > > required for the backup are archived at the end of backup. > > > > Actually, pg_basebackup waits for the all required files to be > > archived, which is an established behavior by commit > > 52f8a59dd9@PG10. However, the same commit seems to have forgot to > > change the doc for pg_basebackup. (The current description is > > introduced by 9a4d51077c@PG10) > > > > The attached is a proposal to rewrite it as the following. > > > > + If you are using -X none, pg_basebackup may wait for a long time for > > + all the required WAL files to be archived. In that case, You may need > > + to call pg_switch_wal() on the primary to complete it sooner. > > I forgot to preserve the description about *primary*. It should be as > the following instead. > > + If you are using -X none, there is no guarantee on the primary that > + all WAL files required for the backup are archived at the end of > + backup. When the standby is configured as archive_mode=always, > + pg_basebackup may wait for a long time for all the required WAL files > + to be archived. In that case, You may need to call pg_switch_wal() on > + the primary to complete it sooner. Hmm. Some words need to be qualified. Attached. regards. -- Kyotaro Horiguchi NTT Open Source Software Center diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml index a3e292d44a..15e48f5a8e 100644 --- a/doc/src/sgml/ref/pg_basebackup.sgml +++ b/doc/src/sgml/ref/pg_basebackup.sgml @@ -83,8 +83,14 @@ PostgreSQL documentation </listitem> <listitem> <para> - If you are using <literal>-X none</literal>, there is no guarantee that all - WAL files required for the backup are archived at the end of backup. + If you are using <literal>-X none</literal>, there is no guarantee on + the primary that all WAL files required for the backup are archived at + the end of backup. When <varname>archive_mode</varname> is set + to <literal>on</literal> on the + standby, <application>pg_basebackup</application> may wait for a long + time for all the required WAL files to be archived. In that case, You + may need to call <function>pg_switch_wal()</function> on the primary to + complete it sooner. </para> </listitem> <listitem>
On 2021/04/21 11:09, Kyotaro Horiguchi wrote: > At Wed, 21 Apr 2021 10:43:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in >> At Tue, 20 Apr 2021 13:32:35 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in >>> Hello. >>> >>> It seems to me that there's a stale description in the documentation >>> of pg_basebackup. I think you're right. > Hmm. Some words need to be qualified. Attached. + If you are using <literal>-X none</literal>, there is no guarantee on + the primary that all WAL files required for the backup are archived at + the end of backup. I don't think that this should be picked up as a limitation of standby backup. Because users basically want to make pg_basebackup wait for all required WAL files to be archived on the standby, in the standby backup case. When <varname>archive_mode</varname> is set + to <literal>on</literal> on the "on" should be "always"? + standby, <application>pg_basebackup</application> may wait for a long + time for all the required WAL files to be archived. In that case, You + may need to call <function>pg_switch_wal()</function> on the primary to + complete it sooner. What about the following description? ------------------- When you are using -X none, if write activity on the primary is low, pg_basebackup may need to wait a long time for all WAL files required for the backup to be archived. It may be useful to run pg_switch_wal on the primary in order to trigger an immediate WAL file switch and archiving. ------------------- Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
Ugg. I was confused. At Wed, 21 Apr 2021 23:06:56 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in > > Hmm. Some words need to be qualified. Attached. > > + If you are using <literal>-X none</literal>, there is no guarantee > on > + the primary that all WAL files required for the backup are archived > at > + the end of backup. > > I don't think that this should be picked up as a limitation of standby > backup. > Because users basically want to make pg_basebackup wait for all > required > WAL files to be archived on the standby, in the standby backup case. Yeah, you're right. I think it is what I thought at first. The last proposal is a result of some confusion.. > When <varname>archive_mode</varname> is set > + to <literal>on</literal> on the > > "on" should be "always"? Yes.. > + standby, <application>pg_basebackup</application> may wait for a > long > + time for all the required WAL files to be archived. In that case, > You > + may need to call <function>pg_switch_wal()</function> on the primary > to > + complete it sooner. > > What about the following description? > > ------------------- > When you are using -X none, if write activity on the primary is low, > pg_basebackup may need to wait a long time for all WAL files required > for > the backup to be archived. It may be useful to run pg_switch_wal > on the primary in order to trigger an immediate WAL file switch and > archiving. > ------------------- Looks far better. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On 2021/04/22 9:25, Kyotaro Horiguchi wrote: >> What about the following description? >> >> ------------------- >> When you are using -X none, if write activity on the primary is low, >> pg_basebackup may need to wait a long time for all WAL files required >> for >> the backup to be archived. It may be useful to run pg_switch_wal >> on the primary in order to trigger an immediate WAL file switch and >> archiving. >> ------------------- > > Looks far better. Patch attached. I appended the following description to assist users to understand why pg_basebackup may need wait a long time when write activity is low on the primary. ------------------ pg_basebackup cannot force the standby to switch to a new WAL file at the end of backup. ------------------ Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
Attachment
At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in > > > On 2021/04/22 9:25, Kyotaro Horiguchi wrote: > >> What about the following description? > >> > >> ------------------- > >> When you are using -X none, if write activity on the primary is low, > >> pg_basebackup may need to wait a long time for all WAL files required > >> for > >> the backup to be archived. It may be useful to run pg_switch_wal > >> on the primary in order to trigger an immediate WAL file switch and > >> archiving. > >> ------------------- > > Looks far better. > > Patch attached. I appended the following description to assist > users to understand why pg_basebackup may need wait a long time > when write activity is low on the primary. > > ------------------ > pg_basebackup cannot force the standby to switch to > a new WAL file at the end of backup. > ------------------ I'm not sure which is the convention here, but I saw that some function names in the doc are followed by parentheses (ie pg_switch_wal()). (prepended?) It seems a bit redundant but also a bit clearer. How about the following simplification? - It may be useful to run pg_switch_wal on the primary in order to - trigger an immediate WAL file switch and archiving. + It may be useful to run pg_switch_wal() on the primary in that case. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On 2021/04/22 11:19, Kyotaro Horiguchi wrote:
> At Thu, 22 Apr 2021 10:56:10 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in
>>
>>
>> On 2021/04/22 9:25, Kyotaro Horiguchi wrote:
>>>> What about the following description?
>>>>
>>>> -------------------
>>>> When you are using -X none, if write activity on the primary is low,
>>>> pg_basebackup may need to wait a long time for all WAL files required
>>>> for
>>>> the backup to be archived. It may be useful to run pg_switch_wal
>>>> on the primary in order to trigger an immediate WAL file switch and
>>>> archiving.
>>>> -------------------
>>> Looks far better.
>>
>> Patch attached. I appended the following description to assist
>> users to understand why pg_basebackup may need wait a long time
>> when write activity is low on the primary.
>>
>> ------------------
>> pg_basebackup cannot force the standby to switch to
>> a new WAL file at the end of backup.
>> ------------------
>
> I'm not sure which is the convention here, but I saw that some
> function names in the doc are followed by parentheses (ie
> pg_switch_wal()).
Either works for me. I didn't add "()" because I just used the same description
as that in func.sgml.
it may be useful to run <function>pg_switch_wal</function> on the
primary in order to trigger an immediate segment switch.)
> (prepended?) It seems a bit redundant but also a bit clearer. How
> about the following simplification?
>
> - It may be useful to run pg_switch_wal on the primary in order to
> - trigger an immediate WAL file switch and archiving.
> + It may be useful to run pg_switch_wal() on the primary in that case.
IMO "in order to..." part is helpful for us to understand why pg_switch_wal
is useful in this case. So I'd like to leave it.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in > Either works for me. I didn't add "()" because I just used the same > description > as that in func.sgml. > > it may be useful to run <function>pg_switch_wal</function> on the > primary in order to trigger an immediate segment switch.) .. > IMO "in order to..." part is helpful for us to understand why > pg_switch_wal > is useful in this case. So I'd like to leave it. Ok, I'm fine with both of them. Thanks for the explanation. regards. -- Kyotaro Horiguchi NTT Open Source Software Center
On 2021/04/22 13:25, Kyotaro Horiguchi wrote: > At Thu, 22 Apr 2021 13:06:50 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in >> Either works for me. I didn't add "()" because I just used the same >> description >> as that in func.sgml. >> >> it may be useful to run <function>pg_switch_wal</function> on the >> primary in order to trigger an immediate segment switch.) > .. >> IMO "in order to..." part is helpful for us to understand why >> pg_switch_wal >> is useful in this case. So I'd like to leave it. > > Ok, I'm fine with both of them. Thanks for the explanation. Pushed. Thanks! -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION