Allow wal_keep_segments to keep all segments - Mailing list pgsql-hackers

Bruce Momjian wrote:
> Robert Haas wrote:
> > On Sat, May 8, 2010 at 10:40 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > Bruce Momjian <bruce@momjian.us> writes:
> > >> Uh, did we decide that 'wal_keep_segments' was the best name for this
> > >> GUC setting? ?I know we shipped beta1 using that name.
> > >
> > > I thought min_wal_segments was a reasonable proposal, but it wasn't
> > > clear if there was consensus or not.
> >
> > I think most people thought it was another reasonable choice, but I
> > think the consensus position is probably something like "it's about
> > the same" rather than "it's definitely better".  We had one or two
> > people with stronger opinions than that on either side, I believe.
>
> Agreed the current name seems OK.  However, was there agreement that
> wal_keep_segments = -1 should keep all WAL segements?  I can see that as
> useful for cases where you are doing a dump to be transfered to the
> slave, and not using archive_command.  This avoids the need for the "set
> a huge value" solution.

The attached patch allows wal_keep_segments = -1 to keep all segements;
this is particularly useful for taking a base backup, where you need all
the WAL files during startup of the standby.  I have documented this
usage in the patch as well.

I am thinking of applying this after 9.0 beta2 if there is no objection.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.280
diff -c -c -r1.280 config.sgml
*** doc/src/sgml/config.sgml    31 May 2010 15:50:48 -0000    1.280
--- doc/src/sgml/config.sgml    2 Jun 2010 19:19:18 -0000
***************
*** 1887,1893 ****
          Specifies the number of past log file segments kept in the
          <filename>pg_xlog</>
          directory, in case a standby server needs to fetch them for streaming
!         replication. Each segment is normally 16 megabytes. If a standby
          server connected to the primary falls behind by more than
          <varname>wal_keep_segments</> segments, the primary might remove
          a WAL segment still needed by the standby, in which case the
--- 1887,1893 ----
          Specifies the number of past log file segments kept in the
          <filename>pg_xlog</>
          directory, in case a standby server needs to fetch them for streaming
!         replication.  Each segment is normally 16 megabytes. If a standby
          server connected to the primary falls behind by more than
          <varname>wal_keep_segments</> segments, the primary might remove
          a WAL segment still needed by the standby, in which case the
***************
*** 1901,1908 ****
          is zero (the default), the system doesn't keep any extra segments
          for standby purposes, and the number of old WAL segments available
          for standbys is determined based only on the location of the previous
!         checkpoint and status of WAL archiving.
!         This parameter can only be set in the <filename>postgresql.conf</>
          file or on the server command line.
         </para>
         </listitem>
--- 1901,1909 ----
          is zero (the default), the system doesn't keep any extra segments
          for standby purposes, and the number of old WAL segments available
          for standbys is determined based only on the location of the previous
!         checkpoint and status of WAL archiving.  If <literal>-1</> is
!         specified, log file segments are kept indefinitely. This
!         parameter can only be set in the <filename>postgresql.conf</>
          file or on the server command line.
         </para>
         </listitem>
Index: doc/src/sgml/high-availability.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/high-availability.sgml,v
retrieving revision 1.70
diff -c -c -r1.70 high-availability.sgml
*** doc/src/sgml/high-availability.sgml    29 May 2010 09:01:10 -0000    1.70
--- doc/src/sgml/high-availability.sgml    2 Jun 2010 19:19:19 -0000
***************
*** 750,756 ****
      If you use streaming replication without file-based continuous
      archiving, you have to set <varname>wal_keep_segments</> in the master
      to a value high enough to ensure that old WAL segments are not recycled
!     too early, while the standby might still need them to catch up. If the
      standby falls behind too much, it needs to be reinitialized from a new
      base backup. If you set up a WAL archive that's accessible from the
      standby, wal_keep_segments is not required as the standby can always
--- 750,760 ----
      If you use streaming replication without file-based continuous
      archiving, you have to set <varname>wal_keep_segments</> in the master
      to a value high enough to ensure that old WAL segments are not recycled
!     too early, while the standby might still need them to catch up. This
!     is particularly important when performing a base backup because the
!     standby will need all WAL segments generated since the start of the
!     backup;  consider setting <varname>wal_keep_segments</> to
!     <literal>-1</> temporarily in such cases.  If the
      standby falls behind too much, it needs to be reinitialized from a new
      base backup. If you set up a WAL archive that's accessible from the
      standby, wal_keep_segments is not required as the standby can always
Index: src/backend/access/transam/xlog.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.414
diff -c -c -r1.414 xlog.c
*** src/backend/access/transam/xlog.c    27 May 2010 00:38:39 -0000    1.414
--- src/backend/access/transam/xlog.c    2 Jun 2010 19:19:20 -0000
***************
*** 7339,7345 ****
       * Delete old log files (those no longer needed even for previous
       * checkpoint or the standbys in XLOG streaming).
       */
!     if (_logId || _logSeg)
      {
          /*
           * Calculate the last segment that we need to retain because of
--- 7339,7345 ----
       * Delete old log files (those no longer needed even for previous
       * checkpoint or the standbys in XLOG streaming).
       */
!     if ((_logId || _logSeg) && wal_keep_segments != -1)
      {
          /*
           * Calculate the last segment that we need to retain because of
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.554
diff -c -c -r1.554 guc.c
*** src/backend/utils/misc/guc.c    2 May 2010 02:10:33 -0000    1.554
--- src/backend/utils/misc/guc.c    2 Jun 2010 19:19:22 -0000
***************
*** 1661,1667 ****
              NULL
          },
          &wal_keep_segments,
!         0, 0, INT_MAX, NULL, NULL
      },

      {
--- 1661,1667 ----
              NULL
          },
          &wal_keep_segments,
!         0, -1, INT_MAX, NULL, NULL
      },

      {