Re: Allow wal_keep_segments to keep all segments - Mailing list pgsql-hackers

Heikki Linnakangas wrote:
> On 03/06/10 15:15, Bruce Momjian wrote:
> > Simon Riggs wrote:
> >> I think its much easier to find out your free disk space than it is to
> >> calculate how much WAL might be generated during backup. Disk space
> >> doesn't vary significantly on a production database.
> >>
> >> If we encourage that laziness then we will get reports that replication
> >> doesn't work and Postgres crashes.
> >
> > Well, we don't clean out the archive directory so I don't see this as
> > anything new.
>
> We leave that up to the DBA to clean out one way or another. We provide
> restartpoint_command and the %r option in restore_command to help with that.
>
> Surely we don't expect DBAs to delete old files in pg_xlog? I agree with
> Simon here, I think it would be better to not provide -1 as an option
> here. At least you better document well that you should only do that
> temporarily or you will eventually run out of disk space.

Using this only temporarily is mentioned in the doc patch.  Do I need
more?

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

  + None of us is going to be here forever. +
Index: doc/src/sgml/config.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/config.sgml,v
retrieving revision 1.280
diff -c -c -r1.280 config.sgml
*** doc/src/sgml/config.sgml    31 May 2010 15:50:48 -0000    1.280
--- doc/src/sgml/config.sgml    2 Jun 2010 19:19:18 -0000
***************
*** 1887,1893 ****
          Specifies the number of past log file segments kept in the
          <filename>pg_xlog</>
          directory, in case a standby server needs to fetch them for streaming
!         replication. Each segment is normally 16 megabytes. If a standby
          server connected to the primary falls behind by more than
          <varname>wal_keep_segments</> segments, the primary might remove
          a WAL segment still needed by the standby, in which case the
--- 1887,1893 ----
          Specifies the number of past log file segments kept in the
          <filename>pg_xlog</>
          directory, in case a standby server needs to fetch them for streaming
!         replication.  Each segment is normally 16 megabytes. If a standby
          server connected to the primary falls behind by more than
          <varname>wal_keep_segments</> segments, the primary might remove
          a WAL segment still needed by the standby, in which case the
***************
*** 1901,1908 ****
          is zero (the default), the system doesn't keep any extra segments
          for standby purposes, and the number of old WAL segments available
          for standbys is determined based only on the location of the previous
!         checkpoint and status of WAL archiving.
!         This parameter can only be set in the <filename>postgresql.conf</>
          file or on the server command line.
         </para>
         </listitem>
--- 1901,1909 ----
          is zero (the default), the system doesn't keep any extra segments
          for standby purposes, and the number of old WAL segments available
          for standbys is determined based only on the location of the previous
!         checkpoint and status of WAL archiving.  If <literal>-1</> is
!         specified, log file segments are kept indefinitely. This
!         parameter can only be set in the <filename>postgresql.conf</>
          file or on the server command line.
         </para>
         </listitem>
Index: doc/src/sgml/high-availability.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/high-availability.sgml,v
retrieving revision 1.70
diff -c -c -r1.70 high-availability.sgml
*** doc/src/sgml/high-availability.sgml    29 May 2010 09:01:10 -0000    1.70
--- doc/src/sgml/high-availability.sgml    2 Jun 2010 19:19:19 -0000
***************
*** 750,756 ****
      If you use streaming replication without file-based continuous
      archiving, you have to set <varname>wal_keep_segments</> in the master
      to a value high enough to ensure that old WAL segments are not recycled
!     too early, while the standby might still need them to catch up. If the
      standby falls behind too much, it needs to be reinitialized from a new
      base backup. If you set up a WAL archive that's accessible from the
      standby, wal_keep_segments is not required as the standby can always
--- 750,760 ----
      If you use streaming replication without file-based continuous
      archiving, you have to set <varname>wal_keep_segments</> in the master
      to a value high enough to ensure that old WAL segments are not recycled
!     too early, while the standby might still need them to catch up. This
!     is particularly important when performing a base backup because the
!     standby will need all WAL segments generated since the start of the
!     backup;  consider setting <varname>wal_keep_segments</> to
!     <literal>-1</> temporarily in such cases.  If the
      standby falls behind too much, it needs to be reinitialized from a new
      base backup. If you set up a WAL archive that's accessible from the
      standby, wal_keep_segments is not required as the standby can always
Index: src/backend/access/transam/xlog.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.414
diff -c -c -r1.414 xlog.c
*** src/backend/access/transam/xlog.c    27 May 2010 00:38:39 -0000    1.414
--- src/backend/access/transam/xlog.c    2 Jun 2010 19:19:20 -0000
***************
*** 7339,7345 ****
       * Delete old log files (those no longer needed even for previous
       * checkpoint or the standbys in XLOG streaming).
       */
!     if (_logId || _logSeg)
      {
          /*
           * Calculate the last segment that we need to retain because of
--- 7339,7345 ----
       * Delete old log files (those no longer needed even for previous
       * checkpoint or the standbys in XLOG streaming).
       */
!     if ((_logId || _logSeg) && wal_keep_segments != -1)
      {
          /*
           * Calculate the last segment that we need to retain because of
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.554
diff -c -c -r1.554 guc.c
*** src/backend/utils/misc/guc.c    2 May 2010 02:10:33 -0000    1.554
--- src/backend/utils/misc/guc.c    2 Jun 2010 19:19:22 -0000
***************
*** 1661,1667 ****
              NULL
          },
          &wal_keep_segments,
!         0, 0, INT_MAX, NULL, NULL
      },

      {
--- 1661,1667 ----
              NULL
          },
          &wal_keep_segments,
!         0, -1, INT_MAX, NULL, NULL
      },

      {