Thread: pg_xlog dir not getting swept
About a day ago, there seems to have been some trouble in the network of my database (postgresql 9.3).
I’m running my db with a streaming replication setup with wall shipping.
I sync wal logs to a mounted networkdrive using archive_command = 'rsync -a %p /mnt/wal_drive/wals/%f </dev/null’. Somehow this command was failing, leading to my pg_xlog dir building up (590Gb). I rebooted the server, and the archiving command seems to succeed now - however - After about an hour of running, the pg_xlog drive has not decreased in size - I would have expect that! I can see that lot’s of files get’s synced to the /mnt/wal_drive/wals dir, but somehow the pg_xlog dir is not swept (yet)? Will this happen automatically eventually, or do I need to do something manually?
Best
PS. I found this blog post http://www.hivelogik.com/blog/?p=513, but I’m unsure if it’s necessary and if it can be dangerous?
On Wed, Jun 29, 2016 at 3:00 AM, Niels Kristian Schjødt <nielskristian@autouncle.com> wrote: > About a day ago, there seems to have been some trouble in the network of my > database (postgresql 9.3). > > I’m running my db with a streaming replication setup with wall shipping. > > I sync wal logs to a mounted networkdrive using archive_command = 'rsync -a > %p /mnt/wal_drive/wals/%f </dev/null’. Somehow this command was failing, > leading to my pg_xlog dir building up (590Gb). I rebooted the server, and > the archiving command seems to succeed now - however - After about an hour > of running, the pg_xlog drive has not decreased in size - I would have > expect that! I can see that lot’s of files get’s synced to the > /mnt/wal_drive/wals dir, but somehow the pg_xlog dir is not swept (yet)? > Will this happen automatically eventually, or do I need to do something > manually? Successfully archived files are only removed by the checkpointer. The logic is quite complex and it can be very frustrating trying to predict exactly when any given file will get removed. You might want to run a few manual checkpoints to see if that cleans it up. But turn on log_checkpoints and reload the configuration first. Cheers, Jeff
Thanks, after a few more hours of waiting, things started get cleaned up. Things are back in order now.

Niels Kristian Schjødt
Co-founder & Developer
E-Mail: nielskristian@autouncle.com
Mobile: +45 28 73 04 93

Den 29. jun. 2016 kl. 21.19 skrev Jeff Janes <jeff.janes@gmail.com>:On Wed, Jun 29, 2016 at 3:00 AM, Niels Kristian Schjødt
<nielskristian@autouncle.com> wrote:About a day ago, there seems to have been some trouble in the network of my
database (postgresql 9.3).
I’m running my db with a streaming replication setup with wall shipping.
I sync wal logs to a mounted networkdrive using archive_command = 'rsync -a
%p /mnt/wal_drive/wals/%f </dev/null’. Somehow this command was failing,
leading to my pg_xlog dir building up (590Gb). I rebooted the server, and
the archiving command seems to succeed now - however - After about an hour
of running, the pg_xlog drive has not decreased in size - I would have
expect that! I can see that lot’s of files get’s synced to the
/mnt/wal_drive/wals dir, but somehow the pg_xlog dir is not swept (yet)?
Will this happen automatically eventually, or do I need to do something
manually?
Successfully archived files are only removed by the checkpointer. The
logic is quite complex and it can be very frustrating trying to
predict exactly when any given file will get removed. You might want
to run a few manual checkpoints to see if that cleans it up. But turn
on log_checkpoints and reload the configuration first.
Cheers,
Jeff