Thread: pg_basebackup failing

On Wed, Jan 15, 2014 at 6:53 AM, Alan Nilsson <anilsson@apple.com> wrote:
> Could someone give me some insight to the following error message:
>
> [mqsql06:/Volumes/SQL_Set] _postgres% pg_basebackup --user=replicate
> --host=mqsql03 -xP -Fp --pgdata=pgsql
> 19439890/65873894 kB (29%), 1/1 tablespace
> pg_basebackup: could not get transaction log end position from server:
> ERROR:  archive member "base/17040/25491" too large for tar format
>
> I find the error message confusing since I have the format explicitly set to
> plain.
pg_basebackup fetches a full backup from server using tar all the time
for performance purposes to reduce the amount of data sent through the
stream. pg_basebackup contains some code to decode this tarball and
generate a plain format from it.
The error you are seeing is triggered because this relation file
exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
is as well the norm for tar.

> Is there a solution/workaround?
Not use a pg_basebackup but an external backup solution or a custom
script. Break this relation file into several pieces by doing some
partitions on it.
> Can I break up the referenced file somehow?
Partitions on the parent table.
Regards,
--
Michael

On Jan 15, 2014 12:07 PM, "Sameer Kumar" <sameer.kumar@ashnik.com> wrote:
>>
>> The error you are seeing is triggered because this relation file
>> exceeds MAX_TAR_MEMBER_FILELEN or 8GB for a single tar member, which
>> is as well the norm for tar.
>
>
> I thought PostgreSQL would break the file if it grows beyond 1GB (1GB is segment size which one can modify while compiling). Am I missing something or have I got it totally wrong?
>

Yeah, how did you actually end up with a file that size? Do you have any non standard compiler options set when you built your server?

/Magnus