Re: Restore fails if using multiple threads and database is dumped through pipe - Mailing list pgsql-admin

From Tom Lane
Subject Re: Restore fails if using multiple threads and database is dumped through pipe
Date
Msg-id 12160.1579535943@sss.pgh.pa.us
Whole thread Raw
In response to Restore fails if using multiple threads and database is dumpedthrough pipe  (Joni Ruuskanen <joni.ruuskanen@likeit.fi>)
List pgsql-admin
Joni Ruuskanen <joni.ruuskanen@likeit.fi> writes:
> After upgrading from Postgresql 11 to 12 I noticed pg_restore fails if -j / jobs parameter is specified and database
isdumped through pipe. 
> This doesn't happen if dump is saved through redirect (for example: > file.dmp) or file parameter (-f).
> Is this intended behaviour?

> Dump to file via stdout: pg_dump -Fc dumptest > dumptest-stdout.dmp
> Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-stdout.dmp
> No errors

> Dump to file via pipe: pg_dump -Fc dumptest | tee dumptest-pipe.dmp >/dev/null
> Restore: pg_restore -j4 -d dumptest -O -Fc --role=tomcat dumptest-pipe.dmp
> pg_restore: error: could not find block ID 5648 in archive -- possibly due to out-of-order restore request, which
cannotbe handled due to lack of data offsets in archive 

Yes, if you don't write directly to the output file then pg_dump has
no way to seek in the file, so it can't go back and fill in the data
offsets in the table of contents after it finishes dumping the tables.
pg_restore can cope with the lack of information as long as it's doing
a simple serial restore; but in parallel mode, not so much.

            regards, tom lane



pgsql-admin by date:

Previous
From: Joni Ruuskanen
Date:
Subject: Restore fails if using multiple threads and database is dumpedthrough pipe
Next
From: Shrikant Bhende
Date:
Subject: Re: pg_upgrade from 9.5.10 to 12 fails with encoding issue