Thread: pgsql: Avoid tricky race condition recording XLOG_HINT
Avoid tricky race condition recording XLOG_HINT We copy the buffer before inserting an XLOG_HINT to avoid WAL CRC errors caused by concurrent hint writes to buffer while share locked. To make this work we refactor RestoreBackupBlock() to allow an XLOG_HINT to avoid the normal path for backup blocks, which assumes the underlying buffer is exclusive locked. Resulting code completely changes layout of XLOG_HINT WAL records, but this isn't even beta code, so this is a low impact change. In passing, avoid taking WALInsertLock for full page writes on checksummed hints, remove related cruft from XLogInsert() and improve xlog_desc record for XLOG_HINT. Andres Freund Bug report by Fujii Masao, testing by Jeff Janes and Jaime Casanova, review by Jeff Davis and Simon Riggs. Applied with changes from review and some comment editing. Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/47c4333189c9eb5ac1f95adc90213e61996f53c4 Modified Files -------------- src/backend/access/rmgrdesc/xlogdesc.c | 6 +- src/backend/access/transam/xlog.c | 232 +++++++++++++++++++------------- src/backend/storage/buffer/bufmgr.c | 4 +- src/include/catalog/catversion.h | 2 +- 4 files changed, 144 insertions(+), 100 deletions(-)
Simon Riggs <simon@2ndQuadrant.com> writes:
> Resulting code completely changes layout of XLOG_HINT WAL records, but
> this isn't even beta code, so this is a low impact change.
> src/include/catalog/catversion.h | 2 +-
Just for the record, the right way to handle that kind of change is to
change XLOG_PAGE_MAGIC, not catversion. A database's catalog version
might not be available to code that is inspecting WAL files and would
like to know what format it should expect.
regards, tom lane
On 8 April 2013 15:29, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Simon Riggs <simon@2ndQuadrant.com> writes:> src/include/catalog/catversion.h | 2 +-
> Resulting code completely changes layout of XLOG_HINT WAL records, but
> this isn't even beta code, so this is a low impact change.
Just for the record, the right way to handle that kind of change is to
change XLOG_PAGE_MAGIC, not catversion. A database's catalog version
might not be available to code that is inspecting WAL files and would
like to know what format it should expect.
I was so happy Andres had remembered to bump the catversion like I often do, it didn't occur to me it was the wrong one. My bad, sorry.
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Simon Riggs <simon@2ndQuadrant.com> writes:
> On 8 April 2013 15:29, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Just for the record, the right way to handle that kind of change is to
>> change XLOG_PAGE_MAGIC, not catversion. A database's catalog version
>> might not be available to code that is inspecting WAL files and would
>> like to know what format it should expect.
> I was so happy Andres had remembered to bump the catversion like I often
> do, it didn't occur to me it was the wrong one. My bad, sorry.
No problem. XLOG_PAGE_MAGIC is already different from what it was in
9.2, so it's mostly academic whether or not we bump it again during
the devel cycle.
regards, tom lane
On 2013-04-08 10:47:50 -0400, Tom Lane wrote: > Simon Riggs <simon@2ndQuadrant.com> writes: > > On 8 April 2013 15:29, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >> Just for the record, the right way to handle that kind of change is to > >> change XLOG_PAGE_MAGIC, not catversion. A database's catalog version > >> might not be available to code that is inspecting WAL files and would > >> like to know what format it should expect. > > > I was so happy Andres had remembered to bump the catversion like I often > > do, it didn't occur to me it was the wrong one. My bad, sorry. Sorry for that. > No problem. XLOG_PAGE_MAGIC is already different from what it was in > 9.2, so it's mostly academic whether or not we bump it again during > the devel cycle. I just didn't want anybodys devel instance crashing because it accessed record data that previously didn't exist. XLOG_PAGE_MAGIC space looks so much more sparse than catversion's ;) Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services