Re: Incorrect handling of OOM in WAL replay leading to data loss - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Incorrect handling of OOM in WAL replay leading to data loss
Date
Msg-id ZMiSKDYZ93e7fFB7@paquier.xyz
Whole thread Raw
In response to Re: Incorrect handling of OOM in WAL replay leading to data loss  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
Responses Re: Incorrect handling of OOM in WAL replay leading to data loss
List pgsql-hackers
On Tue, Aug 01, 2023 at 01:51:13PM +0900, Kyotaro Horiguchi wrote:
> I believe a database server is not supposed to be executed under such
> a memory-constrained environment.

I don't really follow this argument.  The backend and the frontends
are reliable on OOM, where we generate ERRORs or even FATALs depending
on the code path involved.  A memory bounded environment is something
that can easily happen if one's not careful enough with the sizing of
the instance.  For example, this error can be triggered on a standby
with read-only queries that put pressure on the host's memory.

> One issue on changing that behavior is that there's not a simple way
> to detect a broken record before loading it into memory. We might be
> able to implement a fallback mechanism for example that loads the
> record into an already-allocated buffer (which is smaller than the
> specified length) just to verify if it's corrupted. However, I
> question whether it's worth the additional complexity. And I'm not
> sure what if the first allocation failed.

Perhaps we could rely more on a fallback memory, especially if it is
possible to use that for the header validation.  That seems like a
separate thing, still.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: "Hayato Kuroda (Fujitsu)"
Date:
Subject: Fix compilation warnings when CFLAGS -Og is specified
Next
From: Jeff Davis
Date:
Subject: Re: Faster "SET search_path"