Thread: BUG #17865: Logical decoding : JDBC - Out of memory exception (WAL2JSON Format -1) gives a wrong associated LSN
BUG #17865: Logical decoding : JDBC - Out of memory exception (WAL2JSON Format -1) gives a wrong associated LSN
From
PG Bug reporting form
Date:
The following bug has been logged on the website: Bug reference: 17865 Logged by: Reynold Regan Edward Email address: reynoldregan94@gmail.com PostgreSQL version: 13.9 Operating system: Linux Description: This issue happens while reading from the logical replication slot (WAL2JSON plugin). When there are multiple sessions writing to Postgres tables and there is a Postgres StringBuffer limit exceeded exception (processing transactions greater than 1GB size), the associated LSN reported in the SQLException object is incorrect. Steps to reproduce 1. Create a logical replication slot with Wal2Json. 2. Start the client Java Replication API Read. 3. Session 1: start one small transaction & commit Session 2 : start one large transaction - 5million insert Session 3 : start 2nd large transaction - 5 million insert Session 4: When when session 2 and 3 are running , keep doing small transactions. Now, the Java logical decoding API reading from the slot will fail with Out of memory error while processing the session 2's transaction. Exception object received contains irrelevant LSN marked as associated LSN. Note: Incase of single session writing to the database the Associated LSN is clearly of the failing transaction.
Re: BUG #17865: Logical decoding : JDBC - Out of memory exception (WAL2JSON Format -1) gives a wrong associated LSN
From
"Euler Taveira"
Date:
On Thu, Mar 23, 2023, at 9:22 AM, PG Bug reporting form wrote:
This issue happens while reading from the logical replication slot (WAL2JSONplugin).When there are multiple sessions writing to Postgres tables and there is aPostgres StringBuffer limit exceeded exception (processing transactionsgreater than 1GB size), the associated LSN reported in the SQLExceptionobject is incorrect.
If you have a big transaction then it is a format 1 limitation. In this case,
you have 2 options:
* use write-in-chunks parameter. It is only available for format 1;
* use format 2.
Format 1 is the default but format 2 avoid issues with big transactions.
Now, the Java logical decoding API reading from the slot will fail with Outof memory error while processing the session 2's transaction. Exceptionobject received contains irrelevant LSN marked as associated LSN.Note: Incase of single session writing to the database the Associated LSN isclearly of the failing transaction.
I know nothing about JDBC, if the options above does not solve your problem, I
suggest that you ask in pgsql-jdbc mailing list. If after investigation, you
conclude that it is a wal2json limitation/bug, open a issue [1].