Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding - Mailing list pgsql-hackers

From li jie
Subject Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date
Msg-id CAGfChW5Qo2SrjJ7rU9YYtZbRaWv6v-Z8MJn=dQNx4uCSqDEOHA@mail.gmail.com
Whole thread Raw
In response to Proposal: Filter irrelevant change before reassemble transactions during logical decoding  (li jie <ggysxcq@gmail.com>)
Responses Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
RE: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
List pgsql-hackers
> This is just an immature idea. I haven't started to implement it yet.
> Maybe it was designed this way because there
> are key factors that I didn't consider. So I want to hear everyone's
> opinions, especially the designers of logic decoding.

Attached is the patch I used to implement this optimization.
The main designs are as follows:
1. Added a callback LogicalDecodeFilterByRelCB for the output plugin.

2. Added this callback function pgoutput_table_filter for the pgoutput plugin.
    Its main implementation is based on the table filter in the
pgoutput_change function.

3. After constructing a change and before Queue a change into a transaction,
    use RelidByRelfilenumber to obtain the relation associated with the change,
    just like obtaining the relation in the ReorderBufferProcessTXN function.

4. Relation may be a toast, and there is no good way to get its real
table relation
   based on toast relation. Here, I get the real table oid through
toast relname, and
   then get the real table relation.

5. This filtering takes into account INSERT/UPDATE/INSERT. Other
changes have not
   been considered yet and can be expanded in the future.

6.  The table filter in pgoutput_change and the get relation in
ReorderBufferProcessTXN
  can be deleted. This has not been done yet. This is the next step.

Sincerely look forward to your feedback.
Regards, lijie

Attachment

pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: [PoC] pg_upgrade: allow to upgrade publisher node
Next
From: Nazir Bilal Yavuz
Date:
Subject: Re: Show WAL write and fsync stats in pg_stat_io