Re: Avoid detoast overhead when possible - Mailing list pgsql-hackers

From Nikita Malakhov
Subject Re: Avoid detoast overhead when possible
Date
Msg-id CAN-LCVP8GHRMYDV4LLcAjBBjNS_rnx3q2hC8y=PY7xau9YsUzw@mail.gmail.com
Whole thread Raw
In response to Re: Avoid detoast overhead when possible  (Andy Fan <zhihuifan1213@163.com>)
List pgsql-hackers
Hi!

Sorry for the long delay, other tasks required my attention and this project
was postponed, but we have some recent requests from clients and this
work has to be revived.

1) In short the idea behind the Json improvements is to introduce iterators
for Json objects to extract these objects value by value, and modification
of in-memory and on-disk representations to hold partially-detoasted Json
objects (tree nodes with need-detoast state) for in-memory, and to store
key-value map in on-disk representation to extract only required values.

2) Backward compatibility would not be a problem because in on-disk
representation we could distinguish old version from new, there are enough
service bits for that. But servers not updated with this patch could not
parse new data, of course.

3) Yes, it definitely has to. These changes are very complex and invasive,
that's the reason I haven't finished the patch.

4) It doesn't seem worthy at first glance. Do you have any ideas on this?

Thank you for your interest!

On Wed, Oct 30, 2024 at 2:59 AM Andy Fan <zhihuifan1213@163.com> wrote:
zhihuifan1213@163.com writes:

Hello Nikita,

>> There's a view from the other angle - detoast just attributes that are needed
>> (partial detoast), with optimized storage mechanics for JSONb.
>
> Very glad to know that,  looking forward your design & patch!

This is interesting, do you mind to provide a high level design of
this. I'm not pushing to do this, but you asked me how I think about
your proposal[1], so I think a high level design is a must to for these
question. A first set of questions includes:

1. How to optimize the JSONb storage?
2. How to be compatible with older version?
3. How to integreate with existing ExprExecutionEngine? looks your
proposal needs to know which "part" of jsonb fields are needed.
4. Is there any optimization if user want to get a full JSONB, like
SELECT jb_col FROM t?

[1]
https://www.postgresql.org/message-id/CAN-LCVO3GZAKVTKNwwcezoc%3D9Lq%3DkU2via-BM3MXVdOq4tD9RQ%40mail.gmail.com

--
Best Regards
Andy Fan



--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: Index AM API cleanup
Next
From: Heikki Linnakangas
Date:
Subject: Re: AIO writes vs hint bits vs checksums