Thread: Regarding GIN Fast Update Technique
Hello, At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? -- Amit Langote
On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote: > At what point do the entries in the pending list are moved to the main > GIN data structure? > From documentation, I read that overflowing work_mem and vacuum are > two such causes; what about when the concerned backend is to exit and > autovacuum has not yet kicked in? I don't think there's any special handling for that case, nor do I think any is needed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote: > On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote: >> At what point do the entries in the pending list are moved to the main >> GIN data structure? >> From documentation, I read that overflowing work_mem and vacuum are >> two such causes; what about when the concerned backend is to exit and >> autovacuum has not yet kicked in? > > I don't think there's any special handling for that case, nor do I > think any is needed. > Okay, aside from that case, what else would move those to the main structure? They (the entries in the unsorted pending list) are in the local memory (work_mem?) of the backend, right? -- Amit Langote
On 2013-06-07 23:28:56 +0900, Amit Langote wrote: > On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote: > > On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote: > >> At what point do the entries in the pending list are moved to the main > >> GIN data structure? > >> From documentation, I read that overflowing work_mem and vacuum are > >> two such causes; what about when the concerned backend is to exit and > >> autovacuum has not yet kicked in? > > > > I don't think there's any special handling for that case, nor do I > > think any is needed. > > > > Okay, aside from that case, what else would move those to the main > structure? They (the entries in the unsorted pending list) are in the > local memory (work_mem?) of the backend, right? No, it's in the normal heap, pointed to by the metapage. Storing it in local memory would mean we would have to flush it out before commit. That part of gin is actually quite readable code, so I suggest looking there. Start with ginfast.c:ginHeapTupleFastInsert(). Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training & Services
Amit Langote <amitlangote09@gmail.com> writes:
> Okay, aside from that case, what else would move those to the main
> structure? They (the entries in the unsorted pending list) are in the
> local memory (work_mem?) of the backend, right?
No. If they were, it wouldn't be crash-safe.
regards, tom lane
On Fri, Jun 7, 2013 at 11:36 PM, Andres Freund <andres@2ndquadrant.com> wrote: > On 2013-06-07 23:28:56 +0900, Amit Langote wrote: >> On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote: >> > On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote: >> >> At what point do the entries in the pending list are moved to the main >> >> GIN data structure? >> >> From documentation, I read that overflowing work_mem and vacuum are >> >> two such causes; what about when the concerned backend is to exit and >> >> autovacuum has not yet kicked in? >> > >> > I don't think there's any special handling for that case, nor do I >> > think any is needed. >> > >> >> Okay, aside from that case, what else would move those to the main >> structure? They (the entries in the unsorted pending list) are in the >> local memory (work_mem?) of the backend, right? > > No, it's in the normal heap, pointed to by the metapage. Storing it in > local memory would mean we would have to flush it out before commit. > > That part of gin is actually quite readable code, so I suggest looking > there. Start with ginfast.c:ginHeapTupleFastInsert(). > Thanks. -- Amit Langote
On 07/06/13 16:39, Tom Lane wrote: > Amit Langote <amitlangote09@gmail.com> writes: >> Okay, aside from that case, what else would move those to the main >> structure? They (the entries in the unsorted pending list) are in the >> local memory (work_mem?) of the backend, right? > No. If they were, it wouldn't be crash-safe. Thats how it is, but if we could push in wishes, then I would wish that is woulndn't be crash-safe, and be flushed by the backends commit. The way it currently operates is that a "random backend" pays the penalty of other backends pushes to the pending-list and "all queries" pays the penalty of searching the pendinglist in queries. If the pending list were backend only it would not have to be searched by "all queries" since commit needs to flush it and random backends wouldn't be penalized. Allthough we'd still have the benefit of batching up gin-inserts over mulitiple changes to the index done within the same transaction. Jesper -- Jesper