Re: Facing a problem with SPI - Mailing list pgsql-hackers

<span style="font-family: courier new,monospace;">    Just noticed that this was committed by Tom on 21/Nov with this
logmessage:</span><br style="font-family: courier new,monospace;" /><br style="font-family: courier new,monospace;"
/><spanstyle="font-family: courier new,monospace;"><snip></span><br style="font-family: courier new,monospace;"
/><spanstyle="font-family: courier new,monospace;">Prevent intratransaction memory leak when a subtransaction is
aborted</span><br style="font-family: courier new,monospace;" /><span style="font-family: courier new,monospace;">in
themiddle of executing a SPI query.  This doesn't entirely fix the</span><br style="font-family: courier
new,monospace;"/><span style="font-family: courier new,monospace;">problem of memory leakage in plpgsql exception
handling,but it should</span><br style="font-family: courier new,monospace;" /><span style="font-family: courier
new,monospace;">get rid of the lion's share of leakage.<br /></snip><br /><br />Can the memory leaks be avoided
insome other way?<br /><br />Best regards,<br /><br />-- </span><br style="font-family: courier new,monospace;" /><span
style="font-family:courier new,monospace;"> gurjeet[.singh]@EnterpriseDB.com</span><br style="font-family: courier
new,monospace;"/><span style="font-family: courier new,monospace;">singh.gurjeet@{ gmail | hotmail | yahoo }.com
</span><brstyle="font-family: courier new,monospace;" /><br style="font-family: courier new,monospace;" /><div
style="font-family:courier new,monospace;"><span class="gmail_quote">On 12/3/06, <b class="gmail_sendername">Gurjeet
Singh</b> <<a href="mailto:singh.gurjeet@gmail.com">singh.gurjeet@gmail.com</a>> wrote:</span><blockquote
class="gmail_quote"style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi
all,<br/><br />    I am facing a problem with the SPI (spi.c). I have implemented short-lived virtual indexes using
SubTransactions.The Index Adviser, when creating virtual indexes, creates them inside a SubTransaction, and when it is
donewith its advisory stuff, it rolls back the SubTransaction. This achieves two objectives: <br /><br />    (1) Easy
cleanupof dependencies/changes in the catalog.<br />    (2) The virtual indexes are not visible to other backends. This
eliminatesthe possibility of other backends (not running in Advise mode) seeing these indexes, as these don't have any
datain them. <br /><br />    So all the index creation/deletion is done between a pair of BeginInternalSubTransaction()
(BIST())and RollbackAndReleaseCurrentSubTransaction() (RARCST()). This all works well when we are EXPLAINing query in
specialadvise mode. <br /><br />    Now, based on this, I wanted to generate advise for every query that comes for
planning,be it from psql or pl/pgsql or anywhere else. So I made a function call to the index_adviser() from within
pg_plan_query()after the planner() call. <br /><br />    The problem I am facing is that that the queries being planned
bypl/pgsql, come after a call to SPI_connect() and SPI_Prepare(). SPI_prepare() switches to _SPI_current->execCxt
memorycontext using _SPI_begin_call(). <br /><br />    Everything goes well until the index_adviser() calls RARCST() to
rollbackit's SubTransaction. RARCST() ultimately ends up calling AtEOSubXact_SPI(), which, just before returning,
deletesthe execution memory context of the surrounding SPI (which happens to be pl/pgsql's SPI). This, in effect, frees
allthe memory allocated by pl/pgsql for the prepare stage; and the next time the pl/pgsql's data-structures are
referenced,it causes a crash. The code in question is: <br /><br />void<br />AtEOSubXact_SPI(bool isCommit,
SubTransactionIdmySubid)<br />{<br />    bool        found = false;<br /><br />    while (_SPI_connected >= 0)<br
/>   {<br />        _SPI_connection *connection = &(_SPI_stack[_SPI_connected]); <br /><br />        if
(connection->connectSubid!= mySubid)<br />            break;              /* couldn't be any underneath it either
*/<br/><br />        ......<br />        ......<br />    /*<br />     * If we are aborting a subtransaction and there
isan open SPI context <br />     * surrounding the subxact, clean up to prevent memory leakage.<br />     */<br />   
if(_SPI_current && !isCommit)<br />    {<br />        /* free Executor memory the same as _SPI_end_call would
do*/<br />        MemoryContextResetAndDeleteChildren(_SPI_current->execCxt); <br />        ^^^^^^^^^^^<br />       
/*throw away any partially created tuple-table */<br />        SPI_freetuptable(_SPI_current->tuptable);<br
/>       _SPI_current->tuptable = NULL;<br />    }<br />}<br /><br />    The code is doing what the comments say,
buthow I wish it didn't. This code assumes that the latest BIST() came from a SPI invoking module, which the Index
Adviseris not! <br /><br />    I haven't been able to come up with a proof, but I think this situation can be
simulated/reproducedusing some pl/* language that we support.<br /><br />    pl/pgsql is safe, since it doesn't allow
SAVEPOINT/ROLLBACKTO SAVEPOINT inside it's code blocks; but if some pl/xxx allows SAVEPOINTS, I think it can be
reproducedusing something like: <br /><br />    SAVEPOINT xxx            ( should call BIST() )<br />        SAVEPOINT
yyy       ( should call BIST() )<br />        Rollback (to yyy)    ( should call RARCST() ) will free pl/xxx exec
context<br/>    Rollback (to xxx)        ( should call RARCST() ) <br /><br />    Probably just one pair, instead of
twonested ones, should reproduce it. I tried to reproduce it using pl/pgsql's exception block (there we use BIST() and
RARCST());but couldn't succeed.<br /><br />    I hope I have understood the issue correctly, or have I missed
something?Is there a way to work around this? <br /><br />Best regards,<br clear="all" /><span class="sg"><br />-- <br
/>gurjeet[.singh]@EnterpriseDB.com<br />singh.gurjeet@{ gmail | hotmail | yahoo }.com </span></blockquote></div>