Thread: BUG #3302: Crash on gist ltree - PANIC: failed to add item to index page

The following bug has been logged online:

Bug reference:      3302
Logged by:          Cstdenis
Email address:      cstdenis@ctgameinfo.com
PostgreSQL version: 8.2.3
Operating system:   FreeBSD 6.1-RELEASE-p15
Description:        Crash on gist ltree - PANIC:  failed to add item to
index page
Details:

I was creating the following gist index of an ltree column

CREATE INDEX idx_picture_comments_id_tree_gist
        ON picture_comments
        USING gist
        (id_tree);

and the server exited on signal 6 (core dumped)

Came back up just fine after in recovery mode.

Here are the details.


/var/log/messages
---------
May 23 07:57:00 ayu postgres[39969]: [6-2] STATEMENT:  select count(*) from
picture_comments;
May 23 08:01:40 ayu postgres[39969]: [7-1] PANIC:  failed to add item to
index page in "idx_picture_comments_id_tree_gist"
May 23 08:01:40 ayu postgres[39969]: [7-2] STATEMENT:  CREATE INDEX
idx_picture_comments_id_tree_gist^M
May 23 08:01:40 ayu postgres[39969]: [7-3]        ON picture_comments^M
May 23 08:01:40 ayu postgres[39969]: [7-4]        USING gist^M
May 23 08:01:40 ayu postgres[39969]: [7-5]        (id_tree);
May 23 08:02:01 ayu kernel: pid 39969 (postgres), uid 70: exited on signal 6
(core dumped)
May 23 08:02:01 ayu postgres[15722]: [1-1] WARNING:  terminating connection
because of crash of another server process
May 23 08:02:01 ayu postgres[15722]: [1-2] DETAIL:  The postmaster has
commanded this server process to roll back the current transaction and exit,
because another server
May 23 08:02:01 ayu postgres[15722]: [1-3]  process exited abnormally and
possibly corrupted shared memory.
May 23 08:02:01 ayu postgres[15722]: [1-4] HINT:  In a moment you should be
able to reconnect to the database and repeat your command.

Backtrace
---------
ayu# gdb /usr/local/bin/postgres postgres.core

<snip copyright and loading symbols>

(gdb) bt
#0  0x2861a3df in kill () from /lib/libc.so.6
#1  0x2861a37e in raise () from /lib/libc.so.6
#2  0x2861909e in abort () from /lib/libc.so.6
#3  0x082efe2a in errfinish ()
#4  0x082f0367 in elog_finish ()
#5  0x080850f7 in gistfillbuffer ()
#6  0x0808459e in gistmakedeal ()
#7  0x080849e0 in gistmakedeal ()
#8  0x080e2750 in IndexBuildHeapScan ()
#9  0x08084f08 in gistbuild ()
#10 0x082f4205 in OidFunctionCall3 ()
#11 0x080e228e in index_create ()
#12 0x081450a7 in DefineIndex ()
#13 0x0823332e in ProcessUtility ()
#14 0x08231d7e in PortalSetResultFormat ()
#15 0x08231f73 in PortalSetResultFormat ()
#16 0x0823247a in PortalRun ()
#17 0x0822d9e6 in pg_parse_query ()
#18 0x0822fbf8 in PostgresMain ()
#19 0x081f1cda in ClosePostmasterPorts ()
#20 0x081f3d35 in PostmasterMain ()
#21 0x081a3866 in main ()

> CREATE INDEX idx_picture_comments_id_tree_gist
>         ON picture_comments
>         USING gist
>         (id_tree);
>
> and the server exited on signal 6 (core dumped)

Is it reproducible? Pls, send to me dump of ltree column.

> I haven't been able to reproduce it. I think its a race condition
> between some of the other processes running at the time. I'm not sure
> what else was running on it, but the error suggests an insert to me and
> there may have been a vacuum running (plus there are always plenty of
> selects running).

I don't believe in that:  INSERT/UPDATE/DELETE acquire ROW EXCLUSIVE
lock and vacuum - SHARE UPDATE EXCLUSIVE. They are conflicted with SHARE
lock mode acquired by CREATE INDEX command.

It seems to me tat we have bug in defining free space at page.

>
> It is not reasonable for me to send you an ltree dump since the talbe is
> about 2 million rows, but I'll send a small sample and describe it.
You can put dump somewhere in www and send just a link. But you say that
you can't reproduce the bug,  so it isn't needed. But I should some
digger the code.

Teodor Sigaev wrote:
>
>> CREATE INDEX idx_picture_comments_id_tree_gist
>>         ON picture_comments
>>         USING gist
>>         (id_tree);
>>
>> and the server exited on signal 6 (core dumped)
>
> Is it reproducible? Pls, send to me dump of ltree column.

I haven't been able to reproduce it. I think its a race condition
between some of the other processes running at the time. I'm not sure
what else was running on it, but the error suggests an insert to me and
there may have been a vacuum running (plus there are always plenty of
selects running).

It is not reasonable for me to send you an ltree dump since the talbe is
about 2 million rows, but I'll send a small sample and describe it.

The ltree is a tree is id numbers of rows (the table is a linked list of
comments and their replies (think slashdot's comment system) the id
numbers come from a serial and are in most cases 1-2 levels deep.

Here is a sample tho not entirely representative because all very old
entries have no replies (its a newer feature) and are therefore one
level deep.

"1821086"
"1819309"
"1817475"
"1810945"
"1810929"
"1810913"
"1808878"
"1795503"
"1792553"
"1792753"
"1792229"
"1788609"
"1787736"
"1775321"
"1775175"
"1775231"
"1773624"
"1768493"
"1767839"
"116026.116028.116032"
"116021.116048"
"111129.116063"
"116093.116094"
"116095.116100"
"116101.116104"
"116101.116104.116106"
"116101.116104.116106.116114.116117"
"116093.116094.116096.116122"
"116119.116123"
"116120.116126"
"116093.116094.116096.116103.116127"
"116128.116130"
"116101.116104.116106.116114.116117.116121.116131"
"116101.116104.116106.116114.116117.116121.116131.116136"
"116138.116143"
"116135.116146"
"116138.116143.116148"
"115343.115661.116152"
"116163.116166"
"116168.116171"
"116173.116174.116175.116176.116177"
"116173.116174.116175.116176.116177.116178"
"116179.116188.116191"
"116190.116192"
"116190.116192.116198.116199"
"116190.116192.116198.116199.116200"
"116132.116144"
"116244.116246"
"116249.116251"
"116155.116280"
"116279.116291"
"116260.116297"
"116279.116298"
"116257.116259.116262.116300"
"116303.116306"
"116308.116310"
"116308.116310.116312"
"116320.116321"
"116319.116322"
"116320.116321.116324"
"116333.116335"
"116342.116347"
"116343.116353"
"116359.116360"