Thread: PANIC: right sibling 2019 of block 2018 is not next child of 1937 in index "sl_log_2_idx1"

I know I don't have a lot of data right now (still looking for core dump)
Any obvious thoughts or advice until I can get more info?

Dec 30 17:41:57 db-1 postgres[28957]: [ID 748848 local0.crit] [34004622-1] 2009-12-30 17:41:57.825 CST    28957PANIC:
rightsibling 2019 of block 2018 is not next child of 1937 in index "sl_log_2_idx1" 
Dec 30 17:46:17 db-1 postgres[17576]: [ID 748848 local0.info] [34005240-1] 2009-12-30 17:46:17.279 CST    17576LOG:
autovacuumprocess (PID 28957) was terminated by signal 6 
Dec 30 17:46:17 db-1 postgres[17576]: [ID 748848 local0.info] [34005241-1] 2009-12-30 17:46:17.279 CST    17576LOG:
terminatingany other active server processes 

Database recovered itself...and autovacuum of sl_log_2 started up again with no obvious issue right afterwords.

PostgreSQL 8.2.12 on i386-pc-solaris2.10, compiled by GCC gcc (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath)

Oldish version of slony (1.2.10)

--- On Wed, 12/30/09, Jeff Amiel <becauseimjeff@yahoo.com> wrote:
> Subject: PANIC:  right sibling 2019 of block 2018 is not next child of 1937 in index "sl_log_2_idx1"


I am assuming a re-index for that particular index will rebuild/fix the index (if it happens again).  Any other
thoughts? 

Jeff Amiel wrote:
> --- On Wed, 12/30/09, Jeff Amiel <becauseimjeff@yahoo.com> wrote:
>
>> Subject: PANIC:  right sibling 2019 of block 2018 is not next child of 1937 in index "sl_log_2_idx1"
>>
>
>
> I am assuming a re-index for that particular index will rebuild/fix the index (if it happens again).  Any other
thoughts? 
>
>

It appears you have at least one corrupt block.   Where there's one,
there's often more.

I think i'd do a reindex all.   and Imight even do a pg_dumpall,
re-initdb, and restore said dumpall to be safeest.   to be really safe,
stop pg, and take a file system backup first, incase more serious
problems show up during the reindex or dump process

John R Pierce wrote:
> It appears you have at least one corrupt block.   Where there's one,
> there's often more.
>
> I think i'd do a reindex all.   and Imight even do a pg_dumpall,
> re-initdb, and restore said dumpall to be safeest.   to be really
> safe, stop pg, and take a file system backup first, incase more
> serious problems show up during the reindex or dump process

I would add a basic check of the system hardware to that list.  In
particular, I'm always suspicious of memory errors when seeing strange
PANIC messages.  A quick check for SMART errors on the hard drives would
be in order too.

--
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com