Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunknumber 0 for toast value .. in pg_toast_2619 - Mailing list pgsql-general

From Justin Pryzby
Subject Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunknumber 0 for toast value .. in pg_toast_2619
Date
Msg-id 20180519163528.GD30060@telsasoft.com
Whole thread Raw
In response to Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunknumber 0 for toast value .. in pg_toast_2619  (Justin Pryzby <pryzby@telsasoft.com>)
Responses Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619
Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunknumber 0 for toast value .. in pg_toast_2619
List pgsql-general
On Sat, May 19, 2018 at 11:24:57AM -0500, Justin Pryzby wrote:
> On Sat, May 19, 2018 at 11:08:23AM -0400, Tom Lane wrote:
> > Justin Pryzby <pryzby@telsasoft.com> writes:
> > > I'll defer fixing this for awhile in case someone wants me to save a copy of
> > > the relation/toast/index.  From last time, I recall this just needs the right
> > > combination of REINDEX/VACUUM/ANALYZE, and the only complication was me
> > > needing to realize the right combination of affected DB(s).
> > 
> > If you could come up with such a sequence that causes the problem
> > reproducibly, that would be of huge interest, and probably lead to
> > a fix promptly.  But I don't think that we can do much by looking
> > at the post-mortem state --- once the toast rows are gone, they're
> > gone, especially if the table's been vacuumed since.
> 
> This is unlikely to allow reproducing it, but for sake of completeness here's a
> fuller log.  I'll try to trigger on another DB.

Did not take long...

[pryzbyj@database ~]$ while :; do for db in `psql postgres -Atc "SELECT datname FROM pg_database WHERE datallowconn"`;
dofor t in pg_statistic pg_attrdef pg_constraint; do echo "$db.$t..."; PGOPTIONS=-cstatement_timeout='9s' psql $db -qc
"VACUUMFULL $t"; done; done; done
 

...
postgres.pg_statistic...
postgres.pg_attrdef...
postgres.pg_constraint...
template1.pg_statistic...
template1.pg_attrdef...
template1.pg_constraint...
ts.pg_statistic...
ERROR:  canceling statement due to statement timeout
ts.pg_attrdef...
ts.pg_constraint...
postgres.pg_statistic...
ERROR:  missing chunk number 0 for toast value 3372855171 in pg_toast_2619

I'm running this again on another DB, but I wonder if that's enough for anyone
else to reproduce it with some consistency ?  I think that took something like
10min before failing.

Justin


pgsql-general by date:

Previous
From: Justin Pryzby
Date:
Subject: Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunknumber 0 for toast value .. in pg_toast_2619
Next
From: Dale Seaburg
Date:
Subject: View Filtered Rows technique