Re: Add index scan progress to pg_stat_progress_vacuum - Mailing list pgsql-hackers

>    >> If the failsafe kicks in midway through a vacuum, the number indexes_total will not be reset to 0. If
INDEX_CLEANUPis turned off, then the value will be 0 at the start of the vacuum.
 
>    >
>    > The way that this works with num_index_scans is that we "round up"
>    > when there has been non-zero work in lazy_vacuum_all_indexes(), but
>    > not if the precheck in lazy_vacuum_all_indexes() fails. That seems
>    > like a good model to generalize from here. Note that this makes
>    > INDEX_CLEANUP=off affect num_index_scans in much the same way as a
>    > VACUUM where the failsafe kicks in very early, during the initial heap
>    > pass. That is, if the failsafe kicks in before we reach lazy_vacuum()
>    > for the first time (which is not unlikely), or even in the
>    > lazy_vacuum_all_indexes() precheck, then num_index_scans will remain
>    > at 0, just like INDEX_CLEANUP=off.
>    >
>    > The actual failsafe WARNING shows num_index_scans, possibly before it
>    > gets incremented one last time (by "rounding up"). So it's reasonably
>    > clear how this all works from that context (assuming that the
>    > autovacuum logging stuff, which reports num_index_scans, outputs a
>    > report for a table where the failsafe kicked in).

>    I am confused.  If failsafe kicks in during the middle of a vacuum, I
>   (perhaps naively) would expect indexes_total and indexes_processed to go to
>    zero, and I'd expect to no longer see the "vacuuming indexes" and "cleaning
>   up indexes" phases.  Otherwise, how would I know that we are now skipping
>  indexes?  Of course, you won't have any historical context about the index
>  work done before failsafe kicked in, but IMO it is misleading to still
>   include it in the progress view.

After speaking with Nathan offline, the best forward is to reset indexes_total and indexes_processed to 0 after the
startof "vacuuming indexes" or "cleaning up indexes" phase. 
 
Also, if failsafe is triggered midway through a vacuum, the values for both indexes_total and indexes_processed is
(re)setto 0.
 

Revision of the patch is attached.

Below is a test that shows the output.

-[ RECORD 1 ]------+------------------
pid                | 4360
datid              | 5
datname            | postgres
relid              | 16399
phase              | vacuuming indexes
heap_blks_total    | 401092
heap_blks_scanned  | 211798
heap_blks_vacuumed | 158847
index_vacuum_count | 3
max_dead_tuples    | 1747625
num_dead_tuples    | 1747366
indexes_total      | 8                <<<<--- index_vacuum_count  is 3, indexes_total is 8  and indexes_processed so
faris 1
 
indexes_processed  | 1


-[ RECORD 1 ]------+--------------
pid                | 4360
datid              | 5
datname            | postgres
relid              | 16399
phase              | scanning heap
heap_blks_total    | 401092
heap_blks_scanned  | 234590
heap_blks_vacuumed | 211797
index_vacuum_count | 4
max_dead_tuples    | 1747625
num_dead_tuples    | 752136
indexes_total      | 0                <<<<--- index_vacuum_count  is 4 and not in an index phase. indexes_total is 0
andindexes_processed so far is 0
 
indexes_processed  | 0


-[ RECORD 1 ]------+------------------
pid                | 4360
datid              | 5
datname            | postgres
relid              | 16399
phase              | vacuuming indexes
heap_blks_total    | 401092
heap_blks_scanned  | 264748
heap_blks_vacuumed | 211797
index_vacuum_count | 4
max_dead_tuples    | 1747625
num_dead_tuples    | 1747350
indexes_total      | 8
indexes_processed  | 6                            <<<<--- index_vacuum_count  is 4, indexes_total is 8  and
indexes_processedso far is 6
 


--
Sami Imseih
Amazon Web Services