Re: Enhance file_fdw to report processed and skipped tuples in COPY progress - Mailing list pgsql-hackers

From Fabrízio de Royes Mello
Subject Re: Enhance file_fdw to report processed and skipped tuples in COPY progress
Date
Msg-id CAFcNs+qEB96BLQLU4abuOm8+xpoRnvb_OUeZ7cuDCyEp8azr7Q@mail.gmail.com
Whole thread Raw
List pgsql-hackers

On Thu, Oct 3, 2024 at 6:23 AM Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>
> Hi,
>
> Currently, file_fdw updates several columns in the pg_stat_progress_copy view,
> like relid and bytes_processed, but it doesn't track tuples_processed or
> tuples_skipped. Monitoring these would be particularly useful when handling
> large data sets via file_fdw, as it helps track the progress of scan.
>
> The attached patch updates file_fdw to add support for reporting
> the number of tuples processed and skipped (due to on_error = 'ignore')
> in the pg_stat_progress_copy view. What are your thoughts?
>

Awesome... no failures building and it worked as expected:

-[ RECORD 1 ]----+-----------
pid              | 102570
datid            | 16388
datname          | fabrizio
relid            | 16398
command          | COPY FROM
type             | FILE
bytes_processed  | 1359675392
bytes_total      | 2921579128
tuples_processed | 18605306
tuples_excluded  | 0
tuples_skipped   | 0

Regards,

--
Fabrízio de Royes Mello

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: make \d table Collation field showing domains collation if that attribute is type of domain.
Next
From: Jacob Champion
Date:
Subject: Re: Retire support for OpenSSL 1.1.1 due to raised API requirements