CopyMultiInsertInfo.bufferedBytes is not increased in binary format cases - Mailing list pgsql-hackers

From Masahiko Sawada
Subject CopyMultiInsertInfo.bufferedBytes is not increased in binary format cases
Date
Msg-id CAD21AoDvNTmmSL5_tzFLYTdXk7N26raaLs7sUYXoLZsz+HscAg@mail.gmail.com
Whole thread Raw
List pgsql-hackers
Hi,

When using the multi insertion method during COPY FROM, we buffer
input tuples until the number of tuples exceeds MAX_BUFFERED_TUPLES
(=1000) or the total buffered tuple size exceeds MAX_BUFFERED_BYTES
(=64kB). We use line_buf.len as input tuple size as follows and add it
to bufferedBytes:

    /* Add this tuple to the tuple buffer */
    CopyMultiInsertInfoStore(&multiInsertInfo,
                             resultRelInfo, myslot
                             cstate->line_buf.len,
                             cstate->cur_lineno);

However, while it works in text or csv cases, in binary format cases,
bufferedBytes is not increased as we don't use line_buf. So we always
buffer 1000 tuples regardless of tuple size. Is it intentional
behavior? I could not find any related discussion on the original
discussion threads.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com



pgsql-hackers by date:

Previous
From: Melanie Plageman
Date:
Subject: Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Next
From: Melanie Plageman
Date:
Subject: Re: Checkpointer write combining