Thread: Difference between Bulk Load (Multiple inserts or single inserts) and COPY

The following documentation comment has been logged on the website:

Page: https://www.postgresql.org/docs/10/sql-copy.html
Description:

Hello, 

Myself Mayank. I am a Ph.D. student.

I experimented with Bulk load and COPY.
Loading in COPY was very fast. 
However, after COPYing data from a CSV file to PostgreSQL Table. The query
execution took lot of time for 1 of the first 4 queries.
Only this slow query was taking so much time, that even if I had used normal
bulk load, it would have been faster in total.
Then all other Query executions took equal time as it took while querying a
table after the Bulk data load method.

So, I want to know the exact reason what's the issue with COPY.
How exactly they differ? The only thing from the document I could identify
was row security.
But it did not mention anything about indexing. Like, in Bulk load, do
indices(or constraint checks) are created with data loading?
& in COPY it's done after? so when indices are being created that query
slows down??

Please reply soon with more details or send a link where I can read it in
depth.
Thanks.
Mayank.
mayank.l.patel90@gmail.com

On Tue, 2019-11-19 at 18:55 +0000, PG Doc comments form wrote:
> I experimented with Bulk load and COPY.
> Loading in COPY was very fast. 
> However, after COPYing data from a CSV file to PostgreSQL Table. The query
> execution took lot of time for 1 of the first 4 queries.
> Only this slow query was taking so much time, that even if I had used normal
> bulk load, it would have been faster in total.
> Then all other Query executions took equal time as it took while querying a
> table after the Bulk data load method.
> 
> So, I want to know the exact reason what's the issue with COPY.
> How exactly they differ? The only thing from the document I could identify
> was row security.
> But it did not mention anything about indexing. Like, in Bulk load, do
> indices(or constraint checks) are created with data loading?
> & in COPY it's done after? so when indices are being created that query
> slows down??
> 
> Please reply soon with more details or send a link where I can read it in
> depth.

That cannot be answered without knowing the exact statements and the
table definitions.

Yours,
Laurenz Albe
-- 
Cybertec | https://www.cybertec-postgresql.com

On Tue, Nov 19, 2019 at 11:55:44PM +0100, Laurenz Albe wrote:
> On Tue, 2019-11-19 at 18:55 +0000, PG Doc comments form wrote:
> > I experimented with Bulk load and COPY.
> > Loading in COPY was very fast. 
> > However, after COPYing data from a CSV file to PostgreSQL Table. The query
> > execution took lot of time for 1 of the first 4 queries.
> > Only this slow query was taking so much time, that even if I had used normal
> > bulk load, it would have been faster in total.
> > Then all other Query executions took equal time as it took while querying a
> > table after the Bulk data load method.
> > 
> > So, I want to know the exact reason what's the issue with COPY.
> > How exactly they differ? The only thing from the document I could identify
> > was row security.
> > But it did not mention anything about indexing. Like, in Bulk load, do
> > indices(or constraint checks) are created with data loading?
> > & in COPY it's done after? so when indices are being created that query
> > slows down??
> > 
> > Please reply soon with more details or send a link where I can read it in
> > depth.
> 
> That cannot be answered without knowing the exact statements and the
> table definitions.

I wonder if it is the overhead of rewriting all the rows to set the
per-row HEAP_XMIN_COMMITTED bit.  Unfortunately, I don't know a way to
test this hypothesis.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +