Re: speedup COPY TO for partitioned table. - Mailing list pgsql-hackers

From jian he
Subject Re: speedup COPY TO for partitioned table.
Date
Msg-id CACJufxEEsFUHNdSHyO7Zy3yjp3pia2yZVfjTv9b7ZxkTKdQuKQ@mail.gmail.com
Whole thread Raw
In response to Re: speedup COPY TO for partitioned table.  (Kirill Reshke <reshkekirill@gmail.com>)
List pgsql-hackers
hi.

In the V10 patch, there will be some regression if the partition column
ordering is different from the root partitioned table.

because in V10 CopyThisRelTo
+ while (table_scan_getnextslot(scandesc, ForwardScanDirection, slot))
+ {
+ if (map != NULL)
+ {
+ original_slot = slot;
+ root_slot = MakeSingleTupleTableSlot(rootdesc, &TTSOpsBufferHeapTuple);
+ slot = execute_attr_map_slot(map, slot, root_slot);
+ }
+ else
+ slot_getallattrs(slot);
+
+ if (original_slot != NULL)
+ ExecDropSingleTupleTableSlot(original_slot);
+}
as you can see, for each slot in the partition, i called
MakeSingleTupleTableSlot to get the dumpy root_slot
and ExecDropSingleTupleTableSlot too.
that will cause overhead.

we can call produce root_slot before the main while loop.
like the following:
+ if (root_rel != NULL)
+ {
+ rootdesc = RelationGetDescr(root_rel);
+ root_slot = table_slot_create(root_rel, NULL);
+ map = build_attrmap_by_name_if_req(rootdesc, tupdesc, false);
+ }
....
+ while (table_scan_getnextslot(scandesc, ForwardScanDirection, slot))
+ {
+ TupleTableSlot *copyslot;
+ if (map != NULL)
+ copyslot = execute_attr_map_slot(map, slot, root_slot);
+ else
+ {
+ slot_getallattrs(slot);
+ copyslot = slot;
+ }
+
please check CopyThisRelTo in v11. so, with v11, there is no
regression for case when
partition column ordering differs from partitioned.

I have tested 30 partitions, 10 columns, all the column ordering
is different with the root partitioned table.

copy pp to '/tmp/2.txt'
is still faster than
copy (select * from pp) to '/tmp/1.txt';
(359.463 ms versus  376.371 ms)

I am using -Dbuildtype=release
PostgreSQL 18beta1_release_build on x86_64-linux, compiled by gcc-14.1.0, 64-bit

you may see the attached test file.

Attachment

pgsql-hackers by date:

Previous
From: Renan Alves Fonseca
Date:
Subject: Re: PoC: Compute a histogram of prune_xid to support autovacuum improvements
Next
From: Michael Paquier
Date:
Subject: Re: Encapsulate io_uring process count calculation