Yeah, kind of. But obviously if we could make the limit smaller without hurting performance, that would be better.
Per my note yesterday about performance degradation with parallel COPY, I wasn't able to demonstrate that this patch gives a consistent performance benefit on hydra - the best result I got was speeding up a 9.5 minute load to 8 minutes where linear scalability would have been 2 minutes. And I found cases where it was actually slower with the patch. Now maybe hydra is just a crap machine, but I'm feeling nervous.
I see the performance gain when either
"complete data is in SSD"
or "data on MD and WAL on SSD"
or unlogged table.
What machines did you use to test this? Have you tested really large data loads, like many MB or even GB of data?
With INSERT Script within 2 mins run data size is 18GB I am running 5-10 Mins means at least 85GB data.
(Inserts 1000 1KB tuples in each transactions)
With COPY Script within 2 mins run data size is 23GB and I am running 5-10 Mins means at least 100GB data.
(Inserts 10000 tuples in each transactions (tuple size is 1byte to 5 bytes)
Machine Details
-----------------------
I tested in 8 socket NUMA machine with 64 core.
Machine Details:
----------------------
[dilip.kumar@cthulhu ~]$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 8
NUMA node(s): 8
Vendor ID: GenuineIntel
CPU family: 6
Model: 47
Model name: Intel(R) Xeon(R) CPU E7- 8830 @ 2.13GHz
Stepping: 2
CPU MHz: 1064.000
BogoMIPS: 4266.62
If you need some more information please let me know ?
