Re: COPY performance on Windows - Mailing list pgsql-hackers

From Vladlen Popolitov
Subject Re: COPY performance on Windows
Date
Msg-id 18499a33364b8adae0741adf8396d2b4@postgrespro.ru
Whole thread Raw
In response to Re: COPY performance on Windows  (Robert Haas <robertmhaas@gmail.com>)
Responses RE: COPY performance on Windows
List pgsql-hackers
Ryohei Takahashi (Fujitsu) писал(а) 2024-12-11 15:18:
> Hi,
> 
> 
> I continuously investigate the performance problem of COPY on Windows.
> 
> I noticed that not only PG17.0 but also PG16.6 have performance problem 
> compared to PG16.4.
> The performance is 2.5%-5.8% worse, especially when the number of 
> clients is 1 or 2.
> 
> I modified the performance measurement script of the thread in [1].
> * Enabled to run on Windows git bash
> * Enabled to compare PG16.4, PG16.6 and PG17.0
> * Increase the row number to 10 times (about 10GB)
> 
> I measured on Windows Server 2022 machine with 44 core CPU and 512GB 
> memory.
> The results are following.
> 

> 
> (1)
> I attach the performance measurement script.
> If you have a Windows environment, could you please reproduce the same 
> performance problem?
> 
> (2)
> The performance of PG16.6 and PG17.0 are worse than PG16.4.
> So, I think the commits between August and September affects the 
> performance.
> I will analyze these commits.
> 
> 
> [1] 
> https://postgr.es/m/CAD21AoDvDmUQeJtZrau1ovnT_smN940=Kp6mszNGK3bq9yRN6g@mail.gmail.com
> 
> 
> Regards,
> Ryohei Takahashi
COPY FROM code had changes with DEFAULT option and numbers conversion, 
and you suspect disk read/writes also.
I propose to isolate read-write case. You could create RAM DISK in 
Windows server
(Microsoft has documentation on its site how to configure it, googling I 
found this but not tested it
https://learn.microsoft.com/ru-ru/archive/blogs/windowsinternals/how-to-create-a-ram-disk-in-windows-server 
).
You could create database on RAM disk and make benchmarks.
You will exclude disk influence, and probably other changes affected 
performance.
The number conversion patch was tested on many processors, but not on 
all. Probably you run Windows
on one that has worse performance or other issues possible.

P.S. Could you point where is the performance measurement script you 
mantioned in you email?
I will try to play with it.

-- 
Best regards,

Vladlen Popolitov.



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: pg_attribute_noreturn(), MSVC, C11
Next
From: Andres Freund
Date:
Subject: Re: checkpointer: PANIC: could not fsync file: No such file or directory