Re: Locking B-tree leafs immediately in exclusive mode - Mailing list pgsql-hackers

On 2018/07/10 20:36, Alexander Korotkov wrote:
> On Tue, Jul 10, 2018 at 2:19 PM Imai, Yoshikazu
> <imai.yoshikazu@jp.fujitsu.com> wrote:
>> On Mon, July 9, 2018 at 4:44 PM, Alexander Korotkov wrote:
>>>> Firstly, I did performance tests on 72-cores machines(AWS c5.18xlarge) same as you did.
>>>
>>> OK.  But not that c5.18xlarge is 72-VCPU machine, which AFAIK is
>>> close to the performance of 36 physical cores.
>>
>> Thanks for pointing that. I referred to /proc/cpuinfo and understood there are 36 physical cores.
>>
>>> In this case it also looks like we observed 1% regression.  Despite 1%
>>> may seem to be very small, I think we should clarify whether it really
>>> exists.  I have at least two hypothesis about this.
>>>
>>> 1) There is no real regression, observed difference of TPS is less
>>> than error of measurements.  In order to check that we need to retry
>>> the experiment multiple times.  Also, if you run benchmark on master
>>> before patched version (or vice versa) you should also try to swap the
>>> order to make sure there is no influence of the order of benchmarks.
>>> 2) If we consider relation between TPS and number of clients, TPS is
>>> typically growing with increasing number of clients until reach some
>>> saturation value.  After the saturation value, there is some
>>> degradation of TPS.  If patch makes some latency lower, that my cause
>>> saturation to happen earlier.  In order to check that, we need run
>>> benchmarks with various number of clients and draw a graph: TPS
>>> depending on clients.
>>>
>>> So, may I ask you to make more experiments in order to clarify the
>>> observed regression?
>>
>> I experimented 2) with changing clients parameter with 18, 36, 54, 72.
>> While doing experiment, I realized that results of pgbench with 36 clients improve after executing pgbench with 72
clients.
>> I don't know why this occurs, but anyway, in this experiment, I executed pgbench with 72 clients before executing
otherpgbenchs. (e.g. -c 72, -c 18, -c 36, -c 54, -c 72)
 
>> I tested experiments to master and patched unorderly(e.g. master, patched, patched, master, master, patched,
patched,master)
 
>>
>> # results of changing clients(18, 36, 54, 72 clients)
>> master, -c 18 -j 18:  Ave 400410 TPS (407615,393942,401845,398241)
>> master, -c 36 -j 36:  Ave 415616 TPS (411939,400742,424855,424926)
>> master, -c 54 -j 54:  Ave 378734 TPS (401646,354084,408044,351163)
>> master, -c 72 -j 72:  Ave 360864 TPS (367718,360029,366432,349277)
>> patched, -c 18 -j 18: Ave 393115 TPS (382854,396396,395530,397678)
>> patched, -c 36 -j 36: Ave 390328 TPS (376100,404873,383498,396840)
>> patched, -c 54 -j 54: Ave 364894 TPS (365533,373064,354250,366727)
>> patched, -c 72 -j 72: Ave 353982 TPS (355237,357601,345536,357553)
>>
>> It may seem saturation is between 18 and 36 clients, so I also experimented with 27 clients.
>>
>> # results of changing clients(27 clients)
>> master, -c 27 -j 27:  Ave 416756 (423512,424241,399241,420030) TPS
>> patched, -c 27 -j 27: Ave 413568 (410187,404291,420152,419640) TPS
>>
>> I created a graph and attached in this mail("detecting saturation.png").
>> Referring to a graph, patched version's saturation happens earlier than master's one as you expected.
>> But even the patched version's nearly saturated TPS value has small regression from the master's one, I think.
>>
>> Is there another experiments to do about this?
> 
> Thank you for the experiments!  It seems that there is real regression
> here...  BTW, which script were you using in this benchmark:
> script_unordered.sql or script_duplicated.sql?

Sorry, I forgot to write that. I used script_unordered.sql.


Yoshikazu Imai