Re: [HACKERS] Need a builtin way to run all tests faster manner - Mailing list pgsql-hackers
| From | Andrew Dunstan |
|---|---|
| Subject | Re: [HACKERS] Need a builtin way to run all tests faster manner |
| Date | |
| Msg-id | f7de74f2-f7ca-de55-c476-1647fc132243@2ndQuadrant.com Whole thread Raw |
| In response to | Re: [HACKERS] Need a builtin way to run all tests faster manner (Tom Lane <tgl@sss.pgh.pa.us>) |
| Responses |
Re: [HACKERS] Need a builtin way to run all tests faster manner
|
| List | pgsql-hackers |
On 03/13/2017 12:35 AM, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2017-03-11 22:14:07 -0500, Tom Lane wrote:
>>> This looks generally sane to me, although I'm not very happy about folding
>>> the "$(MKDIR_P) output_iso" call into pg_isolation_regress_check --- that
>>> seems weird and unlike the way it's done for the regular regression test
>>> case.
>> Yea, not super happy about that either - alternatively we could fold it
>> into pg_regress.
> Yeah, teaching pg_regress to auto-create the --temp-instance directory
> seems perfectly sane from here.
w.r.t. $subject, I thought it might be useful to get some recent stats
from the buildfarm. Results are below. The bin checks dwarf everything
else. Upgrade checks and isolation check are other items of significant
cost. Upgrade checks could be significantly shortened if we could avoid
rerunning the regression tests.
cheers
andrew
pgbfprod=> select s.branch, s.log_stage, count(*), avg(extract(epoch
from stage_duration)::numeric(15,2))::numeric(15,1),
stddev(extract(epoch from stage_duration)::numeric(15,2))::numeric(15,1)
from public.build_status_log s where sysname <> 'crake' and snapshot >
now() - interval '10 days' and log_stage !~ 'start|stop' group by
s.branch, s.log_stage having count(*) > 20 and avg(extract(epoch from
stage_duration)::numeric(15,2)) > 20 order by log_stage, branch <>
'HEAD', branch desc
; branch | log_stage | count
| avg | stddev
---------------+-----------------------------------------------+-------+--------+--------HEAD | bin-check.log
| 388 |
1739.0 | 1758.5REL9_6_STABLE | bin-check.log | 91 |
1430.5 | 1287.9REL9_5_STABLE | bin-check.log | 87 |
1140.0 | 994.1REL9_4_STABLE | bin-check.log | 72
| 751.0 | 666.8HEAD | check.log | 2305
| 263.1 | 1197.0REL9_6_STABLE | check.log | 610
| 294.7 | 1369.6REL9_5_STABLE | check.log | 627
| 170.1 | 819.6REL9_4_STABLE | check.log | 512
| 140.4 | 535.3REL9_3_STABLE | check.log | 449
| 112.0 | 446.0REL9_2_STABLE | check.log | 406
| 109.2 | 380.9HEAD | check-pg_upgrade.log | 1785
| 319.4 | 1310.5REL9_6_STABLE | check-pg_upgrade.log | 482
| 571.3 | 2811.0REL9_5_STABLE | check-pg_upgrade.log | 484
| 350.5 | 2160.3REL9_4_STABLE | check-pg_upgrade.log | 385
| 240.8 | 1278.9REL9_3_STABLE | check-pg_upgrade.log | 353
| 214.0 | 1188.3REL9_2_STABLE | check-pg_upgrade.log | 314
| 195.6 | 1016.6HEAD | config.log | 2216
| 84.5 | 101.5REL9_6_STABLE | config.log | 576
| 90.0 | 90.8REL9_5_STABLE | config.log | 584
| 114.0 | 358.8REL9_4_STABLE | config.log | 495
| 84.5 | 85.3REL9_3_STABLE | config.log | 431
| 97.9 | 100.7REL9_2_STABLE | config.log | 391
| 93.1 | 94.6HEAD | contrib-install-check-C.log | 2250
| 122.9 | 474.5REL9_6_STABLE | contrib-install-check-C.log | 606
| 124.6 | 410.3REL9_5_STABLE | contrib-install-check-C.log | 622
| 84.7 | 348.8REL9_4_STABLE | contrib-install-check-C.log | 508
| 105.9 | 434.1REL9_3_STABLE | contrib-install-check-C.log | 445
| 61.8 | 273.4REL9_2_STABLE | contrib-install-check-C.log | 403
| 54.3 | 205.7HEAD | contrib-install-check-cs_CZ.UTF-8.log | 184
| 25.7 | 11.8REL9_6_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 46
| 25.7 | 14.0REL9_5_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 42
| 23.8 | 16.6REL9_4_STABLE | contrib-install-check-cs_CZ.UTF-8.log | 36
| 22.9 | 12.4HEAD | contrib-install-check-en_US.8859-15.log | 37
| 173.9 | 32.3HEAD | contrib-install-check-en_US.ISO8859-1.log | 33
| 244.7 | 35.7HEAD | contrib-install-check-en_US.log | 171
| 65.9 | 101.6REL9_6_STABLE | contrib-install-check-en_US.log | 42
| 61.7 | 89.3REL9_5_STABLE | contrib-install-check-en_US.log | 37
| 54.3 | 79.3REL9_4_STABLE | contrib-install-check-en_US.log | 33
| 53.7 | 71.5REL9_3_STABLE | contrib-install-check-en_US.log | 32
| 57.8 | 86.6REL9_2_STABLE | contrib-install-check-en_US.log | 29
| 46.9 | 63.8HEAD | contrib-install-check-en_US.utf8.log | 142
| 29.2 | 31.4REL9_6_STABLE | contrib-install-check-en_US.utf8.log | 30
| 40.2 | 43.7REL9_5_STABLE | contrib-install-check-en_US.utf8.log | 29
| 41.8 | 51.6REL9_4_STABLE | contrib-install-check-en_US.utf8.log | 25
| 56.6 | 75.8REL9_3_STABLE | contrib-install-check-en_US.utf8.log | 24
| 35.7 | 40.4REL9_2_STABLE | contrib-install-check-en_US.utf8.log | 21
| 31.3 | 36.1HEAD | contrib-install-check-en_US.UTF-8.log | 188
| 79.7 | 73.8REL9_6_STABLE | contrib-install-check-en_US.UTF-8.log | 36
| 78.2 | 68.6REL9_5_STABLE | contrib-install-check-en_US.UTF-8.log | 34
| 64.9 | 53.7REL9_4_STABLE | contrib-install-check-en_US.UTF-8.log | 30
| 68.4 | 56.2REL9_3_STABLE | contrib-install-check-en_US.UTF-8.log | 28
| 54.6 | 46.1REL9_2_STABLE | contrib-install-check-en_US.UTF-8.log | 25
| 48.3 | 39.8HEAD | contrib-install-check-POSIX.log | 33
| 227.7 | 42.6HEAD | ecpg-check.log | 2229
| 88.5 | 212.4REL9_6_STABLE | ecpg-check.log | 586
| 97.5 | 139.6REL9_5_STABLE | ecpg-check.log | 591
| 92.5 | 183.5REL9_4_STABLE | ecpg-check.log | 491
| 82.1 | 117.1REL9_3_STABLE | ecpg-check.log | 428
| 77.3 | 102.8REL9_2_STABLE | ecpg-check.log | 388
| 80.7 | 98.3REL9_6_STABLE | initdb-C.log | 606
| 21.2 | 73.1HEAD | initdb-en_US.8859-15.log | 37
| 241.8 | 24.8HEAD | initdb-en_US.ISO8859-1.log | 33
| 251.6 | 39.1HEAD | initdb-en_US.log | 171
| 54.3 | 103.8REL9_6_STABLE | initdb-en_US.log | 42
| 54.9 | 99.7REL9_5_STABLE | initdb-en_US.log | 37
| 61.2 | 112.5REL9_4_STABLE | initdb-en_US.log | 33
| 47.5 | 78.5REL9_3_STABLE | initdb-en_US.log | 32
| 46.0 | 74.6REL9_2_STABLE | initdb-en_US.log | 29
| 41.1 | 62.9HEAD | initdb-POSIX.log | 33
| 251.1 | 31.6HEAD | install-check-C.log | 2241
| 229.5 | 1227.9REL9_6_STABLE | install-check-C.log | 606
| 299.0 | 1601.9REL9_5_STABLE | install-check-C.log | 622
| 173.7 | 1147.2REL9_4_STABLE | install-check-C.log | 508
| 139.3 | 796.4REL9_3_STABLE | install-check-C.log | 446
| 104.7 | 713.3REL9_2_STABLE | install-check-C.log | 403
| 91.3 | 574.6HEAD | install-check-cs_CZ.ISO-8859-2.log | 137
| 37.5 | 1.5REL9_6_STABLE | install-check-cs_CZ.ISO-8859-2.log | 33
| 30.8 | 1.5REL9_5_STABLE | install-check-cs_CZ.ISO-8859-2.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-cs_CZ.ISO-8859-2.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-cs_CZ.ISO-8859-2.log | 24
| 24.9 | 1.2REL9_2_STABLE | install-check-cs_CZ.ISO-8859-2.log | 21
| 22.6 | 0.7HEAD | install-check-cs_CZ.UTF-8.log | 184
| 43.0 | 9.7REL9_6_STABLE | install-check-cs_CZ.UTF-8.log | 46
| 39.1 | 15.1REL9_5_STABLE | install-check-cs_CZ.UTF-8.log | 42
| 39.4 | 14.6REL9_4_STABLE | install-check-cs_CZ.UTF-8.log | 36
| 32.3 | 9.5REL9_3_STABLE | install-check-cs_CZ.UTF-8.log | 34
| 29.7 | 8.2REL9_2_STABLE | install-check-cs_CZ.UTF-8.log | 30
| 27.6 | 9.2HEAD | install-check-cs_CZ.WIN-1250.log | 137
| 37.3 | 1.2REL9_6_STABLE | install-check-cs_CZ.WIN-1250.log | 33
| 30.5 | 1.3REL9_5_STABLE | install-check-cs_CZ.WIN-1250.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-cs_CZ.WIN-1250.log | 25
| 26.3 | 0.7REL9_3_STABLE | install-check-cs_CZ.WIN-1250.log | 24
| 24.9 | 1.4REL9_2_STABLE | install-check-cs_CZ.WIN-1250.log | 21
| 22.3 | 0.6HEAD | install-check-de_DE.utf8.log | 129
| 35.4 | 9.4REL9_6_STABLE | install-check-de_DE.utf8.log | 22
| 25.8 | 7.8REL9_5_STABLE | install-check-de_DE.utf8.log | 21
| 27.4 | 8.0HEAD | install-check-en_US.8859-15.log | 37
| 136.9 | 21.2HEAD | install-check-en_US.ISO8859-1.log | 33
| 319.0 | 29.4HEAD | install-check-en_US.log | 171
| 65.7 | 58.7REL9_6_STABLE | install-check-en_US.log | 42
| 62.7 | 69.4REL9_5_STABLE | install-check-en_US.log | 37
| 59.4 | 56.2REL9_4_STABLE | install-check-en_US.log | 33
| 50.6 | 44.7REL9_3_STABLE | install-check-en_US.log | 32
| 50.2 | 46.8REL9_2_STABLE | install-check-en_US.log | 29
| 45.1 | 39.2HEAD | install-check-en_US.utf8.log | 142
| 47.6 | 40.3REL9_6_STABLE | install-check-en_US.utf8.log | 30
| 58.1 | 54.4REL9_5_STABLE | install-check-en_US.utf8.log | 29
| 61.7 | 57.4REL9_4_STABLE | install-check-en_US.utf8.log | 25
| 57.2 | 59.9REL9_3_STABLE | install-check-en_US.utf8.log | 24
| 51.3 | 47.7REL9_2_STABLE | install-check-en_US.utf8.log | 21
| 45.2 | 41.9HEAD | install-check-en_US.UTF-8.log | 188
| 153.5 | 162.7REL9_6_STABLE | install-check-en_US.UTF-8.log | 36
| 140.8 | 140.9REL9_5_STABLE | install-check-en_US.UTF-8.log | 34
| 145.7 | 144.3REL9_4_STABLE | install-check-en_US.UTF-8.log | 30
| 118.9 | 113.1REL9_3_STABLE | install-check-en_US.UTF-8.log | 28
| 107.0 | 101.8REL9_2_STABLE | install-check-en_US.UTF-8.log | 25
| 93.6 | 87.8HEAD | install-check-POSIX.log | 33
| 189.5 | 76.5HEAD | install-check-sk_SK.ISO-8859-2.log | 137
| 37.4 | 1.2REL9_6_STABLE | install-check-sk_SK.ISO-8859-2.log | 33
| 30.5 | 1.3REL9_5_STABLE | install-check-sk_SK.ISO-8859-2.log | 29
| 30.9 | 1.5REL9_4_STABLE | install-check-sk_SK.ISO-8859-2.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.ISO-8859-2.log | 24
| 24.8 | 1.2REL9_2_STABLE | install-check-sk_SK.ISO-8859-2.log | 21
| 22.7 | 0.6HEAD | install-check-sk_SK.UTF-8.log | 137
| 37.6 | 1.1REL9_6_STABLE | install-check-sk_SK.UTF-8.log | 33
| 31.0 | 1.8REL9_5_STABLE | install-check-sk_SK.UTF-8.log | 29
| 31.3 | 1.8REL9_4_STABLE | install-check-sk_SK.UTF-8.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.UTF-8.log | 24
| 25.0 | 1.2REL9_2_STABLE | install-check-sk_SK.UTF-8.log | 21
| 22.7 | 1.2HEAD | install-check-sk_SK.WIN-1250.log | 137
| 37.4 | 1.2REL9_6_STABLE | install-check-sk_SK.WIN-1250.log | 33
| 30.4 | 0.7REL9_5_STABLE | install-check-sk_SK.WIN-1250.log | 29
| 31.0 | 1.7REL9_4_STABLE | install-check-sk_SK.WIN-1250.log | 25
| 26.2 | 0.4REL9_3_STABLE | install-check-sk_SK.WIN-1250.log | 24
| 25.0 | 1.2REL9_2_STABLE | install-check-sk_SK.WIN-1250.log | 21
| 22.8 | 1.1HEAD | isolation-check.log | 2250
| 175.6 | 457.5REL9_6_STABLE | isolation-check.log | 606
| 201.0 | 527.7REL9_5_STABLE | isolation-check.log | 622
| 172.3 | 846.7REL9_4_STABLE | isolation-check.log | 508
| 146.0 | 602.3REL9_3_STABLE | isolation-check.log | 445
| 114.7 | 504.4REL9_2_STABLE | isolation-check.log | 403
| 57.2 | 300.6HEAD | make-contrib.log | 2176
| 43.7 | 67.6REL9_6_STABLE | make-contrib.log | 572
| 46.2 | 50.3REL9_5_STABLE | make-contrib.log | 579
| 44.9 | 54.2REL9_4_STABLE | make-contrib.log | 491
| 50.0 | 52.3REL9_3_STABLE | make-contrib.log | 428
| 48.6 | 51.4REL9_2_STABLE | make-contrib.log | 388
| 42.2 | 42.8HEAD | make-dist.log | 36
| 56.7 | 0.7HEAD | make-doc.log | 36
| 150.8 | 1.5REL9_2_STABLE | make-install.log | 403
| 22.0 | 29.8HEAD | make.log | 2328
| 363.1 | 581.0REL9_6_STABLE | make.log | 610
| 369.6 | 400.9REL9_5_STABLE | make.log | 627
| 392.8 | 556.8REL9_4_STABLE | make.log | 512
| 335.3 | 329.4REL9_3_STABLE | make.log | 449
| 320.0 | 311.0REL9_2_STABLE | make.log | 406
| 299.8 | 276.3HEAD | pl-install-check-en_US.8859-15.log | 37
| 32.9 | 3.3HEAD | pl-install-check-en_US.ISO8859-1.log | 33
| 34.3 | 5.1HEAD | pl-install-check-POSIX.log | 33
| 33.1 | 8.3REL9_2_STABLE | SCM-checkout.log | 406
| 33.2 | 87.8HEAD | sepgsql-test.log | 63
| 20.7 | 6.8HEAD | test-decoding-check.log | 1905
| 144.9 | 770.6REL9_6_STABLE | test-decoding-check.log | 493
| 226.3 | 1244.0REL9_5_STABLE | test-decoding-check.log | 495
| 147.1 | 1080.2REL9_4_STABLE | test-decoding-check.log | 419
| 168.1 | 1117.6HEAD | testmodules-install-check-C.log | 2233
| 22.6 | 42.3REL9_6_STABLE | testmodules-install-check-C.log | 602
| 25.6 | 35.8HEAD | testmodules-install-check-en_US.8859-15.log | 37
| 57.8 | 5.2HEAD | testmodules-install-check-en_US.ISO8859-1.log | 33
| 58.9 | 10.6HEAD | testmodules-install-check-en_US.log | 171
| 20.2 | 28.7REL9_6_STABLE | testmodules-install-check-en_US.utf8.log | 30
| 30.4 | 46.9HEAD | testmodules-install-check-en_US.UTF-8.log | 188
| 22.8 | 19.4REL9_6_STABLE | testmodules-install-check-en_US.UTF-8.log | 36
| 23.6 | 19.4REL9_5_STABLE | testmodules-install-check-en_US.UTF-8.log | 34
| 20.4 | 16.4HEAD | testmodules-install-check-POSIX.log | 33
| 72.1 | 14.7HEAD | typedefs.log | 96
| 74.8 | 99.1REL9_6_STABLE | typedefs.log | 28
| 106.3 | 119.7REL9_5_STABLE | typedefs.log | 28
| 100.0 | 111.7REL9_4_STABLE | typedefs.log | 24
| 88.1 | 100.7REL9_3_STABLE | typedefs.log | 23
| 85.6 | 95.1
(180 rows)
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
pgsql-hackers by date: