Thread: pgsql-server/src/test/regress expected/random. ...

CVSROOT:    /cvsroot
Module name:    pgsql-server
Changes by:    momjian@svr1.postgresql.org    04/03/15 11:46:25

Modified files:
    src/test/regress/expected: random.out
    src/test/regress/sql: random.sql

Log message:
    Improve random regression tests to fail less frequently.

momjian@svr1.postgresql.org (Bruce Momjian) writes:
>     Improve random regression tests to fail less frequently.

If you're gonna do that you should fix the discussion in the admin
guide.  It presently says that failures in random should be expected
every five to ten trials, and even hints that you should be concerned
if you never see a failure.

            regards, tom lane

Tom Lane wrote:
> momjian@svr1.postgresql.org (Bruce Momjian) writes:
> >     Improve random regression tests to fail less frequently.
>
> If you're gonna do that you should fix the discussion in the admin
> guide.  It presently says that failures in random should be expected
> every five to ten trials, and even hints that you should be concerned
> if you never see a failure.

Oh, I didn't know we had a mention of random there too.  New text is:

    <title>The <quote>random</quote> test</title>

    <para>
     There is at least one case in the <literal>random</literal> test
     script that is intended to produce random results. This causes
     random to fail the regression test occasionally.  Typing
<programlisting>
diff results/random.out expected/random.out
</programlisting>
     should produce only one or a few lines of differences.  You need
     not worry unless the random test always fails in repeated
     attempts.
    </para>

I wanted to reduce the frequency of random failure by doing more random
tests, and reduce the amount of confusion when it fails.  I see no way
of preventing it from ever failing.

--
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073