Thread: Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace
"Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> Tom, could you run this test for different block sizes?
> Up to 32*8k?
>>
>> You mean changing the amount written per write(), while holding the
>> total file size constant, right?
> Yes. Currently XLogWrite writes 8k blocks one by one. From what I've seen
> on Solaris we can use O_DSYNC there without changing XLogWrite to
> write() more than 1 block (if > 1 block is available for writing).
> But on other platforms write(BLOCKS_TO_WRITE * 8k) + fsync() probably will
> be
> faster than BLOCKS_TO_WRITE * write(8k) (for file opened with O_DSYNC)
> if BLOCKS_TO_WRITE > 1.
> I just wonder with what BLOCKS_TO_WRITE we'll see same times for both
> approaches.
Okay, I changed the program tochar zbuffer[8192 * BLOCKS];
(all else the same)
and on HPUX 10.20 I get
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c
$ time a.out
real 1m18.48s
user 0m0.04s
sys 0m34.69s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c
$ time a.out
real 0m35.10s
user 0m0.01s
sys 0m9.08s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c
$ time a.out
real 0m29.75s
user 0m0.01s
sys 0m5.23s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c
$ time a.out
real 0m22.77s
user 0m0.01s
sys 0m1.80s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c
$ time a.out
real 0m22.08s
user 0m0.01s
sys 0m1.25s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c
$ time a.out
real 0m20.64s
user 0m0.02s
sys 0m0.67s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c
$ time a.out
real 0m20.72s
user 0m0.01s
sys 0m0.57s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=32 tfsync.c
$ time a.out
real 0m20.59s
user 0m0.01s
sys 0m0.61s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c
$ time a.out
real 0m20.86s
user 0m0.01s
sys 0m0.69s
So I also see that there is no benefit to writing more than one block at
a time with ODSYNC. And even at half a meg per write, DSYNC is slower
than ODSYNC with 8K per write! Note the fairly high system-time
consumption for DSYNC, too. I think this is not so much a matter of a
really good ODSYNC implementation, as a really bad DSYNC one ...
regards, tom lane
More numbers, these from a Powerbook G3 laptop running Linux 2.2:
[tgl@g3 tmp]$ uname -a
Linux g3 2.2.18-4hpmac #1 Thu Dec 21 15:16:15 MST 2000 ppc unknown
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m32.418s
user 0m0.020s
sys 0m14.020s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m10.894s
user 0m0.000s
sys 0m4.030s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m7.211s
user 0m0.000s
sys 0m2.200s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m4.441s
user 0m0.020s
sys 0m0.870s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m4.488s
user 0m0.000s
sys 0m0.640s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m3.725s
user 0m0.000s
sys 0m0.310s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m3.785s
user 0m0.000s
sys 0m0.290s
[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c
[tgl@g3 tmp]$ time ./a.out
real 0m3.753s
user 0m0.010s
sys 0m0.300s
Starting to look like we should just use ODSYNC where available, and
forget about dumping more per write ...
regards, tom lane
Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace
From
Denis Perchine
Date:
On Saturday 10 March 2001 08:41, Tom Lane wrote: > More numbers, these from a Powerbook G3 laptop running Linux 2.2: Eeegghhh. Sorry... But where did you get O_DSYNC on Linux????? Maybe here? bits/fcntl.h: # define O_DSYNC O_SYNC There is no any O_DSYNC in the kernel... Even in the latest 2.4.x. > [tgl@g3 tmp]$ uname -a > Linux g3 2.2.18-4hpmac #1 Thu Dec 21 15:16:15 MST 2000 ppc unknown > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m32.418s > user 0m0.020s > sys 0m14.020s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m10.894s > user 0m0.000s > sys 0m4.030s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m7.211s > user 0m0.000s > sys 0m2.200s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m4.441s > user 0m0.020s > sys 0m0.870s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m4.488s > user 0m0.000s > sys 0m0.640s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m3.725s > user 0m0.000s > sys 0m0.310s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m3.785s > user 0m0.000s > sys 0m0.290s > > [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c > [tgl@g3 tmp]$ time ./a.out > > real 0m3.753s > user 0m0.010s > sys 0m0.300s > > > Starting to look like we should just use ODSYNC where available, and > forget about dumping more per write ... > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org -- Sincerely Yours, Denis Perchine ---------------------------------- E-Mail: dyp@perchine.com HomePage: http://www.perchine.com/dyp/ FidoNet: 2:5000/120.5 ----------------------------------
Denis Perchine <dyp@perchine.com> writes:
> On Saturday 10 March 2001 08:41, Tom Lane wrote:
>> More numbers, these from a Powerbook G3 laptop running Linux 2.2:
> Eeegghhh. Sorry... But where did you get O_DSYNC on Linux?????
> bits/fcntl.h: # define O_DSYNC O_SYNC
Hm, must be. Okay, so those two sets of numbers should be taken as
fsync() and O_SYNC respectively. Still the conclusion seems pretty
clear: the open() options are way more efficient than calling fsync()
separately.
regards, tom lane