Re: [PERFORM] A Better External Sort? - Mailing list pgsql-hackers

From Josh Berkus
Subject Re: [PERFORM] A Better External Sort?
Date
Msg-id 200510031340.29376.josh@agliodbs.com
Whole thread Raw
In response to Re: [PERFORM] A Better External Sort?  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Tom,

> Raising work_mem to a gig should result in about five runs, needing only
> one pass, which is really going to be as good as it gets.  If you could
> not see any difference then I see little hope for the idea that reducing
> the number of merge passes will help.

Right.  It *should have*, but didn't seem to.    Example of a simple sort
test of 100 million random-number records

1M   3294 seconds
  16M   1107 seconds
  256M   1209 seconds
  512M   1174 seconds
  512M with 'not null' for column that is indexed  1168 seconds

> Umm ... you were raising maintenance_work_mem, I trust, not work_mem?

Yes.

>
> We really need to get some hard data about what's going on here.  The
> sort code doesn't report any internal statistics at the moment, but it
> would not be hard to whack together a patch that reports useful info
> in the form of NOTICE messages or some such.

Yeah, I'll do this as soon as the patch is finished.   Always useful to
gear up the old TPC-H.

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

pgsql-hackers by date:

Previous
From: Simon Riggs
Date:
Subject: Tuning current tuplesort external sort code for 8.2
Next
From: "Jeffrey W. Baker"
Date:
Subject: Re: [PERFORM] A Better External Sort?