Re: [HACKERS] sorting big tables :( - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: [HACKERS] sorting big tables :(
Date
Msg-id 199805201422.KAA14065@candle.pha.pa.us
Whole thread Raw
In response to Re: [HACKERS] sorting big tables :(  (The Hermit Hacker <scrappy@hub.org>)
List pgsql-hackers
>
> On Wed, 20 May 1998, Michal Mosiewicz wrote:
>
> > The Hermit Hacker wrote:
> >
> > > Now, as a text file, this would amount to, what...~50MB?
> > 40M of records to produce a 50MB text file? How would you sort such a
> > *compressed* file? ;-)
>
> My math off?  40M rows at 11bytes each (2xint4+int2+\n?)  oops...ya, just
> off by a factor of ten...still, 500MB is a quarter of the size of the 2gig
> file we started with...

Actually, my description of the use of tape files was somewhat off.
Actually, the file is sorted by putting several batches in each tape
file, then reading the batches make another tape file with bigger
batches until there is one tape file and one big sorted batch.  Also, if
the data is already sorted, it can do it in one pass, without making all
those small batches because of the way the data structure sorts them in
memory.  Only Knuth can do the description justice, but suffice it to
say that the data can appear up to two places at once.

This is the first time I remember someone complaining about it.

--
Bruce Momjian                          |  830 Blythe Avenue
maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
  +  If your life is a hard drive,     |  (610) 353-9879(w)
  +  Christ can be your backup.        |  (610) 853-3000(h)

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [DOCS] Re: FE/BE protocol revision patch
Next
From: Bruce Momjian
Date:
Subject: Re: [HACKERS] sorting big tables :(