Re: Mass-Data question - Mailing list pgsql-general

From Curt Sampson
Subject Re: Mass-Data question
Date
Msg-id Pine.NEB.4.43.0204171219140.444-100000@angelic.cynic.net
Whole thread Raw
In response to Re: Mass-Data question  (Boris Köster <koester@x-itec.de>)
List pgsql-general
On Tue, 16 Apr 2002, [ISO-8859-15] Boris Kster wrote:

> Parallel querying sounds very interesting to me. My current plan was
> to do parallel writing because the hard-drives are not fast enough to
> collect all the data....

If it's really the hard drives that are not fast enough, you've
got a serious problem. The raw write speed of a hard drive is much,
much faster than Postgres.

But even so, it sounds like you have basically the same problem as
I do; how to get loads of data into the system really quickly.

> Normally it sounds not very complex to do parallelized
> reading/writing but getting the results in the right order that is a
> problem.

I don't see why. Just run the queries in parallel and merge the
results as they come in. Just make sure you use the same ORDER BY
on all the queries so you can do a merge sort.

cjs
--
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC


pgsql-general by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: [HACKERS] Testers needed ...
Next
From: Barry Lind
Date:
Subject: Re: j2sdk1.4.0 jdbc compile errors