Re: Merging large volumes of data - Mailing list pgsql-performance

From Gregory Stark
Subject Re: Merging large volumes of data
Date
Msg-id 87lkg0yanc.fsf@oxford.xeocode.com
Whole thread Raw
In response to Re: Merging large volumes of data  (Andreas Kostyrka <andreas@kostyrka.org>)
List pgsql-performance
"Andreas Kostyrka" <andreas@kostyrka.org> writes:

>> (select a,b,c,d,e from table1 order by a,b) union all
>> (select a,b,c,d,e from table2 order by a,b)  union all
>> etc...
>> (select a,b,c,d,e from tablen order by a,b)  order by a,b;
>>
>> PostgreSQL does not seem to realise (maybe it should not be able to do this
>> trick anyway) that the last "order by" clause is merely a final merge step
>> on the ordered data sets.

There's no plan type in Postgres for merging pre-sorted data like this. The
only merge plan type is for joins which isn't going to be what you need.

But the queries as written here would be just as fast or faster to do one big
sort as they would be to do separate sorts and merge the results.

You might want to do it the way you describe if there were selective WHERE
clauses that you've left out that make the intermediate orderings come for
free.

--
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com


pgsql-performance by date:

Previous
From: Andreas Kostyrka
Date:
Subject: Re: Merging large volumes of data
Next
From: Tom Lane
Date:
Subject: Re: Merging large volumes of data