Re: performance while importing a very large data set in to database - Mailing list pgsql-performance

From Kris Kewley
Subject Re: performance while importing a very large data set in to database
Date
Msg-id 0CFC51C4-0916-45D2-AD67-E8F1E705E087@gmail.com
Whole thread Raw
In response to Re: performance while importing a very large data set in to database  (Scott Marlowe <scott.marlowe@gmail.com>)
Responses Re: performance while importing a very large data set in to database
List pgsql-performance
Does postgres have the concept of "pinning" procs, functions, etc to
cache.

As you mention, typically frequently used statements are cached
improving performance.

If waiting for the DBMS to do this is not an option then pinning
critical ones should improve performance immediately following start up.

This is an approach I have used with oracle to address this situation.

Kris

On 5-Dec-09, at 15:42, Scott Marlowe <scott.marlowe@gmail.com> wrote:

> On Wed, Dec 2, 2009 at 4:31 PM, Ashish Kumar Singh
> <ashishkumar.singh@altair.com> wrote:
>> Hello Everyone,
>>
>> I have a very bit big database around 15 million in size, and the
>> dump file
>> is around 12 GB.
>>
>> While importing this dump in to database I have noticed that
>> initially query
>> response time is very slow but it does improves with time.
>>
>> Any suggestions to improve performance after dump in imported in to
>> database
>> will be highly appreciated!
>
> This is pretty normal.  When the db first starts up or right after a
> load it has nothing in its buffers or the kernel cache.  As you access
> more and more data the db and OS learned what is most commonly
> accessed and start holding onto those data and throw the less used
> stuff away to make room for it.  Our production dbs run at a load
> factor of about 4 to 6, but when first started and put in the loop
> they'll hit 25 or 30 and have slow queries for a minute or so.
>
> Having a fast IO subsystem will help offset some of this, and
> sometimes "select * from bigtable" might too.
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
> )
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

pgsql-performance by date:

Previous
From: Pierre Frédéric Caillaud
Date:
Subject: Re: performance while importing a very large data set in to database
Next
From: Greg Smith
Date:
Subject: Re: performance while importing a very large data set in to database