Re: Load experimentation - Mailing list pgsql-performance

From Greg Smith
Subject Re: Load experimentation
Date
Msg-id 4B1D6CB1.3090909@2ndquadrant.com
Whole thread Raw
In response to Re: Load experimentation  (Ben Brehmer <benbrehmer@gmail.com>)
Responses Re: Load experimentation
List pgsql-performance
Ben Brehmer wrote:
By "Loading data" I am implying: "psql -U postgres -d somedatabase -f sql_file.sql".  The sql_file.sql contains table creates and insert statements. There are no indexes present nor created during the load.
COPY command: Unfortunately I'm stuck with INSERTS due to the nature this data was generated (Hadoop/MapReduce).
Your basic options here are to batch the INSERTs into bigger chunks, and/or to split your data file up so that it can be loaded by more than one process at a time.  There's some comments and links to more guidance here at http://wiki.postgresql.org/wiki/Bulk_Loading_and_Restores

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg@2ndQuadrant.com  www.2ndQuadrant.com

pgsql-performance by date:

Previous
From: Karl Denninger
Date:
Subject: Re: RAID card recommendation
Next
From: Greg Smith
Date:
Subject: Re: RAID card recommendation