Thread: ignore duplicate key while using COPY?

I am using COPY public.table_name FROM STDIN  to import data.  it is
very efficient, but if there's any duplicate key exists, the whole
procedure has been stopped.  is there anyway to around this?

why does not postgresql just give a warning and continue the copy?

Copy to a staging table without the constraint and then write a query to consolidate/remove the duplicate data and
insertthe clean data into the final table. 

It doesn't just give a warning because not importing a record from the source data should be a fatal error since it
meansyou have a design mis-match surrounding your import procedure. 

David J.

On Dec 19, 2011, at 1:54, Yan Chunlu <springrider@gmail.com> wrote:

> I am using COPY public.table_name FROM STDIN  to import data.  it is
> very efficient, but if there's any duplicate key exists, the whole
> procedure has been stopped.  is there anyway to around this?
>
> why does not postgresql just give a warning and continue the copy?
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general

On Sunday, December 18, 2011 10:54:21 pm Yan Chunlu wrote:
> I am using COPY public.table_name FROM STDIN  to import data.  it is
> very efficient, but if there's any duplicate key exists, the whole
> procedure has been stopped.  is there anyway to around this?
>
> why does not postgresql just give a warning and continue the copy?

Take a look at pgloader:
http://pgfoundry.org/projects/pgloader/

--
Adrian Klaver
adrian.klaver@gmail.com