Thread: COPY failure

Hi.

I'm trying to run a script that executes multiple \copy commands in order
to import some 5 GB of data. All the input files are computer generated,
simple (5 numeric columns and "\N" for NULL in some cases), use default
delimiters, and _should_ be error free. But I keep getting error messages
for _some_ of the \copy commands.

eg,
pqReadData() -- backend closed the channel unexpectedly.
         This probably means the backend terminated abnormally
         before or while processing the request.
PQendcopy: resetting connection

Questions:

1. Are there any size restrictions on the input files?

2. How do I tell which file or better yet which line is tripping up the
system? I could cut the list in half repeatedly until I find the problem,
but that would be a huge waste of time given how long it takes to import
any of the data. I could setup more verbose logging on the backend, but
will that make a mess if my error is 2 GB into the import?

3. I'm running the Windows/cygwin version of the psql client and a Linux
backend, if that makes a difference.

Any help would be much appreciated.

-Xavier



_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

x <xbdelacour@yahoo.com> writes:

> pqReadData() -- backend closed the channel unexpectedly.
>          This probably means the backend terminated abnormally
>          before or while processing the request.
> PQendcopy: resetting connection
>
> 3. I'm running the Windows/cygwin version of the psql client and a Linux
> backend, if that makes a difference.

What version of the backend?  Is there anything in the log?  Have you
cranked up the debug level?

-Doug
--
The rain man gave me two cures; he said jump right in,
The first was Texas medicine--the second was just railroad gin,
And like a fool I mixed them, and it strangled up my mind,
Now people just get uglier, and I got no sense of time...          --Dylan

x <xbdelacour@yahoo.com> writes:
> pqReadData() -- backend closed the channel unexpectedly.

That shouldn't happen :-(.  Look for a core file in $PGDATA/base/yourdb/core
(if you don't find one, you probably need to restart the postmaster with
"ulimit -c unlimited" to allow core dumps).  Then do
    gdb /path/to/postgres/executable /path/to/corefile
    gdb> bt
    gdb> quit
and send us the results.

BTW, what Postgres version is this, and what are the declarations of the
tables being loaded?

            regards, tom lane

Hmm, what about the postmaster log (stderr output) --- does it indicate
the backend is terminating normally?

            regards, tom lane