Re: [BUGS] BUG #1347: Bulk Import stopps after a while ( - Mailing list pgsql-jdbc

From Oliver Jowett
Subject Re: [BUGS] BUG #1347: Bulk Import stopps after a while (
Date
Msg-id 41BF5554.4040309@opencloud.com
Whole thread Raw
In response to Re: [BUGS] BUG #1347: Bulk Import stopps after a while (  (Markus Schaber <schabios@logi-track.com>)
List pgsql-jdbc
Markus Schaber wrote:
> Hi, Oliver,
>
> On Tue, 14 Dec 2004 09:58:16 +1300
> Oliver Jowett <oliver@opencloud.com> wrote:
>
>
>>Also, it's never going to be 100% without a separate thread, as the
>>server can spontaneously generate output (e.g. because of NOTIFY)
>>regardless of how careful we are with our queries.

I think my analysis here was wrong. It doesn't matter if the server
unexpectedly fills up its output pipe so long as the driver doesn't do
the same.

> Sadly, we cannot abandon compatibility with older JDK releases, this
> would allow us to use the new asynchroneous I/O interfaces.

Hm, is async I/O standardized yet? I thought JSR203 ("more NIO") covered
this, but it's still in the early stages according to jcp.org.

> Async I/O is a great tool when in the right hands, you might google for
> Felix von Leitner and his "fnord" and "gatling" servers that even beat
> the kernel based khttpd.

Current NIO isn't async I/O, but it's still fairly good (use at least
1.4.2_06, though, or you can have GC problems when collecting direct
buffers). We have written several resource adaptors that use it and they
perform well. The main win is that you don't need N threads for N
clients. That can't really be applied to the JDBC driver easily as the
API means that the client controls most of the threading, but the
ability to get at non-blocking primitives would still be useful.

I wonder if we can have some threads-or-NIO abstraction .. the main
thing we need is the ability to call out to some other code to do reads
either when we discover our writes will block (NIO) or when we have data
ready to read (threads). We then read and process messages until we
would block (NIO) or we *do* block (threads).

I doubt I'll have time to look at this in the near future though.

-O

pgsql-jdbc by date:

Previous
From: Csaba Nagy
Date:
Subject: Re: query not working with JDBC driver
Next
From: Eli Bingham
Date:
Subject: [7.4.6] Calling PLpgSQL stored procedures with table row arguments via JDBC?