Thread: Improved JDBC driver part 2
Hello,
Maybe you are interested about this what I done with JDBC
=== Original driver (Text mode) ===
* Memory *
1. Memory usage improvments when using result set input streams (no uneeded
memory copy) - needs few touches for bigger performance.
2. Memory usage improvments for large data, should be no problem to load 1GB
bytea[] when have only 300MB of memory ("threshold" size still hardcoded).
* JDBC 4 *
1. XML are now correctly transformed before send to server - previous version
used normal text-file transformations that is not enaugh.
2. In all modes (text/binary) XMLs are sended in binary mode, so driver don't
need to do special transformation (does it require libxml?), until character
streams are used.
3. JDBC4 exception throwing.
4. XML objects are readable only once, you can't reuse it, update form result
set (silently set to null on RS.updateRow() - shouldn't be silent) returns
null till refreshRow(), but you can write to them after load.
5.Target XML behavior is streaming behavior to don't repeat problems with
bytea.
* JDBC 4.1 *
1. Just started.
* Others *
1. Few additional test cases. Few utils for XML checking (string equals is too
less) no good, but better.
2. Fixed bug, causing inproper time(stamps) encoding for WITH TIME ZONE fields,
after changing default time zone.
=== Binary mode ===
1. Read for almost all data types with arrays.
2. Write for few.
3. Much more restrictive checking when casting form one type to other.
4. Exceptions when casting from one type to other inproper type.
5. Still ResultSet.getString() for XML will return XML - this spec. prohibited
(X - base type conversion, x - possible conversion, no x - no base and
possible = no conversion).
6. No getPriviliges for metadata - no binary output for ACL!!!
7. Many, many tests passed.
8. Data reading is faster for all reads (checked with profiler, against
original driver).
Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is
currently JDK 6 compatible (will be not), compressed patch takes about 136kb
gziped.
Kind regards & have a nice day
----------
Radosław Smogura
http://www.softperience.eu
Hi,
I cannot get the file:
--2010-12-01 12:05:28-- http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz
Resolving www.rsmogura.net... 64.120.14.83
Connecting to www.rsmogura.net|64.120.14.83|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2010-12-01 12:05:29 ERROR 404: Not Found.
On Tue, Nov 30, 2010 at 7:49 PM, Radosław Smogura <mail@smogura.eu> wrote:
Hello,
Maybe you are interested about this what I done with JDBC
=== Original driver (Text mode) ===
* Memory *
1. Memory usage improvments when using result set input streams (no uneeded
memory copy) - needs few touches for bigger performance.
2. Memory usage improvments for large data, should be no problem to load 1GB
bytea[] when have only 300MB of memory ("threshold" size still hardcoded).
* JDBC 4 *
1. XML are now correctly transformed before send to server - previous version
used normal text-file transformations that is not enaugh.
2. In all modes (text/binary) XMLs are sended in binary mode, so driver don't
need to do special transformation (does it require libxml?), until character
streams are used.
3. JDBC4 exception throwing.
4. XML objects are readable only once, you can't reuse it, update form result
set (silently set to null on RS.updateRow() - shouldn't be silent) returns
null till refreshRow(), but you can write to them after load.
5.Target XML behavior is streaming behavior to don't repeat problems with
bytea.
* JDBC 4.1 *
1. Just started.
* Others *
1. Few additional test cases. Few utils for XML checking (string equals is too
less) no good, but better.
2. Fixed bug, causing inproper time(stamps) encoding for WITH TIME ZONE fields,
after changing default time zone.
=== Binary mode ===
1. Read for almost all data types with arrays.
2. Write for few.
3. Much more restrictive checking when casting form one type to other.
4. Exceptions when casting from one type to other inproper type.
5. Still ResultSet.getString() for XML will return XML - this spec. prohibited
(X - base type conversion, x - possible conversion, no x - no base and
possible = no conversion).
6. No getPriviliges for metadata - no binary output for ACL!!!
7. Many, many tests passed.
8. Data reading is faster for all reads (checked with profiler, against
original driver).
Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is
currently JDK 6 compatible (will be not), compressed patch takes about 136kb
gziped.
Kind regards & have a nice day
----------
Radosław Smogura
http://www.softperience.eu
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
I've just started small clean up - now it's there. On Wed, 1 Dec 2010 12:06:19 +0100, Valentine Gogichashvili <valgog@gmail.com> wrote: > Hi, > > I cannot get the file: > > wget http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz > --2010-12-01 12:05:28-- > http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz > Resolving www.rsmogura.net... 64.120.14.83 > Connecting to www.rsmogura.net|64.120.14.83|:80... connected. > HTTP request sent, awaiting response... 404 Not Found > 2010-12-01 12:05:29 ERROR 404: Not Found. > > > On Tue, Nov 30, 2010 at 7:49 PM, Radosław Smogura <mail@smogura.eu> wrote: > >> Hello, >> >> Maybe you are interested about this what I done with JDBC >> >> === Original driver (Text mode) === >> * Memory * >> 1. Memory usage improvments when using result set input streams (no >> uneeded >> memory copy) - needs few touches for bigger performance. >> 2. Memory usage improvments for large data, should be no problem to load >> 1GB >> bytea[] when have only 300MB of memory ("threshold" size still >> hardcoded). >> >> * JDBC 4 * >> 1. XML are now correctly transformed before send to server - previous >> version >> used normal text-file transformations that is not enaugh. >> 2. In all modes (text/binary) XMLs are sended in binary mode, so driver >> don't >> need to do special transformation (does it require libxml?), until >> character >> streams are used. >> 3. JDBC4 exception throwing. >> 4. XML objects are readable only once, you can't reuse it, update form >> result >> set (silently set to null on RS.updateRow() - shouldn't be silent) >> returns >> null till refreshRow(), but you can write to them after load. >> 5.Target XML behavior is streaming behavior to don't repeat problems with >> bytea. >> >> * JDBC 4.1 * >> 1. Just started. >> >> * Others * >> 1. Few additional test cases. Few utils for XML checking (string equals >> is >> too >> less) no good, but better. >> 2. Fixed bug, causing inproper time(stamps) encoding for WITH TIME ZONE >> fields, >> after changing default time zone. >> >> === Binary mode === >> 1. Read for almost all data types with arrays. >> 2. Write for few. >> 3. Much more restrictive checking when casting form one type to other. >> 4. Exceptions when casting from one type to other inproper type. >> 5. Still ResultSet.getString() for XML will return XML - this spec. >> prohibited >> (X - base type conversion, x - possible conversion, no x - no base and >> possible = no conversion). >> 6. No getPriviliges for metadata - no binary output for ACL!!! >> 7. Many, many tests passed. >> 8. Data reading is faster for all reads (checked with profiler, against >> original driver). >> >> Driver is here >> http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gzis >> currently JDK 6 compatible (will be not), compressed patch takes about >> 136kb >> gziped. >> >> Kind regards & have a nice day >> ---------- >> Radosław Smogura >> http://www.softperience.eu >> >> -- >> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-hackers >> -- ---------- Radosław Smogura http://www.softperience.eu
On Tue, Nov 30, 2010 at 19:49, Radosław Smogura <mail@smogura.eu> wrote: > Hello, > > Maybe you are interested about this what I done with JDBC <snip> > Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is > currently JDK 6 compatible (will be not), compressed patch takes about 136kb > gziped. Is there any particular reason why this work can't be maintained as a branch to the main driver? My understanding is your work is based off that one? Being able to work like that would make things a lot easier to review. That said, such a process would also be a lot easier if the JDBC driver wasn't in cvs ;) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Wed, 1 Dec 2010 12:47:13 +0100, Magnus Hagander <magnus@hagander.net> wrote: > On Tue, Nov 30, 2010 at 19:49, Radosław Smogura <mail@smogura.eu> wrote: >> Hello, >> >> Maybe you are interested about this what I done with JDBC > > <snip> > > >> Driver is here >> http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is >> currently JDK 6 compatible (will be not), compressed patch takes about >> 136kb >> gziped. > > Is there any particular reason why this work can't be maintained as a > branch to the main driver? My understanding is your work is based off > that one? Being able to work like that would make things a lot easier > to review. Yes, it's based on this, with CVS subfolders in sources. I don't see any problems to maintain this as branch. Ah only one need to read something about CVS & branching. > That said, such a process would also be a lot easier if the JDBC > driver wasn't in cvs ;) Yes, SVN is much more nicer. > > -- > Magnus Hagander > Me: http://www.hagander.net/ > Work: http://www.redpill-linpro.com/ -- ---------- Radosław Smogura http://www.softperience.eu
On Wed, Dec 01, 2010 at 12:47:13PM +0100, Magnus Hagander wrote: > On Tue, Nov 30, 2010 at 19:49, Radosław Smogura <mail@smogura.eu> wrote: > > Hello, > > > > Maybe you are interested about this what I done with JDBC > > <snip> > > > > Driver is here http://www.rsmogura.net/pgsql/pgjdbc_exp_20101130_C.tar.gz is > > currently JDK 6 compatible (will be not), compressed patch takes about 136kb > > gziped. > > Is there any particular reason why this work can't be maintained as a > branch to the main driver? My understanding is your work is based off > that one? Being able to work like that would make things a lot easier > to review. > > That said, such a process would also be a lot easier if the JDBC > driver wasn't in cvs ;) This brings up an excellent point. Would the people now developing the JDBC driver object to switching to git? Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter <david@fetter.org> wrote: > Would the people now developing the JDBC driver object to > switching to git? If we move to git, don't forget that there is not one repository which has the entire history for PostgreSQL JDBC -- the current repository is missing some work, including releases, from one stable branch. (Was it 7.4?) We'd probably want to merge that in as part of any conversion effort. -Kevin
On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote: > David Fetter <david@fetter.org> wrote: > > > Would the people now developing the JDBC driver object to > > switching to git? > > If we move to git, don't forget that there is not one repository > which has the entire history for PostgreSQL JDBC -- the current > repository is missing some work, including releases, from one stable > branch. (Was it 7.4?) We'd probably want to merge that in as part > of any conversion effort. I guess that depends on our current needs. As pre-split-off JDBC driver history is already preserved in the main git tree, I'd see putting the pre-split history into the JDBC tree as less important than making current and future JDBC development easier, but that's just me. Cheers, David. -- David Fetter <david@fetter.org> http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fetter@gmail.com iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter <david@fetter.org> writes:
> On Wed, Dec 01, 2010 at 10:15:38AM -0600, Kevin Grittner wrote:
>> If we move to git, don't forget that there is not one repository
>> which has the entire history for PostgreSQL JDBC -- the current
>> repository is missing some work, including releases, from one stable
>> branch. (Was it 7.4?) We'd probably want to merge that in as part
>> of any conversion effort.
> I guess that depends on our current needs. As pre-split-off JDBC
> driver history is already preserved in the main git tree, I'd see
> putting the pre-split history into the JDBC tree as less important
> than making current and future JDBC development easier, but that's
> just me.
It was difficult enough to get an accurate conversion from cvs to git
when that was the only constraint. Trying to merge some unrelated
history in at the same time seems like a recipe for trouble. I'd
recommend just converting your existing repo and calling it good.
regards, tom lane