Thread: Removing support for v1 and v2 protocols?
Hi I've just found a bug (followup mail to come) that seems to be tied up in support for the old v1/v2 protocols. Is there any reason to keep all that cruft? v3 was introduced in 7.4, in 2003-11-17 . Also, I'm wondering why psqlODBC does so much of its own protocol handling. It links to libpq, why doesn't it use it? There's a lot of complicated code in there - see, e.g. CC_send_query_append, that seems to duplicate much of what libpq its self does, and mix protocol handling up with client driver logic. I'm sure the bug I've just found isn't the only one lurking in that protocol code. How much of it can just be removed by adopting more of libpq? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Craig Ringer <craig@2ndquadrant.com> writes:
> I've just found a bug (followup mail to come) that seems to be tied up
> in support for the old v1/v2 protocols.
> Is there any reason to keep all that cruft? v3 was introduced in 7.4, in
> 2003-11-17 .
FWIW, I've been thinking of proposing that we drop v2 protocol support
in libpq and the backend.
I don't recall at the moment what the differences were between v1 and v2,
but I think that libpq hasn't had v1 support since 7.4, if not further
back. The backend might still nominally support v1, but that code is
surely pretty untested anymore. So you definitely don't want v1, if
indeed ODBC still contains support for that.
regards, tom lane
On 06/19/2014 11:10 AM, Tom Lane wrote: > Craig Ringer <craig@2ndquadrant.com> writes: >> I've just found a bug (followup mail to come) that seems to be tied up >> in support for the old v1/v2 protocols. > >> Is there any reason to keep all that cruft? v3 was introduced in 7.4, in >> 2003-11-17 . > > FWIW, I've been thinking of proposing that we drop v2 protocol support > in libpq and the backend. Sounds great, and we could cut a bunch of code out of PgJDBC then. That'll make Dave Cramer happy. > I don't recall at the moment what the differences were between v1 and v2, > but I think that libpq hasn't had v1 support since 7.4, if not further > back. The backend might still nominally support v1, but that code is > surely pretty untested anymore. So you definitely don't want v1, if > indeed ODBC still contains support for that. It appears to, in that it has places where it tests for whether it's talking to 6.4+ (!) or not. It might just be scattered remenants, but that's almost worse. *Begins sharpening razor* I'm a bit lost as to why psqlODBC does so much of this its self, instead of just using libpq. PgJDBC does it because it wants to be a JDBC Type 4 driver, i.e. all Java code, so it can't talk to libpq over JNI/JNA. That concern does not apply to psqlODBC, which in fact already links to libpq but doesn't seem to use it for protocol handling. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Jun 19, 2014 at 11:52 AM, Craig Ringer <craig@2ndquadrant.com> wrote: > Hi > > I've just found a bug (followup mail to come) that seems to be tied up > in support for the old v1/v2 protocols. > > Is there any reason to keep all that cruft? v3 was introduced in 7.4, in > 2003-11-17 . > > Also, I'm wondering why psqlODBC does so much of its own protocol > handling. It links to libpq, why doesn't it use it? There's a lot of > complicated code in there - see, e.g. CC_send_query_append, that seems > to duplicate much of what libpq its self does, and mix protocol handling > up with client driver logic. Please see here: http://www.postgresql.org/message-id/CAB7nPqRuxEuBgM-us4oKsc-rB6qN84CgsOC8M0O+pyW2d_GUKA@mail.gmail.com The plan is to wait a couple of weeks before Postgres 9.4 and then integrate that in the driver for the first 9.4-XXXX release of the ODBC driver. For now, this work is periodically rebased and stored on my own pgodbc fork on github. This cleanup is of course perfectible, but that's a first step into a more maintainable code. -- Michael
On Thu, Jun 19, 2014 at 12:49 PM, Craig Ringer <craig@2ndquadrant.com> wrote: > *Begins sharpening razor* My katana is already pretty sharp on the matter, no need to double the amount of work :) -- Michael
Craig Ringer <craig@2ndquadrant.com> writes:
> On 06/19/2014 11:10 AM, Tom Lane wrote:
>> FWIW, I've been thinking of proposing that we drop v2 protocol support
>> in libpq and the backend.
> Sounds great, and we could cut a bunch of code out of PgJDBC then.
> That'll make Dave Cramer happy.
Actually, I thought the JDBC crew would be the most likely source of
push-back on such a proposal :-(. It's been common practice in that
community to tell the JDBC driver to use v2 protocol if you were having
performance problems with prepared statements. I'm not sure if those
issues have been solved to the point where that workaround is no longer
necessary.
regards, tom lane
On 06/19/2014 11:59 AM, Tom Lane wrote: > Actually, I thought the JDBC crew would be the most likely source of > push-back on such a proposal :-(. It's been common practice in that > community to tell the JDBC driver to use v2 protocol if you were having > performance problems with prepared statements. I'm not sure if those > issues have been solved to the point where that workaround is no longer > necessary. The work you committed in 9.2 (?) to re-plan generic paths appears to have resolved the vast majority of that problem. Certainly to the point where it's no longer worth retaining a whole new protocol just for the purpose. At least, that's my opinion on the matter - Dave's the one doing the vast majority of the work on the driver, so it's his opinion that'll really count, but I'll be surprised if he isn't pleased by anything that shrinks and simplifies the driver. I've CC'd him. Dave, this (small) thread begins here: http://www.postgresql.org/message-id/53A2507F.808@2ndquadrant.com -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Thu, Jun 19, 2014 at 4:49 AM, Craig Ringer <craig@2ndquadrant.com> wrote: > > I'm a bit lost as to why psqlODBC does so much of this its self, instead > of just using libpq. PgJDBC does it because it wants to be a JDBC Type 4 > driver, i.e. all Java code, so it can't talk to libpq over JNI/JNA. That > concern does not apply to psqlODBC, which in fact already links to libpq > but doesn't seem to use it for protocol handling. We tried to port psqlODBC to libpq 5 - 10 years ago. It didn't go well, and eventually we went back to the original code. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 06/19/2014 03:49 PM, Dave Page wrote: > We tried to port psqlODBC to libpq 5 - 10 years ago. It didn't go > well, and eventually we went back to the original code. Pity. I'm glad it's been looked at though. I'll see if I can find the threads. Perhaps it's worth seeing if a few progressive enhancements to libpq (hook functions, "legal" access to the raw socket for applications that promise to be good little citizens, etc) can make it possible in future. Maintaining psqlODBC as yet another client library doesn't look like fun from the little I've seen. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 06/19/2014 05:52 AM, Craig Ringer wrote: > Hi > > I've just found a bug (followup mail to come) that seems to be tied up > in support for the old v1/v2 protocols. > > Is there any reason to keep all that cruft? v3 was introduced in 7.4, in > 2003-11-17 . No reason. Michael Paquier has been working on removing it, see thread "Removing support for < 7.4 servers": http://www.postgresql.org/message-id/CAB7nPqRuxEuBgM-us4oKsc-rB6qN84CgsOC8M0O+pyW2d_GUKA@mail.gmail.com > Also, I'm wondering why psqlODBC does so much of its own protocol > handling. It links to libpq, why doesn't it use it? There's a lot of > complicated code in there - see, e.g. CC_send_query_append, that seems > to duplicate much of what libpq its self does, and mix protocol handling > up with client driver logic. Yeah. Historical reasons, I'd say. I'd like to refactor the driver to stop doing that, and use libpq. It's not an easy task, exactly because the current code is quite complicated. - Heikki
On 18 June 2014 23:59, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> FWIW, I've been thinking of proposing that we drop v2 protocol supportActually, I thought the JDBC crew would be the most likely source of
>> in libpq and the backend.
> Sounds great, and we could cut a bunch of code out of PgJDBC then.
> That'll make Dave Cramer happy.
push-back on such a proposal :-(. It's been common practice in that
community to tell the JDBC driver to use v2 protocol if you were having
performance problems with prepared statements. I'm not sure if those
issues have been solved to the point where that workaround is no longer
necessary.
Ya, that is the short version... I'll go through the list and see just what kind of issues we are still solving with v2, if any
If we drop v2 can we start working on v4 or v3+ there are still some issues such as returning the generated keys which I think the server can help us with
As far as dropping code goes, the driver is pretty well written so that dropping v2 should be a matter of just removing the v2 package. We don't really have to muck with that code anymore
* Dave Page (dpage@pgadmin.org) wrote:
> On Thu, Jun 19, 2014 at 4:49 AM, Craig Ringer <craig@2ndquadrant.com> wrote:
> > I'm a bit lost as to why psqlODBC does so much of this its self, instead
> > of just using libpq. PgJDBC does it because it wants to be a JDBC Type 4
> > driver, i.e. all Java code, so it can't talk to libpq over JNI/JNA. That
> > concern does not apply to psqlODBC, which in fact already links to libpq
> > but doesn't seem to use it for protocol handling.
>
> We tried to port psqlODBC to libpq 5 - 10 years ago. It didn't go
> well, and eventually we went back to the original code.
As I recall, one big issue about porting to libpq was the lack of a
streaming-style capability without using cursors. We have the 'single
row mode' option in libpq now which might address that. I'm sure there
were other issues but I don't recall them offhand.
Thanks,
Stephen
Attachment
On 06/19/2014 07:49 PM, Dave Cramer wrote: > > If we drop v2 can we start working on v4 or v3+ there are still some > issues such as returning the generated keys which I think the server can > help us with > As it may be of interest to the ODBC folks, too, I should note on-list that Ian Barwick is working on a RETURNING PRIMARY KEY patch. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, Jun 20, 2014 at 9:27 AM, Craig Ringer <craig@2ndquadrant.com> wrote: > As it may be of interest to the ODBC folks, too, I should note on-list > that Ian Barwick is working on a RETURNING PRIMARY KEY patch. Patch: https://commitfest.postgresql.org/action/patch_view?id=1465 Beginning of thread: http://www.postgresql.org/message-id/53953EFB.8070701@2ndquadrant.com -- Michael
On 06/19/2014 06:49 AM, Michael Paquier wrote: > On Thu, Jun 19, 2014 at 11:52 AM, Craig Ringer <craig@2ndquadrant.com> wrote: >> Hi >> >> I've just found a bug (followup mail to come) that seems to be tied up >> in support for the old v1/v2 protocols. >> >> Is there any reason to keep all that cruft? v3 was introduced in 7.4, in >> 2003-11-17 . >> >> Also, I'm wondering why psqlODBC does so much of its own protocol >> handling. It links to libpq, why doesn't it use it? There's a lot of >> complicated code in there - see, e.g. CC_send_query_append, that seems >> to duplicate much of what libpq its self does, and mix protocol handling >> up with client driver logic. > > Please see here: > http://www.postgresql.org/message-id/CAB7nPqRuxEuBgM-us4oKsc-rB6qN84CgsOC8M0O+pyW2d_GUKA@mail.gmail.com > The plan is to wait a couple of weeks before Postgres 9.4 and then > integrate that in the driver for the first 9.4-XXXX release of the > ODBC driver. For now, this work is periodically rebased and stored on > my own pgodbc fork on github. I'm getting eager to move ahead with this. In addition to Michael's cleanup, I have a bunch of other changes queued up that are such big changes that it would be nice to do them while bumping the major version number: 1. Switch to MSBuild-based build system on Windows. 2. Link msdtc_enlist.cpp directly into the main DLL. http://www.postgresql.org/message-id/53A57080.1070501@vmware.com 3. Deal with Unicode in DSN names and other config options. (Subject "Non-ASCII DSN name troubles", post stalled ATM) I propose the following: * Do one more release of the 09.03.XXXX series, from the current git master. Right now. There are a bunch of bug fixes there that people are probably waiting for. * After that last 09.03.XXXX release, bump the major version number in git master to 09.04.0000 and commit all of those bigger patches. Any objections? Could someone wrap a new release, please? - Heikki
On Sat, Jun 21, 2014 at 8:59 PM, Heikki Linnakangas <hlinnakangas@vmware.com> wrote: > * Do one more release of the 09.03.XXXX series, from the current git master. > Right now. There are a bunch of bug fixes there that people are probably > waiting for. > > * After that last 09.03.XXXX release, bump the major version number in git > master to 09.04.0000 and commit all of those bigger patches. > > Any objections? Could someone wrap a new release, please? I'm fine with this plan, but I don't do the release :) so... I'm pretty sure I'll be able to rebase my 9.4 cleanup branch once again by Monday. -- Michael