Let's use libpq for everything - Mailing list pgsql-odbc

Hi everyone,

In the long-term, I believe psqlodbc would be better off relying on
libpq for all operations, and not touching the socket directly. That
would allow getting rid of all the authentication stuff, and a lot of
other code. In general, less code to maintain is good. This also ties
into my recent work in the PostgreSQL and libpq side to support other
SSL implementations than OpenSSL. If we ever get to do that in
PostgreSQL, psqlodbc will still be stuck with OpenSSL until someone gets
around to adding support for the new libraries. If we pull that off in
PostgreSQL side, *and* switch to doing everything through libpq in
psqlodbc, we have a lot to gain: if we can replace OpenSSL with native
Windows calls, we would no longer need to ship OpenSSL with the
installer, and we would no longer be vulnerable to whatever security
issues OpenSSL happens to have that week.

I went ahead and replaced all the backend-interactions with libpq calls.
I've pushed that to a development branch at:
git@github.com:hlinnaka/psqlodbc.git, branch "libpq-integration4". It's
not 100% complete - a few regression tests are failing - but it mostly
works. Not surprisingly, this makes a lot of code unnecessary:

  46 files changed, 1357 insertions(+), 7545 deletions(-)


The principle is simple: instead of sending FE/BE protocol messages
directly, just call the corresponding libpq functions. There are a few
problems that are worth mentioning:

0. A few regression tests fail currently. Need to investigate; they are
probably some minor bugs, nothing insurmountable.

1. The code structure in some places doesn't make it easy to switch to
libpq. We have code where we send the Parse message in one function, and
in a completely different place in the codebase we send the Sync and
process the results. The corresponding libpq functions couple those two
operations. Some refactoring of the code is required to bend it to the
libpq way. I tried to change as little as possible, to avoid unraveling
the whole ball of spaghetti that the psqlodbc code is, but some changes
were necessary to the way statements are prepared and executed.

2. Using libpq functions requires more round-trips for some operations,
because when speaking the protocol directly, you can "pipeline" some
operations, and libpq doesn't expose functions to do that. I haven't
investigated that thoroughly, but I think we will need an extra
round-trip for some cases where a query is prepared, described, and
executed. We currently send a command sequence: Parse, Describe
(Params), Bind, Execute, Sync. With libpq, the equivalent is to do
PQprepare + PQdescribePrepared + PQexecute, but that requires three
round-trips vs. one with the current approach. That sounds like a lot,
but I believe it only happens when preparing a "permanent" prepared
query for the first time. This probably needs more testing and comparing
to make sure none of the common cases regress too much, but I don't
think it's a show-stopper.

3. I'm not sure if I got all the less common "modes" like
"DisallowPremature=1/0" and "Parse=1/0" to work the same they did
before. I think we need to have a discussion on which modes we still
need to support, and which are just legacy that we could remove.

Does anyone object to this as a goal? If not, I'm going to continue
working on this, fixing the regression failures, and looking closely at
the number of round-trips.

- Heikki