Re: Re: [GENERAL] 7.0 vs. 7.1 (was: latest version?) - Mailing list pgsql-hackers

[Since I've rested over the weekend, I hope I don't come across this
morning as an angry old snarl, like some of my previous posts on this
subject unfortunately have been.]

Bruce Momjian wrote:
> > *     Location-agnostic installation.  Documentation (which I'll be happy to
> > contribute) on that.  Peter E is already working in this area. Getting
> > the installation that 'make install' spits out massaged into an FHS
> > compliant setup is the majority of the RPM's spec file.
> Well, we certainly don't want to make changes that make things harder or
> more confusing for non-RPM installs.  How are they affected here?

They wouldn't be.  Peter E has seemingly done an excellent job in this
area. I say seemingly because I haven't built an RPM from the 7.1 branch
yet, but from what he has posted, he seems to understand the issue. 
Many thanks, Peter.
> > *     Upgrades that don't require an ASCII database dump for migration. This
> > can either be implemented as a program to do a pg_dump of an arbitrary
> > version of data, or as a binary migration utility.  Currently, I'm
> I really don't see the issue here.

At the risk of being redundant, here goes.  As I've explained before,
the RPM upgrade environment, thanks to our standing with multiple
distributions as being shipped as a part of the OS, could be run as part
of a general-purpose OS upgrade.  In the environment of the general
purpose OS upgrade, the RPM's installation scripts cannot fire up a
backend, nor can it assume one is running or is not running, nor can the
RPM installation scripts fathom from the run-time environment whether
they are being run from a command line or from the OS upgrade (except on
Linux Mandrake, which allows such usage).

Thus, if a system administrator upgrades a system, or if an end user who
has a pgaccess-customized data entry system for things as mundane as an
address list or recipe book, there is no opportunity to do a dump.  The
dump has to be performed _after_ the RPM upgrade.

Now, this is far from optimal, I know.  I _know_ that the user should
take pains with their data.  I know that there should be a backup.  I
also know that a user of PostgreSQL should realize that 'this is just
the way it is done' and do things Our Way.

I also know that few new users will do it 'Our Way'.  No other package
that I am aware of requires the manual intervention that PostgreSQL
does, with the possible exception of upgrading to a different file
system -- but that is something most new users won't do, and is
something that is more difficult to automate.

However, over the weekend, while resting (I did absolutely NO computer
work this weekend -- too close to burnout), I had a brainstorm.

A binary migration tool does not need to be written, if a concession to
the needs of some users who just simply want to upgrade can be made.

Suppose we can package old backends (with newer network code to connect
to new clients).  Suppose further that postmaster can be made
intelligent enough to fire up old backends for old data, using
PG_VERSION as a key.  Suppose a NOTICE can be fired off warning the user
that 'The Database is running in Compatibility Mode -- some features may
not be available.  Please perform a dump of your data, reinitialize the
database, and restore your data to access new features of version x.y'.

I'm highly considering doing just that from a higher level.  It will not
be nearly as smooth, but doable.

Of course, that increases maintenance work, and I know it does.  But I'm
trying to find a middle ground here, since providing a true migration
utility (even if it just produces a dump of the old data) seems out of
reach at this time.

We are currently forcing something like a popular word processing
program once did -- it's proprietary file format changed.  It was coded
so that it could not even read the old files.  But both the old and the
new versions could read and write an interchange format.  People who
blindly upgraded their word processor were hit with a major problem. 
There was even a notice in the README -- which could be read after the
program was installed.

While the majority of us use PostgreSQL as a server behind websites and
other clients, there will be a large number of new users who want to use
it for much more mundane tasks.  Like address books, or personal
information management, or maybe even tax records.  Frontends to
PostgreSQL, thanks to PostgreSQL's advanced features, are likely to span
the gamut -- we already have OnShore TimeSheet for time tracking and
payroll, as one example.  And I even see database-backed intranet-style
web scripts being used on a client workstation for these sorts of
things.  I personally do just that with my home Linux box -- I have a
number of AOLserver dynamic pages that use PostgreSQL for many mundane
tasks (a multilevel sermon database is one).

While I don't need handholding in the upgrade process, I have provided
support to users that do -- who are astonished at the way we upgrade. 
Seamless upgrading won't help me personally -- but it will help
multitudes of users -- not just RPM users.  As a newbie to PostgreSQL I
was bitten, giving me compassion on those who might be bitten.

> We can compress ASCII dump files, so
> the space need should not be too bad.

Space isn't the problem.  The procedure is the problem.  Even if the
user fails to do it Right, we should at least attempt to help them
recover, IMHO.

> > *     A less source-centric mindset.  Let's see, how to explain?  The
> > regression tests are a good example.  You need make. You need the source
[snip]
> > it certainly does give the user something to use
> > to get standard output for bug reports.  As a point, I run PostgreSQL in

> Well, no compiler?  I can't see how we would do that without making
> other OS installs harder.  That is really the core of the issue.  We
> can't be making changes that make things harder for other OS's.  Those
> have to be isolated in the RPM, or in some other middle layer.

And I've done that in the past with the older serialized regression
tests.

I don't see how executing a shell script instead of executing a make
command would make it any harder for other OS users.  I am not trying to
make it harder for other OS users.  I _am_ trying to make it easier for
users who are getting funny results from queries to be able to run
regression tests as a standardized way to see where the problem lies. 
Maybe there is a hardware issue -- regression testing might be the only
way to have a standard way to pinpoint the problem.

And telling someone who is having a problem with prepackaged binaries
'Run the regression tests by executing the script
/usr/lib/pgsql/tests/regress/regress.sh and pipe me the results' is much
easier to do than 'Find me a test case where this blow up, and pipe me a
backtrace/dump/whatever' for the new users.  Plus that regression output
is a known quantity.

Or, to put it in a soundbite, regression testing can be the user's best
bug-zapping friend.

> > The documentation as well as many of the examples assume too much, IMHO,
> > about the install location and the install methodology.
> Well, if we are not specific, things get very confusing for those other
> OS's.  Being specific about locations makes things easier.  Seems we may
> need to patch RPM installs to fix that.  Certainly a pain, but I see no
> other options.

I can do that, I guess.  I currently ship the README.rpm as part of the
package -- but I continue to hear from people who have not read it, but
have read the online docs.  I have even put the unpacked source RPM up
on the ftp site so that people can read the README right online.
> Please give us more information about how the current upgrade is a
> problem.  We don't hear that much from other OS's.  How are RPM's
> specific, and maybe we can get a plan for a solution.

RPM's are expected to 'rpm -U' and you can simply _use_ the new version,
with little to no preparation.  At least that is the theory.  And it
works for most packages.

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11