Re: RPMS for 7.3 beta. - Mailing list pgsql-hackers

On Tuesday 17 September 2002 11:51 pm, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> >> How does pg_upgrade work?
> > [ pretty good description ]

> You missed a key point, which is that pg_upgrade does not even try to
> cope with version-to-version system catalog changes.  It assumes it can
> use pg_dump to dump and reload the database schema.  So there is no
> hope, ever, that it will be more reliable than pg_dump.  All pg_upgrade
> tries to do is short-circuit the moving of the bulk data.

Yes, this is a key point and one that shouldn't be overlooked.  If the 
metadata belonging to the user's data didn't have to be pg_dumped, but was 
decoupled somewhat from the system metadata about types, operators, classes, 
and the like, the schema (great, another overloaded term) wouldn't need 
dumping but would travel with its data.

> The bald fact of the matter is that we are still a good ways away from
> the point where we might be willing to freeze the system catalogs.  

Not talking about a freeze.  Talking about separation of system/feature 
metadata from user metadata that wouldn't change in the upgrade anyway -- 
table names, fields, user types, views, triggers, etc, that belong to this 
database and not to the installation as a whole.  If columns need changed or 
added to the user data's metadata, have the upgrade script run the 
appropriate ALTER commands and UPDATES necessary.  The hard parts, I know, 
are the details behind the broad 'appropriate'.

> PG
> is evolving and improving by a substantial amount with every release,
> and the implication of that is that there *will* be some upgrade pain.

Why is it a given conclusion?  It should not be axiomatic that 'there *will* 
be upgrade pain if we improve our features.'  That's fatalistic.

We have innovative solutions in PostgreSQL that solve some pretty hairy 
problems.  WAL.  MVCC.  The subselect code (made my day when I heard about 
that one -- but then had to wait seven months before Red Hat saw fit to 
provide an RPM that I wasn't expecting.....the other reason I began RPM 
building, even though it was two cycles later before I got up the nerve to 
tackle it...).  The PL's. Foreign keys. TOAST (now that's a prime example of 
a 'sideways' solution to a head-on problem).

This is just a different challenge: how to keep the loosely dynamic system 
catalog structure while at the same time allowing the possibility of smooth 
data migration so people can more easily take advantage of the improved 
system catalog structure.  And yes I know that such a change is not for 7.3.  
Too late for that, and maybe too late for 7.4 too.

But unlike Bruce I winced at Oliver's last line -- it hit a little too close 
to home and to many multitudes of bug reports and nastygrams directed my way 
for something I have tried to kludge around in the past.  Yes, nastygrams, in 
the grand old alt.flame tradition.  When you maintain RPM's, you find 
yourself the point man for the entire project in some people's eyes.  The bug 
report about my RPM's trashing a fellow's RPM database was an extreme example 
of that.  I get two-three dozen e-mails a week that I redirect to the web 
site and/or the mailing lists.  I'm sure Oliver is nodding his head in 
understanding on this one.

I don't think seamless upgrading is a pipe dream.  And I think that dismissing 
it out of hand as 'impossible' is a self-fulfilling prophecy.

But I do think it won't work well if it's just tacked-on.

But, like Tom, I really don't have more of an answer than that.  I do 
understand pg_upgrade much better now, though.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11