Thread: sync structures

Hi,
I have a development DB and a production DB.  I need a way to sync the changes
I make to the stucture in the devel DB to the production DB.  I found pgdiff
but can't get it to work.  I would like a solution that would work on windows
and linux.   But I'll take either alone.

postgres 8.3
openSUSE 11.0
windows XP/vista

Thanks in advance for any help.

Johnf

On Monday 28 September 2009 09:06:25 am Filip Rembiałkowski wrote:
> 2009/9/28 John <jfabiani@yolo.com>
>
> > Hi,
> > I have a development DB and a production DB.  I need a way to sync the
> > changes
> > I make to the stucture in the devel DB to the production DB.  I found
> > pgdiff
> > but can't get it to work.  I would like a solution that would work on
> > windows
> > and linux.   But I'll take either alone.
> >
> > postgres 8.3
> > openSUSE 11.0
> > windows XP/vista
> >
> > Thanks in advance for any help.
>
> You could use standard text compare programs. They won't  write a patch for
> you. There's still some niche for live DBAs.
>
> pg_dump -sOx dev_db_name > dev.schema
> pg_dump -sOx prod_db_name > prod.schema
> diff -u dev.schema prod.schema

Thanks that will help.  After all this time I'm surprized that someone hasn't
provide an easy way to get this done.  It's has to be every developers
problem.

Johnf

----- "John" <jfabiani@yolo.com> wrote:

> On Monday 28 September 2009 09:06:25 am Filip Rembiałkowski wrote:
> > 2009/9/28 John <jfabiani@yolo.com>
> >
> > > Hi,
> > > I have a development DB and a production DB.  I need a way to sync
> the
> > > changes
> > > I make to the stucture in the devel DB to the production DB.  I
> found
> > > pgdiff
> > > but can't get it to work.  I would like a solution that would work
> on
> > > windows
> > > and linux.   But I'll take either alone.
> > >
> > > postgres 8.3
> > > openSUSE 11.0
> > > windows XP/vista
> > >
> > > Thanks in advance for any help.
> >
> > You could use standard text compare programs. They won't  write a
> patch for
> > you. There's still some niche for live DBAs.
> >
> > pg_dump -sOx dev_db_name > dev.schema
> > pg_dump -sOx prod_db_name > prod.schema
> > diff -u dev.schema prod.schema
>
> Thanks that will help.  After all this time I'm surprized that someone
> hasn't
> provide an easy way to get this done.  It's has to be every developers
>
> problem.
>
> Johnf

I have looked at but not tried pgmigrate:
http://code.google.com/p/pgmigrate/

It might do what you want.


Adrian Klaver
aklaver@comcast.net

On Sep 28, 2009, at 9:24 AM, John wrote:

> On Monday 28 September 2009 09:06:25 am Filip Rembiałkowski wrote:
>> 2009/9/28 John <jfabiani@yolo.com>
>>
>>> Hi,
>>> I have a development DB and a production DB.  I need a way to sync
>>> the
>>> changes
>>> I make to the stucture in the devel DB to the production DB.  I
>>> found
>>> pgdiff
>>> but can't get it to work.  I would like a solution that would work
>>> on
>>> windows
>>> and linux.   But I'll take either alone.
>>>
>>> postgres 8.3
>>> openSUSE 11.0
>>> windows XP/vista
>>>
>>> Thanks in advance for any help.
>>
>> You could use standard text compare programs. They won't  write a
>> patch for
>> you. There's still some niche for live DBAs.
>>
>> pg_dump -sOx dev_db_name > dev.schema
>> pg_dump -sOx prod_db_name > prod.schema
>> diff -u dev.schema prod.schema
>
> Thanks that will help.  After all this time I'm surprized that
> someone hasn't
> provide an easy way to get this done.  It's has to be every developers
> problem.


It's hard to do by comparing two schemas in general. For example,
if you rename a column from bob to jerry, there's not enough information
left to a tool to tell whether you dropped a column called bob and
created one called jerry, or renamed bob to jerry.

If you choose a more appropriate format than straight DDL, it's fairly
easy to do this sort of thing entirely mechanically.

Three links I have handy are ...

http://xml2ddl.berlios.de/
http://dbmstools.sourceforge.net/
http://www.liquibase.org/

... but there are a bunch of other similar tools too.

They'll all pull from existing databases too, so they can also be
used to diff existing schemas with some slight limitations.

Another approach is to store all your schema versions as
upgrade (and downgrade) scripts, rather than as whole DDL
scripts. Everything else you need can be derived from those
mechanically. It also lets you be a lot smarter about how to
handle existing data when changing a schema.

Cheers,
   Steve

John wrote on 28.09.2009 18:24:

> Thanks that will help.  After all this time I'm surprized that someone hasn't
> provide an easy way to get this done.  It's has to be every developers
> problem.

Have a look at my SQL Workbench. It has a built-in command to generate a diff between two databases. The output is an
XMLfile that can easily be transformed into the approriate SQL scripts (a sample XSLT for Postgres is available) 

http://www.sql-workbench.net

Description of the WbSchemaDiff command:
http://www.sql-workbench.net/manual/wb-commands.html#command-schema-diff

Regards
Thomas

On Monday 28 September 2009 09:31:30 am Adrian Klaver wrote:
> ----- "John" <jfabiani@yolo.com> wrote:
> > On Monday 28 September 2009 09:06:25 am Filip Rembiałkowski wrote:
> > > 2009/9/28 John <jfabiani@yolo.com>
> > >
> > > > Hi,
> > > > I have a development DB and a production DB.  I need a way to sync
> >
> > the
> >
> > > > changes
> > > > I make to the stucture in the devel DB to the production DB.  I
> >
> > found
> >
> > > > pgdiff
> > > > but can't get it to work.  I would like a solution that would work
> >
> > on
> >
> > > > windows
> > > > and linux.   But I'll take either alone.
> > > >
> > > > postgres 8.3
> > > > openSUSE 11.0
> > > > windows XP/vista
> > > >
> > > > Thanks in advance for any help.
> > >
> > > You could use standard text compare programs. They won't  write a
> >
> > patch for
> >
> > > you. There's still some niche for live DBAs.
> > >
> > > pg_dump -sOx dev_db_name > dev.schema
> > > pg_dump -sOx prod_db_name > prod.schema
> > > diff -u dev.schema prod.schema
> >
> > Thanks that will help.  After all this time I'm surprized that someone
> > hasn't
> > provide an easy way to get this done.  It's has to be every developers
> >
> > problem.
> >
> > Johnf
>
> I have looked at but not tried pgmigrate:
> http://code.google.com/p/pgmigrate/
>
> It might do what you want.
>
>
> Adrian Klaver
> aklaver@comcast.net

Thanks Adrian and looks like it's in python too which is a major plus.

Johnf

On Monday 28 September 2009 09:56:33 am Filip Rembiałkowski wrote:
> 2009/9/28 John <jfabiani@yolo.com>
>
> >   After all this time I'm surprized that someone hasn't
> > provide an easy way to get this done.  It's has to be every developers
> > problem.
>
> hmm, maybe because there's no easy way? db schemas can be complicated...
> there are some commercial tools for db comparing but they are not perfect
> too.
>
> BTW, you did not specify what exactly did not work when you tried apgdiff.
> this would help others to help you.

To be honest I could not determine how to start the app.

Johnf

John wrote:
> Hi,
> I have a development DB and a production DB.  I need a way to sync the changes
> I make to the stucture in the devel DB to the production DB.  I found pgdiff
> but can't get it to work.  I would like a solution that would work on windows
> and linux.   But I'll take either alone.
>
> postgres 8.3
> openSUSE 11.0
> windows XP/vista
>

we make our changes via .sql files, and always keep in parallel a 'new
database' .sql file that creates the schema from scratch, and a 'delta'
SQL file which updates from one released version to the next.   these
.sql files live in our source code control, along with the applications.
  We do _not_ diddle the development environment schema interactively,
except for experiments on a scratch copy.

On Mon, 2009-09-28 at 10:08 -0700, John wrote:
> On Monday 28 September 2009 09:56:33 am Filip Rembiałkowski wrote:
> > BTW, you did not specify what exactly did not work when you tried apgdiff.
> > this would help others to help you.
>
> To be honest I could not determine how to start the app.

apgdiff dump1.sql dump2.sql

You may need to flip dump1 and dump2 to get the result you want, but
there isn't much more to it than this.