Thread: Child program using parent program's transaction?

I work with a system designed as lots of little cooperating worker
programs, with boss programs that... well, boss the worker programs
around.

Boss and workers all use the same database.

Sometimes it would be convenient to have a boss start a transaction
and then have the workers do their work in the context of that
transaction.

Each time, I've decided that since I don't know how to do that, that I
never really wanted to do that.  You know how it goes.

Today, it would once again be convenient to have an exec'd program do
its work in the context of its parent program's transaction.  So,
before I once again decide that I don't actually want to do that, can
you tell me... is it possible?  And, would any sane person do it?

We're using Linux.  All of the programs are written in Ruby 1.8, using
the venerable (or is it just ancient?) pgsql library.

I would say that in certain contexts it would definitely be
sane/preferable to have a "worker" process work in its boss's transaction.
In the traditional bank example:
Open a transfer transaction:
One application goes to take money out of one account.
The second application goes to put money into the other account.
A third application might do other things.

This way you can do parallel processing with separate physical
applications, and still keep the integrity of the transaction. The
transaction would then be committed when all of the child applications
came back and reported success and would be rolled back if one reported
a failure.

How to do it is environment specific and I couldn't tell you how to pass
a transaction or connection as a parameter in ruby.

Sim

Wayne Conrad wrote:
> I work with a system designed as lots of little cooperating worker
> programs, with boss programs that... well, boss the worker programs
> around.
>
> Boss and workers all use the same database.
>
> Sometimes it would be convenient to have a boss start a transaction
> and then have the workers do their work in the context of that
> transaction.
>
> Each time, I've decided that since I don't know how to do that, that I
> never really wanted to do that.  You know how it goes.
>
> Today, it would once again be convenient to have an exec'd program do
> its work in the context of its parent program's transaction.  So,
> before I once again decide that I don't actually want to do that, can
> you tell me... is it possible?  And, would any sane person do it?
>
> We're using Linux.  All of the programs are written in Ruby 1.8, using
> the venerable (or is it just ancient?) pgsql library.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq
>

On Wed, Sep 06, 2006 at 03:21:04PM -0700, Wayne Conrad wrote:
> I work with a system designed as lots of little cooperating worker
> programs, with boss programs that... well, boss the worker programs
> around.
>
> Boss and workers all use the same database.
<snip>
> Today, it would once again be convenient to have an exec'd program do
> its work in the context of its parent program's transaction.  So,
> before I once again decide that I don't actually want to do that, can
> you tell me... is it possible?  And, would any sane person do it?

The answer is, not really. If you're only fork()ing you can get away
with it as long as you make sure no two processes are accessing the db
at the same time. There's a little too much state to transfer a
connection from one process to another.

What you could do is have a single process that handles the actual
connection to the database and have the other processes talk to it to
do the queries. That way you have complete control over who accesses
what in which connection.

Hope this helps,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Wed, Sep 06, 2006 at 03:21:04PM -0700, Wayne Conrad wrote:
> Today, it would once again be convenient to have an exec'd program do
> its work in the context of its parent program's transaction.  So,
> before I once again decide that I don't actually want to do that, can
> you tell me... is it possible?  And, would any sane person do it?

This isn't exactly the same thing, but I've seen something similar
done with threads.  What I have to say about it is that it appears to
be easy to do when you decide to do it, but it turns out to be hard
to do well in practice.  I cannot count the number of bugs I've come
across because of poor handling of this sort of situation.  In
particular, in my experience, what you start to realise is that the
handoff period is super ultra critical, and subject to all sorts of
nasty race conditions; so you start locking things up in an effort to
avoid the race.  In no time at all, the whole application has ground
to a halt while everything goes through this serialised global
choke-point.  It is at this point that you decide there's a reason
the system doesn't do this sort of thing out of the box ;-)

A

--
Andrew Sullivan  | ajs@crankycanuck.ca
Everything that happens in the world happens at some place.
        --Jane Jacobs