Re: autovacuum: multiple workers - Mailing list pgsql-patches

From Alvaro Herrera
Subject Re: autovacuum: multiple workers
Date
Msg-id 20070328133947.GA21845@alvh.no-ip.org
Whole thread Raw
In response to Re: autovacuum: multiple workers  ("Simon Riggs" <simon@2ndquadrant.com>)
Responses Re: autovacuum: multiple workers
List pgsql-patches
Simon Riggs wrote:
> On Tue, 2007-03-27 at 17:41 -0400, Alvaro Herrera wrote:
>
> > The main change is to have an array of Worker structs in shared memory;
> > each worker checks the current table of all other Workers, and skips a
> > table that's being vacuumed by any of them.  It also rechecks the table
> > before vacuuming, which removes the problem of redundant vacuuming.
>
> Slightly OT: Personally, I'd like it if we added an array for all
> special backends, with configurable behaviour. That way it would be
> easier to have multiple copies of other backends of any flavour using
> the same code, as well as adding others without cutting and pasting each
> time. That part of the postmaster code has oozed sideways in the past
> few years and seems in need of some love. (A former sinner repents).

I'm not really thrilled about it, each case being so different from the
others.  For the autovac workers, for example, the array in shared
memory is kept on the autovac launcher, _not_ in the postmaster.  In the
postmaster, they are kept in the regular BackendList array, so they
don't fit on that array you describe.  And as far as the other processes
are concerned, every one of them is a special case, and we don't add new
ones frequently.  In fact, the autovac work is the only thing that has
added new processes in a long time, since the Windows port was
introduced (which required the logger process) and the bgwriter.

How would you make it "configurable"?  Have a struct containing function
pointers, each function being called when some event takes place?

What other auxiliary processes are you envisioning, anyway?

In any case I don't think this is something that would be good to attack
this late in the devel cycle -- we could discuss it for 8.4 though.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

pgsql-patches by date:

Previous
From: Gregory Stark
Date:
Subject: Re: LIMIT/SORT optimization
Next
From: Tom Lane
Date:
Subject: Re: [PATCH] add CLUSTER table ORDER BY index