Re: autovacuum process handling - Mailing list pgsql-hackers

Hi,

Alvaro Herrera wrote:
> 1. There will be two kinds of processes, "autovacuum launcher" and
> "autovacuum worker".

Sounds similar to what I do in Postgres-R: one replication manager and 
several "replication workers". Those are called "remote backends" (which 
is somewhat of an unfortunate name, IMO.)

> 6. Launcher will start a worker using the following protocol:
>    - Set up information on what to run on shared memory
>    - invoke SendPostmasterSignal(PMSIGNAL_START_AUTOVAC_WORKER)
>    - Postmaster will react by starting a worker, and registering it very
>      similarly to a regular backend, so it can be shut down easily when
>      appropriate.
>      (Thus launcher will not be informed right away when worker dies)
>    - Worker will examine shared memory to know what to do, clear the
>      request, and send a signal to Launcher
>    - Launcher wakes up and can start another one if appropriate

It looks like you need much less communication between the launcher and 
the workers, probably also less between the postmaster and the launcher.

For Postgres-R, I'm currently questioning if I shouldn't merge the 
replication manager process with the postmaster. Of course, that would 
violate the "postmaster does not touch shared memory" constraint. But it 
would make some things a lot easier:
 * What if the launcher/manager dies (but you potentially still have   active workers)?
   Maybe, for autovacuum you can simply restart the launcher and that   one detects workers from shmem.
   With replication, I certainly have to take down the postmaster as   well, as we are certainly out of sync and can't
simplyrestart the   replication manager. So in that case, no postmaster can run without a   replication manager and
viceversa. Why not make it one single   process, then?
 
 * Startup races: depending on how you start workers, the launcher/   manager may get a "database is starting up" error
whenrequesting   the postmaster to fork backends.   That probably also applies to autovacuum, as those workers
shouldn't  work concurrently to a startup process. But maybe there are other   means of ensuring that no autovacuum
getstriggered during startup?
 
 * Simpler debugging: one process less which could fail, and a whole lot   of concurrency issues (like deadlocks or
invalidIPC messages) are   gone.
 

So, why do you want to add a special launcher process? Why can't the 
postmaster take care of launching autovacuum workers? It should be 
possible to let the postmaster handle *that* part of the shared memory, 
as it can simply clean it up. Corruptions wouldn't matter, so I don't 
see a problem with that.

(Probably I'm too much focussed on my case, the replication manager.)

> Does this raise some red flags?  It seems straightforward enough to me;
> I'll submit a patch implementing this, 

Looking forward to that one.

Regards

Markus