Thread: Many postmasters...
Hi everyone, Using Linux RH7.0 with correct gcc and glibc, PG7.03, Apache 1.3.14 and PHP4. We have several unresolved questions : * Is it normal that ps aux |grep postgres shows (what we want : processes own by postgres) multiple postgres backends (which seems normal to me) *AND* multiple postmaster (same full cmd line). Sometimes we also have "defunct" postgresses. * we start postgres with a /etc/rc.d/init.d script that launches pg_ctl -w <many options here> start When invoked from the shell, this command never returns to the shell by itself, we have to press <enter>. This behaviour prevents the script for terminating properly. Is there a way around this ? Not tried echo | pg_ctl .... yet * every backend created by an Apache session opens many files (in our case, about 80 including the indexes) and many backends will finally generate an "Too many files open" message. We first increased the /proc/sys/fs/file-max to 8192 but that's a lot ! The apache/php server always uses the same connect parameters for every page but it seems php's pg_pconnect() behaves just like pg_connect. Shouldn't we have apache hold a few backends connected ? Thanks for you attention and help. -- Jean-Christophe Boggio cat@thefreecat.org Independant Consultant and Developer Delphi, Linux, Oracle, Perl
On Wed, Dec 06, 2000 at 02:06:01AM +0100, some SMTP stream spewed forth: > Hi everyone, Hi > > Using Linux RH7.0 with correct gcc and glibc, PG7.03, Apache 1.3.14 > and PHP4. We have several unresolved questions : Ah, but do not we all? ;-) > > * Is it normal that Er, it seems so...not sure about the "defunct" business. > ps aux |grep postgres > shows (what we want : processes own by postgres) multiple postgres > backends (which seems normal to me) *AND* multiple postmaster (same > full cmd line). > Sometimes we also have "defunct" postgresses. > > * we start postgres with a /etc/rc.d/init.d script that launches > pg_ctl -w <many options here> start > When invoked from the shell, this command never returns to the shell > by itself, we have to press <enter>. This behaviour prevents the > script for terminating properly. Is there a way around this ? > Not tried echo | pg_ctl .... yet I would need to see the code, but perhaps someone else would not. (?) > > * every backend created by an Apache session opens many files (in our > case, about 80 including the indexes) and many backends will finally > generate an "Too many files open" message. We first increased the > /proc/sys/fs/file-max to 8192 but that's a lot ! > > The apache/php server always uses the same connect parameters for > every page but it seems php's pg_pconnect() behaves just like > pg_connect. Shouldn't we have apache hold a few backends connected ? pg_pconnect creates a persistent connection *for that Apache process*. Id est: 10 backends, 10 persistent connections to 10 Postgres backends. Yep, I hate the situation too, but tcl under AOLServer or Java are not options to me, perhaps they are to you. (The overhead for non-persistent connections is not *that* bad, but it is (to me) noticeable.) Connection pooling would (somewhat) "fix" the situation, but...blah. Unless you are in a situation where you can set MaxClients equal to the max(backends) then I would just use regular connections (as I have been "forced" to do in my current situation). Apache does not (and can not?) do connection pooling in itself, I believe there may be modules that do so, but I do not use them. > > > Thanks for you attention and help. Hey, that's why we are here, eh? I hope that I have been (at least somewhat) helpful. "'Aaaaaaarggggghhhhhh!' went Ford Prefect...", The Hitchhiker's Guide to the Galaxy, Douglas Adams gh > > -- > Jean-Christophe Boggio > cat@thefreecat.org > Independant Consultant and Developer > Delphi, Linux, Oracle, Perl > >
Jean-Christophe Boggio <cat@thefreecat.org> writes:
> * Is it normal that
> ps aux |grep postgres
> shows (what we want : processes own by postgres) multiple postgres
> backends (which seems normal to me) *AND* multiple postmaster (same
> full cmd line).
> Sometimes we also have "defunct" postgresses.
Multiple postmasters sounds fishy. But you haven't looked closely
enough to tell for sure. I don't trust the ps-status stuff a whole lot,
so I don't trust ps' report of whether a process calls itself a
postmaster or a postgres. What I do trust is ps' report of process
parent/child relationships. A true postmaster is always a child of init
(process 1); a true backend is always a child of some true postmaster.
If you see more children of process 1 than you thought you should have,
then that needs looking into.
A <defunct> postmaster or postgres is an indication of trouble, also,
since either init or the parent postmaster (respectively) should have
reaped the dead process immediately. If a zombie process survives more
than a few milliseconds then something is wrong with its parent. Again,
the parent-process link shown by ps is critical information.
> * we start postgres with a /etc/rc.d/init.d script that launches
> pg_ctl -w <many options here> start
> When invoked from the shell, this command never returns to the shell
> by itself, we have to press <enter>. This behaviour prevents the
> script for terminating properly. Is there a way around this ?
That seems broken. I think there was an old version of pg_ctl that
didn't work properly unless -S was one of the switches passed to the
postmaster --- is that what's biting you?
> * every backend created by an Apache session opens many files (in our
> case, about 80 including the indexes) and many backends will finally
> generate an "Too many files open" message. We first increased the
> /proc/sys/fs/file-max to 8192 but that's a lot !
No it's not. Increase file-max or decrease your max number of backends.
regards, tom lane
Jean-Christophe Boggio wrote: > Using Linux RH7.0 with correct gcc and glibc, PG7.03, Apache 1.3.14 > and PHP4. We have several unresolved questions : > * Is it normal that > ps aux |grep postgres > shows (what we want : processes own by postgres) multiple postgres > backends (which seems normal to me) *AND* multiple postmaster (same > full cmd line). > Sometimes we also have "defunct" postgresses. Yes, this would be normal. Due to the fork nature of the backend, you will see with ps, depending upon traffic, the actual postmaster fork before the backend (postgres) is exec'd. I don't see that here due to my use of a pooling webserver, but non-pooled situations will have backends bouncing up and down constantly. The defunct postgres processes are the ones that are going away, but haven't yet been removed from the process table, IIRC. > * we start postgres with a /etc/rc.d/init.d script that launches > pg_ctl -w <many options here> start > When invoked from the shell, this command never returns to the shell > by itself, we have to press <enter>. This behaviour prevents the > script for terminating properly. Is there a way around this ? > Not tried echo | pg_ctl .... yet The init.d script has an & after the pg_ctl line. If it didn't return, your system would never finish booting, due to the sequential nature of the RedHat 7 SysV init setup. Now, pg_ctl is kept running; it just doesn't block the initscript. > * every backend created by an Apache session opens many files (in our > case, about 80 including the indexes) and many backends will finally > generate an "Too many files open" message. We first increased the > /proc/sys/fs/file-max to 8192 but that's a lot ! > The apache/php server always uses the same connect parameters for > every page but it seems php's pg_pconnect() behaves just like > pg_connect. Shouldn't we have apache hold a few backends connected ? Thanks to the non-pooled connection scheme of Apache/PHP, the way the persistent pconnect mechanism works is non-obvious. Each apache _process_ can hold a configured number of connections open -- but that is then multiplied by the number of apache _processes_. So, to run persistent connections in a usable manner on Apache/PHP requires a huge number of backends, requiring an even larger number of open files. File-max at 8192 is probably middle of the road for such a system. Too bad PHP can't use AOLserver's pooled connections -- that would be a big win. PHP can run on AOLserver -- it just doesn't yet use the pooled API. Apache 2.0's multithreaded nature will help this -- unless a mechanism can be devised to share database connections amongst multiple full processes for older Apache's. Multithreading is a big win for clients that generate multiple connections -- it's not a big win for backends that serve multiple connections. IMHO. -- Lamar Owen WGCR Internet Radio 1 Peter 4:11
> Yes, this would be normal. Due to the fork nature of the backend, you > will see with ps, depending upon traffic, the actual postmaster fork > before the backend (postgres) is exec'd. I don't see that here due to my > use of a pooling webserver, but non-pooled situations will have backends We don't do any exec since 6.4 I think, just fork(). I think the reason is shows postmaster for backends is because the part of the process containing the ps args is paged out, and ps will not page it in to get the args, it will simply print the name of the binary that initially started the process. Not a problem. > > case, about 80 including the indexes) and many backends will finally > > generate an "Too many files open" message. We first increased the > > /proc/sys/fs/file-max to 8192 but that's a lot ! > > > The apache/php server always uses the same connect parameters for > > every page but it seems php's pg_pconnect() behaves just like > > pg_connect. Shouldn't we have apache hold a few backends connected ? > PostgreSQL keeps most files open in a LIFO manner to prevent the overhead of re-opening files it just closed. I think it keeps the last 64 file open, but you can configure that lower if needed. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000 + If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania 19026