Thread: do we EXEC_BACKEND on Mac OS X?
See the CAVEATS here: https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/fork.2.html Apparently fork() without exec() isn't all that well supported. Noticed while perusing http://lwn.net/Articles/518306/ -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Wed, Oct 3, 2012 at 01:05:54PM -0300, Alvaro Herrera wrote: > See the CAVEATS here: > > https://developer.apple.com/library/mac/#documentation/Darwin/Referenc > e/ManPages/man2/fork.2.html > > Apparently fork() without exec() isn't all that well supported. > > Noticed while perusing http://lwn.net/Articles/518306/ I think this comment is more relevant: Ah, OK, I found this https://developer.apple.com/library/mac/#documentation/Da... It seems that from 10.5 this caveat was added to the official OS X documentation. In that light I think it'ssafest to conclude that Apple realised fork() is hard (check out the long list of things a current Linuxfork does to retain sanity in the face of threads, asynchronous I/O, capabilities and other fun toys thatdidn't exist at the dawn of Unix) and decided they don't care. It will probably work, but if it doesn't they aren't interested in explaining why/ fixing the problem. On balance I agree this makes OS X a pretty shoddy Unix, but then, I would have been easily persuaded of thatanyway. I am hesitant to avoid fork() on OS/X until someone reports a problem; the slowdown would be significant, and I don't think we use enough OS/X libraries to cause a problem for us, though Bonjour might be a problem. Anyway, good you asked and we should be aware of possible problems. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
Alvaro Herrera <alvherre@2ndquadrant.com> writes: > See the CAVEATS here: > https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man2/fork.2.html > Apparently fork() without exec() isn't all that well supported. This doesn't say fork() doesn't work. It says that Apple's framework libraries aren't meant to work in a forked subprocess --- but we don't use those. > Noticed while perusing > http://lwn.net/Articles/518306/ I'm afraid Brian was just looking for an excuse to dump on Apple. We have a lot of years of Postgres experience showing that fork() works fine on OS X. regards, tom lane
I wrote: > Alvaro Herrera <alvherre@2ndquadrant.com> writes: >> Noticed while perusing >> http://lwn.net/Articles/518306/ > I'm afraid Brian was just looking for an excuse to dump on Apple. We > have a lot of years of Postgres experience showing that fork() works > fine on OS X. BTW, I think the commenter at the bottom of the thread puts his finger on the core of the real problem: > I'd wager most libraries are not fork safe, including such libraries > as SQLite as mentioned in the SQLite FAQ. Libraries that talk to the > outside world contain much state that is not safe to share. To bring that closer to home, suppose you have a program with an open database connection in libpq, and you fork(), and then parent and child both try to use the connection. How well would that work? Is it the fault of fork()? I think Apple is just pointing out that their framework libraries have similar issues. regards, tom lane
On Wed, Oct 3, 2012 at 12:41:37PM -0400, Tom Lane wrote: > I wrote: > > Alvaro Herrera <alvherre@2ndquadrant.com> writes: > >> Noticed while perusing > >> http://lwn.net/Articles/518306/ > > > I'm afraid Brian was just looking for an excuse to dump on Apple. We > > have a lot of years of Postgres experience showing that fork() works > > fine on OS X. > > BTW, I think the commenter at the bottom of the thread puts his finger > on the core of the real problem: > > > I'd wager most libraries are not fork safe, including such libraries > > as SQLite as mentioned in the SQLite FAQ. Libraries that talk to the > > outside world contain much state that is not safe to share. > > To bring that closer to home, suppose you have a program with an open > database connection in libpq, and you fork(), and then parent and child > both try to use the connection. How well would that work? Is it the > fault of fork()? > > I think Apple is just pointing out that their framework libraries have > similar issues. Yes, but those framework libraries are typically supposed to prevent such problems from being seen by applications calling them. This is certainly sloppy practice on Apple's part, and it leave us wondering if we are using anything that might be a problem. The bottom line is that we don't know. Libraries are supposed to document these limitations, as we do with libpq. I wonder if they just documented fork() and now don't feel they need to document these limitations per-library. Anyway, I agree that we need to see a failure before adjusting anything. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
Bruce Momjian <bruce@momjian.us> writes:
> Yes, but those framework libraries are typically supposed to prevent
> such problems from being seen by applications calling them.
How exactly would a library prevent such problems? In particular,
let's see a proposal for how libpq might make it look like a fork
was transparent for an open connection.
> This is
> certainly sloppy practice on Apple's part, and it leave us wondering if
> we are using anything that might be a problem. The bottom line is that
> we don't know.
> Libraries are supposed to document these limitations, as we do with
> libpq. I wonder if they just documented fork() and now don't feel they
> need to document these limitations per-library.
Do we know that they *didn't* document such issues per-library?
Mentioning the risk under fork() too doesn't seem unreasonable.
Not trying to sound like an Apple apologist, but I see a whole lot of
bashing going on here on the basis of darn little evidence.
regards, tom lane
On Wed, Oct 3, 2012 at 01:53:28PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Yes, but those framework libraries are typically supposed to prevent > > such problems from being seen by applications calling them. > > How exactly would a library prevent such problems? In particular, > let's see a proposal for how libpq might make it look like a fork > was transparent for an open connection. I guess that is impossible. > > This is > > certainly sloppy practice on Apple's part, and it leave us wondering if > > we are using anything that might be a problem. The bottom line is that > > we don't know. > > > Libraries are supposed to document these limitations, as we do with > > libpq. I wonder if they just documented fork() and now don't feel they > > need to document these limitations per-library. > > Do we know that they *didn't* document such issues per-library? > Mentioning the risk under fork() too doesn't seem unreasonable. > > Not trying to sound like an Apple apologist, but I see a whole lot of > bashing going on here on the basis of darn little evidence. Well, ideally if Apple is going to brand a Unix function as unsafe, it would be good to mention which libraries are unsafe. I have no idea if they are documenting the problems in the libraries themselves. I guess my point is that the fork() warning was too vague. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Wed, Oct 03, 2012 at 01:57:47PM -0400, Bruce Momjian wrote: > On Wed, Oct 3, 2012 at 01:53:28PM -0400, Tom Lane wrote: > > Bruce Momjian <bruce@momjian.us> writes: > > > Yes, but those framework libraries are typically supposed to prevent > > > such problems from being seen by applications calling them. > > > > How exactly would a library prevent such problems? In particular, > > let's see a proposal for how libpq might make it look like a fork > > was transparent for an open connection. > > I guess that is impossible. Well, not quite impossible. A simple solution would be to use pthread_atfork() to register a handler that puts the socket into an invalid state in either the parent or the child. http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_atfork.html Ofcourse, the backward compatabilty issues prevent us doing that, but it's *possible*. Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > He who writes carelessly confesses thereby at the very outset that he does > not attach much importance to his own thoughts. -- Arthur Schopenhauer
Martijn van Oosterhout <kleptog@svana.org> writes:
> On Wed, Oct 03, 2012 at 01:57:47PM -0400, Bruce Momjian wrote:
>> On Wed, Oct 3, 2012 at 01:53:28PM -0400, Tom Lane wrote:
>>> How exactly would a library prevent such problems? In particular,
>>> let's see a proposal for how libpq might make it look like a fork
>>> was transparent for an open connection.
>> I guess that is impossible.
> Well, not quite impossible. A simple solution would be to use
> pthread_atfork() to register a handler that puts the socket into an
> invalid state in either the parent or the child.
That doesn't make it "transparent" --- that merely ensures that we break
one of the two currently-working use cases (namely, that either the
parent or the child can continue to use the connection as long as the
other doesn't touch it).
regards, tom lane