Re: Threaded PosgreSQL server - Mailing list pgsql-hackers

"D. Hageman" wrote:
> 
> On Thu, 7 Feb 2002, mlw wrote:
> >
> > Going from a "process model" to a "threaded model" is a HUGE
> > undertaking. In the process model, all data is assumed to be private,
> > and shared data must be explicitly shared.  In a threaded model all data
> > is implicitly shared and private data must be explicitly made private.
> > Do not under estimate what this means or how hard it is to convert one
> > to the other.
> >
> 
> I agree with the first and last sentance ... the rest of the paragraph is
> ... well we argued this before - look in the archives.

yes, I know.
> 
> > Also:
> >
> > Think of file handles. In a threaded version of postgreSQL, all
> > connections will be competing for file handles. I think the limit in
> > Linux is 1024.
> 
> Depends on how it is done.

How does it depend? If you have one process with multiple threads, you
will bump up against the process limit of file handles.

> 
> > All threads will be competing for memory mapping. As systems get more
> > and more RAM, on the x86 and other 32 bit machines, process space is
> > limited to 2 to 3 gig. If you have 8 gig in your system, PostgreSQL
> > won't be able to use it.
> 
> Depends on how it is done.

Again, How does it depend? If you have one process, there is a limit to
the amount of memory it can access. 3gig (2gig on older Windows) of
process space it is a classic limitation to x86 operating systems.

> 
> > I think threading connections is a VERY bad idea. I am dubious that the
> > amount of work will result in a decent return on investment.
> 
> Depends on how it is done.  We should be careful to assume that threading
> postgresql instantly equates to threading connections.  That is only *ONE*
> possible type of threading architecture one could choose.  Making broad
> generalized statements doesn't accomplish anything in this debate ...
> instead be more focused with your comments so one can make heads or tails
> out of them.

There are, AFAIK two reasons to thread PostgreSQL:

(1) Run the multiple connections in their own thread with the assumption
that this is more efficient for [n] reasons.
(2) Run a single query across multiple threads, thus parallelizing the
query engine.

There is a mutant of this as well: (1a)  You could have multiple
processes each with [n] connection threads.

As far as PostgreSQL is concerned, I am dubious that (1) or (1a) will
provide any real benefit for the amount of work required to accomplish
it. Work on "pre-forking" would be FAR more productive.

The idea of parallelizing queries could be very worth while. However,
that being said, creating a set of I/O threads that get blocks from disk
devices asynchronously, my be enough with a very limited amount of work.

I guess all I am saying, is that a person's time is really the only
limited resource. Tom, Bruce, Marc, Peter and everyone else have a
limited amount of time. If I could influence how those guys spend their
time, I would hope they spent time working on improving the
functionality of PostgreSQL, not the tedium of making it thread safe.






> 
> --
> //========================================================\\
> ||  D. Hageman                    <dhageman@dracken.com>  ||
> \\========================================================//
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)