Re: dynamically allocating chunks from shared memory - Mailing list pgsql-hackers

From Bruce Momjian
Subject Re: dynamically allocating chunks from shared memory
Date
Msg-id 201008091502.o79F27O26068@momjian.us
Whole thread Raw
In response to Re: dynamically allocating chunks from shared memory  (Markus Wanner <markus@bluegap.ch>)
Responses Re: dynamically allocating chunks from shared memory
Re: dynamically allocating chunks from shared memory
List pgsql-hackers
Markus Wanner wrote:
> Hi,
> 
> On 07/26/2010 07:16 PM, Robert Haas wrote:
> > Of course, there are other parts of the system (a whole bunch of them)
> > that used shared memory also, and perhaps some of those could be
> > modified to use the dynamic allocator as well.  But they're getting by
> > without it now, so maybe they don't really need it.  The SLRU stuff, I
> > think, works more or less like shared buffers (so you have the same
> > set of issues) and I think most of the other users are allocating
> > small, fixed-size chunks.
> 
> Yeah, I see your point(s).
> 
> Note however, that a thread based design doesn't have this problem *at 
> all*. Memory generally is shared (between threads) and you can 
> dynamically allocate more or less (until Linux' OOM killer hits you.. 
> yet another story). The OS reuses memory you don't currently need even 
> for other applications.

[ Sorry to be jumping into this thread late.]

I am not sure threads would greatly help us.  The major problem is that
all of our our structures are currently contiguous in memory for quick
access.  I don't see how threading would help with that.  We could use
realloc(), but we can do the same in shared memory if we had a chunk
infrastructure, though concurrent access to that memory would hurt us in
either threads or shared memory.

Fundamentally, recreating the libc memory allocation routines is not
that hard.  (Everyone has to detach from the shared memory segment, but
they have to stop using it too, so it doesn't seem that hard.)

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com
 + It's impossible for everything to be true. +


pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: is syntax columname(tablename) necessary still?
Next
From: Pavel Stehule
Date:
Subject: Re: is syntax columname(tablename) necessary still?