Re: About these IPC parameters - Mailing list pgsql-hackers

On Thu, 20 Jul 2000, Tom Lane wrote:

> Peter Eisentraut <peter_e@gmx.net> writes:
> > We use three shared-memory segments: One is for the spin locks and is of
> > negligible size (144 bytes currently). The other two I don't know, but one
> > of them seems to be sized about 550kB + -B * BLCKSZ
> 
> The shmem sizes depend on both -B and -N, but the dependence on -B is
> much stronger.  Obviously there's 8K per -B for the buffer itself,
> and there's also some allowance for hashtable entries for the buffer
> indexing tables.  The -N number drives the size of the PROC table plus
> some hashtables --- but a PROC entry isn't very big.
> 
> I believe there's no really fundamental reason why we use three shmem
> segments and not just one.  I've toyed with the idea of trying to
> combine them, but not done anything about it yet...
> 
> > My kernel has the following interesting-looking shared memory settings:
> 
> FWIW, HPUX does not have SHMALL --- and since HPUX began life as SysV
> I would imagine a lot of other SysV derivatives don't either.  The
> relevant parameters here seem to be
> 
> SEMA            Enable Sys V Semaphores
> SEMAEM          Max Value for Adjust on Exit Semaphores
> SEMMAP          Max Number of Semaphore Map Entries
> SEMMNI          Number of Semaphore Identifiers
> SEMMNS          Max Number of Semaphores
> SEMMNU          Number of Semaphore Undo Structures
> SEMUME          Semaphore Undo Entries per Process
> SEMVMX          Semaphore Maximum Value
> SHMEM           Enable Sys V Shared Memory
> SHMMAX          Max Shared Mem Segment (bytes)
> SHMMNI          Number of Shared Memory Identifiers
> SHMSEG          Shared Memory Segments per Process
> 
> Other than shooting yourself in the foot by having SEMA or SHMEM be
> 0 (OFF), it looks like the parameters that could need raising on this
> platform would be SEMMAP, SEMMNI, SEMMNS, SHMMAX.
> 
> > Is there any noteworthy relevance of some of the other parameters? I see
> > FAQ_BSDI talks about SEMUME and SEMMNU.
> 
> AFAIK we don't use semaphore undo, so those are red herrings.

First off, this might be something we need a whole seperate FAQ for, since
I think the concepts are pretty much common across the various OSs?

for instance,  under FreeBSD, I have it set right now as:

====
options         SYSVSHM
options         SHMMAXPGS=4096
options         SHMSEG=256

options         SYSVSEM
options         SEMMNI=256
options         SEMMNS=512
options         SEMMNU=256
options         SEMMAP=256

options         SYSVMSG                 #SYSV-style message queues
====

To run three postmasters, one with '-B 256 -N 128', and the other two just
with '-N 16' ... the thing that I just don't get is how the settings ni my
kernel apply, and trying to find any info on taht is like pulling teeth :(
For instance, I'm allowing for up to 160 clients to connect, max .. does
that make for one semaphore identifier per client, so I need SEMMNI >=
160?  Or ... ?

I grab'd this off a Sun site dealing with Solaris, but it might also be of
aid:

 Name    Default Max             Brief Description ------  ------- --------------
-------------------------------------
 semmap  10      2147483647      Number of entries in semaphore map
 semmni  10      65535           Number of semaphore sets (identifiers)
 semmns  60      2147483647      Number of semaphores in the system                 65535 (usage)
 semmnu  30      2147483647      Number of "undo" structures in the system
 semmsl  25      2147483647      Max number of semaphores, per semaphore id                 65535 (usage)
 semopm  10      2147483647      Max number of operations, per semaphore call
 semume  10      2147483647      Max number of "undo" entries, per process
 semusz  96      *see below*     Size in bytes of "undo" structure
 semvmx  32767   2147483647      Semaphore maximum value                 65535 (usage)
 semaem  16384   2147483647      Adjust on exit maximum value                 32767 (usage)
 Detailed Descriptions ---------------------
 semmap
 Defines the size of the semaphore resource map;  each block of available, contiguous semaphores requires one entry in
thismap.  This is the pool from which semget(2) acquires semaphore sets.
 
 When a semaphore set is removed (deleted), if the block of semaphores to be freed is adjacent to a block of semaphores
alreadyin the resource map, the semaphores in the set being removed are added to the existing map entry; no new map
entryis required.  If the semaphores in the removed set are not adjacent to those in an existing map entry, then a new
mapentry is required to track these semaphores;  if there are no more map entries available, the system has to discard
anentry, 'permanently' losing a block of semaphores (permanence is relative;  a reboot fixes the problem).  If this
shouldoccur, a WARNING will be generated, the text of which will be something like "rmallocmap: rmap overflow, lost
...".  The end result is that a user could later get ENOSPC errors from semget(2) even though it doesn't look like all
thesemaphores are allocated.
 
 semmni
 Defines the number of semaphore sets (identifiers), system wide.  Every semaphore set in the system has a unique
indentifierand control structure. The system pre-allocates kernel memory for semmni control structures;  each control
structureis 84 bytes.  If no more identifiers are available, semget(2) returns ENOSPC.
 
 Attempting to set semmni to a value greater than 65535 will result in generation of a WARNING, and the value will be
setto 65535.
 
 semmns
 Defines the number of semaphores in the system;  16 bytes of kernel memory is pre-allocated for each semaphores.  If
thereis not a large enough block of contiguous semaphores in the resource map (see semmap) to satisfy the request,
semget(2)returns ENOSPC.
 
 Fragmentation of the semaphore map will result in ENOSPC errors, even though there may appear to be ample free
semaphores. Despite attempts by the system to merge free sets (see semmap), the size of the clusters of free semaphores
generallydecreases over time.  For this reason, semmns frequently must be set higher than the actual number of
semaphoresrequired.
 
 semmnu
 Defines the number of semaphore undo structures in the system.  semusz (see below) bytes of kernel memory are
pre-allocatedfor each undo structure; one undo structure is required for every process for which undo information must
berecorded.  semop() will return ENOSPC if it is requested to record undo information and there are no undo structures
available.
 semmsl
 Limits the number of semaphores that can be created for a single semaphore id. If semget(2) returns EINVAL, this limit
shouldbe increased.  This parameter is only used to validate the argument passed to semget(2).  Logically, it should be
lessthan or equal to semmns (see above).  Setting semmsl too high might allow a few identifiers to hog all the
semaphoresin the system.
 
 semopm
 Limits the number of operations that are allowed in a single semop(2) call. If semop(2) returns E2BIG, this limit
shouldbe increased.  This parameter is only used to validate the argument passed to semop(2).
 
 semume
 Limits the number of undo records that can exist for a process.  If semop(2) returns EINVAL, this limit should be
increased. In addition to its use in validating arguments to semop(2), this parameter is used to calculate the value of
semusz(see below).
 
 semusz
 Defines the size of the semaphore undo structure.  Any attempt to modify this parameter directly will be ignored;
semuszis always calculated based upon the value of semume (see above);  semusz = 8 * (semume + 2).
 
 semvmx
 Limits the maximum value of a semaphore.  Due to the interaction with undo structures and semaem (see below), this
tuneableshould not be increased beyond its default value of 32767, unless you can guarantee that SEM_UNDO is never and
willnever be used.  It can be safely reduced, but doing so provides no savings.
 
 semaem
 Limits the maximum value of an adjust-on-exit undo element.  No system resources are allocated based on this value.