AIX FAQ - Mailing list pgsql-patches

Some of this may have already made it in; it's not totally clear...

At any rate, here's a revision to CVS HEAD to reflect some changes by
myself and by Seneca Cunningham for the AIX FAQ.  It touches on the
following issues:

1.  memcpy pointer patch for dynahash.c

2.  AIX memory management, which can, for 32 bit cases, bite people
    quite unexpectedly...

Index: FAQ_AIX
===================================================================
RCS file: /projects/cvsroot/pgsql/doc/FAQ_AIX,v
retrieving revision 1.16
diff -c -r1.16 FAQ_AIX
*** FAQ_AIX    5 Apr 2006 22:55:05 -0000    1.16
--- FAQ_AIX    6 Apr 2006 20:07:28 -0000
***************
*** 113,118 ****
--- 113,180 ----
  http://www.faqs.org/faqs/aix-faq/part4/section-22.html

  http://www.han.de/~jum/aix/ldd.c
+
+ ---
+ From: Christopher Browne <cbbrowne@ca.afilias.info>
+ Date: 2005-11-02
+
+ On AIX 5.3 ML3 (e.g. maintenance level 5300-03), there is some problem
+ with the handling of the pointer to memcpy.  It is speculated that
+ this relates to some linker bug that may have been introduced between
+ 5300-02 and 5300-03, but we have so far been unable to track down the
+ cause.
+
+ At any rate, the following patch, which "unwraps" the function
+ reference, has been observed to allow PG 8.1 pre-releases to pass
+ regression tests.
+
+ The same behaviour (albeit with varying underlying functions to
+ "blame") has been observed when compiling with either GCC 4.0 or IBM
+ XLC.
+
+ ------------ per Seneca Cunningham -------------------
+
+ The following patch works on the AIX 5.3 ML3 box here and didn't cause
+ any problems with postgres on the x86 desktop.  It's just a cleaner
+ version of what I tried earlier.
+
+ *** dynahash.c.orig Tue Nov  1 19:41:42 2005
+ --- dynahash.c  Tue Nov  1 20:30:33 2005
+ ***************
+ *** 670,676 ****
+
+
+             /* copy key into record */
+             currBucket->hashvalue = hashvalue;
+ !           hashp->keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
+
+
+             /* caller is expected to fill the data field on return */
+
+
+ --- 670,687 ----
+
+
+             /* copy key into record */
+             currBucket->hashvalue = hashvalue;
+ !           if (hashp->keycopy == memcpy)
+ !           {
+ !               memcpy(ELEMENTKEY(currBucket), keyPtr, keysize);
+ !           }
+ !           else if (hashp->keycopy == strncpy)
+ !           {
+ !               strncpy(ELEMENTKEY(currBucket), keyPtr, keysize);
+ !           }
+ !           else
+ !           {
+ !               hashp->keycopy(ELEMENTKEY(currBucket), keyPtr, keysize);
+ !           }
+
+
+             /* caller is expected to fill the data field on return */
+
+ ------------ per Seneca Cunningham -------------------
+
  ---

  AIX, readline, and postgres 8.1.x:
***************
*** 185,187 ****
--- 247,367 ----
    IBM Redbook
    http://www.redbooks.ibm.com/redbooks/pdfs/sg245674.pdf
    http://www.redbooks.ibm.com/abstracts/sg245674.html?Open
+
+ -----
+
+ AIX Memory Management: An Overview
+ ==================================
+
+ by Seneca Cunningham...
+
+ AIX can be somewhat peculiar with regards to the way it does memory
+ management.  You can have a server with many multiples of gigabytes of
+ RAM free, but still get out of memory or address space errors when
+ running applications.
+
+ Two examples of AIX-specific memory problems
+ --------------------------------------------
+ Both examples were from systems with gigabytes of free RAM.
+
+ a) createlang failing with unusual errors
+     Running as the owner of the postgres install:
+         -bash-3.00$ createlang plpgsql template1
+         createlang: language installation failed: ERROR:  could not load library
+         "/opt/dbs/pgsql748/lib/plpgsql.so": A memory address is not in the
+         address space for the process.
+
+     Running as a non-owner in the group posessing the postgres install:
+         -bash-3.00$ createlang plpgsql template1
+         createlang: language installation failed: ERROR:  could not load library
+         "/opt/dbs/pgsql748/lib/plpgsql.so": Bad address
+
+ b) out of memory errors in the postgres logs
+     Every memory allocation near or greater than 256MB failing.
+
+
+ The cause of these problems
+ ----------------------------
+
+ The overall cause of all these problems is the default bittedness and
+ memory model used by the postmaster process.
+
+ By default, all binaries built on AIX are 32-bit.  This does not
+ depend upon hardware type or kernel in use.  These 32-bit processes
+ are limited to 4GB of memory laid out in 256MB segments using one of a
+ few models.  The default allows for less than 256MB in the heap as it
+ shares a single segment with the stack.
+
+ In the case of example a), above, check your umask and the permissions
+ of the binaries in your postgres install.  The binaries involved in
+ that example were 32-bit and installed as mode 750 instead of 755.
+ Due to the permissions being set in this fashion, only the owner or a
+ member of the possessing group can load the library.  Since it isn't
+ world-readable, the loader places the object into the process' heap
+ instead of the shared library segments where it would otherwise be
+ placed.
+
+ Solutions and workarounds
+ -------------------------
+ In this section, all build flag syntax is presented for gcc.
+
+ The "ideal" solution for this is to use a 64-bit build of postgres,
+ but that's not always practical.  Systems with 32-bit processors can
+ build, but not run, 64-bit binaries.
+
+ If a 32-bit binary is desired, set LDR_CNTRL to "MAXDATA=0xn0000000",
+ where 1 <= n <= 8, before starting the postmaster and try different
+ values and postgresql.conf settings to find a configuration that works
+ satisfactorily.  This use of LDR_CNTRL tells AIX that you want the
+ postmaster to have $MAXDATA bytes set aside for the heap, allocated in
+ 256MB segments.
+
+ When you find a workable configuration, ldedit can be used to modify
+ the binaries so that they default to using the desired heap size.
+
+ PostgreSQL might also be rebuilt, passing configure
+ LDFLAGS="-Wl,-bmaxdata:0xn0000000" to achieve the same effect.
+
+ For a 64-bit build, set OBJECT_MODE to 64 and pass CC="gcc -maix64"
+ and LDFLAGS="-Wl,-bbigtoc" to configure.  If you omit the export of
+ OBJECT_MODE, your build may fail with linker errors.  When OBJECT_MODE
+ is set, it tells AIX's build utilities such as ar, as, and ld what
+ type of objects to default to handling.
+
+ Overcommit
+ ----------
+
+ By default, overcommit of paging space can happen.  While I have not
+ seen this occur, AIX will kill processes when it runs out of memory
+ and the overcommit is accessed.  The closest to this that I have seen
+ is fork failing because the system decided that there was not enough
+ memory for another process.  Like many other parts of AIX, the paging
+ space allocation method and out-of-memory kill is configurable on a
+ system- or process-wide basis if this becomes a problem.
+
+ References and resources
+ ------------------------
+ "Large Program Support"
+   AIX Documentation: General Programming Concepts: Writing and Debugging Programs
+   http://publib.boulder.ibm.com/infocenter/pseries/topic/com.ibm.aix.doc/aixprggd/genprogc/lrg_prg_support.htm
+
+ "Program Address Space Overview"
+   AIX Documentation: General Programming Concepts: Writing and Debugging Programs
+   http://publib.boulder.ibm.com/infocenter/pseries/topic/com.ibm.aix.doc/aixprggd/genprogc/address_space.htm
+
+ "Performance Overview of the Virtual Memory Manager (VMM)"
+   AIX Documentation: Performance Management Guide
+   http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.doc/aixbman/prftungd/resmgmt2.htm
+
+ "Page Space Allocation"
+   AIX Documentation: Performance Management Guide
+   http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.doc/aixbman/prftungd/memperf7.htm
+
+ "Paging-space thresholds tuning"
+   AIX Documentation: Performance Management Guide
+   http://publib.boulder.ibm.com/infocenter/pseries/v5r3/topic/com.ibm.aix.doc/aixbman/prftungd/memperf6.htm
+
+ "Developing and Porting C and C++ Applications on AIX"
+   IBM Redbook
+   http://www.redbooks.ibm.com/redbooks/pdfs/sg245674.pdf
+   http://www.redbooks.ibm.com/abstracts/sg245674.html?Open

--
(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/linuxxian.html
"What's wrong with 3rd party tools? Especially if they are free?  What
the **** do you think Unix is  anyway? It's a big honkin' party of 3rd
party free tools." -- Bob Cassidy (rmcassid@uci.edu)