Thread: linux server configuration
We are about to build a new database server, our plan is to use Debian. Is there documentation of recommended server configurations for Linux, such as kernel parameters, preferred file system, etc that work best with postgresql? I'm not talking about the pg configuration, which I have seen a lot of documentation about, more on getting the OS ready. Thanks Sim
I don't think that it makes sense to look at PG tuning and server tuning as two separate tasks. XFS was recently benchmarked using bonnie++ by Greg Smith, with interesting results: http://blog.2ndquadrant.com/en/2010/04/the-return-of-xfs-on-linux.html That said, my guess is that the majority of large Linux installations are using ext3 or ext4. It isn't the case that there's a favoured filesystem for Postgres. The kernel parameters that you'll want to change are sysV IPC parameters: http://www.postgresql.org/docs/current/static/kernel-resources.html If you want a guide to choosing hardware for Postgres, I can highly recommend Greg Smith's new book, "Postgresql 9 High performance". -- Regards, Peter Geoghegan
On Thu, Jan 6, 2011 at 4:20 AM, Sim Zacks <sim@compulab.co.il> wrote: > We are about to build a new database server, our plan is to use Debian. > > Is there documentation of recommended server configurations for Linux, such > as kernel parameters, preferred file system, etc that work best with > postgresql? This really depends on your hardware. If you're running a battery backed caching RAID controller it's a good idea to lobotamize the io scheduler since it just gets in the way at that point: echo noop > /sys/block/sda/queue/scheduler On our machines with 128G of ram, we have occasionally seen swap storms for no apparent reason, and have turned off swap. /sbin/swapoff -a On a machine with 16 or more cores, the linux kernel may decide to use the zone reclaim mode, which is supposed to keep memory pinned in the bank accessible by a certain cpu / core for things like virtualization. This leads to disasterous performance on file servers and pg servers. Turn it off in /etc/sysctl.conf: vm.zone_reclaim_mode = 0 For most uses, linux doesn't need a lot of tuning really. Turning the scheduler to noop gained us a bit of performance especially as load increases it's more noticeable that the OS is no longer trying to do the RAID controller's job. If you want to hunt for more tips, search through Greg Smith's site, or buy his excellent pg 9.0 performance book. It'll keep you occupied for days and days with all the cool stuff you can do to make fast pg machines.
As a followup, I'd like to point out that you can probably get more performance wise from hardware upgrades than from tuning your OS. Something as simple as an $800 caching RAID controller can make a workstation class machine into a monster performer, going from 250 tps to 3000 tps with one simple change. Tuning might get you from 250 to 275. Hardware upgrades can take you much much further. Memory upgrades are cheap (I just ordered 8 Gigs for my laptop for $89). 4G ECC RAM is pretty cheap for servers too. Any newly minted database server with less than 16 gigs of ram is inexcusable today. 32 is really the minimum I'd throw at a new server now. Also, very fast 8 or 12 core cpus, both from intel and amd, are cheap too. An 8 core 2.0GHz Magny Cours on newegg is only $275. Put a pair of those in a db server with 32 Gigs ram and a decent caching RAID controller and a handful of decent hard drives and you don't really need to do a lot of tuning. Even a 4 drive machine can give a good account of itself with that hardware. What I'm trying to say is don't spend tons of time tuning slow hardware, you'll get at most a few % gain.
On Thu, Jan 6, 2011 at 9:38 AM, Scott Marlowe <scott.marlowe@gmail.com> wrote: > As a followup, I'd like to point out that you can probably get more > performance wise from hardware upgrades than from tuning your OS. > Something as simple as an $800 caching RAID controller can make a > Totally agree here. Throwing hardware at the problem is a very cost effective solution to performance issues! :)
On Thu, Jan 6, 2011 at 6:20 AM, Sim Zacks <sim@compulab.co.il> wrote: > We are about to build a new database server, our plan is to use Debian. > > Is there documentation of recommended server configurations for Linux, such > as kernel parameters, preferred file system, etc that work best with > postgresql? > > I'm not talking about the pg configuration, which I have seen a lot of > documentation about, more on getting the OS ready. My company has two identical PostgreSQL servers running on Debian (Squeeze) & we didn't tune the kernel and left it as it was installed by Debian. I partitioned the drives as 'ext4' & and the data is stored on a iSCSI NAS (RAID 5) configuration. The servers have been rock solid. Just be super careful that you don't blindly upgrade the server (using apt-get upgrade) and accidentally swap from 8.4 > 9.0. I ran into this problem and it was very messy. This is a PG issue however, not a Debian issue. On a side note I've ran PostgreSQL 8.4 in a production environment on the following: - CentOS 5 64-bit - RHEL 5 64-bit - Arch Linux 64-bit - Ubuntu 10.04 Server - Slackware Linux 13 64-bit - Gentoo Linux (Uggh) 64-bit Out of all those distributions, I can honestly say that between Arch Linux and Debian, no other distribution comes close.