LINBIT Open Source Software - hugetlb


more LINBIT OSS ..

   HugeTLB (HugeTables) for PostgreSQL
   Huge tables
   In order to reduce the TLB misses modern CPUs offer so called huge pages, or
   huge page TLB entries. The usual page size on x86 machines is today 4kB. On
   PostgreSQL instances that are used for web applications it became common to
   have shared memory segments which are many GByte in size.
   Mapping such a big virtual memory area through normal page table entries is
   quite inefficient, and causes a rather low hit ratio in the TLB.
   The Linux kernel offers support for huge pages since some time, and I
   believe this was fostered by Oracle. What is good for Oracle is good for
   PostgreSQL as well. But unfortunately PostgreSQL does not use huge pages
   as it of today (PostgreSQL 8.2).
   To make PostgreSQL (or other applications to use huge pages) to use huge
   tables here is a small LD_PRELOAD_LIBRARY that does this.
   How to Install
   1) Make sure your kernel support huge pages
      grep HUGETLB /boot/config-2.6.18-6-amd64 or
      zgrep HUGETLB /proc/config.gz
      it should say:
   2) Compile hugetlb.c to hugetlb.so by calling make. Install the library
      somewhere on your machine, e.g. /usr/local/lib/hugetlb.so
   3) Tune your kernel to provide some huge pages:
      echo 2700 > /proc/sys/vm/nr_hugepages
      Make sure the right amount of huge pages is available:
      cat /proc/meminfo
      In case not, do 
      echo 3 > /proc/sys/vm/drop_caches
      and retry to set the nr_hugepages.
      Tune your kernel to allow the postgres' group to use huge tables
      id postgres
      uid=31(postgres) gid=32(postgres) groups=32(postgres),107(ssl-cert)
      echo 32 > /proc/sys/vm/hugetlb_shm_group
   4) Make sure that PostgreSQL gets the library preloaded. On debian etch I do
      this with:
      cat <<EOF > /etc/postgresql/8.2/main/environment
   5) Start PostgreSQL.
   Experienced Benefits
   The average we got on our production systems was an reduction of Query times
   by 13%, a reduction of CPU usage by about 15%.

Unless otherwise specified, this program is free software; you can redistribute them and/or modify them under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.   [ Read the licence ]

Imprint: LINBIT Information Technologies GmbH Vivenotgasse 48, A-1120 Vienna, phone +43 1 817 82 92 - 0, office@linbit.com