LIBHUGETLBFS(7) LIBHUGETLBFS(7) NAME libhugetlbfs - preload library to back text, data, malloc() or shared memory with hugepages SYNOPSIS export [environment options] [LD_PRELOAD=libhugetlbfs.so] target_application DESCRIPTION libhugetlbfs is a library that can back application text, data, mal- loc() and shared memory with hugepages. This is of benefit to applica- tions that use large amounts of address space and suffer a performance hit due to TLB misses. Wall-clock time or oprofile can be used to determine if there is a performance benefit from using libhugetlbfs or not. In the preloaded use-case, the library can back malloc() and shared memory. Text and data segments can be partially backed if they are large enough. By linking to the library and using the ld helpers, the ELF segments can be aligned to a hugepage size and this is necessary for effective backing of text and data. Once linked, preloading is no longer neces- sary to back malloc or shared memory but the necessary environment variables will still need to be set. See /usr/share/docs/lib- hugetlbfs/HOWTO for detailed instructions on relinking applications. An API for the direct allocation of hugepage-backed regions is also available when an application is directly linked. Unless otherwise specified, libhugetlbfs will use the default hugepage size to back memory regions. The default size is the value of Hugepage- size displayed in /proc/meminfo. The size can be specified in bytes or in kilobytes, megabytes, or gigabytes by appending K, M, or G respec- tively. It is an error to specify a invalid, unsupported, or otherwise unconfigured huge page size. Kernel 2.6.27 or later is required to specify any pagesize other than the default. See /usr/share/docs/libhugetlbfs/HOWTO for detailed instructions on how the library should be used, particularly when relinking the applica- tion. This manual page provides a brief synopsis of the environment variables for as a quick reference. The following options affect what memory regions are backed by hugepages. In all cases, the environment being unset implies it should not be used. HUGETLB_MORECORE=[yes|pagesize] This enables the hugepage malloc() feature, instructing libhugetlbfs to override libc's normal morecore() function with a hugepage version and use it for malloc(). From this point all malloc()s should come from hugepage memory until it runs out. HUGETLB_SHM=yes When this environment variable is set, the SHM_HUGETLB flag is added to the shmget() call and the size parameter is aligned to back the shared memory segment with hugepages. In the event hugepages cannot be used, small pages will be used instead and a warning will be printed to explain the failure. The pagesize cannot be specified with this parame- ter. To change the kernels default hugepage size, use the pagesize= kernel boot parameter (2.6.26 or later required). HUGETLB_ELFMAP=[no|[R[=pagesize]:[W[=pagesize]]] If the application has been relinked (see the HOWTO for instructions), this environment variable determines whether read-only, read-write, both or no segments are backed by hugepages and what pagesize should be used. HUGETLB_FORCE_ELFMAP=yes Force the use of hugepages for text and data segments even if the application has not been relinked to align the ELF segments on a hugepage boundary. Partial segment remapping is not guaranteed to work and the segments must be large enough to contain at least hugepage for the remapping to occur. The following options affect how libhugetlbfs behaves. HUGETLB_MORECORE_SHRINK=yes By default, the hugepage heap does not shrink and is enabled by setting this environment variable. It is disabled by default as glibc occasion- ally exhibits strange behaviour if it mistakes the heap returned by libhugetlbfs as a foreign brk(). HUGETLB_NO_PREFAULT By default libhugetlbfs will prefault regions it creates to ensure they can be referenced without receiving a SIGKILL. On kernels older than 2.6.27, this was necessary as the system did not guarantee that future faults would succeed on regions mapped MAP_PRIVATE. Prefaulting impacts the performance of malloc() and can result in poor placement on NUMA systems. If it is known the hugepage pool is large enough to run the application or the kernel is 2.6.27 or later, this environment variable should be set. HUGETLB_MORECORE_HEAPBASE=address libhugetlbfs normally picks an address to use as the base of the heap for malloc() automatically. This environment variable fill fix what address is used. HUGETLB_PATH=path The path to the hugetlbfs mount is automatically determined at run- time. In the event there are multiple mounts and the wrong one is being selected, use this option to select the correct one. HUGETLB_SHARE=1 By default, libhugetlbfs uses unlinked hugetlbfs files to store remapped program segment data. If the same program is started multiple times using hugepage segments, multiple hugepages will be used to store the same program data. The reduce this wastage, setting this environ- ment variable will share read-only segments between multiple invoca- tions of a program at the cost of the memory being used whether the applications are running or not. It is also possible that a malicious application inferfere with other applications executable code. See the HOWTO for more detailed information on this topic. The following options control the verbosity of libhugetlbfs HUGETLB_VERBOSE=[1:99] The default value for this is 1. The higher the value, the more verbose the output is. HUGETLB_DEBUG Once set, this will give very detailed output on what is happening in the library and run extra diagnostics. SEE ALSO oprofile(1) AUTHORS libhugetlbfs was written by various people on the libhugetlbfs-devel mailing list. September 27, 2008 LIBHUGETLBFS(7)