Linux内存监控工具.doc

Linux内存监控工具一、free 该工具主要是显示系统里可用和已用的内存Linux通常按一定的算法把常用的数据加载到系统的虚拟内存buffers和cached中，以便于用户程序在访问系统资源更快。而由free 查看到的buffers是用于存放元数据，而cached是用于存放真实的文件内容。由上图free -k的输出结果中可知：系统总物理内存(total)是4144656K(约4G);已用(Mem行对应的used)的物理内存是3871932K(约3.8G,注:这里包含了buffers的152460K(约152M)和cached的2253060K(2.2G).)，他包含系统的buffers和cached的。-/+ buffers/cache对应的used是1466412K(约1.4G),也就是Mem行used(3871932K)-Mem行buffers(152460K)- Mem行cached(2253060K)=1466412K(约1.4G).所以实际上可用于分配的物理内存(-/+ buffers/cache行对应的free)是2678244K(约2.6G).Shared在man手册里提示应该忽略(man free:The shared memory column should be ignored; it is obsolete.)。Mem行对应的free对应的274220K(约274M).其实这个free是有一定限制的:不能低于min_free_kbytes。min_free_kbytes用于计算系统里lowmem zone(物理内存0-896MB之间的zone)的值(This is used to force the Linux VM to keep a minimum number of kilobytes free. The VM uses this number to compute a pages_min value for each lowmem zone in the system. Each lowmem zone gets a number of reserved free pages based proportionally on its size.).计算方式参见mm/page_alloc.c的min_free_kbytes = sqrt(lowmem_kbytes * 16)上述值是一定的公式计算系统的lowmem是872656KBrootcrm_10 /rootgrep LowTotal /proc/meminfoLowTotal: 872656min_free_kbytes=sqrt(872656*16) 约等于 3797二、ps,top 这两个工具在内存监视方面有很大的相似性，所以一并说一下:下面top里的VIRT相当于ps 里的VSZ:指用于这个任务的总虚拟内存(虚拟内存包括物理内存和swap交换分区),包括所有的代码、数据、共享库以及已经被out到swap分区的数据。/* The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.*/而top里的RES 相当于ps 里的RSS: 指用于这个任务的没被out到swap分区的总物理内存/* resident set size, the non-swapped physical memory that a task has used */top里的%MEM: 指这个任务的RES占总物理内存的比率/* Memory usage (RES) A tasks currently used share of available physical memory.*/三、vmstat显示的值跟用free工具查看到的值相似。一般情况下:只要swap一列的si/so数值不超过1024即可。Swap si: Amount of memory swapped in from disk (/s). so: Amount of memory swapped to disk (/s).四:VFS里的meminfo信息:Dirty：是指数据已写入内存，但还没同步到外存(磁盘)的数据量.Slab:为了提供内核空间以页分配对有些调用(只需小内存)不合适的一种内存分配方式,提出Pool的概念。Vmalloc:为了解决非连续性内存的使用，提供的一种内存分配方式(采用链表)。CommitLimit:指当前可以分配给程序使用的虚拟内存(只有当vm.overcommit_memory的值设置为2时，CommitLimit才有意义)CommitLimit: Based on the overcommit ratio (vm.overcommit_ratio), this is the total amount of memory currently available to be allocated on the system. This limit is only adhered to if strict overcommit accounting is enabled (mode 2 in vm.overcommit_memory). The CommitLimit is calculated with the following formula: CommitLimit = (vm.overcommit_ratio * Physical RAM) + Swap For example, on a system with 1G of physical RAM and 7G of swap with a vm.overcommit_ratio of 30 it would yield a CommitLimit of 7.3G. For more details, see the memory overcommit documentation in vm/overcommit-accounting.Committed_AS:指当前已分配给程序使用的总虚拟内存(包含已分配给进程但还没使用的内存)Committed_AS: The amount of memory presently allocated on the system. The committed memory is a sum of all of the memory which has been allocated by processes, even if it has not been used by them as of yet. A process which malloc()s 1G of memory, but only touches 300M of it will only show up as using 300M of memory even if it has the address space allocated for the entire 1G. This 1G is memory which has been committed to by the VM and can be used at any time by the allocating application. With strict overcommit enabled on the system (mode 2 in vm.overcommit_memory), allocations which would exceed the CommitLimit (detailed above) will not be permitted. This is useful if one needs to guarantee that processes will not fail due to lack of memory once that memory has been successfully allocated.HugePagesize:在X86架构下,通常linux给内存分页时，默认是每页是4KB,而有些应用自己可以管理内存(例如db,java),所以可以把页的大小分大些(在32位下:4K或4M,在PAE模式下可以把每页设置为2M.在64位下: 4K, 8K, 64K, 256K, 1M, 4M, 16M,256M),这样可以增加TLB(一个存储线性地址和物理地址对应表的高速缓冲器)存储的条目,这样就可减少线性地址到物理地址转换的过程.可通过vm.hugetlb_shm_group 和vm.nr_hugepages参数进行调整。具体可查看/usr/share/doc/kernel-doc-uname r|cut -d- -f1/Documentation/vm/hugetlbpage.txt(如果你的机器上已经安装了kernel-doc)The intent of this file is to give a brief summary of hugetlbpage support inthe Linux kernel. This support is built on top of multiple page size supportthat is provided by most of modern architectures. For example, IA-32architecture supports 4K and 4M (2M in PAE mode) page sizes, IA-64architecture supports multiple page sizes 4K, 8K, 64K, 256K, 1M, 4M, 16M,256M. A TLB is a cache of virtual-to-physical translations. Typically thisis a very scarce resource on processor. Operating systems try to make bestuse of limited number of TLB resources. This optimization is more criticalnow as bigger and bigger physical memories (several GBs) are more readilyavailable.Linux C/C+ 内存泄漏检测工具：ValgrindValgrind 是一款 Linux下（支持 x86、x86_64和ppc32）程序的内存调试工具，它可以对编译后的二进制程序进行内存使用监测（C语言中的malloc和free，以及C+中的new和delete），找出内存泄漏问题。Valgrind 中包含的 Memcheck 工具可以检查以下的程序错误：使用未初始化的内存 (Use of uninitialised memory)使用已经释放了的内存 (Reading/writing memory after it has been freed)使用超过malloc分配的内存空间(Reading/writing off the end of mallocd blocks)对堆栈的非法访问 (Reading/writing inappropriate areas on the stack)申请的空间是否有释放 (Memory leaks where pointers to mallocd blocks are lost forever)malloc/free/new/delete申请和释放内存的匹配(Mismatched use of malloc/new/new vs free/delete/delete )src和dst的重叠(Overlapping src and dst pointers in memcpy() and related functions)重复free1、编译安装 Valgrind：wget /downloads/valgrind-3.4.1.tar.bz2tar xvf valgrind-3.4.1.tar.bz2cd valgrind-3.4.1/./configure -prefix=/usr/local/webserver/valgrindmakemake install2、使用示例：对“ls”程序进程检查，返回结果中的“definitely lost: 0 bytes in 0 blocks.”表示没有内存泄漏。rootxoyo42 /# /usr/local/webserver/valgrind/bin/valgrind -tool=memcheck -leak-check=full ls /=1157= Memcheck, a memory error detector.=1157= Copyright (C) 2002-2008, and GNU GPLd, by Julian Seward et al.=1157= Using LibVEX rev 1884, a library for dynamic binary translation.=1157= Copyright (C) 2004-2008, and GNU GPLd, by OpenWorks LLP.=1157= Using valgrind-3.4.1, a dynamic binary instrumentation framework.=1157= Copyright (C) 2000-2008, and GNU GPLd, by Julian Seward et al.=1157= For more details, rerun with: -v=1157= bin data0devhomelib64 mediamntopt rootselinuxsys tcsql.db.idx.pkey.decttserver.pidvarbootdata1etclib lost+foundmisc netprocsbinsrvtcsql.dbtmpusr=1157= =1157= ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 5 from 1)=1157= malloc/free: in use at exit: 28,471 bytes in 36 blocks.=1157= malloc/free: 166 allocs, 130 frees, 51,377 bytes allocated.=1157= For counts of detected errors, rerun with: -v=1157= searching for pointers to 36 not-freed blocks.=1157= checked 174,640 bytes.=1157= =1157= LEAK SUMMARY:=1157=definitely lost: 0 bytes in 0 blocks.=1157=possibly lost: 0 bytes in 0 blocks.=1157=still reachable: 28,471 bytes in 36 blocks.=1157= suppressed: 0 bytes in 0 blocks.=1157= Reachable blocks (those to which a pointer was found) are not shown.=1157= To see them, rerun with: -leak-check=full -show-reachable=yes3、使用示例：对一个使用libevent库编写的“httptest”程序进程检查，返回结果中的“definitely lost: 255 bytes in 5 blocks.”表示发生内存泄漏。rootxoyo42 tcsql-0.1# /usr/local/webserver/valgrind/bin/valgrind -tool=memcheck -leak-check=full ./httptest=1274= Memcheck, a memory error detector.=1274= Copyright (C) 2002-2008, and GNU GPLd, by Julian Seward et al.=1274= Using LibVEX rev 1884, a library for dynamic binary translation.=1274= Copyright (C) 2004-2008, and GNU GPLd, by OpenWorks LLP.=1274= Using valgrind-3.4.1, a dynamic binary instrumentation framework.=1274= Copyright (C) 2000-2008, and GNU GPLd, by Julian Seward et al.=1274= For more details, rerun with: -v=1274= =1274= ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1005 from 2)=1274= malloc/free: in use at exit: 402,291 bytes in 74 blocks.=1274= malloc/free: 15,939 allocs, 15,865 frees, 6,281,523 bytes allocated.=1274= For counts of detected errors, rerun with: -v=1274= searching for pointers to 74 not-freed blocks.=1274= checked 682,468,160 bytes.=1274= =1274= 255 bytes in 5 blocks are definitely lost in loss record 17 of 32=1274=at 0x4A0