HP-UNIX-内存热点问题与案例分享

1、HP-UNIX-内存热点问题与 案例分享 HP-UNIX-内存热点问题与案例分享 检查内存检查内存 1. dmesg: Memory Information: physical page size = 4096 bytes, logical page size = 4096 bytes Physical: 1048576 Kbytes, lockable: 790684 Kbytes, available: 918096 Kbytes 2.sam: samPerformance MonitorsSystem PropertiesMemory 3.syslog.log 4.glance HP-U

2、NIX-内存热点问题与案例分享 Memory的角色的角色 三个部分三个部分: 1. Physical memory 2. Available Memory The amount of main memory not reserved for the kernel is termed available memory. Available memory is used by the system for executing user processes. 3. Lockable Memory Pages kept in memory for the lifetime of a process b

3、y means of a system call (such as mlock, plock, or shmctl) are termed locked memory. Locked memory cannot be paged and processes with locked memory cannot be deactivated. Kernel parameter: unlockable_mem. Total available system memory - unlockable_mem = lockable memory HP-UNIX-内存热点问题与案例分享 检查空闲内存检查空闲

4、内存 1. Top Memory: 126108K (79812K) real, 419716K (324852K) virtual, 98184K free 2. Glance 3. Vmstat #vmstat 1 1 procs memory page faults cpu r b w avm free re at pi po fr de sr in sy cs us sy id 1 0 0 93405 5989 5 1 0 0 0 0 0 448 737 133 1 1 99 HP-UNIX-内存热点问题与案例分享 与内存相关的内核参数与内存相关的内核参数 Buffer cache r

5、elated: nbuf and bufpages =0 表示使用动态内存 dbc_max_pct50 dbc_min_pct5 Buffer cache used by . 建议调低dbc_max_pct(尤其大量使用raw device)， 否则占用太多内存。 HP-UNIX-内存热点问题与案例分享 与内存相关的内核参数与内存相关的内核参数(续续) vx_ninode=0 表示使用动态内存,可能占用很大内 存, 造成系统内存资源耗尽，还可能造成 superpagepool corruption Or fragmentation。 unlockable_mem 使用缺省值(=0) HP-UN

6、IX-内存热点问题与案例分享 内存的分配内存的分配 内存分配的三种方式，需要相同数量的swap. plain memory as allocated with malloc(3C) system call. shared memory as allocated with shmget(2) system call. 1.memory mapped files as allocated withmmap(2) system call. HP-UNIX-内存热点问题与案例分享 内存的分配（续）内存的分配（续） 一般内存的分配由一般内存的分配由malloc(3C) 系统调用完成系统调用完成 a) T

7、o report process memory usage, by largest first, run: # ps efl | sort rnk 10 | more And look at 10th column (SZ) to see the amount of memory used by this process for data/text and stack (as this value is in pages, you can multiply by 4096 to determine the size in bytes.) Anytime you see that the siz

8、e (SZ) is a four-digit number, watch it over time and see if it continues to grow. Alternative ps command - Alternatively, you can use the UNIX95 options to look at both Virtual Size as well as the actual Size. Run: # UNIX95=1 ps -efo vsz,sz,pid,args |grep -v grep | sort -rnk 1 | more HP-UNIX-内存热点问题

9、与案例分享 内存的分配（续）内存的分配（续） b) And another way is to sort glances process list (or application list) by RSS (or Res Mem Virt Mem). For example: PROCESS LIST Users= 5 User CPU Util Cum Disk Thd Process Name PID PPID Pri Name ( 100% max) CPU IO Rate RSS Cnt - pax 13819 13818 148 root 2.7/ 5.8 273.3 9.4/32.

10、8 284kb 1 glance 14464 1822 158 root 2.1/ 3.1 3.0 0.0/ 2.1 4.3mb 1 scopeux 1715 1 127 root 1.7/ 0.2 518.4 1.5/ 0.0 4.1mb 1 s 0 0 127 root 1.5/ 0.8 2213.0 0.3/ 0.0 16kb 1 java 10095 1 168 root 1.0/ 2.7 348.7 0.0/ 4.2 42.0mb 28 vxfsd 35 0 138 root 0.2/ 0.1 289.4 1.9/ 1.3 352kb 16 c) Unsupported tools:

11、 procsize & kmeminfo HP-UNIX-内存热点问题与案例分享 内存的分配（续）内存的分配（续） 2) 共享内存的分配共享内存的分配: shmget(2) system call. a) To see shared memory usage, ipcs can be used. For example, run: # ipcs mb | more To total the shared memory usage, run: # ipcs -mb | sed -n /m/p | awk total+=$NFENDprintf(“%dn”, total) b) unsupport

12、ed tool called shminfo HP-UNIX-内存热点问题与案例分享 内存的分配（续）内存的分配（续） 3) 内存映像文件内存映像文件: mmap(2) system call. No tools available but unsupported shminfo. HP-UNIX-内存热点问题与案例分享 用户进程内存限制用户进程内存限制 maxdsiz,maxssiz,maxtsiz Shared memory limit: shmmax swap HP-UNIX-内存热点问题与案例分享 32bit app HP-UNIX-内存热点问题与案例分享 解释 对通常的程序来说： 第

13、一象限用来存放app text 第二象限用来存放data & stack 第三四象限用来存放共享的数据 The default usage for the quadrants is: Quadrant 1 Process text Quadrant 2 Process data Quadrant 3 Shared libraries Shared memory Memory mapped files Quadrant 4 Shared libraries Shared memory Memory mapped files The last part of quadrant 4 is reser

14、ved for system I/O. HP-UNIX-内存热点问题与案例分享 32bit app Private 部分部分 Maxdsiz & maxssiz 所决定的数据段和堆栈段位 于同一个象限中，总和不能大于1GB。 Maxdsiz & maxssiz应该合理配置，否则互相影 响 Maxssiz一般设置成64MB已足够 正常32bit应用，数据段最大为940MB 安装补丁，最大可以1.9GB。 编译选项 ld -N OR “chatr +q3p enable executable_name” to enable third quadrant private HP-UNIX-内存热点问题

15、与案例分享 32bit app 共享部分共享部分 第3、4象限用来做32bit app的共享寻址 最大1.75GB SHMEM_MAGIC 将第二象限用于共享数据(这时 最大为2.75GB) To get SHMEM_MAGIC, the executable needs to have been linked (ld) with EXEC_MAGIC (the -N option) and chatred with SHMEM_MAGIC (the -M option). HP-UNIX-内存热点问题与案例分享 64bit app For 64 bit processes the quadr

16、ant sizes are 4tb, and therefore the maximum sizes for text, data, shared memory, etc, are quite large. For 64 bit 11.00/11i executables there is currently no need to have different memory maps available as the standard one allows up to 4TB for the program text, another 4TB for its private data and

17、a total of 8TB for shared areas. HP-UNIX-内存热点问题与案例分享 考虑一下 16GB物理内存 将多个数据库置于同一台机器 对32位数据库，只有1.75GB内存可用 怎么办？ HP-UNIX-内存热点问题与案例分享 Memory Windows! HP-UNIX-内存热点问题与案例分享 What does the Memory Windows feature do? Enabling memory windows does change the kernels default allocation policy. With only the global

18、memory window configured, the kernel allocates shared addresses from the 4th quadrant first and then the 3rd quadrant. When max_mem_window is set to a non-zero value (enabling memory windows) , the kernel changes the default allocation policy such that the 3rd quadrant is tried first followed by the

19、 4th. HP-UNIX-内存热点问题与案例分享 内核设置内核设置 To enable the use of memory windows the kernel tunable max_mem_window must be set to the desired amount of memory windows. The disabled value is 0. A good default is 256. Note: This tunable is not dynamic. HP-UNIX-内存热点问题与案例分享 内核设置内核设置 To enable 256 memory windows i

20、n the kernel add the following line to the kernel configuration file (/stand/system): max_mem_window 256 Build a new kernel and reboot. Setting max_mem_window to 256 results in creating 256 memory windows, plus the global memory window. Setting max_mem_window to 1 would create 1 memory window, plus

21、the global window, for a total of 2. HP-UNIX-内存热点问题与案例分享 Services.window Modify /etc/services.window . . database1 20 database2 30 database3 40 HP-UNIX-内存热点问题与案例分享 相关命令相关命令 getmemwindow is the command used to extract window ids of user processes from the /etc/services.window file. setmemwindow is th

22、e command that changes the window id of a running process or starts a specified program in a particular memory window. HP-UNIX-内存热点问题与案例分享 限制限制 An application cannot dynamically switch between one window and another. Applications are only allowed to change their window at exec time. Applications are

23、 not required to change their source code, but instead use the memory window command setmemwindow to start an application in a specified memory window. HP-UNIX-内存热点问题与案例分享 例子例子 For example: setmemwindow -i 10 myprog arg1 arg2 arg3 This would start the executable myprog with 3 arguments in the memory

24、 window corresponding to user key 10. There are other options to setmemwindow to control how memory windows are created. HP-UNIX-内存热点问题与案例分享 例子（续）例子（续） getmemwindow is used to extract the user ids/keys from the /etc/services.window a particular string. This avoids having to embed the keys in the scr

25、ipts themselves. syntax: getmemwindow HP-UNIX-内存热点问题与案例分享 例子（续）例子（续） Putting it all together in a real example: # cat startDB1.sh WinId=$(getmemwindow database1) setmemwindow -i $WinId /home/dave/memwinbb/startDB1 DB1 says hello! HP-UNIX-内存热点问题与案例分享 例子（续）例子（续） # ./startDB1.sh writing to segment: DB1

26、 says hello! Key is 1377312562 HP-UNIX-内存热点问题与案例分享 例子（续）例子（续） # ipcs -mob IPC status from /dev/kmem as of Thu Jul 10 08:52:34 2003 T ID KEY MODE OWNER GROUP NATTCH SEGSZ Shared Memory: m 0 0 x4124288b -rw-rw-rw- root root 0 348 m 1 0 x4e000002 -rw-rw-rw- root root 1 61760 m 2 0 x41280050 -rw-rw-rw-

27、root root 1 8192 m 5215 0 x52181f32 -rw-r-r- root sys 0 1024 Our key of 1377312562 converted to hex is 52181f32 HP-UNIX-内存热点问题与案例分享 例子（续）例子（续） # ./startDB2.sh writing to segment: DB2 says Bon Jour! Key is 1377312563 # ./startDB3.sh writing to segment: DB3 says Ola! Key is 1377042734 HP-UNIX-内存热点问题与案

28、例分享 例子（续）例子（续） # more stopDB1.sh WinId=$(getmemwindow database1) setmemwindow -i $WinId /home/dave/memwinbb/stopDB1 # ./stopDB1.sh # ./stopDB2.sh # ./stopDB3.sh HP-UNIX-内存热点问题与案例分享 工具介绍工具介绍 ps top glance HP-UNIX-内存热点问题与案例分享 ps # ps efl | sort rnk 10 | more And look at 10th column (SZ) to see the amo

29、unt of memory used by this process for data/text and stack (as this value is in pages, you can multiply by 4096 to determine the size in bytes.) Anytime you see that the size (SZ) is a four-digit number, watch it over time and see if it continues to grow. Alternative ps command - Alternatively, you

30、can use the UNIX95 options to look at both Virtual Size as well as the actual Size. For example: # UNIX95=1 ps -efo vsz,sz,pid,args |grep -v grep | sort - rnk 1 | more VSZ SZ PID COMMAND 12252 627 2745 /opt/OV/bin/ovdbrun -c /var/opt/OV/share/databases/analysis/ 9060 1214 2362 /opt/omni/lbin/rds -d

31、8808 1892 2677 /opt/hpwebjet-5.5/hpwebjetd HP-UNIX-内存热点问题与案例分享 top Load averages: 0.64, 0.57, 0.56 195 processes: 194 sleeping, 1 running . . Memory: 1444296K (1238320K) real, 1967080K (1468152K) virtual, 84908K free | | | | | 1 2 3 4 5 Memory is not all of physcial, memory, it is: 分配给所有进程的物理内存分配给所有

32、进程的物理内存DEDICATED to text, data or stack DEDICATED to text, data or stack segments segments 所有所有runnable processesrunnable processes的物理内存，不包括的物理内存，不包括sleeping processessleeping processes的。的。 分配给所有进程的虚拟内存分配给所有进程的虚拟内存DEDICATED to text, data or stack DEDICATED to text, data or stack segmentssegments。不是所

33、有都在物理内存，有部分在不是所有都在物理内存，有部分在swapswap区中。区中。 分配给所有进程的虚拟内存，不包括分配给所有进程的虚拟内存，不包括sleeping or stopped processessleeping or stopped processes 1.1. 新进程的可用物理内存，当比较低时，新进程的可用物理内存，当比较低时，s. s. HP-UNIX-内存热点问题与案例分享 Top（续） Size is virtual memory size RES means the part in RAM What if too many CPUs? top h What if I want to get all process info? top n500 f top.out HP-UNIX-内存热点问题与案例分享 glance Total VM : 121.1mb Sys Mem : 13.8mb User Mem: 91.3mb Phys Mem: 144.0mb Active VM: 73.7mb Buf Cache: 26.4mb Free Mem: 12.6mb Total VM: The total private virtual memory (in KBs unless otherwise spec