电子邮件服务器容量规划

1、Exchange 电子邮件效力器容量规划叶晓彬高级顾问微软顾问咨询部门议题容量规划过程Exchange 效力器容量规划和设计根本监控总结Whats Out There?Benchmarks 基准不是容量规划最好的数据来源Why?单效力器不符合实践的配置单一的负荷压力没有额外的软件和冗余思索用来对比效力器的类型，而不是提供容量规划的指南其它方法?压力测试工具LoadSim 2003, JetStress, ESP Sizing toolsVarying levelsRisk of GIGO as solution growsWhite papersKB Articles技术顾问，系统架构师6步容

2、量规划的方法获得根底数据充分了解SLA对于设计一个胜利的工程非常重要建立本人的 benchmarks获得基线数据确定瓶颈平衡系统资源实施，然后自动监控获得数据: 运用场景User profiles转换为可计量的数据IOPS/uMegacycles / mailboxNetwork bytes / mailbox利用峰值作容量规划确定瓶颈反响时间Servers reaction to loadOnly part of the picture吞吐量Messages/secTransactions/sec瓶颈在哪里?High-demand resourceWorkload-dependentNeve

3、r really eliminate; just move# OF USERSTPUTRATE# OF USERSRESPTIME平衡系统资源CPUMemoryPhysical and VirtualI/ODisksNetworkScale up or scale out?议题容量规划过程Exchange 效力器容量规划和设计效力器角色邮箱效力器容量规划I/O 规划工具根本监控总结Exchange 2003 效力器角色不同的角色不同的需求后端邮箱效力器前端效力器衔接器Inbound SMTP gateways / SMTP BridgeheadX.400 / legacy connector

4、serversDL Expansion servers公共文件夹Free/busy 其它(e.g., OAB Generator)选择效力器采用下面的效力器类型:Blade serversFE servers, BE servers with SAN/iSCSIDense serverHighly available, scalable BE, FE serversHigh-capacity serversBridgehead servers, replicated sites/departmentalVirtual serversLab & hardware-independent QA思索

5、:Power consumptionDirect-attached versus network storageHigh-performance serversHigh-availability optionsExchange容量: CPU最正确选择: 依然是4 processorsFamily-dependentYesterday: Pentium III XeonToday: Xeon / Xeon MP / OpteronGoal: Burn 80% (a challenge?)扩展性的要素Hyperthreading / dual-coreClock frequency (3.6 GH

6、z)L2, L3 cache (up to 8 MB L3 cache)Front-side bus / HyperTransportx64 extensions 处置器比较PIII XeonXeon MPOpteronXeon EM64TXeon MP EM64TMax # procs4 (32)4 (32)4+ (single and dual- core)2; 44Clock speed (max)900 MHz3.0 GHz2.6 GHz3.6 GHz3.3 GHzCache2 MB L24 MB L31 MB L21-2 MB L28 MB L3x64?NoNoYesYesYesFS

7、B speed133 MHz400 MHzProc speed800 MHz667 MHz dualHyper-threadingNoYesDual-coreYesYesMemory supportedPC133PC1600PC3200PC2-3200RPC2-3200RTop MMB3N/A9,500 (4P)10,200 (4P single- core)8,208 (2P)10,520 (4P)TimeFrequency, L3 Cache, FSB ComparisonsFront-Side Bus400MHz 533MHzClock speed boostwithout L3 cac

8、heAddition of L3 cache (1MB)Clock speed boostwith L3 cachePlatform: ProLiant DL360G3; 2P, Exchange 2003 RTM; identical configurationWorkload: 3,000 MMB3. Response Time relatively consistentHyper-Threading ResultsPlatform: ProLiant DL580G2; 4P 2.0GHz/2M cache, Exchange 2003 RTMWorkload: MMB31525%boos

9、tExchange容量: 内存物理内存和虚拟内存 4 GB on high-end back-end servers (2,000 mailboxes and above)/3GB (BE and PF servers) with 1 GB其它减少虚拟内存碎片的方法HeapDecommitFreeBlockThresholdWatch Free System Page Table Entries: /USERVA:3030ESE Cache tuning不要在 FE, SMTP上配置Check KBs 815372, 810371Exchange容量: 网络后端效力器Typically, 10

10、0 Mbit full duplex sufficientConsider Gigabit and NIC teaming if:Network backup/restoreiSCSI or NAS storage TCP Offload Engine supportHigh concentration of OWA / POP / IMAP usersConsider NICs with IPSec offloadMAPI compression/buffer packing30 40% improvement in cached mode synchronization, standard

11、 MAPI traffic over E2K前端效力器Dual switched 100 Mbit or Gbit defaultExchange容量: 网络根本思索点Size for I/O *and* capacity思索 SLAArchitect for high availability, backupdont “add-on VSS or replication将数据库和log分放在不同的Performance and disaster recovery considerationsRAID10 everywhere优化延迟Database: fast reads (20ms)Log

12、s: fast writes (10ms)Bursty random I/O patternsDesign for controller cache avoidanceDesign for Monday morning peakDisk IOPsIO per-spindle dependenciesRAID typeLatency requirementsAccess patternRead/write mixSpindle performance (rotational speed)根本原那么 (FC and U320 SCSI)10K disks: 120 IOPS per disk 20

13、ms15K disks: 160 IOPS per disk 20msConservative use of 100 IOPS per disk for design allows room for peak periodsFigure are RAID1+0; RAID5 IOPS drops to 90/ 10K diskTransaction logs require 10% of SG IOPS磁盘瓶颈分析MSA1000; 16 disks/database; 1 DB/LUN/RAID 0+1 Array; increasing load to saturation generate

14、d by JetStress20ms/write10ms/readSaturation存储设计: 前端效力器SMTPCreate a single partition; 30 small msgs/sec/spindleRAID 0+1 recommended; 100% write-back cacheX.400 and legacy connectorsSeparate MTA (RAID 5), logs (RAID 1) and page file, if possibleOther FE serversNot I/O-intensive存储设计: 后端效力器很多需求思索的要素Arra

15、y / LUN designJournaling / ArchivingVSS-based plexes (snapshot, clones)Temp/TMP drivePartition Alignment with physical sectors (Diskpar/Diskpart with SP1)to mention a fewDatabase IOPs/u RatesRough estimates (MAPI corporate users)ISP / “light user: 2.2 GB virtual bytes in clustered environment with n

16、o active mail usersMonitor MSExchangeIS Public | Client Logons and Peak Client LogonsAssume 1.6x scaling factor, and 1:2 R:W mixActive Directory容量规划Typical configuration2 P+, 2 GB+, RAID10 for DIT, RAID1 for logsDepends on replication, type of deploymentThird party AD apps will impactUse /3GB on GC

17、with 1 GB RAMLarger ESE Cache means 20% 40% disk I/O decreaseDedicated AD site for ExchangeHow many clients per Global Catalog Server?1:4 GC/Exchange processor ratioGC co-located with mailbox serversLogin times, DG decomposition, Delegate accessGC with large user populations4,000 clients per GCUse C

18、losestGC registry key for Outlook 2003 clientsMaintenance ConsiderationsIS Online MaintenanceChecking Active Directory for deleted mailboxes Minimal BE server impactScheduling important to minimize AD impactDisk-intensive maintenance activitiesPermanently remove mailboxes / messages older than reten

19、tion policyOnline defragStagger intervals with backup schedulePF Online MaintenanceAllow for 4 hours/day, and full online maintenance weekly议题容量规划过程Exchange 效力器容量规划和设计根本监控主要的对象和计数器总结基线数据分析性能计数器分析细致调查趋势分析协助他进展自动性的容量管理95th percentile instead of average采样、采样、采样ObjectCounterSuggested valuesMemoryAvailab

20、le Mbytes50 at all timesPages/sec10 average, 100 on spikesProcessor% Processor Time80% average at peak times SystemContext Switches / sec50,000 at the 95th percentileProc Queue Length7 Mbps sustained indicates the need for dual-100 Mbit NICsPhysical Disk Average disk sec/read20 ms for 95% of the tim

21、e, w/ spikes 60 msPhysical Disk Average disk sec/write10 ms for 95% of the time, w/ spikes 50 ms性能对象和计数器最小集协议和邮箱计数器 ObjectCounterSuggested valuesMSExchangeISRPC Requests20 for 95% of the time and less than 30 at all times RPC Averaged Latency50ms for 95% of the time Client: RPC Latency2 secMSExchang

22、eIS Mailbox(_Total)Folder opens/sec50 for 95% of the timeMessage Opens/sec80 for 95% of the timeSend Queue Size1% average of the number of active usersMessages Submitted/minMessages Delivered/minMessage Recipients Delivered/minAverage messages/user value approximately one message every five minutes

23、in a large enterprise.RPC Operations/sec1 per connected user at all times (with an average of .2/user in most enterprises)磁盘计数器ObjectCounterSuggested valuesDatabase instanceLog Record Stalls/sec10 for 95% of the timePhysicalDisk()Disk Writes/sec0.2 writes per connected userAvg. Disk sec/Write8 ms fo

24、r 95% of the timePhysicalDisk()Disk Reads/sec0.6 read/sec per connected user at the 95th percentile of the collected dataset Disk Writes/sec0.4 write/sec per connected user at the 95th percentile of the collected dataset Avg. Disk sec/Read20ms for 95% of the time, with spikes 60msAvg. Disk sec/Write

25、10ms for 95% of the time, with spikes 50ms内存计数器ObjectCounterSuggested valuesDatabaseDatabase Cache SizeUsing /3GB: 896 MB Heavily-fragmented VM: 768 MBWithout the /3GB switch: 576 MBIncreasing cache size (max 1200MB) may be beneficial in a log playback environment; however, may result in VM fragment

26、ation.ProcessWorking Set 1.7- 2.2 GB maximum; varies based on server hardware and user loadMemoryPool Paged Bytes200MB, unless a backup/restore is in process; assumes /3GBPool Nonpaged Bytes 1 GB RAM and /3GB switch Free System Page Table Entries5000 (issue with /3GB)MSExchangeISVM Largest Block Siz

27、e200 MB If this block size value drops below 32 MB, a warning event ID=9582 is generated. VM Total 16 MB Free Blocks1VM Total Free BlocksThis Free Blocks value should be monitored for downward trendsVM Total Large Free BlocksThis Free Blocks value refers to the number of free blocks above 16 MB and

28、should be monitored for downward trends.Topics容量规划过程Exchange 效力器容量规划和设计根本监控总结总结Benchmarking: Caveat emptor. Develop own tests/baselines/acceptance criteria.根本规那么Exchange 大多数情况下自我调整分隔角色以更好的支持用户邮箱效力器CPU: 2 4. Consider fastest FSB, larger L3 cache over CPU speed.Memory: 4 GBNetwork: 100 Mbit; dual 100

29、Mbit for FEActive Directory1 GC near each Exchange server and 4,000 users为周一早晨作容量规划，并且思索快速恢复的SLA要求在消费环境中恢复一个数据库存储总结数据库: 采用很多磁盘Cache misses & small random I/O to the disks70/30 read/write ratio, 60/40 for sizingTransaction logging: 快速写才干Small sequential I/O to the logs添加 I/O per second添加更多，更快的磁盘改动 RA

30、ID 级别 (from 5 to 1) 减少延迟(ms)优化HBAs资源Microsoft Exchange homepage: microsoft/exchangeHP ActiveAnswers: hp/solutions/activeanswers/exchangeExchange performance benchmarks: microsoft/exchange/evaluation/performance/default.mspxExchange Server Best Practices Analyzer Tool: microsoft/exchange/exbpaOptimiz

31、ing Storage for Exchange Server 2003: microsoft/downloads/details.aspx?FamilyID=c6084d20-9730-4ffc-805d-b957327604c6&displaylang=enJetstress:microsoft/downloads/details.aspx?FamilyId=94B9810B-670E-433A-B5EF-B47054595E9C&displaylang=en其它资源Troubleshooting ExchangePerformance:microsoft/downloads/detail

32、s.aspx?familyid=8679f6bd-7ff0-41f5-bdd0-c09019409fc0&displaylang=enExchange Replication Support Stance: support.microsoft/default.aspx?scid=kb;en-us;895847Exchange Replication Best Practices:microsoft/technet/prodtechnol/exchange/guides/E2k3DataRepl/bedf62a9-dff7-49a8-bd27-b2f1c46d5651.mspxExchange

33、Public Folder Best Practices: microsoft/technet/prodtechnol/exchange/2003/pubfoldersbp.mspxExchange 2003 Scalability:microsoft/downloads/details.aspx?FamilyID=62fb1297-4c6b-4d84-84cc-060989f2f305&displaylang=en 2005 Microsoft Corporation. All rights reserved.This presentation is for informational pu

34、rposes only. Microsoft makes no warranties, express or implied, in this summary.Storage Sizing Exercise“Back of the NapkinThree scenarios4,000 “medium users8,000 “light users4,000 “heavier usersBaseline assumptionsBaseline disk: 72.8 GB disk = 66 GB useable storage10K spindle = 120 random IOPS15K sp

35、indle = 150 random IOPS0.8 IOPS average; 1.0 IOPS peak4,000 users * 1.0 IOPS = 4,000 IOPS3:1 ratio: 3,000 read and 1,000 write IOPSBackend disk IOPS: RAID 10 calculationTotal IOPS = 3,000 + 2 * 1,000 = 5,00042 drives 10K RPM or 34 drives 15KBackend disk IOPS: RAID 5 calculationTotal IOPS = 3,000 + 4

36、 * 1,000 = 7,00059 drives 10K or 47 drives 15K200 MB mailboxes4,000 users * 200 MB * 1.5 (“fluff factor for deleted items, maintenance, etc.) = 1,172 GB1,172 GB 66 GB/drive = 18 drivesBackend disks: RAID 10 calculation18 drives * 2 (RAID 10) = 36 drivesBest match to 36 drives 73GB 15K RPM (capacity)

37、Backend disks: RAID 5 calculation18 drives * 1.2 (RAID 5) = 22 drivesPoor match to 59 (10K) or 47 (15K)IOPSBack of the Napkin4,000 “medium usersCapacity0.3 IOPS average; 0.5 IOPS peak8,000 users * 0.5 IOPS = 4,000 IOPS3:1 ratio: 3,000 read and 1,000 write IOPSBackend disk IOPS: RAID 10 calculationTo

38、tal IOPS = 3,000 + 2 * 1,000 = 5,00042 drives 10K RPM or 34 drives 15KBackend disk IOPS: RAID 5 calculationTotal IOPS = 3,000 + 4 * 1,000 = 7,00059 drives 10K or 47 drives 15K200 MB mailboxes8,000 users * 200 MB * 1.5 = 2,344 GB2,344 GB 66 GB/drive = 36 drivesBackend disks: RAID 10 calculation36 drives * 2 (RAID 10) = 72 drives 73 GBWould match 36 drives 146 GB 15K RPM or 42 drives 146GB 10K RPM (I/O)Backen