如何架构高端存储系统.ppt

如何架构高端存储系统,程志勇,主题,BackEndDKA,RaidGroupandLDEVFrontEndCHAWorkwithhost,IO的重要性,这是一个典型的OLTP的事务处理:Email,DSS&RichMedia会占用更多的IO资源OLAP,CRM&ERP会占用更多的CPU资源减少IO的时间，提高IO的性能，将较大地提高应用的总体性能注：这里并没有考虑网络传输占用的时间,HDS高端存储设备的结构,Cachepair,Cachepair(s),CHAportworkwithhost,“前端”,“后端”,DKAportworkwithRAID,dataisplacedintocache&pickedupbytheotherend,USP体系结构,BackEndDKARaidGroupandLDEVRAID1andRAID5,USP后端结构,B4-1,B4-18,B4-4,B4-6,B4-3,B4-5,B4-2,B4-8,B4-7,B4-10,B4-9,B4-15,B4-16,B4-17,B4-11,B4-13,B4-14,B4-12,18xB4,eachB4=4x16RAIDor64HDD,ApairofDKAfirstgoesto2verticalB4,thenhorizontallyto2additionalB4(exceptbasicDKA),DKAOption,9900V4xMPand4x1Gport,Cluster1,Cluster2,MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK,(SH281-B),MP-PK(1MP),(SH281-B),MP-PK(1MP),(SH281-B),MP-PK(1MP),(SH281-B),MP-PK(1MP),(SH281-B),WP471-B,DKC-F460I-200,MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),MP,-,PK,(SH350,-,A),Cluster1,Cluster2,WP520,-,B,DKC,-,F510I,-,400,MP,-,PK(2MP),(SH350,-,A),MP,-,PK(2MP),(SH350,-,A),MP,-,PK(2MP),(SH350,-,A),MP,-,PK(2MP),(SH350,-,A),USP8xMPand8x2Gport,DKAUSPFCAL,HDU0,HDU1,HDU2,HDU3,HDU4,HDU5,HDU6,HDU7,RAID12D+2D,RAID53D+1P,D,Onerandomorsequentialwritefromtheserver,storeincache,Twoasynchronouswritestothemirroreddisks.,RAID-12D+2D写惩罚,ThreerandomorsequentialwritestocacherequiresixIOtodisk.,D,D,3,x,2,6,=,Onerandomwritefromtheserver.,Readolddata&oldparityfromdisk.TworeadIO,Writenewdata&newparity.TwowriteIOtodisk,XORnewdata,olddata,oldparitytocreatenewparity,NewD,NewP,OldP,OldD,写没有命中的时候,Eachrandomwritemeans4IO.ThreeInsertwritestocacherequire12IOto/fromdisk.,RAID-53D+1P随机写的写惩罚,3,x,4,12,=,XOR,XOR,写命中的时候,RAID-53D+1P随机写的写惩罚,Onerandomwritefromtheserver.Olddataalreadyincache,Readoldparityfromdisk.OnereadIO,Writenewdata&newparity.TwowriteIOtodisk,XORnewdata,olddata,oldparitytocreatenewparity,NewD,NewP,OldP,OldD,3,x,3,9,=,XOR,XOR,Eachrandomwritemeans3IO.ThreeInsertwritestocacherequire9IOto/fromdisk.,D,P,D,D,Threesequentialwritesfromtheserver.,XORsequentialwritestocreateParity.,FourWritestoDisk.,ThreesequentialwritestocacherequirefourIOtodisk.,RAID-53D+1P顺序写的写惩罚,3,+,1,4,=,XOR,XOR,RAID类型的性能对比,RAID6needs6IOinsteadof4inrandomwrite,4xHDD,8xHDD,8xHDD,8xHDD,LUN的分配要点,尽量不要把一个应用全压到一个RAIDGROUP上，尽可能地分布在不同的RAIDGROUP上尽量避免顺序IO的应用与随机IO的应用分布在同一个RAIDGROUP或LDEV上如果联机事务处理的应用与批处理的应用共享RAIDGROUP的时候，尽量避免它们在同一时间段工作避免ShadowImage的原卷与目标卷在同一个RAIDGROUP上,条带化的作用,1,2,3,P,4,5,P,6,LDEV1,LDEV2,LDEV3,RAID53D+1P,假设一个3D+1PRAID5建立了3x100GOPEN-VLDEVStripeunit(chunk)is64x8=512K,Stripewidthis3,Stripesizeis1536K加入有一个3MB的文件，它会分成6部分当读这个文件的时候，意味着有“6”个I/O发送到3硬盘上,是并行的这样做，肯定这个3MB的比文件放到同一个硬盘上，读速度快得多,1,1,2,2,3,3,4,4,RAID12D+2D,条带化就是尽可能地将数据分布到各个硬盘上(1)使用更多的硬盘放置数据(2)并行读写，提高性能,RAID5的RAIDGROUP间的条带化,ParityGroup1(7D+1P),ParityGroup2(7D+1P),LDEV#A,A-1,A-2,A-3,A-4,B-1,B-2,B-3,A-2,A-4,B-1,B-3,B-4,A-1,A-3,B-4,B-2,ParityGroup1+2concatenation(7D+1Px2),LDEV#B,2ndstripe(3.5M)goestoanotherPG),FrontEndCHA,USP前端板的位置,USPFCCHA,Always1MPfor2ports,8MP,4MP,Workwithhost,LUN是什么？在RAIDGROUP的什么位置,LDEV0=3GB,LDEV1,LDEV2,LDEV71,Volume=hdisk=PV=LUN=LDEV,LDEV=LogicalDEVice,LogicalVolumeManager,应用将数据存放在LV中，相关的LV在一起组成VG，LVM会将LV对应到PV，也就是说对应到真实的磁盘空间。PV=LUN=LDEVLVM的LV采用striping使数据分布到更多的RaidGroup上，是提高IO性能的一个重要的方法SomeAIXAdministratorstilltryingtoallocateLVtoouteredge,outermiddle,center,innermiddle,andinneredgeofPV.MeaninglessifthePVisRAIDLDEV.,LVM采用多大的StripeSize才是最优化？,RAID-1StripeSize,NoParityGenerationTworeadablecopiesofdata,stripeunit=48K*8track=384Kstripesize=384K*2Open-V:stripeunit=64K*8track=512K,stripesize=512K*2,Recall:OPEN-VRAID53D+1Pstoragestripingspread3Mdatato3HDDononeRAIDApplyingOSstripingAssumingOSstripeunitisalso512KPVisLDEV1,LDEV11,LDEV21Readingthis3Mfilemeans“6”I/Oto6HDD,inparallelUNIXoffersstripingbyPPsizeandalso4Kto128Kchunk(AIX5.3has1Mandup).MakesureyoustripeacrossPVondifferentRAIDGROUPTrytomatchOSstripesizewithHDSstripesizeOSstripechunk/unitshallnotbesmallerthanapplicationI/O,StripingfromHost,1,4,LDEV1,LDEV2,LDEV3,5,2,3,6,LDEV11,LDEV12,LDEV13,LDEV21,LDEV22,LDEV23,WhenOSStripesize=32K,PV1,1,2,3,4,5,6,7,8,9,LV,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,33,33,Open-Vstripeisalways512KWithOSstripesize=32KWhathappentoread/write32Kof#21?,512K#2,512K#3,PV2,512K#2,512K#3,1,2,3,4,9,10,11,12,17,18,19,20,25,26,27,28,5,6,7,8,13,14,15,16,21,22,23,24,29,30,31,32,1,2,3,4,5,6,7,8,9,LV,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,33,PV1,PV2,512K#2,512K#3,512K#2,512K#3,Open-Vstripeisalways512KWithOSstripesize=128KWhathappentoread/write32Kof#21?,WhenOSStripesize=128K,1,2,3,4,9,10,11,12,5,6,7,8,13,14,15,16,21,22,23,24,29,30,31,32,LV,33,17,18,19,20,25,26,27,28,PV1,PV2,512K#2,512K#3,512K#2,512K#3,Open-Vstripeisalways512KWithOSstripesize=512KWhathappentoread/write32Kof#21?,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,WhenOSStripesize=512K,Databeingpre-fetchedwillprovidebettercachehitOSLVMstripesize(stripeunit)=多大是最合适的呢?OPEN-V=512KB,StripesizeandSequentialRead,Queue-Depth,Q-depth:numberofconcurrentI/OallowedonaLUNHostOScouldbeusing8ast