媒体存储技术存储基础iSCSI

iSCSI:Past,Present,FutureLi-Gu

ZhuDepartmentofComputerScience,CommunicationUniversityofChinaVeryBriefHistoryLate1997–ideaofstorageoverIP–JulianSatran,IBMresearchLate1999–IBMandCiscostartjointworkonproposalforstandardEarly2000–IETFcreatesIPstorageworkinggroupNovember2000–IETFdraft0postedRestofBriefHistoryJan2001–SNIAcreatesIPstorageforumJuly2001–firstUNHIOLiSCSIPlugtest28companiesattendedTesteddrafts0and6Feb2003–IETFapprovesdraft20June2003–MicrosoftServerwithiSCSIApril2004–IETFpublishesRFC3720TodayiSCSIproductsofferedbyallstorageandplatformvendorsManysmallvendorsinthemarketiSCSInowwellacceptedatlowandmiddleperformanceranges1Gigwire-speedHBAsavailable10GigiSCSIproductsstartingtoappearOtherSANTechnologiesEnterprisedatacentersstillbasedonFibreChannel(4Gig,8Gig)BiginterestinFCoEWillFibreChannelequipmentpricesbelowerinthenearfuture??InfiniBandSCSITransportProtocolforTCPBasedonwidelyused,off-the-shelftechnologySCSI,TCP,IP,IPsec,EthernetFamiliar,alreadyinstalledinfrastructureCommoditycomponents,inexpensivePermitsall-softwareimplementationsEncouragesexperimentationManyfreelydistributedimplementationsiSCSI:RFC3720DocumentLong,informalEnglishproseAmbiguous,canbemisinterpretedTestingislong,hasmanycombinationsNeedforuseofformalmethodsforspecification,verification,testingiSCSIiSCSIoverTCP/IPCommandSCSICommandDataCommandDataTCPHeaderTCPPayloadiSCSIApplicationsConnectivity:iSCSIcanbeusedforDASorSANconnections.iSCSIcapabledevicescouldbeplacedonanexistingLAN(sharedwithotherapplications)inasimilarwaytoNASdevices.iSCSIcapabledevicescouldbeattachedtoaLANwhichisdedicatedtostorageI/O(inotherwordsanIPSAN),orevenonaLANconnectedtoonlyoneprocessor(likeaDAS).iSCSIHBAsandTCPOffloadEngines(TOEs)OffloadsTCPand,optionally,iSCSIprocessingintohardwareRelieveshost

CPUfrom:TCPprocessing—16-bitchecksumperpacketiSCSI—optional32-bitheaderanddatadigests(CRC32C)TCPOffloadiSCSI

andTCPOffloadFileSystemBlockDeviceSCSIGenericHBA

DriverTCP/IP

StackNIC

DriveriSCSI

DriverTOE

DriverSCSIAdapterAdapter

DriverTCP/IP

StackiSCSITCP/IP

StackApplicationsiSCSIProtocolStackDataSCSIiSCSITCPIPEthernetPHYDriverTCPIPEthernetPHYDriverDataSCSIiSCSITCPIPEthernetPHYDriverDataSCSIiSCSIAdapterCardHostProcessingHostBusAdapter(HBA)SoftwareiSCSIiSCSITCPOffloadEngine(TOE)iSCSIHostBusAdapter(HBA)iSCSIPerformanceConsiderationExampleperformanceimpactonCPUutil%FCvs.iSCSITOEvs.iSCSISWDriverCPU%ThroughputMB/sFCHBAiSCSITOEiSCSISWDriver15-35MB/sInflectionpointdeterminedbysystemresources(CPU/Memory)?iSCSIBenefitHostBusAdapter(FibreChannel)HostBusAdapter(iSCSI)10/100/1000BaseTPCIMulti-TCPSessionsHBA:Multi-PathI/OLUN:Logical[Storage]UnitsIsIP-SANsimilartoNAS?Whataretheadvantages,ifany,ofIP-SANvs.NAS?PerformanceAnalysis:iSCSIvs.NAS(softwarebased–noHBA)iSCSINFSSequentialRead/WriteTestsConclusion:Comparableperformanceincharacterread/writeandblockwriteSignificantadvantageofiSCSIinblockread(20-25%)SmallFilesRead/WriteIOGenTest(EmulationofDatabase)FC-SANvs.iSCSISincetheiSCSIapplianceattachestotheexistingEthernetnetwork,NASandiSCSIareverysimilarinnetworkarchitectureHowever,theperformancewouldbesignificantlydifferent.BothiSCSIandSANuseBlockI/Ototransportdata,whereasNASusesFileI/O.SANoffersbetterperformance(c.f.NAS),butismoreexpensiveandrequiresahigherskillsettoimplement.iSCSIandNASofferbetterpricingandskillsmayalreadybeinplacetoimplementthem.BothSANandiSCSIoffertheperformancebenefitofBlockI/O.iSCSI:PerformanceFactorsWorkloadcharacteristicsSequentialstreamingvsrandomaccessRead/write,large/smalltransfersNetworkcharacteristicsSpeed(100,1000,10000Mbps)Distance(LAN,MAN,WAN)ErrorratesCongestioniSCSI:PerformanceMetricsBandwidthutilization–highisdesirableCPUutilization––lowisdesirableLatency–lowisdesirableTransactionrate–highisdesirableiSCSI:PerformanceNumerousstudiesdone,manymoretodoMany,manytunableparametersatalllevelsSCSIiSCSITCPEthernetInteractions/tradeoffswithin/betweenlevelsDynamicparameteradjustmentMemoryaCriticaliSCSIResourceInitiatorPagingtoaniSCSIdiskVMsystemMUSTNOTblockformemoryWithoutcare,standardTCPstackwillblockformemory(buffersandcontrolstructures)Withoutcare,iSCSIdatapathwillblockformemoryTargetmemorystarvationMaygetmultiplecommandsatonceMustholdmemoryuntilreceiptacknowledgedAcknowledgementmaybedelayedindefinitelyiSCSI:CPULoadCPUutilizationisnotnegligibleBiggestpercentfromTCP/IP,notiSCSIorSCSIStandardTOEoff-loadinghelpsoutputiSCSIHBAoff-loadinghelpsinputandoutputSoftwareiSCSICRCisexpensiveforlargePDUsCPUOverheadwithoutHBAInterruptrate1500byteframeevery12microsecson1GE9000byteframeevery5microsecson10GEFrequentcacheflushingExtracopyingTOEshelpmainlyonoutputInputrequiresintermediateTCPbuffersorcostlymemorymappingiSER:iSCSIExtensionsforRDMAInterfacebetweeniSCSIandRDMAiSERadds12-byteheadertocontrolPDUsMakesiSCSIindependentofanyprotocolRDMAP/DDP/MPA/TCP/IPRDMAP/DDP/SCTP/IPInfinibandOthers?(Myrinet?,Quadrics?)iSER:ConceptsTargetcontrolsdataflowiSCSIread=targetRDMAwriteiSCSIwrite=targetRDMAread4newkeysOldkeysfordigests,markersareirrelevantHandlingofiSCSIPDUsR2T,DataOutPDUsreplacedbyRDMAreadDataInPDUsreplacedbyRDMAwriteAllotherPDUscarriedbyRDMAsendiSCSI/iSERErrorHandlingGuaranteedreliable,in-orderdeliveriSERerrorterminatesconnection!AlliSCSIerrorrecoverylevelspossibleLevel1reducedtoalmostnothingDigestandsequenceerrorsnowimpossiblePDUretransmissiontimeoutsdiscouragedSNACKmustnolongerbesentiSCSI:SharingaTargetDeviceMultiplehostseasilyaccesscommontargetEfficientblocktransportdirectlytodiskNonotionoffiles,directories,data,ormetadataNocontentiondetectionorresolutionNoallocationormanagementofblocksObject-basedStorageSystemIdea:pushmoreintelligenceontodiskunitTargetmanagesblockallocationTargetdefinesobjectsandmapstheirblocksTargetmanagesobjectmetadataEnhancementstoSCSIcommandsetMustrewritefilesystemstouseobjectsANSIProjectT10/1355-DSCSIObject-BasedStorageDeviceCommandsFinalRevision10,30July2004“ToprovideefficientoperationofI/Ologicalunitsthatmanagetheallocation,placementandaccessingofvariable-sizedata-storagecontainerscalledobjects.”iSCSI:Object-storeAllOSDcommandsarebi-directional200-byteCDBrequiresuseofiSCSIAHSReadingaPDUheaderrequires2steps:Read48-byteBasicheader,extractAHSLengthReadfollowingAHSLengthbytesHeaderdigest(CRC)isproblematicUseAHSLengthvaluetoreadAHSheadersandCRCUseCRCtocheckcompleteheaderafterreaddoneIfAHSLengthhaserror–inputmayblock!TrackingSCSIStandardsiSCSIversion0(RFC3720)basedon:SAM-2Finalrevision24,11-September-2002SBCFinalrevision8,13-November-1997UpgradetoSAM-3projectT10/1561-D?Finalrevision14,21-September-2004SAM-4projectT10/1683-DFinalrevision14,8-May-2008SAM-5projectT10/2104-DRevision05,11-May-2010iSCSI:ResearchAreasDynamicparameteradjustmentRespondtochangesinapplicationloadRespondtochangesinnetworkconditionsUsingparametersbetweenlevelsE.g.,letiSCSIuseTCP’sRTTtokeeppipefullNegotiatelimitsbutoperateatothervaluesTargetcontrolsburstsizes,outstandingR2TsInitiatorcontrolsconnections,unsoliciteddataiSCSI:ResearchAreasSchedulingmultipleconnectionsinsessionWhatcriteriatouse?Letdifferentconnectionscarrydifferentinfo?ReinstatingsessionsinordertorenegotiateOnlylimiteddifferencesbetweenconnectionsNewfilesystemsNewcachingschemesANYQUESTIONS?THANKYOU!9、静夜四无无邻，荒居居旧业贫。。。12月-2212月-22Saturday,December31,202210、雨中黄叶叶树，灯下下白头人。。。05:53:5005:53:5005:5312/31/20225:53:50AM11、以我独沈沈久，愧君君相见频。。。12月-2205:53:5005:53Dec-2231-Dec-2212、故人江海海别，几度度隔山川。。。05:53:5005:53:5005:53Saturday,December31,202213、乍见翻疑梦梦，相悲各问问年。。12月-2212月-2205:53:5005:53:50December31,202214、他乡乡生白白发，，旧国国见青青山。。。31十十二二月20225:53:50上上午05:53:5012月月-2215、比不了得就就不比，得不不到的就不要要。。。十二月225:53上上午12月-2205:53December31,202216、行动出成果果，工作出财财富。。2022/12/315:53:5005:53:5031December202217、做前，能够够环视四周；；做时，你只只能或者最好好沿着以脚为为起点的射线线向前。。5:53:50上午5:53上上午05:53:5012月-229、没有失败败，只有暂暂时停止成成功！。12月-2212月-22Saturday,December31,202210、很多事情情努力了未未必有结果果，但是不不努力却什什么改变也也没有。。。05:53:5005:53:5005:5312/31/20225:53:50AM11、成成功功就就是是日日复复一一日日那那一一点点点点小小小小努努力力的的积积累累。。。。12月月-2205:53:5005:53Dec-2231-Dec-2212、世间间成事事，不不求其其绝对对圆满满，留留一份份不足足，可可得无无限完完美。。。05:53:5005:53:5005:53Saturday,December31,202213、不知香积积寺，数里里入云峰。。。12月-2212月-2205:53:5005:53:50December31,202214、意志坚强强的人能把把世界放在在手中像泥泥块一样任任意揉捏。。31十二二月20225:53:50上上午05:53:5012月-2215、楚塞三三湘接，，荆门九九派通。。。。十二月225:53上午午12月-2205:53December31,202216、少年十五二二十时，步行行夺得胡马骑骑。。2022/12/315:53:5005:53:5031December202217、空山新雨后后，天气晚来来秋。。5:53:50上午5:53上上午05:53:5012月-2