媒体存储技术-4存储基础iSCSI.ppt

iSCSI:Past,Present,Future,Li-GuZhuDepartmentofComputerScience,CommunicationUniversityofChina,VeryBriefHistory,Late1997ideaofstorageoverIPJulianSatran,IBMresearchLate1999IBMandCiscostartjointworkonproposalforstandardEarly2000IETFcreatesIPstorageworkinggroupNovember2000IETFdraft0posted,RestofBriefHistory,Jan2001SNIAcreatesIPstorageforumJuly2001firstUNHIOLiSCSIPlugtest28companiesattendedTesteddrafts0and6Feb2003IETFapprovesdraft20June2003MicrosoftServerwithiSCSIApril2004IETFpublishesRFC3720,Today,iSCSIproductsofferedbyallstorageandplatformvendorsManysmallvendorsinthemarketiSCSInowwellacceptedatlowandmiddleperformanceranges1Gigwire-speedHBAsavailable10GigiSCSIproductsstartingtoappear,OtherSANTechnologies,EnterprisedatacentersstillbasedonFibreChannel(4Gig,8Gig)BiginterestinFCoEWillFibreChannelequipmentpricesbelowerinthenearfuture?InfiniBand,SCSITransportProtocolforTCP,Basedonwidelyused,off-the-shelftechnologySCSI,TCP,IP,IPsec,EthernetFamiliar,alreadyinstalledinfrastructureCommoditycomponents,inexpensivePermitsall-softwareimplementationsEncouragesexperimentationManyfreelydistributedimplementations,iSCSI:RFC3720Document,Long,informalEnglishproseAmbiguous,canbemisinterpretedTestingislong,hasmanycombinationsNeedforuseofformalmethodsforspecification,verification,testing,iSCSI,iSCSIoverTCP/IP,Command,SCSICommand,Data,Command,Data,TCPHeader,TCPPayload,iSCSIApplications,Connectivity:iSCSIcanbeusedforDASorSANconnections.iSCSIcapabledevicescouldbeplacedonanexistingLAN(sharedwithotherapplications)inasimilarwaytoNASdevices.iSCSIcapabledevicescouldbeattachedtoaLANwhichisdedicatedtostorageI/O(inotherwordsanIPSAN),orevenonaLANconnectedtoonlyoneprocessor(likeaDAS).,iSCSIHBAsandTCPOffloadEngines(TOEs),OffloadsTCPand,optionally,iSCSIprocessingintohardwareRelieveshostCPUfrom:TCPprocessing16-bitchecksumperpacketiSCSIoptional32-bitheaderanddatadigests(CRC32C),TCPOffload,iSCSIandTCPOffload,FileSystem,SCSIGeneric,SCSIAdapter,iSCSIProtocolStack,Data,SCSI,iSCSI,TCP,IP,Ethernet,PHY,Driver,TCP,IP,Ethernet,PHY,Driver,Data,SCSI,iSCSI,TCP,IP,Ethernet,PHY,Driver,Data,SCSI,iSCSI,AdapterCard,HostProcessing,HostBusAdapter(HBA),SoftwareiSCSI,iSCSITCPOffloadEngine(TOE),iSCSIHostBusAdapter(HBA),iSCSIPerformanceConsideration,ExampleperformanceimpactonCPUutil%FCvs.iSCSITOEvs.iSCSISWDriver,CPU%,ThroughputMB/s,FCHBA,?,iSCSIBenefit,HostBusAdapter(FibreChannel),QLogicPCI-EXPRESS4Gb1PTFibreHostBusAdapterSTDFWPCI-Express1Port(FibreChannel)DataRate:4GbStandardFirmwarePriceQuote:$951Source:,HostBusAdapter(iSCSI),TheQLE4060CiSCSIHostBusAdapter(HBA)providesconnectivitytoSANsoverEthernetandTCP/IPnetworkinfrastructuresforPCIExpressbus-basedservers.Usingawidelydeployedandfamiliarnetworkingtechnology,iSCSIcanlowerthetotalcostofownershiptobettermanageandsupportstoragenetworking.Source:Pricequote:$815,10/100/1000BaseTPCI,Multi-TCPSessions,HBA:Multi-PathI/O,LUN:LogicalStorageUnits,IsIP-SANsimilartoNAS?,Whataretheadvantages,ifany,ofIP-SANvs.NAS?,PerformanceAnalysis:iSCSIvs.NAS(softwarebasednoHBA),iSCSI,NFS,SequentialRead/WriteTests,Conclusion:Comparableperformanceincharacterread/writeandblockwriteSignificantadvantageofiSCSIinblockread(20-25%),SmallFilesRead/Write,IOGenTest(EmulationofDatabase),FC-SANvs.iSCSI,SincetheiSCSIapplianceattachestotheexistingEthernetnetwork,NASandiSCSIareverysimilarinnetworkarchitectureHowever,theperformancewouldbesignificantlydifferent.BothiSCSIandSANuseBlockI/Ototransportdata,whereasNASusesFileI/O.SANoffersbetterperformance(c.f.NAS),butismoreexpensiveandrequiresahigherskillsettoimplement.iSCSIandNASofferbetterpricingandskillsmayalreadybeinplacetoimplementthem.BothSANandiSCSIoffertheperformancebenefitofBlockI/O.,iSCSI:PerformanceFactors,WorkloadcharacteristicsSequentialstreamingvsrandomaccessRead/write,large/smalltransfersNetworkcharacteristicsSpeed(100,1000,10000Mbps)Distance(LAN,MAN,WAN)ErrorratesCongestion,iSCSI:PerformanceMetrics,BandwidthutilizationhighisdesirableCPUutilizationlowisdesirableLatencylowisdesirableTransactionratehighisdesirable,iSCSI:Performance,Numerousstudiesdone,manymoretodoMany,manytunableparametersatalllevelsSCSIiSCSITCPEthernetInteractions/tradeoffswithin/betweenlevelsDynamicparameteradjustment,MemoryaCriticaliSCSIResource,InitiatorPagingtoaniSCSIdiskVMsystemMUSTNOTblockformemoryWithoutcare,standardTCPstackwillblockformemory(buffersandcontrolstructures)Withoutcare,iSCSIdatapathwillblockformemoryTargetmemorystarvationMaygetmultiplecommandsatonceMustholdmemoryuntilreceiptacknowledgedAcknowledgementmaybedelayedindefinitely,iSCSI:CPULoad,CPUutilizationisnotnegligibleBiggestpercentfromTCP/IP,notiSCSIorSCSIStandardTOEoff-loadinghelpsoutputiSCSIHBAoff-loadinghelpsinputandoutputSoftwareiSCSICRCisexpensiveforlargePDUs,CPUOverheadwithoutHBA,Interruptrate1500byteframeevery12microsecson1GE9000byteframeevery5microsecson10GEFrequentcacheflushingExtracopyingTOEshelpmainlyonoutputInputrequiresintermediateTCPbuffersorcostlymemorymapping,iSER:iSCSIExtensionsforRDMA,InterfacebetweeniSCSIandRDMAiSERadds12-byteheadertocontrolPDUsMakesiSCSIindependentofanyprotocolRDMAP/DDP/MPA/TCP/IPRDMAP/DDP/SCTP/IPInfinibandOthers?(Myrinet?,Quadrics?),iSER:Concepts,TargetcontrolsdataflowiSCSIread=targetRDMAwriteiSCSIwrite=targetRDMAread4newkeysOldkeysfordigests,markersareirrelevantHandlingofiSCSIPDUsR2T,DataOutPDUsreplacedbyRDMAreadDataInPDUsreplacedbyRDMAwriteAllotherPDUscarriedbyRDMAsend,iSCSI/iSERErrorHandling,Guaranteedreliable,in-orderdeliveriSERerrorterminatesconnection!AlliSCSIerrorrecoverylevelspossibleLevel1reducedtoalmostnothingDigestandsequenceerrorsnowimpossiblePDUretransmissiontimeoutsdiscouragedSNACKmustnolongerbesent,iSCSI:SharingaTargetDevice,MultiplehostseasilyaccesscommontargetEfficientblocktransportdirectlytodiskNonotionoffiles,directories,data,ormetadataNocontentiondetectionorresolutionNoallocationormanagementofblocks,Object-basedStorageSystem,Idea:pushmoreintelligenceontodiskunitTargetmanagesblockallocationTargetdefinesobjectsandmapstheirblocksTargetmanagesobjectmetadataEnhancementstoSCSIcommandsetMustrewritefilesystemstouseobjects,ANSIProjectT10/1355-D,SCSIObject-BasedStorageDeviceCommandsFinalRevision10,30July2004“ToprovideefficientoperationofI/Ologicalunitsthatmanagetheallocation,placementandaccessingofvariable-sizedata-storagecontainerscalledobjects.”,iSCSI:Object-store,AllOSDcommandsarebi-directional200-byteCDBrequiresuseofiSCSIAHSReadingaPDUheaderrequires2steps:Read48-byteBasicheader,extractAHSLengthReadfollowingAHSLengthbytesHeaderdigest(CRC)isproblematicUseAHSLengthvaluetoreadAHSheadersandCRCUseCRCtocheckcompleteheaderafterreaddoneIfAHSLengthhaserrorinputmayblock!,TrackingSCSIStandards,iSCSIversion0(RFC3720)basedon:SAM-2Finalrevision24,11-September-2002SBCFi