中山大学软件学院操作系统概念上课课件第17章.ppt

Chapter17 Distributed FileSystems Chapter17Distributed FileSystems BackgroundNamingandTransparencyRemoteFileAccessStatefulversusStatelessServiceFileReplicationAnExample AFS ChapterObjectives ToexplainthenamingmechanismthatprovideslocationtransparencyandindependenceTodescribethevariousmethodsforaccessingdistributedfilesTocontraststatefulandstatelessdistributedfileserversToshowhowreplicationoffilesondifferentmachinesinadistributedfilesystemisausefulredundancyforimprovingavailabilityTointroducetheAndrewfilesystem AFS asanexampleofadistributedfilesystem Background Distributedfilesystem DFS adistributedimplementationoftheclassicaltime sharingmodelofafilesystem wheremultipleuserssharefilesandstorageresourcesADFSmanagessetofdispersedstoragedevicesOverallstoragespacemanagedbyaDFSiscomposedofdifferent remotelylocated smallerstoragespacesThereisusuallyacorrespondencebetweenconstituentstoragespacesandsetsoffiles DFSStructure Service softwareentityrunningononeormoremachinesandprovidingaparticulartypeoffunctiontoaprioriunknownclientsServer servicesoftwarerunningonasinglemachineClient processthatcaninvokeaserviceusingasetofoperationsthatformsitsclientinterfaceAclientinterfaceforafileserviceisformedbyasetofprimitivefileoperations create delete read write ClientinterfaceofaDFSshouldbetransparent i e notdistinguishbetweenlocalandremotefiles NamingandTransparency Naming mappingbetweenlogicalandphysicalobjectsMultilevelmapping abstractionofafilethathidesthedetailsofhowandwhereonthediskthefileisactuallystoredAtransparentDFShidesthelocationwhereinthenetworkthefileisstoredForafilebeingreplicatedinseveralsites themappingreturnsasetofthelocationsofthisfile sreplicas boththeexistenceofmultiplecopiesandtheirlocationarehidden Locationtransparency filenamedoesnotrevealthefile sphysicalstoragelocationLocationindependence filenamedoesnotneedtobechangedwhenthefile sphysicalstoragelocationchanges NamingSchemes ThreeMainApproaches Filesnamedbycombinationoftheirhostnameandlocalname guaranteesauniquesystem widenameAttachremotedirectoriestolocaldirectories givingtheappearanceofacoherentdirectorytree onlypreviouslymountedremotedirectoriescanbeaccessedtransparentlyTotalintegrationofthecomponentfilesystemsAsingleglobalnamestructurespansallthefilesinthesystemIfaserverisunavailable somearbitrarysetofdirectoriesondifferentmachinesalsobecomesunavailable RemoteFileAccess Remote servicemechanismisonetransferapproachReducenetworktrafficbyretainingrecentlyaccesseddiskblocksinacache sothatrepeatedaccessestothesameinformationcanbehandledlocallyIfneededdatanotalreadycached acopyofdataisbroughtfromtheservertotheuserAccessesareperformedonthecachedcopyFilesidentifiedwithonemastercopyresidingattheservermachine butcopiesof partsof thefilearescatteredindifferentcachesCache consistencyproblem keepingthecachedcopiesconsistentwiththemasterfileCouldbecallednetworkvirtualmemory CacheLocation Diskvs MainMemory AdvantagesofdiskcachesMorereliableCacheddatakeptondiskarestillthereduringrecoveryanddon tneedtobefetchedagainAdvantagesofmain memorycaches PermitworkstationstobedisklessDatacanbeaccessedmorequicklyPerformancespeedupinbiggermemoriesServercaches usedtospeedupdiskI O areinmainmemoryregardlessofwhereusercachesarelocated usingmain memorycachesontheusermachinepermitsasinglecachingmechanismforserversandusers CacheUpdatePolicy Write through writedatathroughtodiskassoonastheyareplacedonanycacheReliable butpoorperformanceDelayed write modificationswrittentothecacheandthenwrittenthroughtotheserverlaterWriteaccessescompletequickly somedatamaybeoverwrittenbeforetheyarewrittenback andsoneedneverbewrittenatallPoorreliability unwrittendatawillbelostwheneverausermachinecrashesVariation scancacheatregularintervalsandflushblocksthathavebeenmodifiedsincethelastscanVariation write on close writesdatabacktotheserverwhenthefileisclosedBestforfilesthatareopenforlongperiodsandfrequentlymodified CacheFSanditsUseofCaching Consistency Islocallycachedcopyofthedataconsistentwiththemastercopy Client initiatedapproachClientinitiatesavaliditycheckServercheckswhetherthelocaldataareconsistentwiththemastercopyServer initiatedapproachServerrecords foreachclient the partsof filesitcachesWhenserverdetectsapotentialinconsistency itmustreact ComparingCachingandRemoteService Incaching manyremoteaccesseshandledefficientlybythelocalcache mostremoteaccesseswillbeservedasfastaslocalonesServersarecontractedonlyoccasionallyincaching ratherthanforeachaccess ReducesserverloadandnetworktrafficEnhancespotentialforscalabilityRemoteservermethodhandleseveryremoteaccessacrossthenetwork penaltyinnetworktraffic serverload andperformanceTotalnetworkoverheadintransmittingbigchunksofdata caching islowerthanaseriesofresponsestospecificrequests remote service CachingandRemoteService Cont CachingissuperiorinaccesspatternswithinfrequentwritesWithfrequentwrites substantialoverheadincurredtoovercomecache consistencyproblemBenefitfromcachingwhenexecutioncarriedoutonmachineswitheitherlocaldisksorlargemainmemoriesRemoteaccessondiskless small memory capacitymachinesshouldbedonethroughremote servicemethodIncaching thelowerintermachineinterfaceisdifferentformtheupperuserinterfaceInremote service theintermachineinterfacemirrorsthelocaluser file systeminterface StatefulFileService MechanismClientopensafileServerfetchesinformationaboutthefilefromitsdisk storesitinitsmemory andgivestheclientaconnectionidentifieruniquetotheclientandtheopenfileIdentifierisusedforsubsequentaccessesuntilthesessionendsServermustreclaimthemain memoryspaceusedbyclientswhoarenolongeractiveIncreasedperformanceFewerdiskaccessesStatefulserverknowsifafilewasopenedforsequentialaccessandcanthusreadaheadthenextblocks StatelessFileServer Avoidsstateinformationbymakingeachrequestself containedEachrequestidentifiesthefileandpositioninthefileNoneedtoestablishandterminateaconnectionbyopenandcloseoperations DistinctionsBetweenStatefulandStatelessService FailureRecoveryAstatefulserverlosesallitsvolatilestateinacrashRestorestatebyrecoveryprotocolbasedonadialogwithclients orabortoperationsthatwereunderwaywhenthecrashoccurredServerneedstobeawareofclientfailuresinordertoreclaimspaceallocatedtorecordthestateofcrashedclientprocesses orphandetectionandelimination Withstatelessserver theeffectsofserverfailuresandrecoveryarealmostunnoticeableAnewlyreincarnatedservercanrespondtoaself containedrequestwithoutanydifficulty Distinctions Cont Penaltiesforusingtherobuststatelessservice longerrequestmessagesslowerrequestprocessingadditionalconstraintsimposedonDFSdesignSomeenvironmentsrequirestatefulserviceAserveremployingserver initiatedcachevalidationcannotprovidestatelessservice sinceitmaintainsarecordofwhichfilesarecachedbywhichclientsUNIXuseoffiledescriptorsandimplicitoffsetsisinherentlystateful serversmustmaintaintablestomapthefiledescriptorstoinodes andstorethecurrentoffsetwithinafile FileReplication Replicasofthesamefileresideonfailure independentmachinesImprovesavailabilityandcanshortenservicetimeNamingschememapsareplicatedfilenametoaparticularreplicaExistenceofreplicasshouldbeinvisibletohigherlevelsReplicasmustbedistinguishedfromoneanotherbydifferentlower levelnamesUpdates replicasofafiledenotethesamelogicalentity andthusanupdatetoanyreplicamustbereflectedonallotherreplicasDemandreplication readinganonlocalreplicacausesittobecachedlocally therebygeneratinganewnonprimaryreplica AnExample AFS Adistributedcomputingenvironment Andrew underdevelopmentsince1983atCarnegie MellonUniversity purchasedbyIBMandreleasedasTransarcDFS nowopensourcedasOpenAFSAFStriestosolvecomplexissuessuchasuniformnamespace location independentfilesharing client sidecaching withcacheconsistency secureauthentication viaKerberos Alsoincludesserver sidecaching viareplicas highavailabilityCanspan5 000workstations ANDREW Cont Clientsarepresentedwithapartitionedspaceoffilenames alocalnamespaceandasharednamespaceDedicatedservers calledVice presentthesharednamespacetotheclientsasanhomogeneous identical andlocationtransparentfilehierarchyThelocalnamespaceistherootfilesystemofaworkstation fromwhichthesharednamespacedescendsWorkstationsruntheVirtueprotocoltocommunicatewithVice andarerequiredtohavelocaldiskswheretheystoretheirlocalnamespaceServerscollectivelyareresponsibleforthestorageandmanagementofthesharednamespace ANDREW Cont ClientsandserversarestructuredinclustersinterconnectedbyabackboneLANAclusterconsistsofacollectionofworkstationsandaclusterserverandisconnectedtothebackbonebyarouterAkeymechanismselectedforremotefileoperationsiswholefilecachingOpeningafilecausesittobecached initsentirety onthelocaldisk ANDREWSharedNameSpace Andrew svolumesaresmallcomponentunitsassociatedwiththefilesofasingleclientAfididentifiesaVicefileordirectory Afidis96bitslongandhasthreeequal lengthcomponents volumenumbervnodenumber indexintoanarraycontainingtheinodesoffilesinasinglevolumeuniquifier allowsreuseofvnodenumbers therebykeepingcertaindatastructures compactFidsarelocationtransparent therefore filemovementsfromservertoserverdonotinvalidatecacheddirectorycontentsLocationinformationiskeptonavolumebasis andtheinformationisreplicatedoneachserver ANDREWFileOperations AndrewcachesentirefilesformserversAclientworkstationinteractswithViceserversonlyduringopeningandclosingoffilesVenus cachesfilesfromVicewhentheyareopened andstoresmodifiedcopiesoffilesbackwhentheyareclosedRea