高能物理系塔塔基础研究所

GRID:ComputingWithoutBordersKajariMazumdarDepartmentofHighEnergyPhysicsTataInstituteofFundamentalResearch,Mumbai.Soft-computingworkshopUniversityofMumbai,December1,2009Disclaimer:Iamonlyaphysicistwhoseresearchfieldinduces&utilizescutting-edgetechnologyIhavemostlyborrowedslidesfromvariousresourcesPlanoftalkGridconceptinsimpletermsRequirementsoftoday’sscientificcommunityEvolutionofGridLHCComputingGrid

TIFRgridcomputingcentreDAEcontributionsOutlookGridcomputinginsimplewords

Gridisanutilityorinfra-structureforcomplex,hugecomputations,whereremoteresourcesareaccessiblethroughweb(internet),fromdesktop,laptop,mobilephone.

Itissimilartopowergrid,wheretheuserdoesnothavetoworryaboutthesourceofthecomputingpower.Imaginemillionsofcomputersownedbyindividuals,institutesfromvariouscountriesacrosstheworldconnectedtoformasingle,huge,super-computer!

Thistechnology,developedsincelastonlyonedecade,isbeingusedpresently,by

HighenergyphysiciststoanalyzedatatobeproducedverysooninLHCexperimentwhereIndianscientistsaretakingpart.

EarthscientiststomonitorOzonelayeractivity(dealsdailywithDatawhosevolumeisequivalentto150CDs).Itisthenaturalevolutionofinternetfacility.1.SharemorethaninformationData,computingpower,applicationsindynamic,multi-institutional,virtualorganizations(IanFoster:AnatomyofGrid)2.Efficientuseofresourcesatmanyinstitutes.Peoplefrommanyinstitutionsworkingtosolveacommonproblem(virtualorganisation).3.Joinlocalcommunities.4.Interactionswiththeunderneathlayersmustbetransparentandseemlesstotheuser.FromWebtoGridComputingComputingrequirementsHigh-endcomputingapplicationWeatherforecastSharedatabetweenthousandsofscientistswithmultipleinterestsLinkmajorandminorcomputercentresEnsurealldataaccessibleanywhere,anytimeGrowrapidly,yetremain

reliable

formorethanadecadeCopewith

differentmanagementpolicies

ofdifferentcentresEnsure

datasecurityBeupandrunningroutinelyNeedtocheckuphealthoffaciltyon24X7basis.Ahugemanpowerisatworkinvisibly.ChallengesinscientificcomputationsEver-increasingdemandPCofearly2000eraisasfastasofsupercomputersof1990’s.Still,formanyapplicationitisnotadequate!

userscontinuetobuynewmachines!StorageavailableinaPCcouldnotbethoughtofduring1990’s

storagecapacitydoublesevery12monthorso!RecentyearsofthisdecadeisseeingmammothscientifcprojectswheredatasizeisseveralPetabytesperyear.ToworkwithacolleagueevenacrossacampusonPetabytescaleweneedultrafastnetwork.EventhoughCPUpower,discstorage,communicationspeedcontinuetoincrease,computingresourcesarefailingtosatisfyusers’demands,theyaredifficulttouse.SupercomputerParallelcomputerPCclusters,multiplePCsClusters:PrimaryITinfrastructuresClustersreplacetraditionalcomputingplatformsandcanbeconfiguredaccordingtotheneedNetworkloaddistributionandloadbalanceHighavailability,Highperformance/computationintensive,..IssuesrelatedtobuildingclustersScalabilityofinterconnectionnetworkScalabilityofsoftwarecomponents(libraries,applications,..)Auto-installation,clustermanagement,trouble-shooting,…Spacemanagement(desktop/rackmounted)Layoutofnodes,noises,cablelayout,cooling,..PowermanagementCentralizedinfrastructuremanagementsoftwarePerformance/Price/Powerconsumption

Costofownershipnotverylow!

PeertoPeer(P2P)computingComputingbasedonideaofsharingdistributedresourceswitheachotherwithorwithoutthesupportfromaserver

Therearemanyunder-utilisedresourcesWithpowerfulpcs,realutilisationtodayis<10%

Inlargeorganizations,withthousandsofPCs,increasingdaybyday

utilisethatincyclestealingmode!Totaldeliveredpoweris>fewMflopsTotalavailablefreediskspace>100TerabytesLatencyandbandwidthofLANenvironmentisquiteadequateforp2pcomputingmostly.Spaceisalsonotaproblem,keepthePCswherevertheyare!InternetcomputingTodayyoucannotrunyourjobsontheinternet.InternetcomputingusingidlePC’sisbecominganimportantcomputingplatform(Seti@home,Napster,..)wwwisthepromisingcandidateforcorecomponentofwide-areadistributedcomputingenvironment.Efficientclient/servermodels/protocolsTransparentnetworking,navigation,GUIwithmultimediaaccessanddisseminationfordatavisualization.Mechanismfordistributedcomputing:CGI,JavaWithimprovedprice/performanceandopensource,freesoftware,web-services,itisbecomingeasytodeveloplooselycoupleddistributedapplications.WorkingtogetherapartEssentialsofGRIDcomputingVirtualorganizations:GRIDTIFR

GridComponentsGridoverviewHowdoesgridwork?GRIDportal/GatewayGridServices:gridmiddlewareLHCandtheGRIDComputingApathologistusesamicroscopetoexaminebloodcells,ofsizeaboutonethousandthofamm,ie,10-6mHighenergyprobesstructureoffundamentalmatter.LHCwillcollidevery,veryhighenergyprotonsforthispurpose.Mammoth,verycomplexdetectors(length30m,dia20m)arethetechnicaleyeofseveralthousandscientiststoprobethesmallestlengthscale.ComplexityofLHCexperimentsWhen2veryhighenergyprotonswillcollideatLHC,mostlythesituationinthedetectorwillbelikethis,verycrowded.About10millionelectricalsignalswillhavetoberecordedintinyfractionofasecond,repeatedlyforalongtime(about10years).Usingcomputers,adigitalimageiscreatedforeachsuchinstance.Imagesizeisabout2MBonaverage,butvariesconsiderably.Butmostofthesepicturesarenotinteresting!Goodthingsarealwaysrare!InLHCexperimentthetaskofthescientistis,to

Lookforaninstancewithpatternsofthistypefrom10thousandBillion(1013)crowdedpictures.Thispicturecontainstheclueaboutouruniverse.Suchajobislike,searchingforaneedleinamillionhaystacks!

Similartolookingforaparticularpersoninathousandworldpopulationsoftoday(6Billion,India’spopulation1.2Billion)Asinglecomputingsystemwillneverscaleuptothechallenge.ConceptofGRIDcomputingdevelopedfromsuchrequirementsLHCwillcollide6-8hundredmillionproton-on-protonpersecondforseveralyears.Only1in20thousandcollisionswillhaveanimportanttaletotell,butwedonotknowwhichone!

sowehavetosearchthroughallofthem!

Hugetask!15PBytes(1015bytes)

ofdataayearAnalysisrequires~100,000computerstogetresultsinreasonabletime.GRIDcomputingisessentialInhardnumbersLHC-CERNDAEcollaboration175Mbps100Mbps100Mbps100MbpsCAF450MB/s(300Hz)30-300MB/s(ag.800MB/s)~150kjobs/day~50kjobs/day50-500MB/s10-20MB/sCMSdetectorTier-0PromptReconstructionArchivalofCopyofRawandFirstRECOdataCalibrationStreams(CAF)DataDistribution

Tier-17Tier-1sRe-ReconstructionSkimmingSecondArchivalofRAWServedCopyofRECOArchivalofSimulationDataDistribution

Tier-2~50Tier-2sPrimaryResourcesforPhysicsAnalysisandDetetectorStudiesbyusersMCSimulation

Tier-1WLCGComputingGridInfrastructureThewayCMSusestheGRID(WLCG)TIER-3TIER-3TIER-3TIER-3100MB/sCMSinTotal:1Tier-0atCERN(GVA)7Tier-1son3continents50Tier-2son4continents33P.Kreuzer-GRIDComputing-MumbaiTier0Tier1NationalcentresTier2RegionalgroupsDifferentUniversities,InstitutesIndividualscientist’sPCExperimentalsiteCERNcomputercentre,GenevaASIA(Taiwan)IndiaChinaKoreaPakistanFranceItalyGermanyUSATIFRDelhiU.PanjabU.UsefulmodelforParticlePhysicsexperiments,butnotnecessaryforothersT2_IN_TIFRTiered/Layered

Structure

connectingcomputersacrosstheglobeHardwareatTIFRsite:T2_IN_TIFRAbout50users/scientistsatpresent,stillgrowing.AnothersimilarTier2centreinKolkataforadifferentexperimentatLHC.GridfacilityhasbeenfunctionalatTIFRforalmostayear.TheCMScollaborationatLHC,CERNhasbeenusingthecomputerresources.Storage:350TB300workernodes.Internetbandwidth:1GBps.

Tobeupgradedinnearfuture.Note,continuousmonitoringessential,wearemanageingwith5engineers,notallarefulltime.Networking,GRIDMiddleware,SitesGRIDMiddlewareServices-StorageElementsComputingElementWorkloadManagementSystemLocalFileCatalogInformationSystemVirtualOrganisationManagementServiceInter-operabilitybetweenGRIDsEGEE,OSG,NorduGriD..NetworkingSiteSpecificities,e.g.Storage/BatchsystemsatCMSTier-1s:Storage:dCache/CastordCache/HPSSdCache/CastorCastor+dCache/ChimeraEnstore Enstore Storm TSMBatch:Condor Torque/MauiBQSTorque/MauiTorque/MauiLSFPBSProFNALRALCCIN2P3PICASGCINFNFZK36P.Kreuzer-GRIDComputing-MumbaiDataTransfersfrom/toTIFRTIFR

T1TransferQuality(last3months):improving,aimforstability:TIFR

T1productiontransfers(lastyear):modestbutreadytogrow!TIFR

ASGC8TBMCdata(custodialstorage)T1

TIFRTot37TBfrom7T1s