OSIsoft PI 时序数据库核心技术

OSIsoftOverview,DevelopmentsinthePISystemFamilyofProductsGreggLeBlancPISystemProductManager,WhyScalability?,MoredatacanbetrackedeverydayPhoneshaveGPSlocationdataReal-timemonitoringinthehomeElectricalusageSecuritysystemsFire/FlooddetectionITApplicationsEverymachineonanetworkneedsmonitoringExample:700,000homesx10datapoints=7,000,000points,Benefits,ITMonitoringcandetectIntrusionsImpendinghardwareproblemsDiagnosesoftwareproblemsVirustrafficExample:BecauseWiredCitycoulddetectandrecordtheamountofinboundtrafficfromtheNimdaworm,theygotarefundfromMCIWorldcom,WhereisPIToday?,100,000+DataStreams100sofsimultaneousclients300+Interfaces6,500+Customers1,000sofeventsrecorded/sec,IssuesWithinScalability,AscalablesystemhasseveralaspectsUsersservedCalculationsandanalysisIntegrationissuesDatacapacityDatathroughputSecurityInfrastructure,UserLoad&Integration,UsersneedfastaccesstothedataClient/ServerusersStatelessUsersClient/ServerusersconnectdirectlytoPIStatelessUsersBroaderaudiencePIICEWebservicesandintegration,ServerManager(PISDK),SafelyexposesPIdataThePITrustTableretainspointbypointsecurityConnectionmanagementbalancesuserrequests,InternetInformationServer(IIS),Browser,SecurityinICE,PIWebService,PISDK,SOAPMessages,PIEnterpriseServer,PIPointSecurity,WindowsAuthentication,(AggregatedQueries),PITrustTable,ConnectionsinICE,Elvispidemo,Nortonpiadmin,Alicepialice,Rickypiadmin,PISystem,Lucypidemo,ICE,Norton,Ricky,Alice,Elvis,Lucy,PISDKServerObjects1peruniquePIuser,1TCP/IPConnectionperserver,PISDK,SessionManager,UnderlyingWebServiceStructure,PIICEWebServicesLayer,ICEWebParts,ICEWebParts,ICEWebParts,ICEWebParts,PISDK,PIServer,OtherHistorians,RelationalDatabases,DCSSCADALIMS,LooselyCoupledApplications,TightlyCoupled,WebServicesLayer,LooselyCoupledComputing,TheideaofWebservicesAllowsforcomputinginanenvironmentthatis:AsynchronousStatelessPlatformindependentGeographicallyindependentWillreplacesome“tightlycoupled”computing,TightlyCoupledComputing,Infrastructure-leveltiesbetweensystemsUsesAPIcallsUsuallyproprietaryNotasflexibleExamples:PIInterfacestootherinformationsystemsOLEDB/ODBCconnectorsPoint-to-Point,Middleware,andotherintegration,WebServiceIntegration,LooselycoupledapplicationsbuiltonWebServiceIntegrationSpace,TraditionalTightlyCoupledIntegrationSpace,ICE,KPI,Reports,Auditing,ERP,Quality,WebServices,PIBatchWebservices,RelationalDBWebservices,PIAuditingWebservices,ERPWebservices,PIICEWebservices,WebServicesIntegrationSpace,Infrastructure,OSIsoftismovingto.NETOlderapplicationswillmigrateNewapplicationswillleveragethe.NETFrameworkManagementtoolswilluse.NETNewtoolscomingforSMT(SystemsManagementTools)PartsofthePISystemwillbeaccessiblethrough.NET,DistributedAnalysis,ServerBasedPerformanceEquationsDesktopBasedSpreadsheetsbasedonDataLinkDistributedCalculationsPIACEServerloadreducedCalculationscanbereused,SecurityConcerns,IsntMicrosoftavulnerability?Overthelastyear26vulnerabilitiesinApache22inMicrosoftsIISKeyissue:systemmaintenanceWhataboutNimda?250,000systemsin9hoursAtleast$2.4BindamagesInfectionstartedJuly13,2001PatchwasavailableJune18,2001(MS01-033),SecurityResponse,Moreaccountability:MicrosoftNIPC(NationalInfrastructureProtectionCenter)workswithNERCandMicrosofttodevelopsecurityproceduresandstandardsKeyprecautionsstillremainGuardagainstsocialengineeringSystemmaintenanceUnknownvulnerabilitiesWiFi,wardrivingUnprotectedmodemlinesusedforsupport,OSIsoftSecurity,PIPointSecurityTrustTableSingledirectionPItoPItransfersAuditingAnewdatabasethatrecordschangestoPIPointsValuesModulechanges,WhatsBeenCovered,HandlinguserloadCalculationsIntegrationdemandSecurityInfrastructureMillionpointPISystems,HistoryoftheHistorian,PI11983HP-10001985Vax-VMS1988PINetClient/ServerArchitecturePI21992Vax-VMS1994Alpha-OpenVMSPI31993DesignWork,HistoryoftheHistorian,PI3Big4UnixHP-UXIBM-AIXDec-OSF/1Sun-SolarisWindowsNT3.51DevelopmentLanguageC+,HistoryoftheHistorian,PI3KeyconceptsofPI2wereusedSnapshotCompressionArchivecacheArchivenavigationPINet,HistoryoftheHistorian,PI3KeynewconceptsMulti-processRPCBasedInter-processcommunicationPISDKHistorizationofmanydifferentdatatypesDoublesStringsBLOBsCommoncodebaseforallplatforms64bitsupport,PI3OriginalRelease,PI3.0ReleasedNovember19951,000to100,000+PointsUpto2000eventsperseconds,PI3.0toPI3.3,Howmuchcanyoudoin7years?ConcentratedonfeaturesAlarmACETotalizerPISDKBatchDatabaseModuleDatabaseAuditNTSecurityCOMConnectorsDevelopmentInfra-structureAutomatedbuildsandtestingBugfixes(justafew),Scaling,1996Typicalsystem100KpointsDatarates2000Events/Second2002Manysystemsapproaching150KpointsUpto10,000Events/Second,Scaling,1995IntelPentiumProProcessor2P200MHz2002IntelXeonProcessor4P1.6GHz,Scaling,ThePIdataratesreflectincreaseinprocessorspeed.PointCountdoesnot.Limitationisnotprocessorspeed.,LimitationsinPI3.3,Memory2GBofvirtualmemoryperprocessArchivecacheInter-processcommunicationCompressedeventsfromsnapshottoarchiveSerializationofRPCsArchivesub-systemhandlesonecallatatime,Memory,PIDatabasesarememoryresidentPointdatabaseModuledatabaseSnapshotPIArchiveCache,PI3.3CacheMemoryIssues,Cacherecordcanbequitelarge.AddingasingleeventrequiresentirecacherecordSystemsreceivingdataformostpointswillhavealargememoryfootprintLotsofcacheactivityPushingrecordsoutofmemoryReadingrecordsintomemory,PI3.4ArchiveCache,TypicalPISystemMajorityofpointsreceivedataregularlyAddeventsNeweventsreceivedinorder,andnearcurrenttime,ThePI3.4Cache,AddressesMemoryIssuesSmallermemoryfootprintMoreefficienthandlingofneweventsSameefficiencyreadingdataDesignTradeoffsMorecomplicated,3.3SnapshotArchiveInter-processCommunication,CurrentMechanismEventreceivedisprocessedbythecompressionalgorithm.Ifpassescompressioneventisaddedtoin-memoryeventqueueEventsinqueuearepackagedandsentviaanRPCtothearchiveprocess,3.3SnapshotBuffering,ArchiveunabletoprocesscompressedeventsBackupsExtremelyhighdataratesSystemproblemsSnapshotmustwriteto“eventqueue”fileArchivemustreadfromtheSnapshoteventqueueResult2physicaldiskI/Oshappenonsamefile,3.4SnapshotArchiveInter-processCommunication,ChangedphysicalfiletomemorymappedfileFilesystemfeatureMappedbySnapshotandArchiveSnapshotwritescompressedeventsto“file”Archivereadscompressedeventsfrom“file”SynchronizationtechniquesrequiredtopreventcorruptionBufferingcasehandledbyfilesystem,Limitation3:RPCSerialization,PI3.3allsub-systemsaresinglethreadedRequeststosub-systemsareserializedArchiveexample3simultaneouscallsArchiveSummaryPlotvaluesCompressedvalues,RPCSerializationSolution,MultipleThreadsThreadsaretheobviousapproachUnixandNThaveverypowerfulandeasytousethreadingmodelsConceptuallySimpleBut,easytowritebugs,Multi-ThreadinginPI3.4,PI3hadsomethreadingsincetheoriginalreleasePINetManagerSub-systemsuseathreadforreadingmessagesPI3.4Sub-systemLevelthreadmodelMain“housekeeping”threadReadthreadMessagepumpPoolofworkerthreads,Multi-ThreadinginPI3.4,PI3.4Sub-systemLevelthreadmodelEverysub-systemismulti-threadedSub-systemmustimplementlocksforoptimalperformanceWorkerthreadpoolisconfigurableRuntimeKillthreadsSuspendthreadsChangepriorityAddworkerthreadsDeleteworkerthreads,Multi-ThreadinginPI3.4,Whatdoesthisgetus?Thearchivecallfrom.Wontkillyoursystem.Whatwill10archivecallsform.dotomysystem?Longtime1=100Longtime2=1134289piar_summary(1023453,&time1,&time2,&rval,&pctgood,ARCTOTAL),PIon64BitWindows,Oneadvantage:largeraddressablememory32bit:2GB64bit:16128GBXP-64:16GB.NetEnterpriseServer:64GB.NetDataCenter:128GB,PIon64BitWindows,Disadvantages:SlowerVerysensitivetoproperlyoptimizedcodeServersonly64bitWindowsisnotintendedtoreplace32bitdesktops.O