数据生命周期模型和概念

CEOS

WorkingGrouponInformationSystemsandServices

DataStewardshipInterestGroup

DataLifeCycleModelsandConcepts

Version1.0

Doc.Ref.:CEOS.WGISS.DSIG.TNO1

Date:26September2011

Issue:1.0

Version7.0August10,2011

INTRODUCTION

Thisisacompilationofdatalifecyclemodelsandconceptsassembledinparttofulfill

CommitteeonEarthObservationSatellites(CEOS)WorkingGrouponInformationSystemsand

Services(WGISS)andtheU.S.GeologicalSurvey(USGS)CommunityforDataIntegration

DataManagementBestPracticesneeds.Itisintendedtobealivingdocument,whichwillevolve

asnewinformationisdiscovered.

CONTENTS

1.DigitalCurationCentre(DCC)LifecycleModel

2.EllynMontgomery,USGS,DataLifecycleDiagram

3.FGDCStagesoftheGeospatialDataLifecyclepursuanttoOMBCircularA-16

4.UniversityofOxfordResearchDataManagementChart

5.NOAAEnvironmentalDataLifeCycleFunctions

6.OpenArchivalInformationSystem(OAIS)Framework

7.USGSScientificInformationManagementWorkshopVocabulary

8.PeterFoxLifecycleDiagrams

9.NationalScienceFoundation

10.NDIIPPPreservingOurDigitalHeritage

11.WhatResearchersWant

12.EPAProjectLifeCycle

13.IWGDD'sDigitalDataLifeCycleModel

14.ScientificDataManagementPlanGuidance

15.LinearDataLifeCycle

16.GenericScienceDataLifecycle

17.CassandraLadinoHybridDataLifecycleModel

18.RayObuchDataManagement-ALifecycleApproach

19.USGSDataManagementPlanFramework(DMPf)-Smith,Tessier,andMcHale

20.BLMDataManagementHandbook

21.ARLJointTaskForceonLibrarySupportforE-Science

22.U.S.DepartmentofHealthandHumanServicesKeyComponents

23.ICPSRPreservationovertheDataLifeCycle

24.WilliamMichenerDataONE:DataLifeCycleManagement

25.IBMAspectsofLifecycleManagement-Research

26.UniversityofCaliforniaSanDiegoDigitalCurationProgram

27.UniversityofMiamiScientificDataLifecycle

28.ManagingResearchDataLifecyclesthroughContext

29.PurdueUniversityResearchRepositoryCollaborativeModel

30.DataLifecycleManagementatDell

31.JohnFaundeen&EllynMontgomery“Spins”

32.DataTrainModel1

33.DataTrainModel2

34.SteveTessierDataManagementCycles

35.PeterFoxFullLifeCycleofData

36.GuidetoSocialScienceDataPreparationandArchiving:BestPracticeThroughout

theDataLifeCycle

37.SteveTessierDataandSystemLifecycleModels

38.UniversityofVirginiaLibrary

39.MarioValle

40.MichiganStateUniversityRecordsLifeCycleModel

41.GeoMAPPGeoarchivingProcessLifecycle

42.JeffdeLaBeaujardiere'sproposedDataManagementFramework

43.SarahDembContinuummModel

44.USGSDataLifeCycleModelElements

1.THEDIGITALCURATIONCENTREMODEL

TheDigitalCurationCentreisbasedinGreatBritain.TheURLfbrtheinformation

belowishttp://www.dcc.ac.uk/resources/curation-lifecycle-model

OurCurationLifecycleModelprovidesagraphical,high-leveloverviewofthestages

requiredfbrsuccessfulcurationandpreservationofdatafrominitialconceptualisationor

receipt.Youcanuseourmodeltoplanactivitieswithinyourorganisationorconsortium

toensurethatallofthenecessarystepsinthecurationlifecyclearecovered.Itis

importanttonotethatthemodelisanideal.Inreality,usersofthemodelmayenterat

anystageofthelifecycledependingontheircurrentareaofneed.Forinstance,adigital

repositorymanagermayengagewiththemodelfbrthisfirsttimewhenconsidering

curationfromthepointofingest.Therepositorymangermaythenworkbackwards

torefinethesupporttheyofferduringtheconceptualisationandcreationprocessesto

improvedatamanagementandlonger-tenncuration.

CONCEPTUAUSE

KeyelementsoftheDCCCurationLifecycleModelData

Data,anyinformationinbinarydigitalform,isatthecentreoftheCurationLifecycle.Thisincludes:

DigitalObjects:simpledigitalobjects(discretedigitalitemssuchastextfiles,imagefilesorsoundfiles,

alongwiththeirrelatedidentifiersandmetadata)orcomplexdigitalobjects(discretedigitalobjectsmade

bycombininganumberofotherdigitalobjects,suchaswebsites)

Databases:structuredcollectionsofrecordsordatastoredinacomputersystem

FullLifecycleActions

DescriptionandRepresentationInformation

Assignadministrative,descriptive,technical,structuralandpreservationmetadata,usingappropriate

standards,toensureadequatedescriptionandcontroloverthelong-term.Collectandassignrepresentation

informationrequiredtounderstandandrenderboththedigitalmaterialandtheassociatedmetadata.

PreservationPlanning

Planfbrpreservationthroughoutthecurationlifecycleofdigitalmaterial.Thiswouldincludeplansfbr

managementandadministrationofallcurationlifecycleactions.

CommunityWatchandParticipation

Maintainawatchonappropriatecommunityactivities,andparticipateinthedevelopmentofshared

standards,toolsandsuitablesoftware.

CurateandPreserve

Beawareof,andundertakemanagementandadministrativeactionsplannedtopromotecurationand

preservationthroughoutthecurationlifecycle.

SequentialActions

Conceptualise

Conceiveandplanthecreationofdata,includingcapturemethodandstorageoptions.

CreateorReceive

Createdataincludingadministrative,descriptive,structuralandtechnicalmetadata.Preservationmetadata

mayalsobeaddedatthetimeofcreation.

Receivedata,inaccordancewithdocumentedcollectingpolicies,fromdatacreators,otherarchives,

repositoriesordatacentres,andifrequiredassignappropriatemetadata.

AppraiseandSelect

Evaluatedataandselectfbrlong-termcurationandpreservation.Adheretodocumentedguidance,policies

orlegalrequirements.

Ingest

Transferdatatoanarchive,repository,datacentreorothercustodian.Adheretodocumentedguidance,

policiesorlegalrequirements.

PreservationAction

Undertakeactionstoensurelong-termpreservationandretentionoftheauthoritativenatureofdata.

Preservationactionsshouldensurethatdataremainsauthentic,reliableandusablewhilemaintainingits

integrity.Actionsincludedatacleaning,validation,assigningpreservationmetadata,assigning

representationinformationandensuringacceptabledatastructuresorfileformats.

Store

Storethedatainasecuremanneradheringtorelevantstandards.

Access,UseandReuse

Ensurethatdataisaccessibletobothdesignatedusersandreusers,onaday-to-daybasis.Thismaybein

theformofpubliclyavailablepublishedinformation.Robustaccesscontrolsandauthenticationprocedures

maybeapplicable.

Transform

Createnewdatafromtheoriginal,forexample:

bymigrationintoadifferentformat,or

bycreatingasubset,byselectionorquery,tocreatenewlyderivedresults,perhapsfbrpublication

OccasionalActions

Dispose

Disposeofdata,whichhasnotbeenselectedfbrlong-termcurationandpreservationinaccordancewith

documentedpolicies,guidanceorlegalrequirements.

Typicallydatamaybetransferredtoanotherarchive,repository,datacentreorothercustodian.Insome

instancesdataisdestroyed.Thedata*snaturemay,fbrlegalreasons,necessitatesecuredestruction.

Reappraise

Returndatawhichfailsvalidationproceduresforfurtherappraisalandre-selection.

Migrate

Migratedatatoadifferentformat.Thismaybedonetoaccordwiththestorageenvironmentortoensure

thedata'simmunityfromhardwareorsoftwareobsolescence.

2.THEELLYNMONTGOMERY,USGS,DATALIFECYCLEDIAGRAM

Providedviaemail18November2010.

Manyofthediagramsouttherehaveappealingelements,butIcouldn'tfindoneIreallylikedsosketched

somethingthat'sacombinationofsomeofTedHabennann^ideasandthematerialat:

http://imageweb.zoo.ox.ac.uk/wiki/index.php/ADMIRAL_Data_Management_PIan_Template.

TheideasItriedtocaptureare:

1)ifsanon-linear(andperhapsmulti-threaded)process

2)multipleloopsorphases(notrestrictedtothenumberdrawn)thatmayoverlapareneeded

3)partsoftheprocessareongoing

4)there'satransitionbetweendataprovideranddatacuratorsomewhereinthemiddleofthe

progressionthismayvarybetweentypesofdataandtheeventualavenueforpublicationanddistribution

3.FGDCStagesoftheGeospatialDataLifecyclepursuanttoOMBCircularA-16

FigureA1belowshowstheFGDClifecyclemodel,whichadvocatescomplianceofOfficeofManagement

andBudget(OMB)CircularA-16,"CoordinationofGeographicInformationandRelatedSpatialData

Activities.nThisframeworkencourages“timelyandhigh-qualitygeospatialdatatosupportbusiness

processesandoperations;strongerpartnershipsacrossalllevelsofgovernmentand,whenappropriate,

theprivatesector,toincreasecostefficiencyandreturnoninvestment;andimprovestrategiesfor

completingandmaintainingnationallysignificantthemesanddatasetsassociatedwithOMBCircularA-16

toenhanceservicetocitizensM(FGDC,2010).

—HarnessingthePowerofDigitalData:TakingtheNextStep.IIScientificDataManagement(SDM)for

GovernmentAgencies:ReportfromtheWorkshoptoImproveSDM.WorkshopheldJune29-July1,2010,

WashingtonD.C.March2011.ReportNo.CEND1/2011-1.Co-sponsoredbytheEnvironmentalProtectionAgency

(EPA),CENDI(TheFederalSTIManagersGroup),andtheInteragencyWorkingGrouponDigitalData(IWGDD)

4.UniversityofOxfordResearchDataManagementChart

Foundathttp://www.admin.ox.ac.uk/rdm/

ResearchDataManagement

Goodpracticeindatamanagementisoneofthecoreareasofresearchintegrity,ortheresponsibleconduct

ofresearch.Thefollowingdiagramprovidesfurtherinsighttosomeofthestagesinvolvedinresearchdatamanagement,andthe

facilitiesandservicesavailabletohelp,bothwithintheUniversityandfromexternalproviders

Data

DepositManagement

yourPlanning

dataChecklist

ToshareFunder

ornotto7

share?

Training,

DAdviceand

DataContentofData

SecuritySupportiManagement

Plans

DataEthical$

BackupandLegal

Considerations

Organisation

and

Documentation

5.NOAAEnvironmentalDataLifeCycleFunctions

ProvidedbyPeterSteurer,NOAA,NationalClimaticDataCenter,2009.

NOAAEnvironmentalDataLifeCycleFunctions

Governance,RequirementsManagement,ArchitectureManagement,Secunty

Developingandmaintainingrichmetadatatoaccompanythedata2

Establishingmechanjsmsthatallowforuserrequirementsandfeedback1

tewardshipOverarchesObservingOperations,Archive2,Access2,Use

PlanningofNew-Allongoing,iterativeprocessesthatimprove:

ObservingorDataa)data&metadatacontent(includesreprocessingdata)2

Management

Systemsb)access&userunderstanding2

-RequirementsDefinitionUse

-AnalysisofAlternativesObserving

23

23Access1-Integrationw/other

-SystemDesignOperationsArchive

information(NOAA,

-ActualObservation-Discovery

-Integrationwith-Ingest(Receipt)others)

observingsystems(catalogs,registries,

-Transmission/-ArchivalStorage-Assimilationinto

(NOAA,Interagency/metadata)

Processing/QA-Datamanagementmodels

State,International)

-Integrationwithother(populatingcatalogs,-Disseminationto

Users(webservices,-Productcreation

-Determiningwhattodatatocreateproductsregistries,metadata)

archiveandassociatedlegacysystems,-Makedecisions

(e.g.,models)-Preservation

fundingstandardformats)(Policy,emergency,

-Disseminationtoreal­

Planning(migrationothers)

-Buy/buildtimeSubscribers,tonewtechnologies)

-Scientificdiscovery

-DeliverytoArchive1-FeedbacktoNOAA

1DefinedbyISO15489-1InformabonandDocumentation-Records^nogement(2001)

2DefinedbyNRCEnvironmentalDataMonagementatNOAA-Archiving.Stowardship,andAccess(2007)

3DefinedbyISO14721.ReferenceModelforanOpenArchivalInformationSystem(OAIS)(20021

6.OpenArchivalInformationSystem(OAIS)Framework

IntroductionbyLarryBaumefromNARA.OAISReferencemodelfoundat

/publications/archive/650x0bl.pdf

OAISFrameworkforDevelopinga"CommunityofPractice"

LarryBaume

NARA

October27,2009

OAISFramework

•ISOStandard14721:2002

•Conceptualframeworkdescribingtheenvironment,functionalcomponents,and

informationobjectswithinasystemresponsibleforthelong-termpreservation

ofdigitalmaterials

•Lifecyclemodelfordataarchives

•Widelyrecognizedinscientific,datamanagement,andarchivalcommunities

•IntegrateswithotherISOstandardssuchatISO9000,ISO15489

OAISFramework

P

R

O

D

U

C

E

R

TRACReport

•TrustedRepositoryAuditandCertificationmethodology,2007

•OCLC,DPC,andNARA

•RestructuredandadoptedinOct.2009asCCSDSRecommendedPractice

(RedBook)

•NestswithOAISFramework

•Widelyrecognizedinarchivalandlibrarycommunities

CCSDSRedBook

•Auditandcertificationcriteriaformeasuring"trustworthiness"ofadigital

repository

•Listscriteriaandevidenceneededtosupportthecriteria

•Assumesanexternalcertificationprocesswhichdoesnotexistnow

•Canbeusedasaself-assessmentguide

DRAMBORA

•"DigitalRepositoryAuditMethodologyBasedonRiskAnalysis,"Feb.2007

•DigitalConservationCentre(DCC)andDigitalPreservationEurope(DPE),

•Proposesamethodologicalframework,guidelinesandaudittoolstosupportthe

identification,assessmentandmanagementofrisksinadigitalrepository

•Self-assessmentchecklist,nocertificationatthistime

•IntegrateswithOAISframework

Challenges

•Nointegrated"communityofpractice"forpromotingtrustedrepositories

•CCDSCRedBookandDRAMBORAarenotwidelyknowninthescientific,IT,

anddatamanagementcommunities

•Theydonotaddressdesignofdatasystemsanddatamanagement,e.g.

systemsplanning,funding,systemrequirements

•NoCaseStudiesandLessonsLearned

7.USGSScientificInformationManagementWorkshopVocabulary

FromTomGuntherandDaveGovoni,USGS,2006.

AVocabularyforScientificInformationandKnowledgeManagement

Producerperspective

CollectAnalyzePublishPreserve

referstoreferstoreferstorefersto

FieldworkPreparation&

Analysis,synthesisPreservation&

(insitu,invitro,distribution

&interpretationarchiving

insilico)(viaanymedium)

includesincludesincludesincludes

Direct&remoteLaboratoryRecords

Publications,data,

observation,experiments,management,

talks,seminars,

monitoring&modeling,datarescue,physical

models,libraries

recordingvisualizationsamplepreservation

Consumerperspective

FindUnderstand

IIII

referstoreferstoreferstorefersto

AcquisitionEvaluationUsefor/lntegration

Resourcediscovery

(contact,determine(assessrelevance,intostudies,models,

(search,browse,

restrictions,access,&quality,significance,visualizations,

mine)

download,etc.)suitability)experiments,etc.

dependsondependsondependsondependson

Catalogs;controlledDocumentedContextual

Documented

vocabularies,exchangeformats&information,e.g.,

dataschema,service

ontologies;geospatialprotocols;documentation,

models,protocols,

topical&preservationadministrativereports;quality&

etc.

metadata&legalmetadatafunctionalitymetadata

Source:Govoni,D.L.andT.M.Gunther,2006.ScientificInformationManagementattheU.S.Geological

Survey:Issues,Challenges,andaCollaborativeApproachtoIdentifyingandApplyingSolutions

(Abstract).Geoinformatics2006—Abstracts.ScientificInvestigationsReport2006-5201,p.19-20.U.S.

GeologicalSurvey,Reston,Virginia.

8.PeterFoxLifecycleDiagrams

PresentedbyPeterFox,RennsselaerPolytechnicInstituteatUSGSCD!August13,2010

Denver,CO.

Needtobehere

RawVBUA

Level0

AppliesPlFacility^

_______Sensor

-Designer

一一一"Instrument

Addsred,oDeveloper,

Deployer

CalibratonApplies

Standards

^AlQtyithfY^

AddsManager,

Auditor

Analyst:eedsInto

Asamilaton

Syntne***AppliesOften'get'

i+rm

Contextua-zat»

AggregaicnManager,the(meta)data

StAfxUrd*Auditor.

^AJgonttvnsAnalysthere

Adds

Toomany,getit

here

Compacson

loterpretatxy

Artifacts(oftenfiles)Roles

Rawvwe

ProcessesLevel0

FeedsIntoAppliesPI.Facility,

QualityControlSensor

enerates_________Designer

Instrument

Deliveredto

Developer,

Deployer

ButbacktoAnalystFeedsInto

/CalibratonXApplies

(StandardsJGenerites________PI,

gHh吧，-xData

realityto.

AddsJ,Manager.

Level2Auditor

FragmentationAnalystFeedsInto

Aesf1.^1--

Comparison

Syntne«<fApplies

Generates

DisconnectionData

.a

AggregationManager.

StandardsAuditor,

Analyst

EncapsulationDanved