2024数据管理趋势报告 Trends in Data Management A 2024 DATAVERSITY Report

TrendsinDataManagement

A2024DATAVERSITY®Report

DonnaBurbankandMichelleKnight

TABLEOFCONTENTS

1.ExecutiveSummary5

2.ResearchandDemographics6

A.ScopeofResearch6

B.PrincipalDemographics7

1.JobFunction7

2.IndustryRepresentation8

3.CompanySize9

3.GoalsandDriversforDataManagement10

A.GoalsandDrivers10

B.ChallengesandPriorities13

4.CurrentStateofDataManagement15

A.TheScopeofDataManagement15

B.RolesDrivingDataManagement16

C.ComponentsofDataManagement17

D.ConcludingComments20

5.TechnologyandTraining21

A.ToolsandTechnology21

B.Training22

6.DataGovernanceandMetadataManagement24

A.DataGovernance24

B.MetadataManagement25

C.ConcludingComments27

7.DataArchitecture28

A.StateofDataArchitectureWithinDataManagement28

B.ValueofDataArchitecture29

C.ConcludingComments30

©2024DATAVERSITYEducation,LLC.Allrightsreserved.3

8.DataModeling31

A.StateofDataModelingWithinDataManagement32

B.DataModelingMethods33

C.TypesofModelsandDiagrams34

D.ConcludingComments35

9.DataPlatformsandStorage36

A.DataPlatforms36

B.FuturePlansforDataPlatformsandStorage37

C.ConcludingComments38

10.NewTechnologiesandTrends39

A.OrganizationalTrends39

B.FutureTrends40

C.ConcludingComments40

11.Conclusion41

12.Producers42

13.AbouttheAuthors43

4©2024DATAVERSITYEducation,LLC.Allrightsreserved.

1.EXECUTIVESUMMARY

Today’seconomyisdrivenbydata,withartificialintelligence(AI)anddigitalcommercebeinghouseholdconcepts.Notableinthisyear’ssurveyisthatthetop-of-mindconcernshavelesstodowiththetransformationalpowerofAIanddigitaltechnologies,butmorewithconcernsabouthowtogovernthedatasupportingthesetechnologiesandtoensurehigh-qualitydata.Datagovernanceandtheneedforcross-functional,data-literateteamsmanagingenterprisedataisastrongthemethroughoutthisyear’ssurvey.Keyhighlightsfromresponsesinclude:

•63%ofrespondentsfeelthattheirorganizationtreatsdataasacorporateasset.

•Gaininginsightsthroughreportingandanalyticswasthetopdriverfordatamanagement,with72%oftheresponses.

•Thenumberofdatasiloswasthetopconcern,with68%ofrespondentslistingitasanissueintheirorganization.

•Thetopinitiativesplannedinthenext1-2yearsaredatagovernance(39%)anddatastrategy(37%)asorganizationslooktodrivemorevaluefromdatawhilemaintainingtrusted,secureinformationassets.

©2024DATAVERSITYEducation,LLC.Allrightsreserved.5

2.RESEARCHANDDEMOGRAPHICS

A.ScopeofResearch

DATAVERSITY®’s2024TrendsinDataManagementSurvey(TDM)offersinsightsintobusinesses’directionsandconcernsasdatamanagementevolves.Thisreportrepresentsthesixthina

seriesstartedin2019.

Theoverallstructureandmethodologyofthe2024studyhaveremainedlikethoseofpreviousyearsfrom2019onward.Whenrelevant,wewillcomparepreviousTrendsinDataManagementsurveyswith2024’sresultstodeterminelong-termtrends.

Thisyear’ssurveyhadatotalof334respondentsfrom45countriesacrosstheglobe.

Geographically,54%ofthosesurveyedcamefromtheUnitedStates,followedby9%fromCanada,6%fromtheUnitedKingdom(U.K.),3%fromAustralia,and3%fromIndia.

Businessesofallsizesweresurveyed,alongwithavarietyofindustriesandroles.Most

respondentsworkedindatagovernancepositions.Additionally,abroadselectionofindustrieswasrepresented,includingmanufacturing,education,andretail.

Thissurveyhad26questions.Threeofthequestionswereopen-ended,withtheremainingquestionsofferingaselectionofanswersandinstructionsforsometocheckoffallanswersthatapply.

Thequestionsofferingchecked-offresponsestypicallyincludedan“Other(pleasespecify)”box,withspaceavailableforashortdescription.Thosecommentswillbeincludedwhererelevanttotheanalysis.

Surveyquestionsweresortedinto10topicsandwerefollowedbyspacesforadditionalcomments.Thesectionsare:

•GeneralDemographics(fourquestions)

•GoalsandDriversforDataManagement(fourquestions)

•TheCurrentStateofDataManagement(fourquestions)

•TechnologyandTraining(twoquestions)

•DataGovernanceandMetadataManagement(threequestions)

•DataArchitecture(twoquestions)

•DataModeling(threequestions)

•DataPlatformsandStorage(twoquestions)

•NewTechnologiesandTrends(twoquestions)

DATAVERSITYrecruitedsurveyparticipantsthroughanemailcampaignusingtheData

EducationMonthresources.Thosewhorespondedreceivedlinksforthe2024TrendsinDataManagementSurvey,andtherewasnotimelimitforansweringthequestions.

6©2024DATAVERSITYEducation,LLC.Allrightsreserved.

The334participantsdidnotreceivemonetarycompensation.Instead,theyreceivedthecompiledpreliminaryresultsfromthestudy.Pleasenotethatthesurvey’sresponsesandcommentsrepresentonlytheviewsoftheseparticipants.

B.PrincipalDemographics

First,surveyrespondentsansweredageneralquestionabouttheirjobtitleandcountryoforigin.Then,theyrepliedtothreemaindemographicquestionsregardingtheirjobfunction,theirindustry,andhowmanyemployeesworkattheircompany.Weincludemoredetailsonthesebelow.

1.JobFunction

Mostrespondentsinthe2024survey(andprevioussurveys)helddata-centricpositions[Figure1]:

Figure1:JobTitle/Function

Byprofession,thetopthreecareergroupsrespondingin2024are:

•Information/DataGovernance:35%

•Dataand/orInformationArchitecture:21%

•Consultant:11%

Theremainingrolesspannedfrombusinessexecutivestoacademic/researcherstotechnical/engineeringroles.

©2024DATAVERSITYEducation,LLC.Allrightsreserved.7

2.IndustryRepresentation

In2024,over40industrieswererepresented[Figure2]:

Figure2:Industry

Thetopfivesectorsincluded:

•Consulting:14%

•Government:12%

•Education:8%

•Finance:7%

Resultsshowthatorganizationsofalltypescanbenefitfromdatamanagement.While

historicallydatamanagementwaslimitedtoindustriessuchasfinanceandgovernment,wearenowseeingawidedistributionofindustriesofalltypes,frommanufacturingtoeducationtoretail,andmore.

8©2024DATAVERSITYEducation,LLC.Allrightsreserved.

3.CompanySize

Althoughbusinessesofmanysizesparticipatedinthesurvey,overtwo-thirdsoftherespondentscamefromcompanieswithmorethan1,000workers[Figure3]:

Figure3:NumberofEmployees

TheTop3segmentsconsistedofthefollowing:

•10,000andover:28%

•1,000–4,999:24%

•5,000–9,999:16%

Inthepast,onlythelargestorganizationswereinterestedinandabletoafforddata

management.Now,datamanagementisofinteresttoandbenefitsorganizationsofallsizes.Also,theavailabilityoftoolsandsolutions,particularlythoseinthecloud,hashelpedmakefunctionalityavailabletosmallorganizations,whichwasnotpossibleinthepast.

©2024DATAVERSITYEducation,LLC.Allrightsreserved.9

3.GOALSANDDRIVERSFORDATAMANAGEMENT

DAMAInternational’s“DataManagementBodyofKnowledge”(

DAMADMBoK

)hastherecognizedindustry-standarddatamanagementdefinition.Itsays:

“Datamanagementisthedevelopment,execution,andsupervisionofplans,policies,programs,andpracticesthatdeliver,control,protect,andenhancethevalueofdataandinformationassetsthroughouttheirlifecycles.”

WeaffirmtheauthorityoftheDAMA-DMBoKasthedefinitionusedinthisreport.Togainanunderstandingofdatamanagement’simportance,weaskedfourquestionsaboutgoals,priorities,andchallenges.

A.GoalsandDrivers

Tounderstandtheimportanceofdatamanagementtoorganizations,thesurveyasked[Figure4]:

▶“Whatareyourmainbusinessgoalsanddriversforimplementingdatamanagementinyourorganization?”

Figure4:BusinessGoalsandDrivers

Thetopfivegoalsanddriversin2024presentasfollows:

•Gaininginsightsthroughreporting&analytics:72%

•Savingcostandincreasingefficiency:63%

•Supportingdigitaltransformation:62%

•Complyingwithregulations:55%

•Reducingrisk:52%

10©2024DATAVERSITYEducation,LLC.Allrightsreserved.

Thegoalofbeingadata-drivenorganizationandmakingdecisionsbasedondataisonceagaintheleadingdriverfordatamanagement,asithasconsistentlybeenyearoveryear.Gaining

insightsthroughreportingandanalyticsisthenumberonedriverfordatamanagement,

with72%ofrespondentslistingthisasapriority.Moreandmoreorganizationsarerelyingonbusinessintelligenceandadvancedanalyticstodrivedecision-making.

Theuseofdataisnotlimitedtoreportingandanalytics,however,andorganizationsareusingdatatoimprovetheircorebusinessoperations.Thegoalofsavingcostandincreasingefficiencywasthesecondmostpopularresponse,with63%oforganizationschoosingthisresponse.

Notonlydoesstrongdatamanagementimprovetheefficiencyofdataoperationsasdata

teamsspendlesstimeonmanualorinefficienteffort,butperhapsmoreimportantly,data

improvestheefficiencyofbusinessoperationsaswell.Improvedsupplychainoperationsandpricingefficienciesaredrivenbystrongproductmasterdata.Streamlinedsupplieronboardingandoptimizedpurchasingcanbedrivenbycentralizedsuppliermasterdata.Nearlyevery

aspectoftheorganizationcanbemademoreefficientwithastrongdatafoundation.

Similarly,digitaltransformation–thethirdtopdriverwith62%oftheresponses–is

supportedbyastrongdatafoundation.Thewaysorganizationslooktousedatainthedigitalworldareasdiverseastheorganizationsthemselves.Thispushtobecomedata-drivenis

onlygrowingovertime.

Forexample,ascompanieslooktosellmoreproductonline,astrongproductmasterdatacatalogisrequired.Organizationsthatarelookingtooptimizethecustomerjourneyinthedigitalworldtypicallydosowithastrongviewof“customer360”drivenbycustomermasterdataandthedigitalthreadofcustomerinteractions.

Complyingwithregulationsandreducingriskaretopdriversthatappearseveraltimes

throughoutthesurvey.Withtheriseinregulations,particularlyarounddataprivacy(e.g.,

CCPA,GDPRetc.),organizationsarerequiredtobolstertheirdatamanagementcapabilities.Forexample,withdatasubjectaccessrequestsrequirements,organizationsneedtohavefull

traceabilityofconsumerdataandneedtoincreasetheirfocusinareassuchasdatagovernance,metadatamanagement,andmasterdatamanagement(MDM).

“Gaininginsightsthroughreportingandanalyticsisthenumberonedriverfordatamanagement,with72%ofrespondentslistingthisasapriority.”

©2024DATAVERSITYEducation,LLC.Allrightsreserved.11

Wealsowantedtoknowhoworganizationsvaluedataandasked[Figure5]:

▶“Doesyourorganizationtreatdataasacorporateasset?”

Figure5:DataasaCorporateAsset

Thissurveyquestionshowedthefollowingresults:

•Yes:63%

•No:28%

•Don’tKnow:9%

Whileit’sclearfromthepreviousquestionthattherearenumerousbenefitstotheorganizationfromdatamanagement,only63%oforganizationsfeeltheirdataistreatedasacorporateassetandmanagedaccordingly.

12©2024DATAVERSITYEducation,LLC.Allrightsreserved.

B.ChallengesandPriorities

Everyonesurveyed,includingthosewhovalueddataasanasset,faceddifficultiesachievingdatamanagementgoals.Togainabetterunderstandingofthesechallenges,weasked

participantsthefollowingquestion[Figure6]:

▶“Whatarethebiggestdatamanagementchallengesfacedbyyourorganization?”

Figure6:DataManagementChallenges

Participantschosethefollowingtopfivechallengesfor2024:

•Numberofdatasilos:68%(61%in2023)

•Dataliteracy:58%(57%in2023)

•Dataqualityissues:56%(55%in2023)

•Lackofdatagovernance:51%(50%in2023)

•Skillsshortage:41%(44%in2023)

Datasilostopthelistasthenumberonechallengefacingorganizations.Thisissuehas

increasedby7%since2023.Achievinganenterprise-wide“singleversionofthetruth”is

stillonlyavisionformany.Silosareexacerbatedbyseveralfactors:organizationalreportingstructures,softwareapplicationswithoutinteroperability,culturalfactors,andmore.Thelackofdatagovernance–whichisthefourthtopchallenge–alsocontributestotheinabilityto

resolvedatamanagementchallengesatanenterpriselevel.

©2024DATAVERSITYEducation,LLC.Allrightsreserved.13

Dataliteracyandashortageofskillsappearassecondandfifthonthelist.Asorganizations

looktobecomedata-drivenandmakestrategicdecisionsbasedondata,theyneedastrong

teamofbusinessandtechnicalstakeholderswhounderstandhowtoanalyze,manage,and

governtheorganization’sdata.Whilemanyorganizationshavestartedformaldatatrainingandawarenessprograms,skillsandknowledgearestillbehindrequiredlevels.

Dataqualityisacommonconcernthatappearsthroughoutthesurvey.Withouttrustindata,theabilitytomakedata-drivendecisionsisdiminished.Withoutastrongdatagovernanceprogram,dataqualityisdifficulttoachieve.

Tobetterunderstandthechallenges,wefollowed-upwithanopen-endedquestionaboutorganizations’prioritiesandgoalsfordatamanagementin2024-2025.

▶“Whatareyourtop2-3datamanagementpriorities/goalsfor2024-2025?”

Thetopprioritywasoverwhelminglydatagovernance,followedcloselybydataquality.Thenextmostpopularwrite-inresponsesweremetadatamanagementandthedatacatalog,aswellasmasterandreferencedatamanagement,whicharecapabilitiesthatstronglysupportdatagovernanceandquality.

“Weseeanintensificationandrenewalofdatamanagementeffortsinthefaceof

renewedregulatoryfocus.AsorganizationsincreasinglyadoptAIinitiatives,the

needforeffectivedatagovernancebecomesmorecriticaltoensurethereliability,transparency,andethicaluseofdatainAIsystems.Tosupportthisintensifiedfocusondatamanagementandgovernance,organizationscanbenefitfromadoptingdatamanagementindustrystandardsandleveragingimprovedautomatedtools.”

14©2024DATAVERSITYEducation,LLC.Allrightsreserved.

4.CURRENTSTATEOFDATAMANAGEMENT

Togainamoreholisticunderstandingofdatamanagement,DATAVERSITYaskedfourquestionsaboutorganizationalscope,roles,andcomponents.Someparticipantsleftadditional

commentsinthespaceprovided.

A.TheScopeofDataManagement

Tounderstandhowbroadlydatamanagementimpactsorganizations,thesurveyasked[Figure7]:

▶“Whatisthescopeofdatamanagementinyourorganization?”

Figure7:ScopeofDataManagement

Surveyresultsshowedthefollowing:

•Wehaveanenterprise-wideeffort:51%

•Wedonothaverobustdatamanagementpracticesinplace:21%

•Weimplementdatamanagementforindividualprojectsonly:19%

•Other(pleasespecify):7%

•Don’tknow:2%

Eventhoughtherearestrongbusinessneedsandbenefitsfromdata,only51%of

organizationsclaimtohaveanenterprise-widedataeffortinplace.Thisalignswiththe

earlierresponsethatthetopchallengeinorganizationsisdatasilos.(SeeSection3:Goals

andDriversforDataManagement)Despitethevalueofdata,orperhapsbecauseofitsvalue,departmentsandteamsacrosstheorganizationarechallengedwithworkingtogetherin

©2024DATAVERSITYEducation,LLC.Allrightsreserved.15

acohesivewaytomanagedataeffectively.Thisspeakstotheneedfordatagovernancetoalignallkeystakeholdersacrosstheorganizationandgivevoicetothedisparateneedsandprioritiesfordata.

B.RolesDrivingDataManagement

Responsibilitiesindevelopingadatamanagementprogramspanawiderangeofjob

descriptions.Togainanunderstandingofwhoisguidinganddevelopingdatamanagement,weasked[Figure8]:

▶“Whoisdrivingdatamanagementinyourorganization?”

Figure8:WhoisDrivingDataManagement?

TheTop10responsesinclude:

“Thedatagovernanceleadis

mostoftentheroledrivingdata

managementacrosstheorganization,whichhasbeentrueyearoveryear.”

•DataGovernanceLead:39%(41%in2023)

•ChiefInformationOfficer(CIO):34%(29%in2023)

•ChiefDataOfficer(CDO):30%(23%in2023)

•DataArchitect:21%(22%in2023)

•DataAnalyticsLeader:18%(19%in2023)

•BusinessStakeholders:18%(18%in2023)

•ITManager:14%(15%in2023)

•CEO:13%(11%in2023)

•ProjectManager:10%(7%in2023)

•ChiefOperatingOfficer:9%(6%in2023)

16©2024DATAVERSITYEducation,LLC.Allrightsreserved.

Thedatagovernanceleadismostoftentheroledrivingdatamanagementacrossthe

organization,whichhasbeentrueyearoveryear.Thisisnotsurprising,asthedatagovernanceleadisoftenachampionfordataacrosstheorganization,bringingtogetherbothbusinessandtechnicalstakeholders.Whilethedatagovernanceleadmightbethemostcommonroledrivingdatamanagement,itisclearfromtheresponsesthatitrequirescollaborationfromnumerousdisparaterolesworkingtogethertoimprovedatawithintheorganization.

C-levelrolessawthelargestincreasefromthepreviousyear,withrepresentationfromthe

ChiefDataOfficer(CDO)rising7%,andtheChiefInformationOfficer(CIO)seeinga5%increase.Additionally,weseetheChiefOperatingOfficer(COO)increasingby3%andtheChiefExecutiveOfficer(CEO)rising2%.Asdataisincreasinglyseenasastrategicassetusedforcompetitive

advantage,itisnotsurprisingthatmoreC-levelstakeholdersareinvolvedinitsmanagement.

C.ComponentsofDataManagement

Next,wewantedtoknowwhichdatamanagementcomponentsorganizationsusetodayandplanforinthefuture.Weasked[Figure9]:

▶“Whichofthefollowinghaveyoualreadyimplementedinyourorganization?”

Figure9:CurrentImplementationofDataManagement

©2024DATAVERSITYEducation,LLC.Allrightsreserved.17

Thetop10implementationsrankasfollows.Notethatitemswith0responseswerenotaskedin2023:

•BusinessIntelligenceandReporting:72%(65%in2023)

•DataWarehouse:64%(63%in2023)

•DataGovernance:59%(57%in2023)

•DataSecurity:55%(60%in2023)

•DataArchitecture:52%(47%in2023)

•DataStrategy:51%(46%in2023)

•DataModeling:48%(41%in2023)

•DataQuality:45%(44%in2023)

•CloudServices:42%(44%in2023)

•DataIntegration:42%(38%in2023)

Whenlookingatthekeyinitiativesthatorganizationsarecurrentlyimplementing,“BusinessIntelligence(BI)andReporting”remainthetopeffortsyearoveryear,withanincreaseof7%from2023.Datawarehousing,theunderlyingstoragebehindbusinessintelligence(BI)andreporting,isaclosesecond.

Whiledatawarehousingisthedefactostandardtosupportreporting,someorganizationsarenotyetatafulllevelofmaturitytoleverageadatawarehouseandmaybereportingdirectlyagainstsourceapplications.Thiscandriveshort-termbenefitsbuttypicallyhasdifficulty

scalingovertime.

Withthegrowthofreportingcomestheneedfordatagovernancetoensurethatreportingisbasedontrusted,well-defined,traceableinformation.Responsesshow59%oforganizationshadadatagovernanceinitiativeinplace,reporting2%increasefrom2023.

18©2024DATAVERSITYEducation,LLC.Allrightsreserved.

Thesurveyfollowedupwithparticipantsabouttheirplansforimprovementsinthefuture.Itasked[Figure10]:

▶“Whichofthefollowingareyouplanningonimplementinginthenext1–2yearsinyourorganization?”

Figure10:FutureImplementationofDataManagement

Between2025and2026,businessesplantoimplementthefollowingtop10components.Notethatitemswith0responseswerenotaskedin2023:

•

•

•

•

•

•

•

•

•

•

DataGovernance:39%(43%in2023)

DataStrategy:37%(40%in2023)

DataQuality:33%(38%in2023)

DataArchitecture:31%(32%in2023)

Self-ServiceReporting&Analytics:28%(34%in2023)

MasterDataManagement:26%(38%in2023)

MetadataManagement:25%(35%in2023)

DataScience(IncludingAIorMachineLearning):25%(26%in2023)

DataModeling:20%(26%in2023)

GenerativeAI:20%(Notaskedin2023)

“Datamanagementisverysiloedwithineachsectionofthedivision.Everyoneishandlingitdifferently,andITdoesnotallowdatastewardstheauthoritytouseproperdatamanagementtools.This

lackofconsistencyandempowermentresultsininefficientdataprocessingandmanagement.”

©2024DATAVERSITYEducation,LLC.Allrightsreserved.19

WhilethehottopicsofgenerativeAIanddatascience(includingAIormachinelearning)

didmakeittothetop10for2024,itisinterestingtonotethatthemoreconservativeand

foundationalcapabilitiesofdatagovernance(39%),dataquality(33%),anddataarchitecture(31%)werethetopconcernsbyasignificantmargin.Datastrategy(37%)wasthesecond-mostchoseninitiative.Organizationsarelookingtoprovideavisionandexecutionpaththat

balancesboththeoffensiveapproaches,suchasreportingandanalytics,andgenerativeAI,withthedefensiveapproachesofdatagovernance,dataquality,etc.

D.ConcludingComments

Write-inresponseshighlightedtheneedfordatagovernancetosupportthegrowingAIinitiatives,aswellasconsideringincreasedregulatoryfocus,particularlyaroundconsumerdata.

Commentsinclude:

•“Ever-evolvingchangeshappentothedataanddataenvironments.Thesechanges

increasetheneedforupskillingtheorganization’sdataliteracy.Wearemovingtowardstreatingdataasaproduct,withgreaterself-servicecapabilities.Thisshiftincreasestheneedtoenhancegovernance.”

•“Datamanagementisverysiloedwithineachsectionofthedivision.Everyoneishandlingitdifferently,andITdoesnotallowdatastewardstheauthoritytouseproperdata

managementtools.Thislackofconsistencyandempowermentresultsininefficientdataprocessingandmanagement.”

•“Weseeanintensificati