生成式人工智能重置：2024年重新布线将潜力转化为价值（英）-2024.3

March2024

Mcsey

Quartery

AgenerativeAIreset:

Rewiringtoturnpotentialintovaluein2024

ThegenerativeAIpayoffmayonlycomewhencompaniesdodeeperorganizationalsurgeryontheirbusiness.

byEricLamarre,AlexSingla,AlexanderSukharevsky,andRodneyZemmel

It’stimeforagenerativeAI(genAI)reset.Theinitialenthusiasmandflurryofactivityin2023isgivingwaytosecondthoughtsandrecalibrationsascompaniesrealizethatcapturinggenAI’senormouspotentialvalueisharderthanexpected.

With2024shapinguptobetheyearforgenAItoproveitsvalue,companiesshould

keepinmindthehardlessonslearnedwithdigitalandAItransformations:competitiveadvantagecomesfrombuildingorganizationalandtechnologicalcapabilitiestobroadlyinnovate,deploy,andimprovesolutionsatscale—ineffect,rewiringthebusinessfor

distributeddigitalandAIinnovation.

CompanieslookingtoscoreearlywinswithgenAIshouldmovequickly.ButthosehopingthatgenAIoffersashortcutpastthetough—andnecessary—organizationalsurgery

arelikelytomeetwithdisappointingresults.Launchingpilotsis(relatively)easy;gettingpilotstoscaleandcreatemeaningfulvalueishardbecausetheyrequireabroadsetofchangestothewayworkactuallygetsdone.

Let’sbrieflylookatwhatthishasmeantforonePacificregiontelecommunications

company.ThecompanyhiredachiefdataandAIofficerwithamandateto“enablethe

organizationtocreatevaluewithdataandAI.”ThechiefdataandAIofficerworkedwith

thebusinesstodevelopthestrategicvisionandimplementtheroadmapfortheusecases.Afterascanofdomains(thatis,customerjourneysorfunctions)andusecaseopportunitiesacrosstheenterprise,leadershipprioritizedthehome-servicing/maintenancedomainto

pilotandthenscaleaspartofalargersequencingofinitiatives.Theytargeted,inparticular,thedevelopmentofagenAItooltohelpdispatchersandserviceoperatorsbetterpredict

thetypesofcallsandpartsneededwhenservicinghomes.

2

Leadershipputinplacecross-functionalproductteamswithsharedobjectivesand

incentivestobuildthegenAItool.Aspartofanefforttoupskilltheentireenterpriseto

betterworkwithdataandgenAItools,theyalsosetupadataandAIacademy,which

thedispatchersandserviceoperatorsenrolledinaspartoftheirtraining.Toprovide

thetechnologyanddataunderpinningsforgenAI,thechiefdataandAIofficeralso

selectedalargelanguagemodel(LLM)andcloudproviderthatcouldmeettheneedsofthedomainaswellasserveotherpartsoftheenterprise.ThechiefdataandAIofficer

alsooversawtheimplementationofadataarchitecturesothatthecleanandreliable

data(includingservicehistoriesandinventorydatabases)neededtobuildthegenAItoolcouldbedeliveredquicklyandresponsibly.

OurbookRewired:TheMcKinseyGuidetoOutcompetingintheAgeofDigitalandAI(Wiley,June2023)providesadetailedmanualonthesixcapabilitiesneededtodeliverthekindof

broadchangethatharnessesdigitalandAItechnology.Inthisarticle,wewillexplorehowtoextendeachofthosecapabilitiestoimplementasuccessfulgenAIprogramatscale.Whilerecognizingthatthesearestillearlydaysandthatthereismuchmoretolearn,ourexperiencehasshownthatbreakingopenthegenAIopportunityrequirescompaniestorewirehowtheyworkinthefollowingways.

FigureoutwheregenAIcopilotscangiveyouarealcompetitiveadvantage

ThebroadexcitementaroundgenAIanditsrelativeeaseofusehasledtoaburstof

experimentationacrossorganizations.Mostoftheseinitiatives,however,won’tgenerateacompetitiveadvantage.Onebank,forexample,boughttensofthousandsofGitHub

Copilotlicenses,butsinceitdidn’thaveaclearsenseofhowtoworkwiththetechnology,progresswasslow.Anotherunfocusedeffortweoftenseeiswhencompaniesmove

toincorporategenAIintotheircustomerservicecapabilities.Customerserviceisa

commoditycapability,notpartofthecorebusiness,formostcompanies.WhilegenAImighthelpwithproductivityinsuchcases,itwon’tcreateacompetitiveadvantage.

Tocreatecompetitiveadvantage,companiesshouldfirstunderstandthedifference

betweenbeinga“taker”(auserofavailabletools,oftenviaAPIsandsubscriptionservices),a“shaper”(anintegratorofavailablemodelswithproprietarydata),anda“maker”(abuilderofLLMs).Fornow,themakerapproachistooexpensiveformostcompanies,sothesweetspotforbusinessesisimplementingatakermodelforproductivityimprovementswhile

buildingshaperapplicationsforcompetitiveadvantage.

MuchofgenAI’snear-termvalueiscloselytiedtoitsabilitytohelppeopledotheir

currentjobsbetter.Inthisway,genAItoolsactascopilotsthatworksidebysidewithanemployee,creatinganinitialblockofcodethatadevelopercanadapt,forexample,ordraftingarequisitionorderforanewpartthatamaintenanceworkerinthefield

canreviewandsubmit(seesidebar“CopilotexamplesacrossthreegenerativeAI

archetypes”).Thismeanscompaniesshouldbefocusingonwherecopilottechnologycanhavethebiggestimpactontheirpriorityprograms.

3

Copilotexamplesacrossthree

generativeAI

archetypes

•“Taker”copilotshelp

realestatecustomers

siftthroughproperty

optionsandfindthemostpromisingone,write

codeforadeveloper,

andsummarizeinvestor

transcripts.

•“Shaper”copilotsprovide

recommendationstosales

repsforupsellingcustomersbyconnectinggenerativeAItoolstocustomerrelationshipmanagementsystems,

financialsystems,and

customerbehaviorhistories;createvirtualassistantsto

personalizetreatmentsforpatients;andrecommendsolutionsformaintenanceworkersbasedonhistoricaldata.

•“Maker”copilotsarefoundationmodels

thatlabscientistsat

pharmaceuticalcompaniescanusetofindandtest

newandbetterdrugs

morequickly.

Someindustrialcompanies,forexample,haveidentifiedmaintenanceasacriticaldomainfortheirbusiness.

Reviewingmaintenancereportsandspendingtimewithworkersonthefrontlinescanhelpdeterminewhere

agenAIcopilotcouldmakeabigdifference,suchas

inidentifyingissueswithequipmentfailuresquickly

andearlyon.AgenAIcopilotcanalsohelpidentify

rootcausesoftruckbreakdownsandrecommend

resolutionsmuchmorequicklythanusual,aswellas

actasanongoingsourceforbestpracticesorstandardoperatingprocedures.

Thechallengewithcopilotsisfiguringouthowto

generaterevenuefromincreasedproductivity.In

thecaseofcustomerservicecenters,forexample,companiescanstoprecruitingnewagentsanduseattritiontopotentiallyachieverealfinancialgains.

Definingtheplansforhowtogeneraterevenuefromtheincreasedproductivityupfront,therefore,iscrucialto

capturingthevalue.

Upskillthetalentyouhave

butbeclearaboutthegen-AI-specificskillsyouneed

Bynow,mostcompanieshaveadecentunderstandingofthetechnicalgenAIskillstheyneed,suchasmodelfine-tuning,vectordatabaseadministration,prompt

engineering,andcontextengineering.Inmany

cases,theseareskillsthatyoucantrainyourexistingworkforcetodevelop.ThosewithexistingAIand

machinelearning(ML)capabilitieshaveastronghead

start.Dataengineers,forexample,canlearnmultimodalprocessingandvectordatabasemanagement,MLOps(MLoperations)engineerscanextendtheirskillsto

LLMOps(LLMoperations),anddatascientistscan

developpromptengineering,biasdetection,andfine-tuningskills.

Thelearningprocesscantaketwotothreemonthsto

gettoadecentlevelofcompetencebecauseofthe

complexitiesinlearningwhatvariousLLMscanandcan’tdoandhowbesttousethem.Thecodersneedtogain

experiencebuildingsoftware,testing,andvalidating

4

answers,forexample.Ittookonefinancial-servicescompanythreemonthstotrainitsbestdatascientiststoahighlevelofcompetence.Whilecoursesanddocumentation

areavailable—manyLLMprovidershavebootcampsfordevelopers—wehavefound

thatthemosteffectivewaytobuildcapabilitiesatscaleisthroughapprenticeship,

trainingpeopletothentrainothers,andbuildingcommunitiesofpractitioners.Rotatingexpertsthroughteamstotrainothers,schedulingregularsessionsforpeopletoshare

learnings,andhostingbiweeklydocumentationreviewsessionsarepracticesthathaveprovensuccessfulinbuildingcommunitiesofpractitioners(seesidebar“AsampleofnewgenerativeAIskillsneeded”).

It’simportanttobearinmindthatsuccessfulgenAIskillsareaboutmorethancoding

proficiency.OurexperienceindevelopingourowngenAIplatform,Lilli,showedusthat

thebestgenAItechnicaltalenthasdesignskillstouncoverwheretofocussolutions,

contextualunderstandingtoensurethemostrelevantandhigh-qualityanswersare

generated,collaborationskillstoworkwellwithknowledgeexperts(totestandvalidate

answersanddevelopanappropriatecurationapproach),strongforensicskillstofigure

outcausesofbreakdowns(istheissuethedata,theinterpretationoftheuser’sintent,the

qualityofmetadataonembeddings,orsomethingelse?),andanticipationskillstoconceiveofandplanforpossibleoutcomesandtoputtherightkindoftrackingintotheircode.A

purecoderwhodoesn’tintrinsicallyhavetheseskillsmaynotbeasusefulateammember.

Whilecurrentupskillingislargelybasedona“learnonthejob”approach,weseearapid

marketemergingforpeoplewhohavelearnedtheseskillsoverthepastyear.Thatskill

growthismovingquickly.GitHubreportedthatdeveloperswereworkingongenAIprojects“inbignumbers,”andthat65,000publicgenAIprojectswerecreatedonitsplatformin

2023—ajumpofalmost250percentoverthepreviousyear.IfyourcompanyisjuststartingitsgenAIjourney,youcouldconsiderhiringtwoorthreeseniorengineerswhohavebuiltagenAIshaperproductfortheircompanies.Thiscouldgreatlyaccelerateyourefforts.

Formacentralizedteamtoestablishstandardsthatenableresponsiblescaling

ToensurethatallpartsofthebusinesscanscalegenAIcapabilities,centralizing

competenciesisanaturalfirstmove.Thecriticalfocusforthiscentralteamwillbeto

developandputinplaceprotocolsandstandardstosupportscale,ensuringthatteamscanaccessmodelswhilealsominimizingriskandcontainingcosts.Theteam’swork

couldinclude,forexample,procuringmodelsandprescribingwaystoaccessthem,developingstandardsfordatareadiness,settingupapprovedpromptlibraries,andallocatingresources.

WhiledevelopingLilli,ourteamhaditsmindonscalewhenitcreatedanopenplug-inarchitectureandsettingstandardsforhowAPIsshouldfunctionandbebuilt.They

developedstandardizedtoolingandinfrastructurewhereteamscouldsecurely

experimentandaccessaGPTLLM,agatewaywithpreapprovedAPIsthatteamscouldaccess,andaself-servedeveloperportal.Ourgoalisthatthisapproach,overtime,can

5

AsampleofnewgenerativeAI

skillsneeded

Thefollowingareexamplesofnewskillsneededforthe

successfuldeploymentof

generativeAItools:

•datascientist:

–promptengineering

–in-contextlearning

–biasdetection

–patternidentification

–reinforcementlearningfromhumanfeedback

–hyperparameter/largelanguagemodelfine-

tuning;transferlearning

•dataengineer:

–datawranglinganddatawarehousing

–datapipelineconstruction

–multimodalprocessing

–vectordatabasemanagement

helpshift“Lilliasaproduct”(thatahandfulofteamsusetobuildspecificsolutions)to“Lilliasaplatform”(thatteamsacrosstheenterprisecanaccesstobuildotherproducts).

ForteamsdevelopinggenAIsolutions,squad

compositionwillbesimilartoAIteamsbutwithdata

engineersanddatascientistswithgenAIexperienceandmorecontributorsfromriskmanagement,compliance,

andlegalfunctions.Thegeneralideaofstaffingsquadswithresourcesthatarefederatedfromthedifferent

expertiseareaswillnotchange,buttheskillcompositionofagen-AI-intensivesquadwill.

Setupthetechnologyarchitecturetoscale

BuildingagenAImodelisoftenrelativelystraightforward,butmakingitfullyoperationalatscaleisadifferentmatterentirely.We’veseenengineersbuildabasicchatbotin

aweek,butreleasingastable,accurate,andcompliantversionthatscalescantakefourmonths.That’swhy,ourexperienceshows,theactualmodelcostsmaybeless

than10to15percentofthetotalcostsofthesolution.

Buildingforscaledoesn’tmeanbuildinganewtechnologyarchitecture.Butitdoesmeanfocusingonafewcore

decisionsthatsimplifyandspeedupprocesseswithoutbreakingthebank.Threesuchdecisionsstandout:

•Focusonreusingyourtechnology.Reusingcode

canincreasethedevelopmentspeedofgenAIuse

casesby30to50percent.Onegoodapproachis

simplycreatingasourceforapprovedtools,code,

andcomponents.Afinancial-servicescompany,for

example,createdalibraryofproduction-gradetools,

whichhadbeenapprovedbyboththesecurityandlegalteams,andmadethemavailableinalibraryforteams

touse.Moreimportantistakingthetimetoidentifyandbuildthosecapabilitiesthatarecommonacrossthe

mostpriorityusecases.Thesamefinancial-services

company,forexample,identifiedthreecomponentsthatcouldbereusedformorethan100identifiedusecases.Bybuildingthosefirst,theywereabletogeneratea

significantportionofthecodebaseforalltheidentifiedusecases—essentiallygivingeveryapplicationabig

headstart.

6

•FocusthearchitectureonenablingefficientconnectionsbetweengenAImodels

andinternalsystems.ForgenAImodelstoworkeffectivelyintheshaperarchetype,theyneedaccesstoabusiness’sdataandapplications.Advancesinintegrationandorchestrationframeworkshavesignificantlyreducedtheeffortrequiredtomake

thoseconnections.Butlayingoutwhatthoseintegrationsareandhowtoenable

themiscriticaltoensurethesemodelsworkefficientlyandtoavoidthecomplexity

thatcreatestechnicaldebt(the“tax”acompanypaysintermsoftimeandresourcesneededtoredressexistingtechnologyissues).Chiefinformationofficersandchief

technologyofficerscandefinereferencearchitecturesandintegrationstandardsfortheirorganizations.Keyelementsshouldincludeamodelhub,whichcontainstrainedandapprovedmodelsthatcanbeprovisionedondemand;standardAPIsthatactas

bridgesconnectinggenAImodelstoapplicationsordata;andcontextmanagement

andcaching,whichspeedupprocessingbyprovidingmodelswithrelevantinformationfromenterprisedatasources.

•Buildupyourtestingandqualityassurancecapabilities.OurownexperiencebuildingLillitaughtustoprioritizetestingoverdevelopment.Ourteaminvestedinnotonly

developingtestingprotocolsforeachstageofdevelopmentbutalsoaligningtheentire

teamsothat,forexample,itwasclearwhospecificallyneededtosignoffoneachstageoftheprocess.Thissloweddowninitialdevelopmentbutspeduptheoveralldelivery

paceandqualitybycuttingbackonerrorsandthetimeneededtofixmistakes.

Ensuredataqualityandfocusonunstructureddatatofuelyourmodels

TheabilityofabusinesstogenerateandscalevaluefromgenAImodelswilldependonhowwellittakesadvantageofitsowndata.Aswithtechnology,targetedupgradesto

existingdataarchitectureareneededtomaximizethefuturestrategicbenefitsofgenAI:

•Betargetedinrampingupyourdataqualityanddataaugmentationefforts.While

dataqualityhasalwaysbeenanimportantissue,thescaleandscopeofdatathatgen

AImodelscanuse—especiallyunstructureddata—hasmadethisissuemuchmore

consequential.Forthisreason,it’scriticaltogetthedatafoundationsright,from

clarifyingdecisionrightstodefiningcleardataprocessestoestablishingtaxonomiessomodelscanaccessthedatatheyneed.Thecompaniesthatdothiswelltietheir

dataqualityandaugmentationeffortstothespecificAI/genAIapplicationanduse

case—youdon’tneedthisdatafoundationtoextendtoeverycorneroftheenterprise.

Thiscouldmean,forexample,developinganewdatarepositoryforallequipment

specificationsandreportedissuestobettersupportmaintenancecopilotapplications.

•Understandwhatvalueislockedintoyourunstructureddata.Mostorganizationshave

traditionallyfocusedtheirdataeffortsonstructureddata(valuesthatcanbeorganizedintables,suchaspricesandfeatures).ButtherealvaluefromLLMscomesfromtheirabilitytoworkwithunstructureddata(forexample,PowerPointslides,videos,and

text).Companiescanmapoutwhichunstructureddatasourcesaremostvaluableandestablishmetadatataggingstandardssomodelscanprocessthedataandteamscan

7

findwhattheyneed(taggingisparticularlyimportanttohelpcompaniesremovedatafrommodelsaswell,ifnecessary).Becreativeinthinkingaboutdataopportunities.Somecompanies,forexample,areinterviewingsenioremployeesastheyretire

andfeedingthatcapturedinstitutionalknowledgeintoanLLMtohelpimprovetheircopilotperformance.

•Optimizetolowercostsatscale.Thereisoftenasmuchasatenfolddifference

betweenwhatcompaniespayfordataandwhattheycouldbepayingiftheyoptimized

theirdatainfrastructureandunderlyingcosts.Thisissueoftenstemsfromcompanies

scalingtheirproofsofconceptwithoutoptimizingtheirdataapproach.Twocosts

generallystandout.Oneisstoragecostsarisingfromcompaniesuploadingterabytes

ofdataintothecloudandwantingthatdataavailable24/7.Inpractice,companies

rarelyneedmorethan10percentoftheirdatatohavethatlevelofavailability,and

accessingtherestovera24-or48-hourperiodisamuchcheaperoption.Theother

costsrelatetocomputationwithmodelsthatrequireon-callaccesstothousandsof

processorstorun.Thisisespeciallythecasewhencompaniesarebuildingtheirown

models(themakerarchetype)butalsowhentheyareusingpretrainedmodelsand

runningthemwiththeirowndataandusecases(theshaperarchetype).Companies

couldtakeacloselookathowtheycanoptimizecomputationcostsoncloudplatforms—

forinstance,puttingsomemodelsinaqueuetorunwhenprocessorsaren’tbeingused(suchaswhenAmericansgotobedandconsumptionofcomputingserviceslikeNetflixdecreases)isamuchcheaperoption.

Buildtrustandreusabilitytodriveadoptionandscale

BecausemanypeoplehaveconcernsaboutgenAI,thebaronexplaininghowthesetoolsworkismuchhigherthanformostsolutions.Peoplewhousethetoolswanttoknowhowtheywork,notjustwhattheydo.Soit’simportanttoinvestextratimeandmoneytobuildtrustbyensuringmodelaccuracyandmakingiteasytocheckanswers.

Oneinsurancecompany,forexample,createdagenAItooltohelpmanageclaims.As

partofthetool,itlistedalltheguardrailsthathadbeenputinplace,andforeachanswerprovidedalinktothesentenceorpageoftherelevantpolicydocuments.ThecompanyalsousedanLLMtogeneratemanyvariationsofthesamequestiontoensureanswer

consistency.Thesesteps,amongothers,werecriticaltohelpingendusersbuildtrustinthetool.

PartofthetrainingformaintenanceteamsusingagenAItoolshouldbetohelpthem

understandthelimitationsofmodelsandhowbesttogettherightanswers.Thatincludes

teachingworkersstrategiestogettothebestanswerasfastaspossiblebystartingwith

broadquestionsthennarrowingthemdown.Thisprovidesthemodelwithmorecontext,

anditalsohelpsremoveanybiasofthepeoplewhomightthinktheyknowtheanswer

already.Havingmodelinterfacesthatlookandfeelthesameasexistingtoolsalsohelps

usersfeellesspressuredtolearnsomethingneweachtimeanewapplicationisintroduced.

Gettingtoscalemeansthatbusinesseswillneedtostopbuildingone-o