【会议演讲PPT】Gartner-开始您的生成式AI之旅（APAC）

Gartner

Webinars

Gartnerdeliversactionable,objective

insight,guidanceandtoolstoenable

strongerperformanceonyour

organization’smissioncriticalpriorities

Enhanceyourwebinarexperience

Aska

Question

Download

Attachments

Watch

Again

2

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ConnectwithGartner

GetStartedonYourGenerativeAI

Journey(APAC)

AlbertGauthier

SrDirectorAnalyst

RESTRICTEDDISTRIBUTION

3

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

WhatisArtificialIntelligence

•AIin2023describesstatisticalanalysisthathasbeenaroundfor50years.

•Setoftoolswrappedaroundto“tune”thestatisticalanalysis.

•Asrawcomputingcapabilitiesimprove,theabilitytoquicklyanalyzelargedatasetsandmodifythosedatasetshasimproved.

•ArtificialIntelligenceisthe“latest”descriptionofoldtechnologies.

•CurrentMLisbasedalmostexclusivelyonStatisticalAnalysistofindpatterns

•Notintelligent.

4

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Distinctions

•Theterm“AI”hasbeenusedtodescribemanythings.

–NLP(Alexa,Google)

–Linear/PolynomialRegressionAnalysis

–Probability

–NeuralNetworks(RemembertheTerminator)

–MachineLearning

–Stochasticbasedmodels(LLMincludingChatGPT)

5

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

UtopianandDystopianViewsofAI-Orthis.

6

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

TheHypeofGenerativeAI

•Headline:

HowgenerativeAIisrevolutionizingthefutureofsmartcities

•“Inconclusion,generativeAIhasthepotentialtorevolutionizethewayweplan,develop,andmanagesmartcities.Byprovidinginsightsintocitizenbehavior,trafficpatterns,andenvironmentalfactors,generativeAIcanhelpcitiesbecomemoreefficient,sustainable,andaccessible.”

RESTRICTEDDISTRIBUTION

7©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

WhataboutChatGPTandotherLLM?

•ChatGPTisa“LargeLanguageModel”(LLM)orFoundationalMachineLearningModel.OriginalmodelsbuiltonStochastics(somecallthestochasticparrots).

•ConversationalchatbotwithGenerativePretrainedTransformer(GPT).

•Is“generative”AImeaning,itcraftsaresponsewith“new”contentandtriestoformatitintonaturallanguage.

•(Stochasticsisthestudyofdatasetswithrandomprobabilitydistributionsthatcanbeanalyzedstatisticallybutnotpredicted.)Duetotheuncertaintypresentinastochastic

model,theresultsprovideanestimateoftheprobabilityofvariousoutcomes.

•Interesting,entertainingandwrappedinalotofhype(anyonerememberthemetaverse).

•GARTNER:IsChatGPTartificialgeneralintelligence?No.

RESTRICTEDDISTRIBUTION

8©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

StochasticModelling-5to95%probability

9

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Howdoesitwork?

•Classifies“intent”likeAlexaorGooglewithconfidencescores(statistics)BUTusingstochastic-likeanalysis.

•Produces“constraints”toboundtheresponse.

•Trainedwithupto300BillionWordsfromvarioussources.

•Cansummarizeresponseswithmarginaldegreesofaccuracy(usecautiously)conditionaluponinput.

•Modelsarefine-tunedbyyourfeedback(unlessyouusetheAPI)

•Generatesoutputsbasedontrainedfoundationalmodels(i.e.Ifthemodelisnottrainedinaparticulararea,itdoesn’twork).

•Usesprobabilityanalysis.

•Determinethebest(mostprobable)pathbasedonyourinput.

RESTRICTEDDISTRIBUTION

10©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Morespecifically,modelstrengthsinclude

•Generateandaugmentproseornarratives

•Codedevelopment,translation,explanationandaugmentation

•Summarizeandsimplifylong-formtexts.

•Classifycontentforsentimentorbytopicarea.

•Answerquestions,

•Translateandconvertlanguage(includingprogramminglanguages).

•Writtencontentaugmentationandcreation.

11

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Whatitisn’t

•Accuratemuchofthetime.

•Equallystrongacrossalldomains…onlywhereit’strained.

•Sentient(Perceptive)…itisnotAI

•Insightfuli.e.)Givesyouthesameanswerifyouaskhowtobuildahigh-performanceteamofplumbersorbrain-surgeons.

•Reliable&Trustworthy(ieRequiresexpertreview).

•Abletobecustomizedortrainedwithyourdata.

•Notparticularlyinsightfulmuchofthetime.Regurgitatesprescribedpathsthroughthemodel.

12

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Salesandmarketing:Engagewithpotentialcustomers

onawebsiteorinachatbot,andprovide

recommendationsandproductdescriptions.

HR:Createinterviewquestions,writeofferlettersandjobdescriptions,summarizeemployeesurveyresultsand

suggestemployeeengagementactivities.

Customerservice:Improvecustomer-facingchatbotbreadthandquality,effectivelyrespondtocustomerinquiriesand

complaints,andgeneratepersonalizedresponses.

Softwareprogramming:Generatecomputercodefromprose,convertcodefromoneprogramminglanguagetoanother,correcterroneouscodeandalsoexplaincode.

PopularUseCasesofChatGPT

ChatGPTCapabilities

✔Createwrittencontent.

✔Answerquestions(noncomputational)anddiscoverinformation.

✔Transformthetone,formalityorwriting

genreoflanguageonrequest.

✔Summarizeandclassifytext.

✔Compareparagraphsandcorrect

grammar.

✔Generateideas,suggestionsandkeypointsondifferenttopics.

✔Classifyandcategorizecontentbasedontheexampleprovided.

✔Generate,translate,explainandverifycomputercode.

✔Translatetexttoinstructions,queryordifferentlanguage.

RESTRICTEDDISTRIBUTION

13©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

SelectEnterpriseUseCasesofChatGPT

Legalandcompliance:Draftandsummarizelegaldocuments,andcreatedraftcompliancepoliciesandtrainingmaterial.

Gartner’sGenerativeAIDefinition:

•Createsnewlyderivedcontent,strategies,designsandmethods.

•Learnsfromlargerepositoriesoforiginalsourcecontent.

RisksExecutivesShouldbeWatching

ΔHallucinations

ΔNoAttribution

ΔDataLeakage

WhatExactlyisGenAIinaProfessionalContext?

GenAICreates&LearnsWhatUseCasesAreEmergingforCXOs?

GartnerUseCasePrismforGenerativeAI

Gartner’sAIDefinition:

•Analyzesdatawithlogic-basedtechniqueslikeMachinelearning(ML)

•Interpretsevents,supportsandautomatedecisions(carefulhere).

14

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

KeyIssueTake-Away:

Foundationmodelsrepresentahugestep

changeinthefieldofAI,duetotheirmassivepretraining,whichmakesthemeffectiveat

few-shotandzero-shotlearning,enablingthemtobeversatile.

15

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

But…

•Toomanyorganizationsarejumpingintothetechnologywithoutunderstandingtheproblemandtheusecase

•ThisisgoingtocreatefailedPOCs

•Manyorganizationshaveasolutionlookingforaproblem.

•Drivenbyleadershipandthehypecycle.

•Havecompletemisunderstandingofhowthemodelsarebuilt,usecasesandlimitation.

16

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

GettingStartedinGenAIPilot

•Whatisyourusecase?

•WillOOTBmodelsuffice?

–PromptEngineering&TokenFiltering?

–ModelSelection?

–TextBasedUI(ChatGPT)

–API’sandApplicationEmbedding.

•IsModelAugmentationrequired?

–ModelSelection?

–ModelTraining,TestingandFeedback?

-APIsusedforTraining?

17

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ShareableSummary

KeyFindings

•Themostsuccessfulpilotsfocusondemonstratingbusinesspotential,notontechnical

feasibility.OrganizationstendtoruntechnicaIpilotsthatsimplydemonstratethatitis

possibletobuildsomethingwithgenerativeAI,leadingtoonlyincrementalimprovementsandignoringthetransformativepotentialofthistechnology.

•ITleadersstruggletoidentifyandprioritizeimpactfulgenerativeAIusecasesduetothebroadandemergingnatureofthetechnology.

•MatureAIorganizationsinvolvebusinesspartnersandsoftwareengineersaskeymembersoftheirAIprojectsandpilotteams.

ViewDocument

RESTRICTEDDISTRIBUTION

18©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

©2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

TextGeneration,Q&A,Summarization,Search,Classification,EntityExtraction,IntentRecognition,Translation,Rewrite,

TexttoSpeech

TexttoImage,ImageClassification,ObjectDetection,VideoClassification,ImagetoText

TexttoCode,CodeCompletion

•DrugDiscovery,GenomicSequencing,ChemicalFormulation

•Human-RobotInteraction

UseCasesforFoundationModels

NLP

ComputerVision

SoftwareEngineering

GeneralSciences

&Others

RESTRICTEDDISTRIBUTION

19©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

EnterpriseChatGPT/GPTUsageAreas:ProsandCons

Out-of-the-BoxModelUsage

•UsesChatGPTservice“asis,”nodirectaccesstoGPT-3.5model.

•Pro:Fasttomarket;limitedinvestments;gainexperience.

•Con:Limiteddifferentiation;controlrangeislimited.

Prompt

Engineering/

•Usestoolstocreate,tune,andevaluatepromptinputsandoutputs.

•Pro:BettertargetedChatGPTandGPT3results;lowstartupcosts.

InContextLearnigCon:Mustintegratewithbusinesssystemstointroducedata.

Deployment/FineTuningofCustom

Models

•Uses(builds/finetunes/licenses)GPTorotherlanguagemodelsdirectly.

•Pro:Customizedoroptimizedmodels,data,parametersandtuning.

•Con:Requiresaddedfundingandskills.ThisisnotChatGPT.

20

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Out-of-the-BoxModelUsage

•Thisformofusageisbyfarthemostaccessibleandcommontoday.

•Text-basedwebchatinterface().APIrecentlyavailable.

•Formostusecases,outputmustbereviewedbyahuman,asitmaycontaininaccuraciesorunacceptablecontent.

•Enterprisesmayachieveusefulresultswithlimitedinvestmentsandskills.Butbecausemanyusersareinexperienced,theyriskoverlookingdata,securityandanalyticsrisks.

•Alimitationisthatthemodelcannotincludereal-time,currentorcustomdata.Nordoesitcoverrecenthistoricalevents(thoseafterDecember2021).However,newdatacanbeaddedviaa

promptatthetimeofinteraction.

21

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

PromptEngineering/InContextLearning

•PromptengineeringcanbeappliedtobothChatGPTandGPTusecases.Itinvolvesdevelopingasystematicapproachtocreating,tuning,andevaluatingresultsintermsofinputsandoutputstoandfromChatGPT.

•InChatGPT,thepromptisthecriticalelementdrivingresults.Smallchangestoaprompt’schoiceofwordsandwordordercanresultinsignificantchangesinoutput.Apromptcanalsocontain

datathatshouldbeincorporatedorconsideredwhengeneratingaresponse.

•Leadersshouldanticipatethatpromptengineeringisanewtechnicalskillthatwillneedtobedeveloped,alongwithrelatedtools.

•Insomecases,thisrequirementwillextendtobuildingaseparatelearningmodeltooptimizeprompts.

•InContextLearning,leveragingRetrievalAugmentedGeneration,isthedominantmodelinusebyorganizationsthatmustkeepdatasecureandregularlyupdatedatainanLLMcontext

22

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Deployment/FineTuningofCustomModels

•Thisisthelikelylong-termapproachforsophisticatedsolutions.

•ThisapproachisnotpossiblewithChatGPT,asitdoesnotprovideuserswithaccesstocustomizeitsunderlyingmodel.

•BesidesGPT,otherfoundationmodelsexist.Somearespecialized.

•Customizingfoundationmodelsisacomplextaskthatrequiressignificantskills,datacurationandfunding.

•Enterprisesshouldanticipatearobustmarketforthird-partymodelscustomizedfordifferentusecases.

•Plannersshouldanticipatetheemergenceofthird-party,fit-for-purpose,specializedmodels.Buyingoneofthesemayproveabetterapproachformanyenterprisesthancustomizingamodelthemselves.

•Applicationsmayalsohaveprebuiltmodelsfortheirusers.

23

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ViewDocument

RESTRICTEDDISTRIBUTION

24

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

5

©2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

VALUE

●Benefits:

○Thesimplicityandversatilityofthisdesignpattern—ageneralpurposenaturallanguagegenerationtool—makesithighvaluein

complementingworkflowsoflanguageandsoftwareproduction.

●Drawbacks:

○Riskofincorrectorbiasedoutputs,requiringhumanqualitycontrolofgeneratedresponse.

○PotentialprivacyriskswhensharingIPorconfidentialinformation.

USECASES

●Codegeneration

●Ideageneration/brainstorming

●Copywriting/contentcreation

●Generalknowledgediscovery/search

●Basictranslation/NLPtasks

UseLLMs“As-Is”

Prompt

Response

SimplePrompt

User

LLM

Prompt+Content

Response

PromptWithContent

User

LLM

VALUE

●Benefits:

○ThepotentialtolinkLLMswithinternaldocumentdatabases,unlockinginsightsfrominternaldatawithLLMcapabilities.

○Thepotentialtohavemuchmoreaccurateandrecentinformation.

○Theresultingsystemcouldincludereferences/citationstotheoriginalsourcedocumentsfromwhichtheresponsewasgenerated.

●Drawbacks:

○LLMretrievalmodelscanstillbeinaccurateandhallucinate,albeittypicallylessthanwhenusingLLMswithoutretrieval.

○Requiresastronginformationclassificationtomitigateprivacyrisks.

○DataleakageriskifLLMandsearcharenotinthesameinfrastructure.

USECASES

●UsingLLMstoanswerquestionsaboutaninternal,privatedocumentdatabase

●AugmentingLLMswithwebsearchresults

LLMWithDocument

RetrievalorSearch

LLMandRetrieval

Prompt

Response

Retrieval/

Search

Model

LLMAPI

Prompt+Context

User

Interface

Query

Top

Docs

Document

Database

PrepTheServices

UserPrompt

4

IndexyourinternaldocswithAzureCognitive

services

1

AzureCognitiveServices

ConvertedChatGPT

Prompt

2

AzureOpenAI

MasterPrompt

Service

SetupyourownPrivate

InstanceofChatGPTw/API

8

3

9

Howitworks

Interactwithservices

PrompttranslatedtoaQueryof

IndexedData

5

6

MasterPrompt

7

ListofDocument

snippetsfromprivate

dataindex

HiddenfromUser

GeneratedSummary

Source1Source2

Groundingw/sources

DESIGNPATTERNAPPLICATION

EmbedLLM“As-Is”Into

anApplicationFrame

●Name:EmbedLLM“as-is”intoanapplicationframe

●Description:ExposingLLMcapabilitiesviaanapplicationframethatmakesAPIcallstotheLLMonthebackend

●Motivation:TobettercontrolandsecureadoptionofLLMcapabilities

●Solution:ServicecalledviaAPIandresultspresentedinUIframeinsidehostapplication

●Implementation:Implementedasanon-demanddiscoveryorcontentgenerationtool(inessence,anewtoolinaframeawaitingapromptfromuser)

VALUE

●Benefits:

○TakesadvantageofthebetterprivacyandsecurityprotectionsincludedinAPIofferings(ascomparedtotheend-userapplications).

○Easiertomonitorcompliancebyrecordingusageviatheproprietaryuserinterface.

○APIsgivemoreflexibilityforcreatingcomplexworkflows(forexample,addingautomatedcontrolsbeforesendingdatatotheAPI).

●Drawbacks:

○Volumeofuseandpricing:APIcostsneedtobemonitored.

○PrivateinstancesofLLMscouldbeeventuallybeoffereddirectlyby

vendors,changingthecost-benefitofbuildingaprivateuserinterface.

USECASES

●EnablingemployeeaccesstoLLMsinacontrolledenvironment

●AlltheusecasesintheUsingLLMsAs-Isapplyhere:codegeneration,Ideageneration/brainstorming,copywriting/contentcreation,generalknowledgediscovery,basicNLPtasks

Request

App

Frame

AppUserInterface

APICall

PromptResponse

LLMExposedWithinanApplicationFrame

User

Response

LLM

API

Non-LLM

prompt

UI

DESIGNPATTERNAPPLICATION

EmbedLLMIntoanApplicationWorkflow

●Name:LLMembeddedinanapplicationworkflow

●Description:Embeddingas-isLLMaspartofabroaderapplicationworkflow.ThisdiffersfromtheApplicationFramepatterninthatthisisnotjustawayto

exposeLLMAPIs,butawaytointegratethemaspartofacomplexapplication

●DataConsiderations:PotentialinconsistencybetweentheLLMandthehostapplicationcontextanddata

●Motivation:ToexpandthefunctionalityofanapplicationwithLLMcapabilities

●Solution:LLMcalledviaAPIbyapplicationandresultsprocessedbytheapplication

●Implementation:Canbeimplementedintwoways:

○Asasecondarysourceofcontentproactivelyqueriedbyapplicationandpresentedtotheuser

○WheretheLLMoutputdrivesanotherprocessintheapplicationandmayormaynotpresentresultsintheUI

VALUE

●Benefits:

○Enhancesthefunctionalityofanapplication

●Drawbacks:

○UsagecontrolsneedtobeimplementedtokeepAPIcostsundercontrol

○UsingtheLLMasacomponentofanapplicationmightbetooriskyforsomeusecases,requiringcarefulguardraildesign

USECASES

●EmbeddingLLMsintoproductivitysoftwareorcollaborationtools

●PresentingLLMoutputsalongsideexistingsearchresults

●Embeddedintoacontentmanagementsystem

●ChatbotapplicationexpandingvirtualassistantnetworkwithLLMs

Aproprietaryapp

augmentedwith

LLMs

Prompt

LLM

API

Response

LLMinanApplicationWorkflow

User

AppWorkflowTrigger

ProcessLLMResponse

Application

AppUserInterface

DESIGNPATTERNAPPLICATION

LLMasaSecondaryChatbot

●Name:LLMasasecondaryconversationalagent

●Description:AconversationalsystemroutesrequeststoanexistingchatbotortheLLMAPI.Thishandovercouldalsobedonefromtheexistingchatbot.

●Motivation:

○ToaddabroadgeneralknowledgeexperiencetoaconversationalUI

○Toenableopen-endedconversations

●Solution:Therearetwobroadapproaches:

○WhereachatbotorchestrationfunctionroutesauserquerytoeitheranexistingchatbotortheLLMAPI

○WhereexistingchatbothaslowconfidenceandhandsqueryovertotheLLM

●Implementation:Theincumbentconversationalsystemisresponsiblefor

invokingtheLLMbasedoncontextorenablingthechatbotsinitsnetworktofallback/handovertotheLLMbasedonconfidencelevels(orsomeotherfactor).

VALUE

●Benefits:

○Extendtheconversationalcapabilitiesofanexistingchatbotecosystem

●Drawbacks:

○LowconsistencyinresponsebetweentheLLMandtheexistingchatbot

○Riskinlowaccuracy/hallucinationscomingfromtheLLMresponses

○ExternalchatbotsmayrequiresendingcustomerdataintotheLLMAPI,potentiallycreatingaprivacyrisk

USECASES

●Improvingcustomerservicechatbots

●Augmentingnonplayablecharactersinvideogames

User

Interface

Orchestration&

RoutingLogic

1

LLM

API

OtherChatbot

2

Response

Handling

Option1.Routetoappropriatebot

Option2.Handoverwhenchatbot

confidenceislow

Response

LLMasSecondary

Agent

Non-LLM

promptUI

ShareableSummary

Recommendations

AsanITleaderfocusedonleveraginggenerativeAItocreatebusinessvalue,youshould:

•Runaworkshoptogenerateuse-caseideaswiththebusiness,focusingonthedisruptivepotentialofgenerativeAIandthewayinwhichitcanenablestrategicobjectives.

•Prioritizetheusecasesforyourpilotagainsttheirpotentialbusinessvalueandfeasibility.FocusonnomorethanafewusecasesforyourgenerativeAIpilot.

•Assembleasmallbutdiverseteam,includingbusinesspartners,softwaredevelopersandAIexperts.Dedicatethisfusionteamforthedurationofthepilot.

•Createaminimumviableproducttovalidateeachusecase.Identifythetargetbusinesskeyperformanceindicator(KPI)improvementhypothesis,anddefinethedeploymentapproachesandriskmitigationsrequiredtoquicklytestthishypothesis.

ViewDocument

4©2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

Mitigate

Hallucinations

•DocumentLimitations

•ModelMonitoring

CollaborateAcrossStakeholders

•SeekDiversity

•PublishLessonsLearned

InstillResponsibleAIPractices

PreventMisuse

•UsageGuidelines

•Enforcement

32

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

AI

Experts

AllowUserstoReportIssues

AI

Services

AIUsers

CreateaFeedbackLoop

Communicate

Limitations

RESTRICTEDDISTRIBUTION

33©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ConductAdversarial

Testingvia“RedTeaming”

Insteadofdoingextensive

annotation,theredteamconductsadversarialtesting,activelyseekingoutexampleswhereitfails.

Themodelisretrainedonthese

examples,withtheteamadding

newadversarialexamples—

continuingthisprocessuntilthey

closethelooponfindingfailures.

34

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

UpskillingRecommendations

•AlignyourskillsdevelopmentwiththedeploymentpatternsforLLMsthatbestfitsyourorganizationsusecaseandmaturity.Formostorganizations,thiswillconsistoftheadoptionofCOTSapplicationsincorporatingLLMs,orLLMapplications

thatleverageretrievalaugmentedgeneration(RAG).

•Crossfunctionaltechnicalteamsshouldupskillpromptengineering,knowledgegraphandLLMOpsskills.CitizenDataScientistsshoulddevelopprompt

engineeringskills.

•Learnfromproductmanagementbestpracticesandspendmoretimeon

discoverybeforeyoujumpintodelivery.Figuringoutwhattobuild,howtobuildit,andhowtobringittomarket,evenwithhelpfromasmartAIcopilot,isstilla

highlychallengingactivity.

•Architects:focusonimprovingTeam,ProcessesandOrganizationdesignwithmethodssuchasAgile,TeamTopologiesandWardleymappingtoenablethevelocity,serviceorientationandadaptabilitytochangerequiredbyAIadoption.

RESTRICTEDDISTRIBUTION

35©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

PromptEngineeringMethods&Skills

TechnicalSkills

SoftSkills

Tools

Core

Prompt

Formulation/Chaining

Writing/CommunicationBusinessDomain

Knowledge

PromptManagement

Valuable

AdvancedPromptingMethods

Prompt

Monitoring/RelevanceScoring

Creativity

Reasoning

ProductSense

ThinkingEnd-To-EndCollaboration

Prompt

Engineering/PromptInfrastructure

Search/Indexing/VectorDatabases

Specialized

SemanticSearch

Knowledge

Engineering

AdversarialPromptingPromptOptimization

Architecture

UserEmpathy

DesignThinking

Persuasion

Automation/Workflow

Platforms

SymbolicAIPluginsDataLabeling

36

RESTRICTEDDISTRIBUTION

©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Domainspecific

orGeneral

Purpose

ModelSize&Benchmarks

EcosystemorBuildyouown

OrganizationalSupport

ModelUpdateoptions

Execution

performance&

latency

LargeLanguageModel(LLM)

RolesandSkills

Considerations

QualityofDataSources

ArchitectureTransparency

Deploymentoptions

RESTRICTEDDISTRIBUTION

37©2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

FutureofGenerativeAI

•Rawpotentialisenormous—morepowerfulandversatilemodels,butsafetyandveracityremainquestionable.

•Willberapidlyembeddedintoconsumerandbusinessapps.

•Modelsizeswillcontinuetoscalebutclientswillprioritizecost,simplicity,security,transparencyanddomainspecificity.

•Emergenceofnewbusinessmodels&ecosystems.

•Growthinmultimodalmodels.

•Theconcentrationofpowerthatthisphenomenonentailsanditseffectsaren’tfullyunderstoodtoday.

•EverythingclaimstobepoweredbyAI.

RESTRICTEDDISTRIBUTION

38©2023G