面向AI智能体的设计模式：构建可扩展、事件驱动的系统 Design Patterns for Agentic AI Building Scalable,Event-Driven Systems

DesignPatternsforAgenticAI:

BuildingScalable,Event-DrivenSystems

May1,2025

Moderatedby:ErikCostlow,InfoQEditor

TYLERJEWELL

CEO@Akka

RICHARDLI

FounderofAmorphous

Data

Today’sagenda

01LLMs,agents,agenticsystems

02Systemengineeringchallenges

03Systemengineeringpractices

04Resourcesandnextsteps

Q&A

05

2

Pollquestion

3

Agenticisreal,but…thereisalottolearn

Visitakka.io

Thebasics:

WhatisagenticAI?

Userstories:

AgenticAIcustomerstories

Webinar:

AblueprintforagenticAIservices

Samples

Production-readyagents

Blogs:

AgenticAIblogs

News:

AkkalaunchesnewdeploymentoptionsforagenticAIatscale

GetStarted:

Developyourownagenticapp

4

Agentsandagenticsystemsare

distributedsystems,poweredbyAI

…thatmustdeliverreliableoutcomes

…whiledependinguponunreliableLLMs.

5

AIAgency

Capacitytomakemeaningfromyourenvironment

LowAgency

RPA

→lowautonomy

→coded

decisions

→humancontrol

Agents

→partialautonomy

→LLMadvice

→humanguidance

Agentic

→highautonomy

→distributed

decisions

→group

coordination

Humans

HighAgency

adaptive

static

reactive

proactive

tasks

goals

supervised

autonomous

“Abiggapexistsbetweencurrent

LLM-basedassistantsandfull-fledgedAIagents,butthisgapwillcloseas

welearnhowtobuild,governandtrustagenticAIsolutions.”

–Gartner

economicproductivity-

cost

App

Ecosystem

Cloud-nativeApplications

+

AgenticAIServices

AparadigmshifttoAI-fueledappecosystems

AIagentsandappsbecomepartofasymbioticexistence

By2028,33%ofenterprisesoftwareapplicationswillincludeagenticAI,upfromlessthan1%in2024.

Gartner,TSP2025Trends:AgenticAI—TheEvolutionofExperience,24February2025

EnhancedUserExperience

OperationalEfficiency

Scalability

AIagentspersonalizeinteractionstoincreasesatisfaction

AIagentsautomateroutine

taskstoallowhumanstofocusonstrategicinitiatives

AI-drivenSaaSadaptto

businessneedswithout

proportionalincreasesincost

7

LLM-poweredappservicesareintelligent

Modelscanbepromptedtoperformarangeofuser&systemtasks

Input

LLMautomationvariesbydatatype

Response

SaaSappusecasesandbehavior

AppEcosystem

Cloud-nativeApplications

+

algorithmdescription

AgenticAIServices

audio/video

metrics

questions

state/datachanges

template

document

1

LLM

1

ML

1

LLM

1

LLM

1

LLM

1

LLM

userbehaviorLLM

parametersLLM

functions

LLM

1

LLM

interpret/summarize/analyzetrendprojection

answers

recommendationspopulatefields

validateagainstaschemapersonalization

routingdecision(pathaorb?)toolselection+invocation

compilable,runnablecode

Rethinkinghowyoursystemmakesdecisions

Solveproblemswheredeterministicandrule-basedapproachesfallshort

Multi-faceted

decisionmaking

Worklowsinvolvingjudgement,

exceptionsorcontext-sensitivedecisions,forexamplewhentoescalateasupport

ticket

Constantly

changingrules

Systemswhoserulesetsfrequentlychange,haveextensiveconditions,orburdensometomaintain,suchas

identifyinginappropriatelanguage

Relianceon

unstructureddata

Extractingmeaningfromcontent,

interpretinglanguage,audioorimages,andconversationalresponses,suchaswithasupportchatbot

FromLLMstoAgenticSystems

AgentsgivestructuretoLLMs;agenticsystemsgivescaletoagents

LLM

client

chunkedresponse

prompt

LLM

Stateless,long-running,computationallyintensiveresourcesthatcananalyze,

reason,andplan

LLM

Tools,APIs

agent

VectorDB

Agent

Memory

Structuredenrichmentloopthatbuildscontext,invokestools,takesaction,andgathershumanfeedback

Humans

AgenticSystem

agent

multi-agent

protocol

agent

Networksofmultipleagentsorchestratedtosolvecomplextasks

Patternsforagenticsystemscreateintelligence

Agentcollaborationenablesreliable,goal-drivenreasoning

promptchaining

sub-agent

out

pass

agent1

exit

check

fail

Tasksthatcaneasilybedecomposedtosubtasks:

e.g.writeablogthentranslatetoFrench.

routing

taskagent1

which

out

taskagent2

classifieragent

AnLLMrouterclassifiesataskforroutingtoanLLMspecialist:

e.g.classifythissupportcallaseithersalesortechnical

parallelization

taskagent1

blast

taskagent2

vote

out

LLMsubtaskscanbedividedforspeedormultipleruns:

e.g.executesecuritytestsfromdifferentpovs,withsuccessvoting

synthesizer

orchestratoragent

taskagent1

taskagent2

combine

out

AnorchestratorLLMbreaksdowntasksnotknowninadvance:

e.g.gatheringinformationfromtargetsidentifiedbyorchestratorLLM

evaluator-optimizer

solution

evaluatoragent

out

accepted

generatoragent

feedback

OneLLMgeneratesaresponsewhileanotherprovidesfeedback:

e.g.atranslationLLMthathasnuancecheckingfromevaluatorLLM

Createandexecuteacomplexplanwhilestaying“grounded”withfeedback:

e.g.createatravelitineraryandbookallreservationsforavacation

serviceagent

Multi-agentsystemsareorchestrated

Traceable,auditable,debuggable,withpoint-in-timerecovery

Agenticsystemsareworklows

reliableexecutionofAItaskswithvisibilityintorequest/responsedata,built-inretries,anderrorcompensation

agent

worklow

monitor

timeout

trigger

storage

policies

domainlogic

plug-ins

lightsearch

hotelsearch

sequential

parallel

event-driven

statemachine

human-in-the-loop

1.Picklocale

2.Sharedates

3.Travelplan

builditinerary

done

rain

forecasted

recommend

indoor

activities

rulesrulesrules

activities

adjust

budget

plan

favebudgetitinerary

dates

proposed

itinerary

Single-agentenrichmentloop

Prompt→retrieve→enrich→repeatisarepetitivecycle&pattern

enrichment

agenticloop

prompt

out

toolresponse

in

agent

LLM

MCP,APIs

gatherenvor

1.augment

humanfeedback

saveprompt

prompt

tosendbacktothe

agent

API,

functions,orprograms

memory

vectorDB,contextDB

2.initialLLMpass

callLLM

calltool,please

saveresponse

calltool

3.calltoolto

takeaction

4.addresults

toprompt

5.augmented

LLMcall

updatepromptw/results

results

callLLMagain

moreresults

saveresponse

repeat,ifmultiple

toolscalled

LLMsarestateless.Contextisassembled.

Agenticservicesaugmentpromptswithdatafrommanysources.

availablein

for

vectordb

agenticservice

containsknowledge,facts

Facts,knowledge,andrelated

informationaboutvarious

placesthatonecantravelto

prompt

Givenmytravelhistoryandthe

weather,whatisthebestlocation

formetotravel.Bookthesame

typeofhotelwediscussedinour

previousinteraction,butdonot

bookitbeforegettingmy

permission.

Cancelthereservationifthe

weatherchanges.

UsetheWeatherUnderground

APItogatherweatherinfo.

retrievaltype

semanticsearch

thisinteraction’shistory

workingmemory

memorydb

HereisthehotelthatI

recommend,canIhaveyour

permissiontobookit?

long-termmemory

Inthelastconversationwehad,

a5starhotelwas

recommendedbytheLLM.

previousinteractions

external

API

callingAPIsorcode

tools

retrieve&augment

Instructtheagenttocallthe

WeatherUndergroundAPI

withspecificparametersand

location.

event

stream

updatingcontext

events

Thereisastormalertforthe

Bahamasnextweek.

Agenticsystemsaredistributedsystems

Architecturaltechniquesandpracticesrequiredforscaleandresilience

LLM

client

chunkedresponse

prompt

LLM

→Async,non-blockinginvocation

→Event-based,streamingresponses→Backpressure

LLM

→Event-drivenarchitecture

Tools,APIs

agent

→Human-in-the-loopinteraction→Streamingreal-timeingest

VectorDB

Agent

Memory

→Retries,circuitbreakers,timeouts→Memory&toolintegration

Humans

→CQRS

→Replicationandfailover

AgenticSystem

agent

multi-agent

protocol

agent

→Durableworklows

→Distributedtracing

→Discovery&meshnetworking

→Multi-agentprotocols:A2A,BeeAI

Pollquestion

16

Today’sagenda

01LLM,agents,agenticsystems

02Systemengineeringchallenges

03Systemengineeringpractices

04Resourcesandnextsteps

Q&A

05

17

BumpypathfromPOCtoproduction

Topthreeenterprisechallenges:uncertainty,privacy,andscale

52%

failtoreachproduction

8+months

POCtoproduction

“Leadersreportedthatonly48%ofAI

POCs(ProofOfConcept)makeitinto

production,andtheytakeanaverageof8.2monthstogofromPOCtoproduction.”

Uncertainty:Fromdeterministictostochastic

Scalingmakesitevenharder

●LLMsareslow,expensive,andlimitedbytokenwindows.

●Youneedstreaming,chunking,caching,windowing,reranking,fallback.

●You’renotcallingamodel–yourorchestratingadistributedsystem.

●Cough,cough-whyAkka:)

Debuggingisablackbox

●Nostacktraces.

●Noexplanations.

●Logsgiveyouinput/output,notreasons.

●Prompttweakscancausesideeffectsfarfromwhereyoumadechange.

Expectations≠reality

●Peopleexpectmemory,perfectinstructions,stableoutputs,andtruth.

●LLMsforget,hallucinate,anddriftbasedonsampling.

●Withoutscaffolding,theywillbe(andfeel)brittleandinconsistent.

Ifyou’relookingforvibes,itwillbeshortlived

●LLMsareprobabilisticpatternmatchers-notdeterministicAPIs.

●Buildingwiththemmeansthinkinginsystems,notfunctions.

●Itmeanscontrollingchaos,noteliminatingit.

Randomnesscannotbeeliminatedandmustbeembraced

LLMsarenotdeterministiccomponents

●Sameprompt≠sameoutput.

●Theypredicttokens,notanswers.

●Youdon’tpassparameters–youdesignprompts.

●Hardtopredictoutputs,validatecorrectness,orreproducebehavior.

Promptingisn’tprogramming

●Nofunctionsignatures.Nomodularreuse.

●Tinypromptchangescanbreakresults.

●Longpromptsincreaselatency.

●Andpromptsdon’talwaysworkthesameacrossworklowsorchains.

Retrievaladdsmoreuncertainty

●InRAG,you’recombiningsemanticsearch,reranking,andformatting.

●Eachstepaddsnoise.

●Generatingoverpossiblyirrelevantcontext.

●Nowthesystemisdoublystochastic:retrieval+generation.

TestingLLMsisn’tstraightforward

●There’sno.assertEqual().

●Heuristicmetricsarelawed.

●Humanevalsareexpensiveandinconsistent.

●Evenstableoutputsmightstillbewrong.

Privacyandcompliancehorrorshow

LLMsareleakysievescreatingnumerousholesforsecuritytoplug

LLMsmemorizeorreproducePII

fromtrainingdata

Privateorproprietarydatamishandledbyagents

Trainingdatalackingexplicitconsentfromindividuals

InsecuredatahandlinginAPIs

AnonymizeddatacanbereconstructedbyLLMs

Regulationswithdatadeletion

requirementsarenotLLM-enforced LLMdecisionsusingpersonaldatacanintroducebiasandethicsissuesLLMsmayexposePIIbyintegrating

withunapprovedsystems

Establishclearsecurity&

complianceguidelines

Enableenterprise-gradedatacontrols

Implementagenticservice

interactionandloggingpolicies

Chooseagenticplatformwithtracingandreasoningauditing

Implementriskmitigationwithcontentfiltering

Implementagentidentitywithrolesandpermissions

Memoryhardeningwithtrustcontrolsandmin-access-policy

Enterpriseagenticscalerequiresefficiency

Moretxs:eachslower,lesspredictableandmorecostly

SaaS

Agentic

Users

billions

20x

TPS

10,000

100x

p(99)Latency

10-80ms

15-400x

Cost/LLMtx

cheap

10-10,000x

Mar25:thebestperformingLLM@86%MMLUaccuracycosts$98/1Mtokens,or~850,000xmoreexpensivethantheaveragedatabasetransaction.TheworstperformingLLM@36%MMLUaccuracycosts$.01/1M

tokens,or7xmoreexpensive.

Pollquestion

22

Today’sagenda

01LLM,agents,agenticsystems

02Systemengineeringchallenges

03Systemengineeringpractices

04Resourcesandnextsteps

Q&A

05

23

Agenticsystemsengineeringforreliability

1.ExecuteaDDDandAI-DDprocess

→Producecontextmap,ubiquitouslanguage,andboundedcontexts→Defineoverallorchestrationandflowacrossboundedcontexts

→Developlocalizedworkflowsforeachboundedcontext

2.Definedatasovereigntyandscope

→Company-specificrequirements(e.g.,retentionpolicies,auditlogging)→Countryorregionalregulations(e.g.,GDPR,HIPAA,financialdatarules)

3.Establishevaluationstrategy

→Makereasoningvisibleandmeasurablefromthestart→Buildsyntheticevaluationsetstotestreasoningsteps

4.SelecttherightAImodels

→Reasoningmodels:OpenAIo3,ClaudeSonnet,DeepSeek→Generalmodels:OpenAIGPT-4o,GeminiPro,LLaMa

→Smalllanguagemodels:Phi-4,Mistral7B,ClaudeHaiku,GeminiFlash→Fine-tunedindustrymodels:DeepSeek-Coder,CodeLlama

5.Selectagenticplatformarchitecture

→Chooseplatformthatenablesservicesthattransactandreason

→Rqmts:Durableexecution,event-driven,memory,streaming,andtoolssupport→Rqmts:Elastic,<20msp99latencies,resilient,multi-regionfailover

6.Builddeveloperworkflowandagents

→Refinedeveloperworkflow

→Buildinitialversionsofyouragent(s)

7.Deployandobserve

→Release,monitor,andrefineagentbasedonreal-worldbehavior

Techniquesforreducinguncertainty

Designtoanticipaterandomnesswhileembracingfailureasexpected

Leverage

strategiesthatcreatelayersofcertainty

Createreasoninglayersthatbreak

complexplansintostages,steps,or

sub-tasksthatcanbevalidatedorcheckedbydownstreamagentsorhumans.

Incorporateeval-drivendevelopment

Continuoustestingandexperimentationofdifferentinputs(real-world,synthetic,adversarial)totrackandvalidateaccuracy.

ChooseanAgenticAIPlatformproventooperateservicesscalably,safely

Leverageaframeworkandplatformbaseduponprovenruntimethatsupports

distributedorchestration,event-drivenbehaviors,backpressure,streaming,andembeddedmemory.

Createlayersofcertainty

Incorporatemulti-agentandhumanverificationstrategies

Humanintheloop

Delegatedecisionstohumanswithworklow

Agenticawareness

TakemoreLLMthinkingtimewhenobservinguncertainty

Checkandbalance

Getsecondopinionsfromotheragents

Specialization

LimitLLMstomakingdecisionsinoneareaofexpertise

Restricteddecisioning

LimitLLMstoafinitesetofoutcomes

Incorporateevaluation-driven

ytemloispomnlyearteliableandaccurateasitsevaluationframework

varyinputs(adversarial,synthetic,real-world)andmeasureoutputvalidity

Givena

setof

inputs…

Guardrails

Post-Inference

Model(s)

Prompt(s)

Context(s)

…istheoutputaccurate

andreliable?

ChooseaprovenAgenticAIPlatform

LLMsunlockreasoning–butthereisnofreelunch

LLMsarestatelessNorecallofpriorinteractions

Needamemorysystem

LLMsneedcontextMustbetoldeverythingupfront

Toolsintegration

Knowledgeintegration(e.g.,vectordatabases)

LLMsarestochasticSameinput,differentoutputs

RelyondeterministicworkflowsasmuchaspossibleDesignforuncertainty

LLMsareunreliableMayfailtorespondortimeoutunderload

AdoptadistributedsystemsmindsetUseadurableexecutionframework

LLMsareslow

Highlatency,limitedconcurrency

Usestreamingtoimproveresponsiveness

28

Humans

StreamingEndpoints

AnyProtocol|In/Out|CustomAPIs

Efficient

70%lesscompute

API+agenticcombo

Elastic

5MTPS

akkaclustering

AgentConnectivity&Adapters

Non-Blocking|Backpressure|LoadBalanced

SemanticSearchMulti-LLM,A2AIntegration&MCP

Agile

Prodindays

SDK+opsenvs

IOTDevicesAudio/VideoMetrics

Data,APIandAgenticAI

Services

Secure|Observable|Scalable

AgentLifecycleMgmt

Agent

Orchestration

1

2

3

MemoryDatabase

PR

Resilient

Prompt

Events

2

3

0-0RTO/RPO

1

multi-region,

LLMs

multi-masterdata

OtherSystems

replication

VectorDB

TheAkkaagenticadvantage

✓Agentic,AI,apps&data✓Hardenedruntime

✓Simple,expressiveSDK✓Multi-region

✓Automatedops

Streamingendpoints

→Sharedcompute:agentic

co-executionwithAPIservices

→

HTTPandgRPCcustomAPI

endpoints

→Customprotocols,mediatypes,

andedgedeployments

→Real-timestreamingingest,

benchmarkedtoover1TB

Memorydatabase

→Agenticsessionswithinfinitecontext→Contextsnapshotpruningtoavoid

LLMtokencaps

→In-memorycontextsharding,load

balancing,andtrafficrouting

→Multi-regioncontextreplication

→Memoryfiltersforregion-pinningand

cross-sessioncontextcreation

→Embeddedcontextpersistencewith

Postgreseventstore

Agentconnectivity&adaptersAgentorchestrationAgentlifecyclemanagement

→Non-blocking,streamingLLM

inferenceadapterswithbackpressure

→Multi-LLMselection

→LLMada