云应用集成的N种成功模式(Richard)

1

Thanksforhavingme…

Forme…someinterestingcloudintegrationsuitestoinvestigatefurther…

/what

-

we

-

do/integration

-

platform/snapcenter.php

/UsageScenarios/ApplicationIntegration/CloudIntegrat

ion.aspx

/resources/whitepapers

2

WhoIam…

3

[Wherearewe]

•Increasedpopularitytowardsusingpackagedsoftwareasaserviceorbuildingappsincloudplatforms

•Morethan95percentoforganizationsexpecttomaintainorincreasetheirinvestmentsinsoftwareasaservice(SaaS)andmorethanone-thirdhavemigrationprojectsunderwayfromon-premisestoSaaS,accordingtoasurveybyGartner,Inc.

•Currently,communications(52percent),utilities(51percent),andbankingandsecurities(49percent)industriesrankhighestwithrespecttoSaaSdeployedacrossthehorizontalandvertical-specificcategoriessampled.In2012,thoseindustriesrankinghighestwithrespecttotheirplanstouseSaaSincludefederalgovernment(33percent),bankingandsecurities(22percent)andwholesaletrade(20percent).Beyond2012,topindustriesconsideringSaaSaremanufacturingandnaturalresources(37percent),wholesaletradeandretail(each29percent).

4

[What’stheproblem?]

•SaaSnotreallybuiltwithdataexchangeinmind

•InformationWeeksurvey(

/news/cloud

-

computing/software/231002362?queryText=SaaS+Integration

)ofSaaSsatisfactionareasshowed“easeofdeployment”#1andintegrationlast

•Easytoseetheseas“offsite”andeitherdolimitedintegration,or,irregularbatchloads

•Threescenarios:on-premisetocloud,cloudtoon-premise,andcloudtocloud

•Eachhaveuniquechallenges,buteacharevaluableandnecessaryscenarios

5

[wheredowewanttobe?]

•Therearetriedandtrueintegrationstylesforsharingdata/processesbetweensystems

•We’retakingthatreliablechassisandapplyingittonewtechnologies

6

[howdowegetthere?]

AseminalbookonthetopicistheEIPbookbyHohpeandWoolf

Discussesahandfulofcommunicationstylesandintegrationpatterns.We’llfocusonthreethatarecalledoutthere:

•Shareddatabase

•Remoteprocedureinvocation

•Messaging/async

Foreachpattern…

•Defineit

•Whenitmakessense

•Constraints

•Cloudconsiderations

•Cloudtechnologies

•Demo

7

Whenitmakessense…

•SharingviaETLorfileisn’ttimely

•Whatifyouneedthelatestdata?

•Couldbeforreporting,referencedataortransactions

•Usewhenyouhavecommondata(and/orstructure)butdifferentCONSUMINGinterfaces

•Thinkofmulti-tenantappswhereyoushareaDB,butpartitionbyuser

•CouldbeanODSordatamartwhereyouwantcommonschemausedbyothers

•Getasingleviewofthedata(noneedtoforceeachendtodefineadataformat)

•Alldependentsystemsareconsistentatthesametime

•Singledataaccessstrategy(SQLsyntax)

8

Challenges

•**Design**

•Toughtoagreeonformats(forODS,mart,transactionalrecords)

•Unifiedschemathatsatisfieseveryone?Commodityonly,or,strongleadershiptoformastandard(orextensibility)

•**Contention/Performance**

•Lesslikelytousewhenhavemultipleappsmanipulatingsametransactionaldata

•Couldhavedatarecordsdefinedwhereyoucouldchangepartsofonerecordwhilesomeonechangespartsofothers

•Maygetfewerinconsistencies,butstillhaveissuesofsimultaneousupdates

•Cantrytohandlewithtransactions,buttransactionsareoftentheenemyofscalability

•

/~cs223/papers/cidr07p15.pdf

•Trytoavoidupdates/conflictsbetweenapplicationssharingthedata

•Newrowsonly,orreadonly(reportingscenario,orreferencedata)

•Avoiddeadlocks

•CouldgetpoorperformanceifappsaredistributedandallaccessingoverWAN

•**COTSsupport**

•Packagedappsrarelyacceptanexternaldatabaseasitssource

•Oppositeof“Sharednothing”wherenodesareselfsufficient

/wiki/Shared_nothing_architecture

•Maymeansharding

/database

-

sharding/

9

CloudConsiderations

•**Accessprotocols**

•CloudprovideseitherDBorwebprotocolaccess

•Havechoicesastohowtoaccess

•Ifremediatingexistingapps,maywanttheoptionofusing“standard”ODBCAPIsandnothavetochangemuchcode

•Fornewer(orrebuilt)apps,manyservicesprovideaRESTfulAPIfordataaccess

•**Identity**

•Identityproviders

•Ifshared,howdoyouapplygranularaccess?

•**Performance**

•Mayhavegoodperformance

•Shardingbuiltintomostofthese(splitbasedondomains/groups)

•Insomecaseshaveeventualconsistentreads,butSimpleDBsupportsoption

tohaveconsistentreads

/articles/3572?_encoding=UTF8&jiveRedirect=1

•ConditionalPUTandDELETEwithexpectedvalues(topreventoverwritingfresherdata)

•CanstilldoeventuallyconsistentreadandtrusttheConditionalstoenforceconsistency

•CoulduseversionortimestampvalueaspartofConditional

9

•**DifferentDBstorageoptions**

•CouldusecloudRDMSifyoudon’thavemassivescaleneeds

•Canuseaschema-lessproductlikeSimpleDB

•**Providerlimits**

•WritesarethrottledtoSimpleDB,socanuseshardeddomainsifexpectmorethanXYZputs/second(dobatch)

•

/post/712653349/simpledb

-

essentials

-

for

-

high

-

performance

-

users

-

part

-

1

•Mayhavelimitedtransactions

•Acrosstables/entitiesforschema-lesslikeAzuretables

•**Options**

•DBs

•StructuredRDMS

•AmazonRDS

•SQLAzure

•D

•DatabasehostedonIaaSplatformlikeAWS

•Schema-less

•SimpleDB

•AzureTables

•GoogleAppEngineDataStore

/appengine/docs/python/datastore/overview.

html

•Optimisticconcurrency

•Distributesdatawhennecessary

•Limitsinnumberofcallsperminute

•Blobs

•S3

•AzureBlobStorage

10

Demo

•Slideshows“beforeandafter”

•Demostart…

•ShowdbvaluesfromVS2010;ShowIAMandcredentialsspecifictoauseraccount

•.NETapp

•Retrieveitems;Openappanddon’tputcustomerIDin;thenputjustthe

30010accountin

•CloudFoundry

•RubyappwhereIquerythedatabaseviaRESTafterbuildingsignedstring

•

http://seroter

-

/lookup/30010

•

http://seroter

-

/lookup/30014

•ShowRubyclass;VMCtoshowrunningappinstances

•SFDC

•CustomcodethatlooksatsharedDB

•

/apex/InteractionHistory?id=003A0000001YN

Pe

(30010)

•

/apex/InteractionHistory?id=003A0000001Yy

cD

(30014)

•ShowSFDCcode

•.NETapp

•Addnewitem;RefreshSDFCandCloudFoundryapps

11

CanviewmyAmazonWebServices(AWS)SimpleDBcontentviaVisualStudio2010plugin.NoticethatIhavefourrowsofdatafortwodifferentcustomers.

12

Icanalsodogranularrole-basedpermissionsinAWSandrestrictwhocan

create/update/readdata

13

IbuiltaclientapplicationthatpullsalltheinteractionsfromAWSSimpleDBforagivenuser

14

I’vealsobuiltaRubywebapplicationhostedinVMWare’spublic/privatecloudcalledCloudFoundry

15

MyCloudFoundryRubycodeconsumestheAWSSimpleDBdataviawebservices

16

YoucanseehowmanyinstancesofmywebapparedeployedintheVMWarecloud

17

Withonecommand,I*instantly*jumptotwoinstances.Immediatelyloadbalanced.

18

Icanthen*instantly*movebacktoasingleinstance.

19

I’mnowinmySaccountwhereIbuiltacustompagetopullSimpleDBdata

20

Byclickingthebuttononmypage,Iretrievedalltheinteractionsforthis

Scustomer

21

MycodeshowthatIconsumetheAWSSimpleDBserviceinaverysimilarwaytoCloudFoundryapp

22

Withinmyon-premisesapp,IaddedanewrecordtomySimpleDBdatabase

23

ThatrowisnowimmediatelyvisibleinS

24

…andwithinmyCloudFoundryapp!

25

Whenitmakessense…

•Havemultipleapps,builtindependentlyanddata/processesneedtobeshared

•Justsharingdatanotalwaysenough;processes,workflowstoo

•Datastayswithit’ssource

•SOA–businessservicesthatinitiateactionbasedoninvocation

•Encapsulationthathidesinternals

•Hideunderlyingchanges

•Canberesponsivetonecessarychangesvs.gettingeveryoneonboardwith(shared)databasechanges

•Abstractiongivesyoucoarsefunctionsinsteadofgranularfunctioncalls

•Interoperability

•RelyonHTTPandXML/JSON

•Reusability

•Veryfamiliarpatternfordevelopers(request/response)

•Goodforfinegrainedfunctionsandmashupservices

•Wantdatabeforemovingtonextstep

•Oftendon’tneedguaranteeddeliveryorabrokersinceyoucanjustretrytherequest

26

Challenges

•**Coupling**

•Notasbadasusingashareddatabase

•Stillhaverelativelytightcoupling

•Hardtochangeonepiece

•**Security/Capabilitygaps**

•Havetosupportawidervarietyofcapabilitiesbyserviceproviders

•Lackofuniformsecuritystrategies,inconsistentsupportfortransactions

•Dependsonprotocolsthatcanbeconsumed

•**Locking**

•Blockingcallforthesender

•Receiverprocessesexceptions(proandcon)

•**Maybetransientdata**

•Ifastraightlookupwithoutpersistence…notavailableforreports,workflowsetc

•**COTSsupport**

•ManyCOTSsystemsdon’tnativelyexposetheircapabilitiesasservices

•Somehavegottenbetter,butyoustillseehyper-granularorabstractservicesexposedbyleadingvendors

27

CloudConsiderations

•**Security**

•Howdoyouconsumeonsiteservices?

•Howfederatewhendoingcloud-to-cloud

•**Latency**

•Mayrequiredoublehopifwefirstgetatokenandthenmakeactualrequest

•Giventhatthisisablockingcall,mayneedtouseAJAXdesign

•**Access**

•Mayrequirespecificprotocols(HTTP)

•NotgoingtoseebroadsupportforDB-specificprotocols

•**Option**

•Cloudtoon-premises

•Coulduseinternetfacingproxyservicethatforwardsrequesttoback-endsystem/DB

•CoulduseVPNbetweencallerandtargetsystem

•ReliesoncloudappthatsupportsVPN

•AmazonVPC,GoogleDataConnector,AzureConnect

•CoulduseWindowsAzureAppFabricforcloudbasedrelayservice(notdurable)

•Cloudtocloud

•Leveragewebservicesonbothsides;Uwithcloudintegrationprovider

28

Demo

•Showonpremiseservice

•RESTcontract

•Implementation

•Showwhatmakesit“cloudy”

•Cloudbindings

•Startservice

•Showinregistry

•

•

/apex/DiscountLookup?id=001A000000YBX3C

•ShowSFDCcall(withtokenforsecurity)

•Callservice

•CallservicefromaccountwithdifferentID

29

I’vebuiltacustomWCFservicethatusesaRESTful(vs.SOAP)webservicestrategy

30

TheimplementationofthisservicesimplysaysiftheuserIDis200,thediscountis10%.Otherwise,5%.

31

ThisserviceconnectstoWindowsAzureAppFabricviathisconfiguration.Whatthisdoesiscreateasecure,two-waytunneltotheMicrosoftcloud.

Messagessentto/DiscountServicewillget“relayed”tomyon-premiseswebservice!

32

Istartupmyon-premisesservice,whichinitiatesthebindingtothecloud

33

Icanseemyon-premisesserviceexposedinmycloudregistry

34

BecauseIhaveturnedoffcallersecurity,Icantestmyserviceusinganybrowser.

35

NotethatIgetadifferentvaluewhenIuseacustomerIDbesides200

36

WithinS,Ihaveacustompagewhichconsumesthecloudrelayserviceandexecutesmyon-premisebusinesslogicinrealtime!

37

ForadifferentaccountID(besides200),adifferentdiscountrateisreturned

38

Behindthescenes,thiscodeconnectstotherelayserviceandoptionallypassesinasecuritytoken.

39

Whenitmakessense…

•Sharedatabetweensystemsinaresponsiveandscalableway

•Callerdoesn’thavetowait

•Thinkofsendinganemail;itdoesn’tblockOutlookuntilyourrecipientgetsit

•Wanttodobroadcast(pub/sub)ormulticast(definedrecipients)

•Callerdoesn’tneedtocarewhereitgoes

•Supportsdisconnectedapplications(notonlineatthesametime)

•Havemultiplereplicableunits(busscenario)

40

Challenges

•**Notrealtimesynchronization**

•Inherentlyincludeseventualconsistency

•Nosimultaneousorinstantupdates

•**Mayneeddurability/reliability**

•Withoutdurabilityinthemiddle,standthechancetolosedata

•Aroutercanbeusedifyouwanttodirectthemessagestoplacesunknowntothecaller

•Routercanalsodoactivitieslikedatatransformation,protocolbridging,orworkflow

•Idempotenceneededinmanycases

•Datasharedmytellreceivingsystemtogogetdata(handledupesok)

•**Appsupportforreceivingorsendingasyncmessages**

•Fewapplicationsnativelysharedataasynchronously

41

CloudConsiderations

•**Appsupport**

•Seemsevenfewercloudappsshareorreceiveasync

•**Security**

•Maystillneedtopokeaholeinthefirewall

•**Providerlimits**

•Cloudvendorlimitsonpolling(e.g.SFDC)

•Maywanttoputlotsintoqueueand“peek”toretrieve/deleteonlycertainones

•Distributednatureofcloudarchcouldresultinunexpected(lack)ofdata

•Distributedqueuesmaynotreturnallitemsoneachpoll(machinesampleviaAmazon)

•**Options**

•SomeSaaSplatformsbakeinasyncpush(e.g.)

•Couldleverageacloudqueuethateveryonetalksto

•Orothercloudbroker(DB,messagerouter)

•Couldworkforbot