分布式消息引擎apache rocketmq最佳实践

1、Trillion Messaging Engine Apache RocketMQAlibaba vongosling© 201 7 A l i b a b a M i d d l e w a r e G r o u pPROFILEMaster of Computer Science,Over 7 plus years of industry software experience, especially in distributed tech., reliable web architecture and performance tuning.Possess many paten

2、ts in distributed system,recommender etc.Open source fanatic, multiple open-source projects contributor, committer onand Apache. Focus on distributed, cloud computing and big data fields. Apache RocketMQ creator.Software craftsman, architect at alibaba,currently in charge of Aliware MQ.:vongoslingap

3、© 201 7 A l i b a b a M i d d l e w a r eG r o u pCONTENTS01 Distributed Messaging and Strea02 RocketMQ History、ArchitectuDomainvanced Features03 Typical Scene、Best Practice、Performance Tuning04 RocketMQ 4.X© 201 7 A l i b a b a M i d d l e w a r e G r o u p01Distributed Messaging

4、and StreaDomain 合抱之木生于毫末；九层之于累土；千里之行始于足下© 201 7 A l i b a b a M i d d l e w a r e G r o u pDistributed Systems Domain© 201 7 A l i b a b a M i d d l e w a r e G r o u pDistributed Messaging and Strea© 201 7 A l i b a b a M i d d l e w a r e G r o u pDomainDistributed Messaging and Str

5、eaProjects© 201 7 A l i b a b a M i d d l e w a r e G r o u p02RocketMQ History、Architectuvanced Features千磨万击还坚韧，西南北风© 201 7 A l i b a b a M i d d l e w a r e G r o u pApacheRocketMQNapoliActiveMQ kernal,used for B2B Trading and other scenariosMetaQ v2.0 v3.0RocketMQ v3.0OS2016200720102011

6、20122015NotifyBorn from multicoloredstone project, used forB2C TradingAliware MQ v1.0Notify v3.0MetaQ v1.0Design for orderly message, especial for messages accumulation© 201 7 A l i b a b a M i d d l e w a r e G r o u pMetaQOrderly message，Pull mRocketMQTranPush mion message,Professional Busine

7、ss Cloud MQNotifyAliware MQ© 201 7 A l i b a b a M i d d l e w a r e G r o u p1400Total number of Message(Unit: Billion)650194.233.2102013 double 11 day2015 double 11 day2014 double 11 day2016 double 11 day2012 double 11 day© 201 7A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l

8、 i b a b a M i d d l e w a r e G r o u pAliware MQ© 201 7 A l i b a b aM i d d l e w a r e G r o u pComponent M© 201 7 A l i b a b a M i d d l e w a r e G r o u pDomain M© 201 7 A l i b a b a M i d d l e w a r e G r o u pServer Macro Architecture© 201 7 A l i b a b a M i d d l e

9、w a r e G r o u pSDK Macro Architecture© 201 7 A l i b a b a M i d d l e w a r e G r o u pDeploy Architecture© 201 7 A l i b a b a M i d d l e w a r eG r o u pCovanced features Producer tranion© 201 7 A l i b a b a M i d d l e w a r e G r o u pCovanced features HA© 201 7 A l i b

10、a b a M i d d l e w a r e G r o u pCovanced features Message Tracing© 201 7 A l i b a b a M i d d l e w a r e G r o u pCovanced features Relaying© 201 7 A l i b a b a M i d d l e w a r e G r o u p03Typical Scene、Best Practice行路难，行路难，多歧路，今。长风破浪会有时，直挂云帆济沧海。© 201 7 A l i b a b a M i d d

11、l e w a r e G r o u pHeterogeneous Application Integration12loosen the coupling between applications3Backbone for EDA or CEPData ReplicatorChannel45Strea Channel© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p² Producersend backoff

12、 send reliability send oneway² Consumerconsume duplicate consume slow consume batch© 201 7A l i b a b aM i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u pü File Descriptors (FD) Limitsü Cffiü Memory lock limitsü NUMA Disableü PageC

13、ache flush tuningü Deadline I/O Schedule© 201 7 A l i b a b a M i d d l e w a r eG r o u p04RocketMQ 4.X咬定青山不放松，立根原在破岩中© 201 7 A l i b a b a M i d d l e w a r e G r o u pE-commerce DomainIOT DomainMassive Terminal Online Massive ThroughputHigh Concurrency High ReliabilityFinal DomainB

14、ig data Domain© 201 7A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u pCommuEc

15、osystem© 201 7A l i b a b a M i d d l e w a r e G r o u pCommuEcosystemü Workshop,Meetupsü Code Marathonü Committers© 201 7A l i b a b a M i d d l e w a r e G r o u pcollaborative software development; commercial-friendly standard license; consistentlyhigh-quality software;

16、respectful, honest, technical-based interaction; faithfulimplementation of standards; security as a mandatory feature© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r e G r o u p© 201 7 A l i b a b a M i d d l e w a r