codis3.0.3数据同步和高可用及压力测试综述

1、codis3.0.3数据同步和高可用及压力测试1 .基本信息三台机器：192.168.30.113192.168.30.114192.168.30.115zk1zk2zk3codis-dashboardcodis-fecodis-hacodis-proxy1codis-proxy2codis-proxy3codis_group1_M(6379)codis_group2_M(6379)codis_group3_M(6379)codis_group3_S(6380)codis_group1_S(6380)codis_group2_S(6380)2 .主从数据同步测试group1：redis-cli

2、-h192.168.30.113-p6379authxxx192.168.30.113:6379>authxxxxOK192.168.30.113:6379>seta1OKredis-cli-h192.168.30.115-p6380192.168.30.115:6380>authxxxOK192.168.30.115:6380>geta"1"group2:redis-cli-h192.168.30.114-p6379authxxx192.168.30.114: 6379>setb2OKredis-cli-h192.168.30.113-p63

3、80authxxx192.168.30.113:6380>getbgroup3:redis-cli-h192.168.30.115-p6379authxxx192.168.30.115: 6379>setc3OKredis-cli-h192.168.30.114-p6380authxxx192.168.30.114: 6380>getc"3"三.高可用测试1 .将proxy1offline(192.168.30.113)后，验证通过proxy2(192.168.30.114)的19000端口访问redis关闭proxy1:codisappmvxl2579c

4、odis$chmodu+x*codisappmvxl2579codis$ps-ef|grepproxycodisapp3089810May31?00:08:46/codisapp/svr/codis/bin/codis-proxy-ncpu=4-config=/codisapp/conf/codis/proxy.toml-log=/codisapp/logs/codis/proxy.log-log-level=WARNcodisappmvxl2579codis$./stop_codis_proxy.shcodisappmvxl2579codis$ps-ef|grepproxycodisapp1

5、05208345013:36pts/000:00:00grepproxycodisappmvxl2580$redis-cli-h192.168.30.114-p19000192.168.30.114:19000>authxxxxOK192.168.30.114: 19000>setd4OK192.168.30.115: 19000>getd"4"2.将proxy1offline后，且将codisserver1的6379和6380关掉，验证通过proxy3的19000端口访问redis再接着关闭server1的redis:codisappmvxl2579co

6、dis$ps-ef|grepcodis-servercodisapp105848345013:44pts/000:00:00grepcodis-servercodisapp2891910May31?00:01:28/codisapp/svr/codis/bin/codis-server*:6379codisapp2915110May31?00:01:18/codisapp/svr/codis/bin/codis-server*:6380codisappmvxl2579codis$./stop_codis_server.shcodisappmvxl2579codis$ps-ef|grepcodi

7、s-servercodisapp106268345013:44pts/000:00:00grepcodis-server通过proxy3访问正常：codisappmvxl2580$redis-cli-h192.168.30.115-p19000192.168.30.115: 19000>authxxxxxxxOK192.168.30.115: 19000>sete5OK192.168.30.116: 19000>gete"5"3.HA主从自动切换测试将servergroup3(192.168.30.115)上的6379进程kill,查看HA是否能自动实现主

8、从切换,及原主库启动后，如何处理？mvxl2530主机上将6380对应的进程kill:codisappmvxl2581codis$ps-ef|grepcodis-servercodisapp813454015:58pts/000:00:00grepcodis-servercodisapp2168210May31?00:01:17/codisapp/svr/codis/bin/codis-server*:6379codisapp2170110May31?00:01:14/codisapp/svr/codis/bin/codis-server*:6380codisappmvxl2581codis$

9、kill21682codisappmvxl2581codis$ps-ef|grepcodis-servercodisapp849454015:58pts/000:00:00grepcodis-servercodisapp2170110May31?00:01:14/codisapp/svr/codis/bin/codis-server*:6380以下显示：group3中的从库已有自动提升为主库，并且原主库已下线，从组中剔除GroupGroupID1.9999GroupID1,9999G-1ServerMasterMemory10.1630.103:6379NOONEQ34527r.1B10.16

10、.30.1Q563B010.16.30.103：6379：UpQ2.432MB/NewGroupG-2ServerMasterMemory10.163C.104637SNOONEQsynced34527MGB10163C.10363S01016.30104：6379：up2432MBServerMaster10.1630104:6330INOONEaG-3Memory2472r.i下线的原主库redis需要先启动服务，再重新加入到组中，并点击帮手小图标进行同步:G*1ServerMasterMemory10.16.30.103.6379MO：ONEQ34.527FJB；1010.1630.10

11、5:638010.1630.l03：6379zup2.432MB/10.(G-2Serv&rMasterMemory10.1630.104:6379NO:ONEsynced34527ME，1GB10.1630.103:63801016.30104l6379：UpQ2432MB/1CG-3ServerMasterMemory10.16.30.1046380NCTONEQ34.491MB；10.16.30.10563791O.l6.3O.1D4.633a：upQ2.432MB/三.大数据量下的主从同步测试插入20W个key（每次测试运行前，需要更改INSTANCE_NAME）vimredi

12、s-key.sh#!/bin/bashREDISCLT="redis-cli-h192.168.30.113-p19000-a"xxxxxxx"-n0set”ID=1while$ID-le50000doINSTANCE_NAME="i-2-$ID-VM”UUID='cat/proc/sys/kernel/random/uuid'CREATED='date"+%Y-%m-%d%H:%M:%S"'$REDISCLTvm_instance:$ID:instance_name"$INSTANCE_NA

13、ME”$REDISCLTvm_instance:$ID:uuid"$UUID"$REDISCLTvm_instance:$ID:created"$CREATED”$REDISCLTvm_instance:$INSTANCE_NAME:id"$ID"ID='expr$ID+1'done执行上面脚本，查看面板，各组主从数据同步正常，QPS达到了800以上，显示如下:1000750500250016:55:0016:55:0516:55:10.416:55:15,16:55:20ProxyIDTokenProxyAdmin1fe2e0

14、afb962642f25d*2411dd04&31&e：s10.16.30.103:1900010.16.30.103:110802633203a9ba753e5e927260d5ced936&710.16.30.104:1900010.16.30.104:11080321Sees9b567b604af0a3939b34f2529110.16.30.105:1900010.16.30.105:11080GroupGroupID1,9999roupID1,9999G-1ServerMasterMemory10.1630,103:6379NO:ONEQ43.132MB/GB

15、10.16.30.105:6380101630.103：63797Upa11073M3/GBG-2ServerMasterMemory101630104:6379MO：ONEsynced43.137MBGBr01016.30103:638010.1630.104:6379:upQ11076MBGBG=3ServerMaster10.16.30.104：6380NO：ONEQ口10.16.30.1051637910.1630.104：63BO：upsyncedMemory46.449fJBGB14.424f；lBGB五.压力测试1、/codisapp/svr/codis/bin/redis-be

16、nchmark-h192.168.30.113-p19000-a"xxxxxxc"100-n100000100个并发连接，100000个请求，检测host为localhost端口为6379的redis服务器性能。以下显示100%的请求都在15ms或13ms或1ms内处理完成。codisappmvxl2579codis$/codisapp/svr/codis/bin/redis-benchmark-h192.168.30.113-p19000-a"xxxxxxx"-c100-n100000=PING_INLINE=100000requestscomplet

17、edin0.83seconds3bytespayloadkeepalive:1100.00%<=1milliseconds120627.27requestspersecond=PING_BULK=100000requestscompletedin0.79seconds100parallelclients3bytespayloadkeepalive:1100.00%<=20milliseconds127064.80requestspersecond=SET=100000requestscompletedin1.08seconds100parallelclients3bytespayl

18、oadkeepalive:1100.00%<=14milliseconds92506.94requestspersecond=GET=100000requestscompletedin1.02seconds100parallelclients3bytespayloadkeepalive:1100.00%<=15milliseconds97560.98requestspersecond=INCR=100000requestscompletedin0.80seconds100parallelclients3bytespayloadkeepalive:1100.00%<=19mil

19、liseconds124843.95requestspersecond=LPUSH=100000requestscompletedin0.81seconds100parallelclients3bytespayloadkeepalive:1100.00%<=18milliseconds123762.38requestspersecond=LPOP=100000requestscompletedin0.82seconds100parallelclients3bytespayloadkeepalive:1100.00%<=6milliseconds121212.12requestspe

20、rsecond=SADD=100000requestscompletedin0.93seconds100parallelclients3bytespayloadkeepalive:1100.00%<=6milliseconds108108.11requestspersecond=SPOP=100000requestscompletedin0.91seconds100parallelclients3bytespayloadkeepalive:1100.00%<=16milliseconds110011.00requestspersecond=LPUSH(neededtobenchma

21、rkLRANGE)=100000requestscompletedin0.88seconds100parallelclients3bytespayloadkeepalive:1100.00%<=15milliseconds114155.25requestspersecond=LRANGE_100(first100elements)=100000requestscompletedin3.76seconds100parallelclients3bytespayloadkeepalive:1100.00%<=19milliseconds26588.67requestspersecond=

22、LRANGE_300(first300elements)=100000requestscompletedin11.85seconds100parallelclients3bytespayloadkeepalive:1100.00%<=53milliseconds8438.82requestspersecond=LRANGE_500(first450elements)=100000requestscompletedin17.56seconds100parallelclients3bytespayloadkeepalive:1100.00%<=61milliseconds5693.46

23、requestspersecond=LRANGE_600(first600elements)=100000requestscompletedin22.23seconds3bytespayloadkeepalive:1100.00%<=74milliseconds4499.44requestspersecond=MSET(10keys)=100000requestscompletedin3.24seconds100parallelclients3bytespayloadkeepalive:1100.00%<=20milliseconds30873.73requestspersecond2、/codisapp/svr/codis/bin/redis-benchmark-h192.168.30.113-p19000-a"xxxxxxx"-q-d100测试存取大小为100字节的数据包的性能codisappmvxl2579codis$/codisapp/svr/codis/bin/redis-benchmark-h192.168.30.113-p19000-a"xxxxxxx"-q-d100PING_INLINE:124223.60requestspersecondPING_BULK:129533.68requestsper