Cosbench工具使用介绍

Cosbench简介COSBench主要是用于测试云存储系统中的对象存储，在ceph中，测试的是存储池pool的性能，不是针对某个rbd块设备。其组件主要包括driver和controller；COSbenchconsistsoftwokeycomponents:Driver(alsoreferredtoasCOSBenchDriverorLoadGenerator):oResponsibleforworkloadgeneration,issuingoperationstotargetcloudobjectstoragandcollectingperformancestatistics.（负责实际的性能测试）oCanbeaccessedviahttp://<driver-host>:18088/driver/index.html.Controller(alsoreferredtoasCOSBenchController):oResponsibleforcoordinatingdriverstocollectivelyexecuteaworkload,collectinganaggregatingruntimestatusorbenchmarkresultsfromdriverinstances,andacceptinworkloadsubmissions.（负责管理Driver和添加工作负载）oCanbeaccessedviahttp://<controller-host>:19088/controller/index.html.安装方法2.1安装Java环境mkdir/usr/java拷贝jdk1.7.0_79.tar.gz到/usr/java路径下##解压tar-zxfjdk1.7.0_79.tar.gz##配置环境变量vi/etc/profile添加上下面4句JAVA_HOME=/usr/java/jdk1.7.0_79CLASSPATH=.:$JAVA_HOME/lib.tools.jarPATH=$JAVA_HOME/bin:$PATHexportJAVA_HOMECLASSPATHPATH##使环境变量马上生效./etc/profile##验证Java环境配置好了[root@iozone-102java]#java-versionjavaversion"1.7.0_79"Java(TM)SERuntimeEnvironment(build1.7.0_79-b15)JavaHotSpot(TM)64-BitServerVM(build24.79-b02,mixedmode2.2.安装cosbench环境2.2.1启动脚本前，还需要确认ncat是否已安装成功[root@localhost0.4.2]#rpm-qa|grepncatnmap-ncat-6.40-4.el7.x86_64[root@localhost0.4.2]#2.2.2解压tar-zxfcosbench-master.tar.gz。cd/root/cosbench-master/0.4.2，centos7上直接运行启动脚本会报错，需要先修改下cosbench-start.sh脚本的1个参数：[root@localhost0.4.2]#vicosbench-start.sh#!/bin/bash…TOOL="nc"TOOL_PARAMS=""---原来TOOL_PARAMS="-i0"不修改可能会有以下报错：Launchingosgiframwork...Successfullylaunchedosgiframework!Bootingcosbenchdriver...Ncat:Invalid-itimeout(mustbegreaterthan0).QUITTING..Ncat:Invalid-itimeout(mustbegreaterthan0).QUITTING..Ncat:Invalid-itimeout(mustbegreaterthan0).QUITTING.2.2.3执行unsethttp_proxy，并将firewall关闭掉。[root@localhost0.4.2]#systemctlstatusfirewalld.servicefirewalld.service-firewalld-dynamicfirewalldaemonLoaded:loaded(/usr/lib/systemd/system/firewalld.service;enabled)Active:active(running)since六2016-09-1022:09:19CST;1h44minagoMainPID:762(firewalld)CGroup:/system.slice/firewalld.service└─762/usr/bin/python-Es/usr/sbin/firewalld--nofork--nopid9月1022:09:19localhost.localdomainsystemd[1]:Startedfirewalld-dynamicfirewalldaemon.[root@localhost0.4.2]#systemctlstopfirewalld.service#停止firewall[root@localhost0.4.2]#systemctldisablefirewalld.service#禁止firewall开机启动[root@localhost0.4.2]#systemctlstatusfirewalld.service2.2.4启用服务，假如我有四台客户端11-21，把cosbench-master.tar.gz的拷贝到每台客户端的/root路径下，然后解压tar-zxfcosbench-master.tar.gz。cd/root/cosbench-master/0.4.2，有几个脚本：[root@localhost0.4.2]#ll*.shcli.shcosbench-start.shcosbench-stop.shstart-all.sh---启动所有服务start-controller.sh---启动constroller服务，会自动调用cosbench-start.shstart-driver.sh---启动driver服务，会自动调用cosbench-start.shstop-all.shstop-controller.shstop-driver.sh[root@localhost0.4.2]###启动每台客户端的driver：cd/root/cosbench-master/0.4.2[root@swift-1040.4.2]#shstart-driver.shLaunchingosgiframwork...Successfullylaunchedosgiframework!Bootingcosbenchdriver......Startingcosbench-log_0.4.1[OK]Startingcosbench-tomcat_0.4.1[OK]Startingcosbench-config_0.4.1[OK]Startingcosbench-http_0.4.1[OK]Startingcosbench-cdmi-util_0.4.1[OK]Startingcosbench-core_0.4.1[OK]Startingcosbench-core-web_0.4.1[OK]Startingcosbench-api_0.4.1[OK]Startingcosbench-mock_0.4.1[OK]Startingcosbench-ampli_0.4.1[OK]Startingcosbench-swift_0.4.1[OK]Startingcosbench-keystone_0.4.1[OK]Startingcosbench-httpauth_0.4.1[OK]Startingcosbench-s3_0.4.1[OK]Startingcosbench-gcs_0.4.1[OK]Startingcosbench-librados_0.4.1[OK]Startingcosbench-scality_0.4.1[OK]Startingcosbench-cdmi-swift_0.4.1[OK]Startingcosbench-cdmi-base_0.4.1[OK]Startingcosbench-driver_0.4.1[OK]Startingcosbench-driver-web_0.4.1[OK]Successfullystartedcosbenchdriver!Listeningonport/:18089...Persistencebundlestarting...Persistencebundlestarted.----------------------------------------------!!!Servicewilllistenonwebport:18088!!!----------------------------------------------##看到以上，说明启动driver成功我们需要起一个controller来管理这4个driver，我们选择11这台来启动一个controllercd/root/cosbench-master/0.4.2viconf/controller.conf----修改controller.conf[controller]drivers=4log_level=INFOlog_file=log/system.logarchive_dir=archive[driver1]name=driver1url=11:18088/driver[driver2]name=driver2url=12:18088/driver[driver3]name=driver3url=13:18088/driver[driver4]name=driver4url=14:18088/driver##看到以下，说明启动controller成功[root@swift-10.4.2]#shstart-controller.shLaunchingosgiframwork...Successfullylaunchedosgiframework!Bootingcosbenchcontroller....Startingcosbench-log_0.4.1[OK]Startingcosbench-tomcat_0.4.1[OK]Startingcosbench-config_0.4.1[OK]Startingcosbench-core_0.4.1[OK]Startingcosbench-core-web_0.4.1[OK]Startingcosbench-controller_0.4.1[OK]Startingcosbench-controller-web_0.4.1[OK]Successfullystartedcosbenchcontroller!Listeningonport/:19089...Persistencebundlestarting...Persistencebundlestarted.----------------------------------------------!!!Servicewilllistenonwebport:19088!!!----------------------------------------------实例说明3.1 ConfiguringWorkloadsWorkloads配置文件在conf目录底下，在提交Workload之前需要，先修改配置文件，里面配置文件较多，有些是关于其他存储类型（mock|swift|ampli|s3|sproxyd）的测试，这里不做介绍。只关注Storagetype为librados的测试。语法结构如下：“containers”,“objects”,and“sizes”有多种格式，修改此参数可以灵活完成不同场景的组合测试。“containers”,“objects”,and“sizes”所支持的格式类型排列组合如下：SelectorversusElement:Keyconstant(c(num))uniform(u(min,max))range(r(min,max))sequential(s(min,max))histogram(h(min|max|ratio))containersobjectssizesSelectorversusWork:Keyinitpreparenormal(read)normal(write)normal(delete)cleanupdisposecontainersr(),s()r(),s()c(),u(),r(),s()c(),u(),r(),s()c(),u(),r(),s()r(),s()r(),s()objects

r(),s()c(),u(),r(),s()c(),u(),r()c(),u(),r(),s()r(),s()

sizes

c(),u(),h()

c(),u(),h()

Note：不同workstage，“containers”,“objects”,and“sizes”所支持的格式类型不同。storagetype="librados"的自带的配置文件模板如下：[root@localhost0.4.2]#moreconf/librados-config-sample.xml<?xmlversion="1.0"encoding="UTF-8"?><workloadname="librados-sample"description="samplebenchmarkforlibrados"><storagetype="librados"config="accesskey=<accesskey>;secretkey=<scretkey>;endpoint=<endpoint>"/><workflow><workstagename="init"><worktype="init"workers="1"config="containers=r(1,2)"/></workstage><workstagename="prepare"><worktype="prepare"workers="1"config="containers=r(1,2);objects=r(1,10);sizes=c(64)KB"/></workstage><workstagename="main"><workname="main"workers="8"runtime="30"><operationtype="read"ratio="80"config="containers=c(1,2);objects=u(1,10)"/><operationtype="write"ratio="20"config="containers=u(1,2);objects=u(11,20);sizes=c(64)KB"/></work></workstage><workstagename="cleanup"><worktype="cleanup"workers="1"config="containers=r(1,2);objects=r(1,20)"/></workstage><workstagename="dispose"><worktype="dispose"workers="1"config="containers=r(1,2)"/></workstage></workflow></workload>[root@localhost0.4.2]#参数说明：<storagetype="librados"config="accesskey=<accesskey>;secretkey=<scretkey>;endpoint=<endpoint>"/><endpoint>设置为集群的monitor节点的Publicip；accesskey=<accesskey>;secretkey=<scretkey>：这是用于身份认证的，ucs2.0中没有启动认证，配置时，此参数不需要；workstagename中最重要的<workstagename="main"><workstagename="main"><workname="main"workers="8"runtime="30"><operationtype="read"ratio="80"config="containers=c(1,2);objects=u(1,10)"/><operationtype="write"ratio="20"config="containers=c(1,2);objects=u(11,20);sizes=c(64)KB"/></work>workers:工人数，会平均分布在n个driver上并行工作，workers是一个关键属性，通常用来控制负载。runtime：运行时间；operationtype：可选read，write，writefile，deleteratio:比率，所有operation相加，ratio不能超过100；config：配置container的前缀、名称以及对象前缀，名称、对象大小，上述模板中没有前缀的部分，实例中需要加入。{cprefix:指定poolname的前缀，如：pool_containers=c(1,2)：指定poolname的后缀，所以这里写入的poolname为:pool_1，pool_2oprefix=myobjects_:指定object名称的前缀objects=u(1,1000)：指定object名称的后缀，所以这里写入的objectname为：myobjects_1，myobjects_2，....,myobjects_1000size:指定对象的大小。}division:控制worker的分配，使worker分配到无重叠的数据分区，可设置的值有3个：container|object|none；当division="container"，worker分配不同的container，即不同的pool，此时不允许workers比containers大；当division="object"，worker分配到所有的container上不同的object，此时不允许workers比objects还大。当division="none"，worker有可能操作到所有的containersorobjects。（If"division=none",itisusedtoturnoffdivisionsothateachworkerdoesexactlywhattheworkhasspecified—thereisnopartitionsofthework,soeachworkermaytouchallcontainersorobjects.）worktype：上述模板文件，worktype=normal，但没有显式声明，除了normal，还有5种特殊的类型：init(creatingspecificcontainersinbulk)<worktype="init"workers="4"config="containers=r(1,100)"/>Parameterlist:ParameterTypeDefaultCommentcontainersStringContainerselectionexpression;forexample:

c(1),r(1,100)cprefixStringmycontainers_ContainerprefixcsuffixString<null>Containersuffixprepare(insertingspecificobjectsinbulk)<worktype="prepare"workers="4"config="containers=r(1,10);objects=r(1,100);sizes=c(64)KB"/>Parameterlist:ParameterTypeDefaultCommentcontainersStringContainerselectionexpression;forexample:

c(1),u(1,100)cprefixStringmycontainers_ContainerprefixcsuffixString<null>ContainersuffixobjectsStringObjectselectionexpression;forexample:

c(1),u(1,100)oprefixStringmyobjects_ObjectprefixosuffixString<null>ObjectsuffixsizesStringSizeselectionexpressionwithunit(B/KB/MB/GB);forexample:

c(128)KB,u(2,10)MBchunkedBooleanFalseUploaddatainchunkedmode(ornot)contentString“random”(default)”zero”Fillobjectcontentwithrandomdataorall-zeroscreateContainerBooleanFalseCreaterelatedcontainerifitdoesnotexisthashCheckBooleanFalseDoworkrelatedtoobject-integritycheckingcleanup(removingspecificobjectsinbulk)<worktype="cleanup"workers="4"config="containers=r(1,10);objects=r(1,100)"/>Parameterlist:ParameterTypeDefaultCommentcontainersStringContainerselectionexpression;forexample:

c(1),u(1,100)cprefixStringmycontainers_ContainerprefixcsuffixString<null>ContainersuffixobjectsStringObjectselectionexpression;forexample:

c(1),u(1,100)oprefixStringmyobjects_Objectprefixosuf