Windows环境下Kafka与Zookeeper安装配置与启动过程.doc

Windows环境下Kafka与Zookeeper安装配置与启动过程1.软件下载1.1下载jdk1.8.0_601.2 下载zookeeper-3.4.9.tar.gz1.3下载kafka_2.12-.tgz2.环境准备安装配置java环境，配置jdk1.8为操作系统java运行时环境2.1安装jdk1.8将网站下载的jdk1.8.0_60压缩包解压缩到 c:program filesjava目录下,如下图：2.2 配置java环境如图在“控制面板-系统和安全-系统”页，鼠标点击“高级系统设置”，在弹出的“系统属性”页面选择高级tab页：点击“环境变量”按钮后，在弹出的环境变量对话框中点击“系统变量”栏中的“新建”按钮，在弹出的“新建系统变量”窗口中输入如下：点确定确定后，再配置JRE_HOME同时打开一个cmd窗口在其中输入：SET JAVA_HOME=C:Program FilesJavajdk1.8.0_60SET JRE_HOME=%JAVA_HOME%jreSET CLASSPATH=.;%JAVA_HOME%/lib/dt.jar;%JAVA_HOME%/lib/tools.jar;%JRE_HOME%lib;%JRE_HOME%librt.jar;%JRE_HOME%libjce.jar;%JRE_HOME%libmetadata-extractor-2.4.0-beta-1.jar;%JRE_HOME%libmediautil-1.0.jar;SET Path=%JAVA_HOME%bin;%JAVA_HOME%jrebin;在命令行输入java -version，输出如下，表明系统已启用java配置3.Zookeeper安装配置3.1 解压与目录设计在目录e:apachezookeeper下解压文件zookeeper-3.4.9.tar.gz,并将解压文件夹改名为zk0,复制该文件夹到相同目录下分别改名为zk1、zk2.3.2在zk0zk1zk2目录下都创建空文件夹data和logs，如下图3.2在data目录中创建文件myid，设置zookeeper服务器的序号。Myid的内容分别为0、1、23.3配置zookeeper的属性配置zk0zk1zk2目录下的config/zoo.cfg文件。zk0/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk0/datadataLogDir=E:/Apache/zookeeper/zk0/logsclientPort=2181server.0=:2888:3888server.1=:2889:3889server.2=:2890:3890zk1/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk1/datadataLogDir=E:/Apache/zookeeper/zk1/logsclientPort=2182server.0=:2888:3888server.1=:2889:3889server.2=:2890:3890zk2/config/zoo.cfg文件内容为：tickTime=2000initLimit=5syncLimit=2dataDir=E:/Apache/zookeeper/zk2/datadataLogDir=E:/Apache/zookeeper/zk2/logsclientPort=2183server.0=:2888:3888server.1=:2889:3889server.2=:2890:38903.4启动zookeeper server开三个cmd窗口，依次执行如下指令：E:cd E:Apachezookeeperzk0binzkServer.cmdE:cd E:Apachezookeeperzk2binzkServer.cmdE:cd E:Apachezookeeperzk1binzkServer.cmd在三个窗口未同时启动前，前面开的窗口会报警：三个窗口都启动后，如下表明zookeeper正常启动：3.5启动zkclient测试 zk server新开一个cmd窗口，执行如下指令E:cd E:Apachezookeeperzk0binzkCli.cmd -server :2182在新开的windows的cmd窗口执行如下命令：create /zk hellocreate /zk/subzkget /zkLs /zk反馈如下：至此，zookeeper安装完毕4.Kafka安装配置4.1 目录配置解压kafka_2.12-.tgz的内容到E:Apachekafka目录下，并将文件命名为kafka0,复制该文件夹到相同目录，命名为kafka1,如图在这两个文件夹下创建logs和matrix文件夹，如图4.2 perties配置修改两个kafka实例的config/perties文件，内容分别如下：kafak0/config/pertiesbroker.id=0port=9092work.threads=2num.io.threads=2socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=./logsnum.partitions=2erval.messages=10000erval.ms=1000log.retention.hours=168log.segment.bytes=536870912erval.mins=10zookeeper.connect=:2181,:2182,:2183zookeeper.connection.timeout.ms=1000000erval.secs=5kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporterkafka.csv.metrics.dir=E:/Apache/kafka/kafka0/matrixkafka.csv.metrics.reporter.enabled=falsekafak1/config/pertiesbroker.id=1port=9093work.threads=2num.io.threads=2socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=./logsnum.partitions=2erval.messages=10000erval.ms=1000log.retention.hours=168log.segment.bytes=536870912erval.mins=10zookeeper.connect=:2181,:2182,:2183zookeeper.connection.timeout.ms=1000000erval.secs=5kafka.metrics.reporters=kafka.metrics.KafkaCSVMetricsReporterkafka.csv.metrics.dir=E:/Apache/kafka/kafka1/matrixkafka.csv.metrics.reporter.enabled=false4.3 启动kafka server在新开的windows的cmd窗口执行如下命令：E:cd E:Apachekafkakafka1.binwindowskafka-server-start.bat .perties在新开的windows的cmd窗口执行如下命令：E:cd E:Apachekafkakafka0.binwindowskafka-server-start.bat .perties4.4 创建主题创建主题，命名为“test2”，replication factor=2（因为2个Kafka服务器在运行）。如果集群中所运行的Kafka服务器不止2个，可以相应增加replication-factor，从而提高数据可用性和系统容错性。在新开的windows的cmd窗口执行如下命令：E:cd E:Apachekafkakafka0binwindowskafka-topics.bat -create -zookeeper :2181,:2182,:2183 -replication-factor 2 -partitions 2 -topic test24.5启动producer在新开的windows的cmd窗口执行如下命令：E:cd E:Apachekafkakafka0binwindowskafka-console-producer.bat -broker-list :9092,:9093 -topic test24.6 启动consumer在