CentOS6.5安装elasticsearchlogstashkibana详解

1、CentOS6.5 安装 elasticsearch+logstash+kibana部署中心节点如果有防火墙需要放开这些端口：port 80 (for the web interface)port 5544 (to receive remote syslog messages)port 9200 (so the web interface can access elasticsearch)安装java 环境和 Apache 服务yum install -y java-1.7.0-openjdk httpd安装 ES 环境 elasticsearch (JVM版本建议： Java 8 updat

2、e 20 or later, or Java 7 update 55 or later)wgethttps:/download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.6.0.noarch.rpm && yum install elasticsearch-1.6.0.noarch.rpm -y安装 logstash环境wgethttps:/download.elastic.co/logstash/logstash/packages/centos/logstash-1.5.1-1.noarch.rpm &

3、;& yum install logstash-1.5.1-1.noarch.rpm -y安装 kibana 环境wget /kibana/kibana/kibana-3.1.2.zip && unzip kibana-3.1.2.zip && mv kibana-3.1.2 /var/www/html/kibana安装 elasticsearch插件cd /usr/share/elasticsearch/ && ./bin/plugin -installmobz/elas

4、ticsearch-head&& ./bin/plugin-installlukas-vlcek/bigdesk/2.5.0修改 elasticsearch配置文件vim /etc/elasticsearch/: elasticsearch去掉前面 ”#”号http.cors.enabled: truehttp.cors.allow-origin:“01”启动 elasticsearch/etc/rc.d/init.d/elasticsearch start测试 elasticsea

5、rchrootlocalhost # curl :9200"status" : 200,"name" : "Agamotto","cluster_name" : "elasticsearch","version" : "number" : "1.4.1","build_hash" : "89d3241d670db65f994242c8e8383b169779e2d4","

6、;build_timestamp" : "2015-11-26T15:49:29Z","build_snapshot" : false,"lucene_version" : "4.10.2","tagline" : "You Know, for Search"rootlocalhost #配置 logstash，如下是Logstash的配置文件rootlocalhost conf.d# vim/etc/logstash/conf.d/logstasg_access.

7、confinput file type => "apache"path => "/etc/httpd/logs/*_log","/var/log/nginx/*.log"file type => "syslog"path => "/var/log/messages", "/var/log/syslog" file type => "nova"path => "/var/log/nova/*.log"

8、file type => "cinder"path => "/var/log/cinder/*.log" file type => "neutron"path => "/var/log/neutron/*.log" file type => "ceilometer"path => "/var/log/ceilometer/*.log" filter if type = "syslog" grok match =>

9、"message" =>"%SYSLOGTIMESTAMP:syslog_timestamp %SYSLOGHOST:syslog_hostname %DATA:syslog_program(?:%POSINT:syslog_pid)?: %GREEDYDA TA:syslog_message" add_field => "received_at", "%timestamp" add_field => "received_from", "%host" sys

10、log_pri date match => "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" else if type = "apache" if path = "access" mutate replace => loglevel =>"apache_access" grok match => "message" =>"%COMBINEDAPACHELO

11、G" date match => "timestamp" ,"dd/MMM/yyyy:HH:mm:ss Z" else if path = "error" mutate replace => loglevel => "apache_error" else mutate replace => loglevel =>"apache_random_logs" else grok match =>"message","%TIME

12、STAMP_ISO8601:logtime %NUMBER:pid %WORD:log level %DATA:process %GREEDYDATA:other"output stdout codec => rubydebug elasticsearch_http host => "01"port => "9200"启动 logstashrootlocalhost bin#/opt/logstash/bin/logstash -f/etc/logstash/conf.d/logstasg_acces

13、s.conf配置 kibanavim/var/www/html/kibana/config.js修改其中一行es 的 IP 地址为 ES 服务器的 IP 地址elasticsearch: "01:9200”,apache 配置如下：<VirtualHost 01:80>ServerAdmin adminDocumentRoot /var/www/html/kibanaServerName ErrorLog logs/-error_logCustomLog logs/-access_log common<

14、/VirtualHost>启动 apache/etc/rc.d/init.d/httpd start完成之后会在 logstash中看到如下文件!elk(/img/ELK1.png)01 web 页面中展示elkelk删除 ES 日志存储：rootlocalhost bin# curl -XDELETE'01:9200/logstash-2015.06.03*'两个插件路径：01:9200/_plugin/head/01:9

15、200/_plugin/bigdesk/部署多个logstash节点首先卸载低版本的java 环境，然后安装java 环境yum install -y java-1.7.0-openjdk安装 logstash环境wget/logstash/logstash/packages/centos/logstash-1.4.2-1_2c0f5a1.noarch.rpm && yum installlogstash-1.4.2-1_2c0f5a1.noarch.rpm -y配置 logstash，如下是Logstash的配

16、置文件rootlocalhost conf.d# vim/etc/logstash/conf.d/logstasg_access.confinput file type => "apache"path => "/etc/httpd/logs/*_log","/var/log/nginx/*.log"file type => "syslog"path => "/var/log/messages", "/var/log/syslog" file type

17、=> "nova"path => "/var/log/nova/*.log" file type => "cinder"path => "/var/log/cinder/*.log" file type => "neutron"path => "/var/log/neutron/*.log" file type => "ceilometer"path => "/var/log/ceilometer/

18、*.log" filter if type = "syslog" grok match => "message" =>"%SYSLOGTIMESTAMP:syslog_timestamp %SYSLOGHOST:syslog_hostname %DATA:syslog_program(?:%POSINT:syslog_pid)?: %GREEDYDA TA:syslog_message" add_field => "received_at", "%timestamp"

19、 add_field => "received_from", "%host" syslog_pri date match => "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" else if type = "apache" if path = "access" mutate replace => loglevel =>"apache_access" grok match => "message" =>"%COMBINEDAPACHELOG" date match =&g