rabbitmq相规避方案iptables内核参数

1、1. rabbitmq 重启方案1.1重启 rabbitmq 影响AMQP 协议是实现 openstack 底层消息通信交互的重要组件，rabbitmq 是实现 AMQP 协议的实现，如果 rabbitmq 服务有异常，会造成许多服务响应异常，现象表现为：1.2.instance 的 VNC 打不开或者无法响应，提示错误；novaresize 功能无法使用，执行 resize 之后，一直在 resizing，无法执行成功；cinder 无法创建 volume，创建 volume 后一直处在 creating 中； instance 无法关联和卸载 volume，一直处在 attaching 中

2、或者很慢； instance 启动过程中，无法获取 IP 地址和无法获取元数据信息；3.4.5.原因分析：rabbitmq 作为 openstack 底层通信的重要环节，一般消费者(如 nova)都具有重连机制，当应用程序有异常的时候，会尝试去连接消息队列，从而确保服务可以正常使用。然而，目前在 openstack oslo 项目中存在一个问题：当底层的消息队列出现异常时，消费者尝试重连消息队列会失败，或者连接上功能也会异常，目前社区尝试在解决这个问题，当前解决改问题的方法：重启所有与消息队列相关的服务，确保服务能够重新连接至消息队列。1.2 影响的服务列表1、 api 相关服务列表nova

3、服务：openstack-nova-certopenstack-nova-conductor openstack-nova-consoleauth openstack-nova-novncproxy openstack-nova-schedulercinder 服务：openstack-cinder-scheduler openstack-cinder-backup openstack-cinder-volumemanila 服务：openstack-manila-scheduler openstack-manila-shareceilometer 服务：openstack-ceilomete

4、r-collector openstack-ceilometer-notification openstack-ceilometer-alarm-notifierneutron 服务：管理服务列表1.3 重启服务的工具#!/bin/bash# Ver:v1.0# Author:HappyLua #:hai# Copyright EasyStack, Inc. #s.# Used to restart all servirelated to rabbitmq-server when rabbitmq-server have been restarted or stoppedNODES_LIST=

5、node-1 node-2 node-3SERVI_LIST=openstack-nova-cert openstack-nova-conductor openstack-nova-consoleauth openstack-nova-novncproxy openstack-nova-scheduler openstack-cinder-scheduler openstack-cinder-backup openstack-cinder-volume openstack-manila-scheduler openstack-manila-share openstack-ceilometer-

6、collector openstack-ceilometer-notification neutron 服务：clone_p_neutron-l3-agent clone_p_neutron-dhcp-agent clone_p_neutron-metadata-agent clone_p_neutron-openvswitch-agentbilling 服务：billing-centralheat 服务：p_openstack-heat-engine ceilometer 服务：p_openstack-ceilometer-central p_openstack-ceilometer-ala

7、rm-evaluatorneutron-serveropenstack-ceilometer-alarm-notifier neutron-server_SERVI_LIST=clone_p_neutron-l3-agent clone_p_neutron-dhcp-agent clone_p_neutron-metadata-agent clone_p_neutron-openvswitch-agent billing-central p_openstack-heat-engine p_openstack-ceilometer-central p_openstack-ceilometer-a

8、larm-evaluator function resstart_rabbitmq_service_list()for node in $NODES_LIST dofor service in $SERVI_LIST dossh root$node service $service restart 2/dev/null return_code=$?if $return_code -ne 0 ;thenecho -e 03331m$node $service restart fail,please checkit0330mcontinuefi sleep 2done donefunction r

9、estart_service_list()for service in $_SERVI_LIST doresource restart $service if $? -ne 0 ;thenecho -e 03331m$service restart fail,please check0330m continuefi sleep 2done2.iptables 标准配置文档*filter:INPUT ACCEPT 0:0:FORWARD ACCEPT 0:0:OUTPUT ACCEPT 4:653-A INPUT -p icmp -m commentment 000 accept all icm

10、p requests -j ACCEPT-A INPUT -i lo -m commentment 001 accept all to loerface -j ACCEPT-A INPUT -m commentment 002 accept related established rules -m se -se RELATED,ESTABLISHED -j ACCEPT-A INPUT -s /32 -p tcp -m multiport -sports 4369,5672,15672,41055,55672,61613 -m comment ment 003 remote rabbitmq

11、-j ACCEPT-A INPUT -p tcp -m multiport -sports 8140 -m commentment 004 remote puppet -j ACCEPT-A INPUT -p tcp -m multiport -dports 6800:7100 -m commentment 011 ceph-osd allow -j ACCEPT-A INPUT -p tcp -m multiport -ports 22 -m commentment 020 ssh -j ACCEPT-A INPUT -p tcp -m multiport -ports 80,443 -m

12、commentment 100 http -j ACCEPT-A INPUT -p tcp -m multiport -ports 3306,3307,4567,4568,49000 -m commentment 101-j ACCEPT-A INPUT -p tcp -m multiport -ports 5000,35357 -m commentment 102 keystone -j ACCEPT-A INPUT -p tcp -m multiport -ports 8080,6000,6001,6002 -m commentment 103 swift -j ACCEPT-A INPU

13、T -p tcp -m multiport -ports 9292,9191,8773 -m commentment 104 glance -j ACCEPT-A INPUT -p tcp -m multiport -ports 8774,8775,8776,6080 -m commentment 105 nova -j ACCEPT-A INPUT -p tcp -m multiport -ports 4369,5672,5673,41055 -m commentment 106 rabbitmq -j ACCEPT-A INPUT -p tcp -m multiport -ports 11

14、211 -m commentment 107 memcached tcp -j ACCEPT-A INPUT -p udp -m multiport -ports 11211 -m commentment 107 memcached udp -j ACCEPT-A INPUT -p tcp -m multiport -ports 873 -m commentment 108 rsync -j ACCEPT-A INPUT -p tcp -m multiport -ports 3260 -m commentment 109 iscsi -j ACCEPT-A INPUT -p tcp -m mu

15、ltiport -ports 9696 -m commentment 110 neutron -j ACCEPT-A INPUT -p udp -m multiport -ports 67 -m commentment 111 dhcp-server -j ACCEPT-A INPUT -p udp -m multiport -ports 53 -m commentment 111 dns-server -j ACCEPT-A INPUT -p udp -m multiport -ports 123 -m commentment 112 ntp-server -j ACCEPT-A INPUT

16、 -p udp -m multiport -ports 5404 -m commentment 113 corosync-input -j ACCEPTfunction main()restart_rabbitmq_service_list restart_service_listmain关于 iptables 规则的说明：1、 ceph 相关，需要放行 ceph osd 的端口，否则扩容后会造成 hearbeat 心跳异常，影响其他 osd 节点，编写如下规则：2、 zabbix相关3、 iperf 测试相关注意事项：通过 roller 扩容计算/融合节点时，由于考虑到 ceph 同步问题，

17、扩容的时候只选择 compute 角色而没有勾选 osd 角色，会导致 compute 节点扩容完之后，并未添加放行 ceph osd 的 iptables 的规则，需要在手动扩容 ceph 的时候添加对应的 iptables规则！3.内核参数优化1、kernel 优化参数kernel.sched_min_granularity_ns = 4000000kernel.sched_latency_ns = 20000000kernel.sched_wakeup_granularity_ns = 4000000-A INPUT -p tcp -dport 5001 -m commentment i

18、perf -j ACCEPT-A INPUT -m tcp -p tcp -dport 10050 -m commentment zabbix agent -j ACCEPT-A INPUT -p tcp -m multiport -dports 6800:7100 -m commentment 011 ceph-osd allow -j ACCEPT-A INPUT -p udp -m multiport -ports 5405 -m commentment 114 corosync-output -j ACCEPT-A INPUT -p udp -m multiport -ports 58

19、882 -m commentment 115 openvswitch db -j ACCEPT-A INPUT -p tcp -m multiport -ports 5666 -m commentment 116 nrpe-server -j ACCEPT-A INPUT -p tcp -m multiport -ports 16509 -m commentment 117 libvirt -j ACCEPT-A INPUT -p tcp -m multiport -ports 49152:49215 -m commentment 118 libvirt migration -j ACCEPT

20、-A INPUT -s /24 -p tcp -m multiport -ports 5900:6100 -m commentment 118 vnc ports-j ACCEPT-A INPUT -p tcp -m multiport -ports 8777 -m commentment 119 ceilometer -j ACCEPT-A INPUT -p gre -m commentment 334 accept gre -j ACCEPT-A INPUT -p tcp -dport 5001 -m commentment iperf -j ACCEPT-A INPUT -m tcp -

21、p tcp -dport 10050 -m commentment zabbix agent -j ACCEPT-A INPUT -p tcp -m tcp -d /24-m multiport -destination-ports 7888,8080 -j ACCEPT -m commentment balant monitor-A INPUT -p tcp -m tcp -s /24-dport 7999 -j ACCEPT -m commentment balant monitor-A INPUT -m commentment 999 drop all other requests -j

22、 DROPCOMMIT内核参数说明：1、 部分参数是系统自适应调整，根据系统环境自动调整参数，所以可能会有一些参数不一kernel.core_pipe_limit = 0kernel.keys.root_maxkeys = 1000000kernel.keys.root_maxbytes = 25000000kernel.slow-work.max-threads = 322、netfilter 优化参数filter.nf_conntrack_icmp_timeout = 30filter.nf_conntrack_max = 1048576filter.nf_conntrack_count

23、= 10678net.nf_conntrack_max = 10485763、网络优化参数 net.ipv4.ip_nonlocal_bind = 1net.ipv4.tcp_keepalive_time = 30net.ipv4.tcp_keepalive_probes = 8 net.ipv4.tcp_keepalive_vl = 3 net.ipv4.tcp_retries2 = 5net.ipv4.tcp_mem = 139383361858444827876672net.ipv4.udp_mem = 139383361858444827876672net.ipv4.conf.all.forwarding = 1net.ipv4.conf.all.accept_redirects = 0net.ipv4.conf.default.forwarding