CentOS6_Mysql_HA搭建(corosync+pacemaker).doc

CentOS6.5 Mysql HA安装手册一 环境规划系统：CentOS 6.5 64bit两台主机名称及Ip:host1(51)host2(52)对外IP（心跳IP）50网关：子网掩码：预定Mysql数据存放位置：/var/lib/mysql_drbd二 安装步骤1. 安装2台主机，操作系统为CentOS6.5 64bit1.1 确认两台主机网卡名一致，如eth0;1.2 存放Mysql数据的空间先预留，不进行划分。为了之后的配置文件统一，所以将两台主机的LVM群组改名为一样，并且留下1G为分空间作为Mysql数据空间。（这个空间也可以留在sda上，不过个人认为LVM对扇区的调整较方便。）2. 关闭服务关闭iptables及NetworkManager服务【两台主机】service iptables stop【两台主机】chkconfig iptables off【两台主机】service NetworkManager stop【两台主机】chkconfig NetworkManager off关闭selinux【两台主机】vi /etc/selinux/config将SELUNUX=ENFORCING改为SELINUX=disabled保存退出。3. 修改hosts文件【两台主机】vi /etc/hosts添加如下内容：51 host1 host1.localdomain52 host2 host2.localdomain4. 安装drbd套件4.1 安装drbd【两台主机】cd /tmp【两台主机】wget /el6.5-x86_64/atrpms/stable/atrpms-repo-6-7.el6.x86_64.rpm 【两台主机】rpm -Uvh atrpms-repo*rpm 【两台主机】yum install drbd4.2 创建drbd扇区【两台主机】lvcreate -l 100%free -n /dev/vg_host/lv_drbd执行成功后，显示结果：Logical volume lv_drbd created重启主机，使扇区生效。查看扇区命令：lvscan4.3 配置drbd现在host1上编写drbd配置文件【host1】mv /etc/drbd.conf /etc/drbd.conf.bak #将drbd.conf进行备份【host1】cp /etc/drbd.d/global_common.conf /etc/drbd.conf【host1】vi /etc/drbd.conf #编辑drbd配置文件修改drbd.conf成以下内容：global usage-count yes; # minor-count dialog-refresh disable-ip-verificationcommon protocol C; handlers # These are EXAMPLE handlers only. # They may have severe implications, # like hard resetting the node under certain circumstances. # Be careful when chosing your poison. pri-on-incon-degr /usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b /proc/sysrq-trigger ; reboot -f; pri-lost-after-sb /usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b /proc/sysrq-trigger ; reboot -f; local-io-error /usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o /proc/sysrq-trigger ; halt -f; fence-peer /usr/lib/drbd/crm-fence-peer.sh; # split-brain /usr/lib/drbd/notify-split-brain.sh root; # out-of-sync /usr/lib/drbd/notify-out-of-sync.sh root; # before-resync-target /usr/lib/drbd/snapshot-resync-target-lvm.sh -p 15 - -c 16k; # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh; startup # wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb degr-wfc-timeout 120; #2 minutes outdated-wfc-timeout 2; #2 seconds options # cpu-mask on-no-data-accessible disk # size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes # disk-drain md-flushes resync-rate resync-after al-extents # c-plan-ahead c-delay-target c-fill-target c-max-rate # c-min-rate disk-timeout on-io-error detach; net # protocol timeout max-epoch-size max-buffers unplug-watermark # connect-int ping-int sndbuf-size rcvbuf-size ko-count # allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri # after-sb-1pri after-sb-2pri always-asbp rr-conflict # ping-timeout data-integrity-alg tcp-cork on-congestion # congestion-fill congestion-extents csums-alg verify-alg # use-rle cram-hmac-alg sha1; shared-secret clusterdb; after-sb-0pri disconnect; after-sb-1pri disconnect; after-sb-2pri disconnect; rr-conflict disconnect; syncer rate 10M; al-extents 257; on-no-data-accessible io-error; resource clusterdb_res #clustedb_res为资源名，可以自己命名 on host1 #这个名字为uname n指令所执行的结果 device /dev/drbd0; disk /dev/vg_host/lv_drbd; address 51:7788; #host1的IP地址 meta-disk internal; on host2 #这个名字为uname n指令所执行的结果 device /dev/drbd0; disk /dev/vg_host/lv_drbd; address 52:7788; #host2的ip地址 meta-disk internal; 以scp将drbd.conf复制到host2上【host1】scp /etc/drbd.conf /etc/创建供DRBD记录信息的数据块【两台主机】drbdadm create-md clusterdb_res4.4 启动drbd【两台主机】service drbd start【任意一台主机】service drbd status执行之后会看到如下画面：设置主备节点，指定host1为正师机（主机）【host1】drbdadm - -overwrite-data-of-peer primary all若这时立马输入指令【host1】service drbd status会看到如下画面：其中百分比表示host1（正式机）和host2（备机）正在同步中。格式化host1正式机扇区【host1】mkfs -t ext4 /dev/drbd05. 安装Mysql5.1安装 【两台主机】yum install mysql-server mysql 变更目录及目录权限 【两台主机】mkdir /var/lib/mysql_drbd 【两台主机】chown mysql /var/lib/mysql_drbd 【两台主机】chgrp mysql /var/lib/mysql_drbd 【两台主机】chown mysql /var/lib/mysql 【两台主机】chgrp mysql /var/lib/mysql5.2配置 在host1挂载分区及创建修改配置文件 【host1】mount /dev/drbd0 /var/lib/mysql_drbd 【host1】mkdir /var/lib/mysql_drbd/data 【host1】cp /usr/share/mysql/f /var/lib/mysql_drbd/f 【host1】vi /var/lib/mysql_drbd/f在 mysql 这个区块加入datadir位置，如下mysqldatadir = /var/lib/mysql_drbd/datasocket = /var/lib/mysql/mysql.sockpid = /var/lib/mysql/mysql.pid5.3Mysql database初始化【host1】mysql_install_db -no-defaults -datadir=/var/lib/mysql_drbd/data -user=mysql执行之后，结果如下：【host1】cd /var/lib/mysql_drbd【host1】chmod -R uog+rw *【host1】chown mysql f【host1】chmod og-w f【host1】chmod 644 f【host1】chown -R mysql dataumount扇区【host1】cd 【host1】umount /var/lib/mysql_drbd将正式机转换为备机状态【host1】drbdadm secondary clusterdb_res #主机转为备机另：drbdadm primary clusterdb_res #备机转为主机6. pacemaker和crmsh的安装配置6.1安装更新pacemaker检查两台主机是否安装pacemaker【两台主机】rpm -q pacemaker显示结果若为：套件pacemaker尚未安装则执行：yum install pacemaker显示结果若为：pacemaker-1.x.x. #后面为任意版本号则执行：yum update pacemaker6.2安装crmsh 方法一： 【两台主机】wget -P /etc/yum.repos.d/ /repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/network:ha-clustering:Stable.repo 【两台主机】yum install crmsh 如果方法一不能安装，请按照方法二进行安装 方法二：下载【host1】cd /usr/tmp【host1】wget /repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/crmsh-2.1-1.6.x86_64.rpm【host1】scp /usr/tmp/crmsh*rpm host2:/usr/tmp/安装crmsh需要依赖pssh和python-lxml检查是否安装【两台主机】rpm -q pssh【两台主机】rpm -q python-pssh下载安装pssh【host1】wget /repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/pssh-2.3.1-4.1.x86_64.rpm【host1】wget /repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/python-pssh-2.3.1-4.1.x86_64.rpm【host1】scp /usr/tmp/pssh*rpm host2:/usr/tmp/【host1】scp /usr/tmp/python*rpm host2:/usr/tmp/【两台主机】rpm -ivh /usr/tmp/python*rpm【两台主机】rpm -ivh /usr/tmp/pssh*rpm安装python-lxml【两台主机】yum install python-lxml安装crmsh【两台主机】rpm -ivh /usr/tmp/crmsh*.rpm6.3配置【host1】cp /etc/corosync/corosync.conf.example /etc/corosync/corosync.conf【host1】vi /etc/corosync/corosync.conf修改内容如下：totem version: 2 secauth: off threads: 0 interface ringnumber: 0 bindnetaddr: #注意前3个数字与IP同网段，第4个数字为0 mcastaddr: mcastport: 5405 ttl: 1 . #其他不做修改建立pcmk【host1】vi /etc/corosync/service.d/pcmk内容如下：service #Load the Pacemaker Cluster Resource Manager name: pacemaker ver: 1以指令的方式建立/etc/cluster/cluster.conf【host1】cd 【host1】ccs -f /etc/cluster/cluster.conf -createcluster pacemaker1【host1】ccs -f /etc/cluster/cluster.conf -addnode host1【host1】ccs -f /etc/cluster/cluster.conf -addnode host2【host1】ccs -f /etc/cluster/cluster.conf -addfencedev pcmk agent=fence_pcmk【host1】ccs -f /etc/cluster/cluster.conf -addmethod pcmk-redirect host1【host1】ccs -f /etc/cluster/cluster.conf -addmethod pcmk-redirect host2【host1】ccs -f /etc/cluster/cluster.conf -addfenceinst pcmk host1 pcmk-redirect port=host1【host1】ccs -f /etc/cluster/cluster.conf -addfenceinst pcmk host2 pcmk-redirect port=host2复制之前的3个文件至host2：【host1】scp /etc/corosync/corosync.conf host2:/etc/corosync/【host1】scp /etc/corosync/service.d/pcmk host2:/etc/corosync/service.d【host1】scp /etc/cluster/cluster.conf host2:/etc/cluster/6.4启动pacemaker 【两台主机】service pacemaker start查看执行结果【两台主机】crm_mon -1出现以下结果6.5配置对外IP（心跳IP） 【host1】crm configure property no-quorum-policy=ignore 【host1】crm configure rsc_defaults resource_stickiness=100 【host1】crm configure property stonith-enabled=false 停止DRBD 【host1】service drbd stop 设置crm 【host1】crm configure进入crm的编辑模式，进行编辑，如下：crm(live)configure# primitive p_drbd_mysql ocf:linbit:drbd params drbd_resource=clusterdb_res op monitor interval=15scrm(live)configure# commitcrm(live)configure# ms ms_drbd_mysql p_drbd_mysql meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=truecrm(live)configure# primitive p_fs_mysql ocf:heartbeat:Filesystem params device=/dev/drbd0 directory=/var/lib/mysql_drbd fstype=ext4下面的指令，ip为对外ip，nic为网卡名称crm(live)configure# primitive p_ip_mysql ocf:heartbeat:IPaddr2 params ip=50 cidr_netmask=24 nic=eth0下面指令很关键，先另外凯恩终端机窗口，查处mysql所在目录，查看那mysql目录命令为：whereis mysqld。以我的主机为例，我的mysqld在/usr/libexec/mysqld,其余如config、datadir、pid、socket等都在前面的/var/lib/mysql_drbd/f中有配置。crm(live)configure# primitive p_mysql ocf:heartbeat:mysql params binary=/usr/libexec/mysqld config=/var/lib/mysql_drbd/f datadir=/var/lib/mysql_drbd/data pid=/var/lib/mysql/mysql.pid socket=/var/lib/mysql/mysql.sock user=mysql group=mysql additional_parameters=-bind-address=50 -user=mysql op start timeout=120s op stop timeout=120s op monitor interval=20s timeout=30scrm(live)configure# group g_mysql p_fs_mysql p_ip_mysql p_mysql crm(live)configure# colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:Mastercrm(live)configure# order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start保存设定crm(live)configure# commit退出编辑crm(live)configure# exit【两台主机】init 0重启后查看状态【两台主机】crm_mon -1结果如下：如果跑出来的结果和上图不同，有可能是某些地方输入错误，可透过一下指令检查修改：roothost1crm configure edit如果结果和上图一致，那输入指令：roothost1service drbd status则会看到drbd服务被pacemaker重新启动因为在crm configure阶段我们已改变了mysqld的启动方式，所以不要以service mysqld status指令去检查mysqld是否成功启动，因为会看到mysqld已停止的响应。题外话：/usr/lib/ocf/resource.d/heartbeat/这个目录，放着许多server套件的script，这应该就是pacemaker目前支持的service。7. 建立验证数据库roothost1 # mysql -u root -e GRANT ALL ON *.* to root% roothost1 # mysql -h 50 -P3306 -u rootmysql create database clusterdb; USE clusterdb;显示相应Database changedmysql create table simples (id int not null primary key);mysql insert into simples values (1),(2),(3),(4);mysql exit8. 建立网络短信时的机制roothost1 # crm configurecrm(live)configure# primitive p_ping ocf:pacemaker:ping params name=ping multiplier=1000 host_list= op monitor interval=15s timeout=60s op start timeout=60scrm(live)configure# clone cl_ping p_ping meta interleave=truecrm(live)configure# location l_drbd_master_on_ping ms_drbd_mysql rule $role=Master -inf: not_defined ping or ping number:lte 0crm(live)configure# commitcrm(live)configure# exit输入命令进行查看，如下：设置开机启动的service【两台主机】chkconfig drbd off【两