Openstack云计算-xt.docx

Openstack云计算实训谢 涛电气信息工程学院网络工程系2015年11月27日Openstack手册 目录1 Openstack安装42 安装完成需要的准备工作42.1 修改配额42.2 修改云主机类型（交换分区设置为0）42.3 修改访问与安全（开启tcp udp icmp）52.4 创建网络（内网 外网）52.5 制作镜像(centos)102.6 制作镜像（win2003）102.7 上传镜像112.8 安装虚拟机112.9 挂载磁盘给虚拟机142.10 Openstack备份和迁移152.11 openstack调整云主机大小162.12 openstack优化token162.13 Openstack开发环境搭建172.14 openstack限速 限制cpu 磁盘使用172.15 虚拟机如何安装到指定的计算节点193 扩展服务203.1 安装cinder203.2 Swift对象存储搭建214 Openstack排错方法274.1 日志排错274.2 通过nova排错275 已知版本的错误285.1 Maximum number of ports exceeded285.2 Stderr: iscsiadm: No session found.n286 参考文件291 Openstack安装1.1 在CentOS上一键安装1.单节点安装安装wget yum install wget更改为163源1.下载repo文件wget /.help/CentOS6-Base-163.repo2.备份并替换系统的repo文件rootlocalhost # cd /etc/yum.repos.d/rootlocalhost # mv CentOS-Base.repo CentOS-Base.repo.bakrootlocalhost # mv CentOS6-Base-163.repo CentOS-Base.repo3.执行yum源更新rootlocalhost # yum clean allrootlocalhost # yum makecacherootlocalhost # yum -y update安装软件源sudo yum install -y /repos/openstack/openstack-liberty/rdo-release-liberty-2.noarch.rpmyum install -y /repos/openstack/openstack-kilo/openstackrdo-release-kilo-1.noarch.rpm安装packstacksudo yum install -y openstack-packstack一键安装packstack -allinone等待安装完毕，keystonerc_admin的文件在/root目录下版本：openstack-juno/ openstack-kilo/ openstack-liberty/ openstack-trunk/2.多节点安装准备双网卡的机器若干，1个做控制节点，其他做计算节点。假设em1是第一个网卡，em2是第二个网卡（将来做vm的内网）。有以下两种部署方案：2.1 增量部署计算节点首先，按照单节点安装的方法，部署控制节点。之后，再增量部署计算节点。增量部署的手册可参考 /Adding_a_compute_node在控制节点上，编辑单节点安装生成的配置文件 packstack-answer-$date-$time.txt 修改 CONFIG_NOVA_COMPUTE_PRIVIF 和 CONFIG_NOVA_NETWORK_PRIVIF 为 em2（第二个网卡）修改CONFIG_NOVA_COMPUTE_HOSTS为计算节点的em1的ip地址。可以逗号分隔，列出多个计算节点运行sudo packstack -answer-file=$youranswerfile2.2 直接安装多节点在控制节点，参照单节点安装软件源和packstack后，直接运行 sudo packstack慢慢的回答问题就可以了，大部分可以用默认选项。只需要在私有网卡的名字替换为em2，以及提供需要安装nova compute服务的机器ip3.错误处理3.1 多节点时nagios安装错误可以暂时不安装，只是个额外的监控软件，将packstack-answer文件的对应项设置为n。packstack似乎在github上的版本已经修了这个bug3.2 偶尔安装过程中网络异常有时候，因为网络问题，会链接不上远程机器，造成安装失败。只需要重新运行packstack就可以。3.3 cinder中创建volume失败等删除/var/lib/cinder文件夹，重新运行packstack1.2 修改配额1.3 修改云主机类型（交换分区设置为0）1.4 修改访问与安全（开启tcp udp icmp）1.5 创建网络（内网 外网）创建网络（外网 在管用员界面）创建网络（内网 在管用员界面）创建子网查看拓扑1.6 制作镜像(centos)1 yum install kvm23 wget /centos/6.4/isos/x86_64/CentOS-6.4-x86_64-minimal.iso4 准备好后，执行以下命令：5 qemu-img create -f raw centos.img 10G67 /usr/libexec/qemu-kvm -m 1024 -cdrom CentOS-6.4-x86_64-minimal.iso -drive file=centos.img,if=virtio,index=1 -boot d -net nic -net user -nographic -vnc :28 用vnc工具连接安装系统9 安装完成 kill掉之前的进程10 /usr/libexec/qemu-kvm -m 1024 -drive file=centos.img, -boot c -net nic -nographic -vnc :2 11 压缩镜像12 qemu-img convert -c -f raw -O qcow2 centos.img centos64.img1.7 制作镜像（win2003）制作win2003 镜像需要加载磁盘驱动和网卡驱动 制作方法也上面类似 （请在ubuntu下完成）1.8 上传镜像注意格式和公有勾选1.9 安装虚拟机依次点击 项目-云主机-启动云主机-按需求选择和填写云主机的配置当点击运行会提示选择网络请点击lan 到已选择的网络框里 这里不要点击wan 接着点击运行到这里控制中心就会收到刚刚提交的各种参数 分配这台虚拟机到某个计算中心 大概20s安装完成。这里默认是优先安装到空隙的计算中心，如果需要指定安装到某个计算中心 需要手工到控制中心执行命令现在开始给虚拟机配置外网的ip点击如图的选项 绑定浮动ip如果这里有ip可以选择 就请选择ip 如果没有点击旁边的加号按钮会弹出如下的框点击分配ip再点击关联即可现在可以去ping刚刚关联的这个ip了 一般都可以ping到这里可以远程访问虚拟机或者使用web后台的vnc工具访问1.10 挂载磁盘给虚拟机填写磁盘大小和名称即可 选择创建云磁盘点击编辑附件点击连接云磁盘 即可1.11 Openstack备份和迁移拔掉计算节点1的网线 模拟这台机器宕机这台虚拟机是在计算节点1上现在模拟恢复1 到控制中心查询这台虚拟机的uuid2 到计算节点2 将备份的93992c63-3d1f-4702-a2cb-4a7240cafb4f放到instances下3 到控制中心修改该虚拟机的节点位置UPDATE nova.instances SET host = compute2,launched_on = compute2,node = compute2 WHERE uuid =93992c63-3d1f-4702-a2cb-4a7240cafb4f4 到dashboard界面重启这台机器mount一下网盘即可1.12 openstack调整云主机大小在Openstack中生成VM后，由于业务的变更或业务量的增加，需要对VM进行扩展。目前Openstack中提供了Resize功能，对Openstack中的源码进行了测试和分析，并进行了一定的修改同一机器中进行resize需要在配置文件nova.conf中添加：-allow_resize_to_same_host=True重启服务：service nova-api restartservice nova-compute restart1.13 openstack优化token优化表token默认情况keystone将token数据写入mysql 但又不会清除 导致这个表会变的很大 也对数据库不好备份解决方法1 Havana新增操作keystone-manage token_flush用于清除DB中过期的token信息，这样能防止数据表持续增大影响性能2 将token写入到memcache中 编辑/etc/keystone/keystone.conf的Token字段：tokendriver = keystone.token.backends.memcache.Token/etc/init.d/openstack-keystone restart1.14 Openstack开发环境搭建在生产环境下dashboard的静态文件由apache支持登录控制节点cd /data/bin/cp -Rfp /usr/share/openstack-dashboard openstack-dashboardcd /data/openstack-dashboard/openstack_dashboard/localcp /etc/openstack-dashboard/local_settings local_settings.pyvi /data/openstack-dashboard/openstack_dashboard/urls.pyurlpatterns = patterns(, url(r$, openstack_dashboard.views.splash, name=splash), url(rauth/, include(openstack_auth.urls), url(r, include(horizon.urls), url(rstatic/(?P.*)$, django.views.static.serve,document_root: /data/openstack-dashboard/static)加上这段python manage.py runserver :891.15 openstack限速 限制cpu 磁盘使用这个其实看上去比较简单，其实非常复杂。目前默认有5个Flavor，你可以编辑Flavor，创建Flavor.目前支持指定某个Flavor给租户使用。当用户要求的虚拟机不在Flavor里，那么你就可以使用这种方式，为特定的租户创建一个Flavor。目前AWS，国内的青云，应该都是类似这种方式。Public=true，表示所有租户都可以使用，Public=False 表示只有某些租户可以使用。Flavor extra Specs：这个地方看上去不起眼，不过很多功能和需求，都是可以通过这个地方满足，例如你希望对虚拟机的cpu资源限制，网络带宽限制，磁盘IO限制，都可以通过这里设置参数。nova-manage flavor set_key -name m1.small -key quota:read_bytes_sec -value 10240000nova-manage flavor set_key -name m1.small -key quota:write_bytes_sec -value 10240000nova-manage flavor set_key -name m1.small -key quota:cpu_quota -value 5000nova-manage flavor set_key -name m1.small -key quota:cpu_period -value 2500nova-manage flavor set_key -name m1.small -key quota:inboud_average -value 10240nova-manage flavor set_key -name m1.small -key quota:outboud_average -value 10240设置好后可以在这里查看到可以看到libvirt.xml中添加了qos的信息： 1.16 虚拟机如何安装到指定的计算节点nova help boot 查看nove boot帮助nova-manage service list查询有哪些计算节点net-id为nova network-list查询nova boot -flavor 1 -image 1e1de3d1-ba32-472d-9a03-db72b572d84b -availability-zone nova:compute1 -nic net-id=0fee3fd4-340f-4d21-8aff-01f11972d1fd scoke添加外网ipwan为网络配置中的外网名称nova floating-ip-create wan查看申请到的ipnova floating-ip-list绑定到虚拟机nova add-floating-ip ww-e5e348a0-8c6d-4d97-b534-5d51ae58f843 42 扩展服务2.1 安装cinderCinder主要用来作为云主机的磁盘控制节点service openstack-cinder-volume stopservice tgtd stopchkconfig openstack-cinder-volume offchkconfig tgtd offcinder节点wget /repos/openstack/openstack-havana/rdo-release-havana-6.noarch.rpmrpm -ivh rdo-release-havana-6.noarch.rpmwget /pub/epel/6/x86_64/epel-release-6-8.noarch.rpmrpm -ivh epel-release-6-8.noarch.rpmfdisk -l/etc/lvm/lvm.confdevices .filter = a/xvdc/, a/xvdb/, r/.*/.pvcreate /dev/xvdbvgcreate cinder-volumes /dev/xvdbvgdisplay(查看vg name和当前空间)vgextend cinder-volumes /dev/xvdcvgdisplay(查看当前空间)yum install openstack-cinder openstack-utils openstack-selinux/etc/cinder/api-paste.inifilter:authtokenpaste.filter_factory = keystoneclient.middleware.auth_token:filter_factoryauth_host=controllerauth_port = 35357auth_protocol = httpadmin_tenant_name=serviceadmin_user=cinderadmin_password=cinderopenstack-config -set /etc/cinder/cinder.conf DEFAULT rpc_backend mon.rpc.impl_qpidopenstack-config -set /etc/cinder/cinder.conf DEFAULT qpid_hostname controlleropenstack-config -set /etc/cinder/cinder.conf database connection mysql:/cinder:cindercontroller/cindervi /etc/tgt/targets.confinclude /etc/cinder/volumes/*service openstack-cinder-volume startservice tgtd startchkconfig openstack-cinder-volume onchkconfig tgtd on2.2 Swift对象存储搭建主要用作云存储比如对文件备份的存放 作为云空间 对其二次开发架构图如下Proxy负责解释控制控制中心的指令（对文件添加，删除，更新）后将文件放入到node中node是可以无限扩展的，既保证了文件的稳定性也保证了整个系统的冗余和增大了整个空间的存储能力控制节点是将整个swift集成到openstack通过指令可以对openstack中各种文件进行备份搭建如下在keystone节点执行1keystone user-create -name=swift -pass=swift -email=keystone user-role-add -user=swift -tenant=service -role=adminkeystone service-create -name=swift -type=object-store -description=Swift Object Store Service service_id 上面返回的idkeystone endpoint-create -service_id ab7c955f52554ec88c70ee09e940e577 -publicurl 01:8888/v1/AUTH_$(tenant_id)s -adminurl 01:8888/v1 -internalurl 01:8888/v1/AUTH_$(tenant_id)skeystone endpoint-create -service_id ab7c955f52554ec88c70ee09e940e577 -publicurl 01:8888/v2/AUTH_$(tenant_id)s -adminurl 01:8888/v2 -internalurl 01:8888/v2/AUTH_$(tenant_id)s在proxy节点01/etc/init.d/iptables stopchkconfig iptables offSELINUX=disabled/usr/sbin/ntpdate wget /repos/openstack/openstack-havana/rdo-release-havana-6.noarch.rpmrpm -ivh rdo-release-havana-6.noarch.rpmwget /pub/epel/6/x86_64/epel-release-6-8.noarch.rpmrpm -ivh epel-release-6-8.noarch.rpmyum install openstack-swift-proxy memcached python-swiftclient python-keystone-auth-token openstack-utils openstack-keystonecat /etc/swift/swift.conf swift-hashswift_hash_path_suffix = 45aa0f3c07b1e166808acd /etc/swiftopenssl req -new -x509 -nodes -out cert.crt -keyout cert.keyservice memcached startchkconfig memcached onrootlocalhost swift# cat /etc/swift/proxy-server.conf DEFAULTbind_port = 8888workers = 8user = swiftpipeline:mainpipeline = healthcheck cache authtoken keystoneauth proxy-serverapp:proxy-serveruse = egg:swift#proxyallow_account_management = trueaccount_autocreate = truefilter:cacheuse = egg:swift#memcachememcache_servers = :11211filter:catch_errorsuse = egg:swift#catch_errors filter:healthcheckuse = egg:swift#healthcheckfilter:keystoneauthuse = egg:swift#keystoneauthoperator_roles = Member,admin,swiftoperatorfilter:keystonepaste.filter_factory = keystone.middleware.swift_auth:filter_factory# swift-ring-builder account.builder rebalanceoperator_roles = admin, SwiftOperatoris_admin = truecache = swift.cachefilter:authtokenpaste.filter_factory = keystoneclient.middleware.auth_token:filter_factorydelay_auth_decision = trueadmin_tenant_name = adminadmin_user = adminadmin_password = adminadmin_token = 45aa0f3c07b1e166808aauth_host = 1auth_port = 35357auth_protocol = httpservice_port = 5000service_host = 1signing_dir = /tmp/keystone-signing-swiftauth_token = 45aa0f3c07b1e166808acd /etc/swiftswift-ring-builder account.builder create 18 2 1swift-ring-builder container.builder create 18 2 1swift-ring-builder object.builder create 18 2 1swift-ring-builder account.builder add z1-02:6002/xvdb1 100swift-ring-builder account.builder add z2-03:6002/xvdb1 100swift-ring-builder container.builder add z1-02:6001/xvdb1 100swift-ring-builder container.builder add z2-03:6001/xvdb1 100swift-ring-builder object.builder add z1-02:6000/xvdb1 100swift-ring-builder object.builder add z2-03:6000/xvdb1 100swift-ring-builder account.builderswift-ring-builder container.builderswift-ring-builder object.builderswift-ring-builder account.builder rebalanceswift-ring-builder container.builder rebalanceswift-ring-builder object.builder rebalance将生成的gz文件拷贝到存储节点上tar zcf ss.tar.gz *gzswift-init proxy-server restartchkconfig openstack-swift-proxy on存储节点上执行/etc/init.d/iptables stopchkconfig iptables offSELINUX=disabled/usr/sbin/ntpdate wget /repos/openstack/openstack-havana/rdo-release-havana-6.noarch.rpmrpm -ivh rdo-release-havana-6.noarch.rpmwget /pub/epel/6/x86_64/epel-release-6-8.noarch.rpmrpm -ivh epel-release-6-8.noarch.rpmyum install openstack-swift openstack-swift-account openstack-swift-container openstack-swift-objectcat /etc/swift/swift.conf swift-hashswift_hash_path_suffix = 45aa0f3c07b1e166808achown -R swift:swift /etc/swift/yum install openstack-swift-account openstack-swift-container openstack-swift-objectinstall xfsprogsfdisk /dev/xvdb n-p-1-wmkfs.xfs -i size=1024 /dev/xvdb1echo /dev/xvdb1 /srv/node/xvdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0 /etc/fstab mkdir -p /srv/node/xvdb1mount /srv/node/xvdb1chown -R swift:swift /srv/nodevi /etc/rsyncd.confuid = swiftgid = swiftlog file = /var/log/rsyncd.logpid file = /var/run/rsyncd.pidaddress = 02accountmax_connections = 2path = /srv/node/read only = falselock file = /var/lock/account.lockcontainermax_connections = 2path = /srv/node/read only = falselock file = /var/lock/container.lockobjectmax_connections = 2path = /srv/node/read only = falselock file = /var/lock/object.lockrsync -daemonecho rsync -daemon /etc/rc.localcat /etc/swift/account-server.confDEFAULTbind_ip = 02bind_port = 6002workers = 1pipeline:mainpipeline = account-serverapp:account-serveruse = egg:swift#accountaccount-replicatoraccount-auditoraccount-reapercat /etc/swift/container-server.confDEFAULTbind_ip = 02bind_port = 6001workers = 1pipeline:mainpipeline = container-serverapp:container-serveruse = egg:swift#containercontainer-replicatorcontainer-updatercontainer-auditorcontainer-synccat /etc/swift/object-server.confDEFAULTbind_ip = 02bind_port = 6000workers = 1pipeline:mainpipeline = object-serverapp:object-serveruse = egg:swift#objectobject-replicatorobject-updaterobject-auditorfor i in $( ls openstack-swift-* ); do service $i start; donefor i in $( ls openstack-swift-* ); do chkconfig $i on; doneswift 命令都在1上执行swift statswift upload test_container XenCenter.msi 如果是大文件swift upload test_container -S 1073741824 centos6.img swift list test_containerswift download test_container cirros-0.3.1-x86_64-disk.imgswift delete test_container cirros-0.3.1-x86_64-disk.imgswift delete test_container cirros-0.3.1-x86_64-disk.img centos6.imgglance使用swift作为后端存储删除之前的镜像vi /etc/glance/glance-api.confdefualt_store = swiftswift_store_auth_address = :5000/v2.0/swift_store_user = service:glanceswift_store_key = glanceswift_store_crea