在RHEL 5上安装并配置iSCSI Initiator软件.doc

在RHEL 5上安装并配置iSCSI Initiator软件RHEL 5已开始在内核中加入了对iSCSI的支持，使用的 iSCSI Initiator软件是Open-iSCSI Initiator，支持万兆网卡，其配置方式与RHEL 4及更早的RedHat Linux发行版中的iSCSI Initiator有很大的区别。一、安装并配置iSCSI Initiator软件以下以RHEL 5 x86版本为例介绍如何安装并配置iSCSI initiator。1.安装iSCSI Initiator把RHEL5 x86第一张安装光盘挂载到/mnt目录下，之后进入/mnt/Server目录进行安装。rootpe03 Server# cd /mnt/Server/rootpe03 Server# rpm -ivh iscsi-initiator-utils-42-0.5.el5.i386.rpmwarning: iscsi-initiator-utils-42-0.5.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186Preparing. # 100%1:iscsi-initiator-utils # 100%这个安装将iscsidiscsiadm安装到/sbin目录下，它还将把默认的配置文件安装到/etc/iscsi目录下：/etc/iscsi/iscsid.conf 所有刚发起的iSCSI session默认都将使用这个文件中的参数设定，可以使用iscsiadm命令针对Target的一些参数进行设置。/etc/iscsi/initiatorname.iscsi 软件iSCSI initiator的intiator名称配置文件。2.确认iscsi服务随系统启动而启动用chkconfig检查iscsi和iscsid服务在运行级别3和5中随系统的启动而自动启动rootpe03 Server# chkconfig -list |grep iscsiiscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:offiscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:off如果iscsid和iscsi没有随系统的启动而启动，则用chkconfig设置这两个服务在系统运行级别为3和5时随系统启动而启动rootpe03 Server# chkconfig iscsi -level 35 onrootpe03 Server# chkconfig iscsid -level 35 on3.设置InitiatorName用vi编辑/etc/iscsi/initiatorname.iscsi文件，文件内容如下InitiatorName=.max:pe03本例中InitiatorName设置为.max:pe03注意：l InitiatorName这个单词必须注意大小写，同时，必须顶格写，xxxx代表要设 置的initiator名称，请遵循iqn命名规范。l iqn规范定义的InitiatorName格式为iqn.domaindate. :optional name，例如：.h3c:dbserver。4.启动iscsi服务用service iscsi start启动iSCSI服务。rootpe03 Server# service iscsi startTurning off network shutdown. Starting iSCSI daemon: OK OK Setting up iSCSI targets: OK 用service iscsi status及service iscsid status查看iscsi相关服务的运行状态rootpe03 Server# service iscsi statusiscsid (pid 3697 3696) is running.rootpe03 Server# service iscsid statusiscsid (pid 3697 3696) is running.5.分配存储资源，在Linux上执行target的发现RHEL 5上当前的iSCSI Initiator版本只支持sendtargets 的发现方式，不支持SLP和iSNS 假设后端存储为IX5000，iSCSI业务IP地址为00，则使用下面的命令执行target的发现：rootpe03 Server# iscsiadm -m discovery -t sendtargets -p 00:3260因为此时还没有在IX5000上创建改initiator并分配卷，这个命令执行成功后没有任何输出，但是此时到IX5000上会查看到有新的initiator生成。在IX5000上把卷分配给Linux服务器后，再次执行target的发现：rootpe03 Server# iscsiadm -m discovery -t sendtargets -p 00:3260iscsiadm: unexpected SendTargets data: 00:3260,1 :h3c:200realm.rhel5此时发现了一个新的target，target名称为:h3c:200realm.rhel5注：在IP SAN存储设备上把相应的存储空间分配给RedHat Linux服务器的具体操作请参照各存储设备相关的指导书6.登录targetrootpe03 Server# iscsiadm -m node -T :h3c:200realm.rhel5 -p 00:3260 l这里-T后面跟target名字，最后的l是login这个词的第一个字母，不是数字1用iscsiadm -m session i查看SiCSI session和设备信息 rootpe03 # iscsiadm -m session -iiscsiadm version 2.0-742*Session (sid 0) using module tcp:*TargetName: :h3c:200realm.rhel5Portal Group Tag: 1Network Portal: 00:3260iSCSI Connection State: LOGGED INInternal iscsid Session State: NO CHANGE*Negotiated iSCSI params:*HeaderDigest: NoneDataDigest: NoneMaxRecvDataSegmentLength: 65536MaxXmitDataSegmentLength: 65536FirstBurstLength: 65536MaxBurstLength: 262144ImmediateData: NoInitialR2T: YesMaxOutstandingR2T: 1*Attached SCSI devices:*Host Number: 3 State: runningscsi3 Channel 00 Id 0 Lun: 0Attached scsi disk sdb State: runningOpen-iSCSI是通过以下iSCSI数据库文件来实现永久配置的：Discovery (/var/lib/iscsi/send_targets)：在/var/lib/iscsi/send_targets目录下，生成一个目标服务器信息文件，文件名为“目标服务的IP，端口号”（例如“00,3260”）。此文件用来记录目标服务器信息。Node (/var/lib/iscsi/nodes)：在/var/lib/iscsi/nodes目录下，生成一个或多个以目标服务器上的Target名命名的文件夹，每文件夹下有一个用来记录目标服务器上特定Target信息的文件。iscsiadm是用来管理（更新、删除、插入、查询）iSCSI配置数据库文件的命令行工具，用户能够用它对iSCSI nodes、sessions、connections和discovery records进行一系列的操作。二、.对新发现的磁盘进行分区并创建文件系统1、先用fdisk l查看新的磁盘名称，这里我们发现了一个100GB的磁盘，设备名为/dev/sdbrootpe03 Server# fdisk l.Disk /dev/sdb: 107.3 GB, 107373133824 bytes255 heads, 63 sectors/track, 13054 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDisk /dev/sdb doesnt contain a valid partition table2、用fdisk /dev/sdb进行分区，本例中我们把整个磁盘分成一个主分区/dev/sdb1rootpe03 Server# fdisk /dev/sdbDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel. Changes will remain in memory only,until you decide to write them. After that, of course, the previouscontent wont be recoverable.The number of cylinders for this disk is set to 13054.There is nothing wrong with that, but this is larger than 1024,and could in certain setups cause problems with:1) software that runs at boot time (e.g., old versions of LILO)2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK)Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)Command (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-13054, default 1): 此处回车，使用默认值1 Using default value 1Last cylinder or +size or +sizeM or +sizeK (1-13054, default 13054): 此处回车使用默认值13054Using default value 13054Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.3、用mkfs命令在/dev/sdb1上创建ext3文件系统rootpe03 Server# mkfs -t ext3 /dev/sdb1mke2fs 1.39 (29-May-2006)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)13107200 inodes, 26214055 blocks1310702 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=0800 block groups32768 blocks per group, 32768 fragments per group16384 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872Writing inode tables: done Creating journal (32768 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 35 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.4、用tune2fs修改文件系统的属性，去掉自动检查的属性：rootpe03 Server# tune2fs -c -1 -i 0 /dev/sdb1tune2fs 1.39 (29-May-2006)Setting maximal mount count to -1Setting interval between checks to 0 secondsLinux上的ext文件系统有一个特性，对某个分区mount、umount很多次后或者隔一个固定的时间后，系统会对该分区进行检测，这就会导致硬盘反映速度很慢，影响业务，本操作的目的就是去掉文件系统自动检查的属性。三、设定文件系统的自动挂载本例中我们将要把/dev/sdb1挂载到/data目录下1、手动创建一个目录/datarootpe03 Server# mkdir /data2、用tune2fs查看文件系统的UUID：rootpe03 Server# tune2fs -l /dev/sdb1tune2fs 1.39 (29-May-2006)Filesystem volume name: Last mounted on: Filesystem UUID: 3f0a00b7-4939-4ad2-a592-0821bb79f7c6Filesystem magic number: 0xEF53Filesystem revision #: 1 (dynamic)Filesystem features: has_journal resize_inode dir_index filetype sparse_super large_file.3、用vi编辑/etc/fstab文件，设置自动挂载：在/etc/fstab文件中增加下面蓝色的一行文字：/dev/VolGroup00/LogVol00 / ext3 defaults 1 1LABEL=/boot /boot ext3 defaults 1 2devpts /dev/pts devpts gid=5,mode=620 0 0tmpfs /dev/shm tmpfs defaults 0 0proc /proc proc defaults 0 0sysfs /sys sysfs defaults 0 0/dev/VolGroup00/LogVol01 swap swap defaults 0 0UUID=3f0a00b7-4939-4ad2-a592-0821bb79f7c6 /data ext3 _netdev 0 0注意：l 挂载选项使用的是“_netdev”l UUID要顶格写。l Linux系统重启后，磁盘设备的名称可能会发生变化，从而引起文件系统不能挂载上来或者不能正确挂载，使用UUID的方式进行挂载可以解决这个问题，也可以使用给文件系统设置卷标的方式来解决，具体操作步骤可以参见KMS-12541：在Linux上使用文件系统卷标解决磁盘名称发生变化引起的文件系统不能正确自动挂载的问题4、用mount a挂载文件系统rootpe03 Server# mount a5、用df查看文件系统已经挂载成功rootpe03 /# df -ThFilesystem Type Size Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00 ext3 65G 2.4G 59G 4% /dev/sda1 ext3 99M 12M 82M 13% /boottmpfs tmpfs 471M 0 471M 0% /dev/shm/dev/sdb1 ext3 99G 188M 94G 1% /data6、条件允许的话，重启一下操作系统，使用df命令确认系统重启后文件系统能够自动挂载上来。四、FAQ1、如果存储上的Target下新挂载了一个卷，如何在服务器上连接并识别？如果一个Target下新增了一个卷，在服务器上使用iscsiadm m session R命令可以刷新当前连接的session以看到新增的卷：rootas5 as5-target# iscsiadm -m session R可以在/var/lib/iscsi/nodes目录下相应的target目录下使用命令lsscsi或者fdisk -l来查看连接过来的卷，例如下面的sdb为新连接的SAN卷：rootas5 as5-target# lsscsi 1:0:0:0 disk H3C H3C ISCSI DISK v1.0 /dev/sda1:0:0:1 disk H3C H3C ISCSI DISK v1.0 /dev/sdb附1：Open-iSCSI modules附2：Open-iSCSI README文件1. In This Release=This file describes the Linux* Open-iSCSI Initiator. 1.1. Features - highly optimized and very small-footprint data path; - persistent configuration database; - SendTargets discovery; - CHAP; - PDU header Digest; - multiple sessions; For the most recent list of features please refer to: 2. Introduction=Open-iSCSI project is a high-performance, transport independent,multi-platform implementation of RFC3720 iSCSI.Open-iSCSI is partitioned into user and kernel parts.The kernel portion of Open-iSCSI is a from-scratch code licensed under GPL. The kernel part implements iSCSI data path (that is, iSCSI Read and iSCSI Write), and consists of three loadable modules: scsi_transport_iscsi.ko, libiscsi.ko and iscsi_tcp.ko.User space contains the entire control plane: configuration manager, iSCSI Discovery, Login and Logout processing,connection-level error processing, Nop-In and Nop-Out handling, and (in the future:) Text processing, iSNS, SLP, Radius, etc.The user space Open-iSCSI consists of a daemon process called iscsid, and a management utility iscsiadm.3. Installation=To install the iSCSI tools run:rpm -ivh iscsi-initiator-utils-.rpmThis will install iscsid and iscsiadm to /sbin. It will also installdefault config files to /etc/iscsi:/etc/iscsi/iscsid.conf - All new session will inherit settings fromthis file when they are first discovered. To override a value for a specifictarget see the iscsiadm op command below and in the iscsiadm man page.See Configuration section below./etc/iscsi/initiatorname.iscsi - The default initaitor name for softwareiSCSI initiator.4. Open-iSCSI daemon=The daemon implements control path of iSCSI protocol, plus some management facilities. For example, the daemon could be configured to automatically re-start discovery at startup, based on the contents of persistent iSCSI database (see next section).For help, run: ./iscsid -helpUsage: iscsid OPTION -c, -config=path Execute in the config file (/etc/iscsi/iscsid.conf). -f, -foreground run iscsid in the foreground -d, -debug debuglevel print debugging information -u, -uid=uid run as uid, default is current user -g, -gid=gid run as gid, default is current user group -h, -help display this help and exit -v, -version display version and exit5. Open-iSCSI Configuration Utility=Open-iSCSI persistent configuration is implemented as a set of iSCSI database files.- Discovery (/var/lib/iscsi/send_targets);- Node (/var/lib/iscsi/nodes).The iscsiadm utility is a command-line tool to manage (update, delete, insert, query) the persistent database.The utility presents a set of operations that a user can perform on iSCSI nodes, sessions, connections, and discovery records.Note that some of the iSCSI Node and iSCSI Discovery operations do not require iSCSI daemon (iscsid) loaded.For help, run: ./iscsiadm -helpUsage: iscsiadm OPTION -m, -mode specify operational mode op = -m discovery -type=type -portal=ip:port -login perform type discovery for target portal with ip-address ip and port port. Initiate Login for each discovered target if -login is specified -m discovery display all discovery records from internal persistent discovery database -m discovery -portal=ip:port -login perform discovery based on portal in database -m discovery -portal=ip:port -op=op -name=name -value=value perform specific DB operation op for specific discovery portal. It could be one of: new, delete, update or show. In case of update, you have to provide name and value you wish to update -m node display all discovered nodes from internal persistent discovery database -m node -targetname=name -portal=ip:port -login|-logout -m node -targetname=name -portal=ip:port -op=op -name=name -value=value perform specific DB operation op for specific portal on target. It could be one of: new, delete, update or show. In case of update, you have to provide name and value you wish to update -m node -logoutall=all,manual,automatic Logout all the running sessions or just the ones with a node or conn startup value manual or automatic. Nodes marked as ONBOOT are skipped. -m node -loginall=all,manual,automatic Login all the running sessions or just the ones with a node or conn startup value manual or automatic. Nodes marked as ONBOOT are skipped. -m session display all active sessions and connections -m session -sid=sid -info | -rescan | -logout -op=op -name=name -value=value perform operation for specific session with session id sid. If no sid is given the operation will be performed on all running sessions if possible. -logout, -rescan, -op only work on single session. -logout and -op work like they do in node mode, but in session mode targetname and portal info is is not passed in. If no sid and no operation is given print out the running sessions. -d, -debug debuglevel print debugging information -V, -version display version and exit -h, -help display this help and exitUsage examples (using the one-letter options): 1) SendTargets iSCSI Discovery: iscsiadm -m discovery -t sendtargets -p :3260 2) iSCSI Login: login to a specific target: iscsiadm -m node -T .max -p :3260 -l login to all targets: iscsiadm -m node -loginall=all 3) iSCSI Logout: logout of a specific target: iscsiadm -m node -T .max -p :3260 -u logout of all targets: iscsiadm -m node -logoutall=all 4) Changing iSCSI parameter: iscsiadm -m node -T .max -p :3260 -o update -n x0.iscsi.MaxRecvDataSegmentLength -v 65536 5) Adding custom iSCSI Node: iscsiadm -m node -o new -p :3260 6) Removing iSCSI Node: iscsiadm -m node -o delete -T .max -p :3260 7) Display iSCSI Node configuration: iscsiadm -m node -T .max -p :3260 or iscsiadm -m node -o show -T .max -p :3260 8) Show all node records: iscsiadm -m node 9) Show all records in discovery database: iscsiadm -m discovery 10) Display discovery record setting: iscsiadm -m discovery -p :3260 11) Display session statistics: iscsiadm -m session -r 1 -stats 12) Display session and device info: iscsiadm -m session i6. Configuration=The default configuration file is /etc/iscsi/iscsid.conf. This file contains only configuration that could be overwritten by iSCSI Discovery, or manualy updated via iscsiadm utility. Its OK if this file does not exist in which case compiled-in default configuration will take place for newer discovered Target nodes. See the example file for the current syntax.7. Getting Started=There are three steps needed to set up a system to use iSCSI storage:1. automate iSCSI startup using the init script.2. discover targets.3. automate target logins for future system reboots.1. automate iSCSI startup using the init script-To start the iSCSI daemon and log into