hadoop_kerberos 配置权限验证.docx

(一)相关版本及服务器：软件包CentOS 5.4 x86_64hadoop-0.20.2-cdh3u3.tar.gzjdk-6u31-linux-x64.binapache-maven-2.2.1-bin.tar.gzapache-ant-1.8.3-bin.zip硬件29 31Kerberos服务器Hadoop服务器(二)软件安装目录结构名称安装目录Jdk/usr/local/jdk1.6.0_31/hadoop/usr/local/hadoop-0.20.2-cdh3u3/maven/usr/local/apache-ant-1.8.3/ant/usr/local/ apache-maven-2.2.1(四)环境变量rootnode1 # vim /etc/profileexport JAVA_HOME=/usr/local/jdk1.6.0_31export ANT_HOME=/usr/local/apache-ant-1.8.3export M2_HOME=/usr/local/apache-maven-2.2.1export PATH=$M2_HOME/bin:$ANT_HOME/bin:$PATH(五)hadoop目录结构hdfs-site.xml参数名称目录说明.dir/data/dfs/nn名称节点目录dfs.data.dir/data/dfs/dn数据节点目录mapred-site.xml参数名称目录说明mapred.local.dir/data/mapred/localmap的local目录core-site.xml参数名称目录说明fs.checkpoint.dir/data/dfs/secondnameSecondarynamenode 目录（六）单机运行的hadoop1）配置文件hdfs-site.xml dfs.replication 1 .dir /data/dfs/nn dfs.data.dir /data/dfs/dn core-site.xml hdfs:/node1:9000 fs.checkpoint.dir /data/dfs/secondname mapred-site.xml mapred.job.tracker node1:9001 mapred.local.dir /data/mapred/local hadoop-env.sh 添加export JAVA_HOME=/usr/local/jdk1.6.0_31export HADOOP_DATANODE_USER=hdfsexport HADOOP_TASKTRACKER_USER=mapred2)创建hdfs 和 mapred 用户及目录rootnode1 # useradd G hadoop hdfsrootnode1 # useradd G hadoop mapredrootnode1 # mkdir p /data/mapred/ /data/dfs/rootnode1 # mkdir p /data/mapred/ /data/dfs/rootnode1 # chown hdfs:hadoop R /data/dfsrootnode1 # chown mapred:hadoop R 3）格式化节点rootnode1 # su hdfshdfsnode1 # cd /usr/local/hadoop-0.20.2-cdh3u3/bin/hdfsnode1 # ./hadoop namenode -format 4）启动namenode、datanode、secondarynamenode并创建临时目录、创建mapred.system.dir目录hdfsnode1 # ./start-dfs.shhdfsnode1 # ./hadoop fs mkdir /tmphdfsnode1 # ./hadoop fs chmod R 1777 /tmphdfsnode1 # ./hadoop fs mkdir /mapred/systemhdfsnode1 # ./hadoop fs chown mapred:hadoop /mapred/system 5)启动jobtracker、tasktrackerhdfsnode1 # ./ start-mapred.sh确认是否运行正常。然后下一步。（七）搭建kerboeros服务器1）根据文档 CDH3_Security_Guide_u2.pdf 文档说明，Krb5-1.6.1 在 Red Hat Enterprise Linux5 及 CentOS 5 通过验证故先检查本机服务器的krbrootkerberos # rpm -qa|grep krbpam_krb5-2.2.14-10krb5-libs-1.6.1-36.el5krb5-libs-1.6.1-36.el5krb5-server-1.6.1-36.el5krb5-devel-1.6.1-36.el5krb5-devel-1.6.1-36.el5krb5-workstation-1.6.1-36.el5krb5-auth-dialog-0.7-12）配置文件内容/etc/krb5.conflogging default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.loglibdefaults default_realm = EXAMPLE.COM dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h forwardable = yesrealms EXAMPLE.COM = kdc = :88 admin_server = :749 default_domain = EXAMPLE.COM domain_realm . = EXAMPLE.COM = EXAMPLE.COM kdcprofile=/var/kerberos/krb5kdc/kdc.confappdefaults pam = debug = false ticket_lifetime = 36000 renew_lifetime = 36000 forwardable = true krb4_convert = false /var/kerberos/krb5kdc/kdc.confkdcdefaults v4_mode = nopreauth kdc_tcp_ports = 88realms EXAMPLE.COM = #master_key_type = des3-hmac-sha1 acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /var/kerberos/krb5kdc/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal des-cbc-crc:v4 des-cbc-crc:afs3 配置域名rootkerberos # vim /etc/hosts29 kerberos29 kdc31 node1 3）生成kerberos数据库并 启动rootkerberos # kdb5_util create -r EXAMPLE.COM srootkerberos # service krb5kdc start 至此kerbose配置完成， 详细配置、介绍、测试，可参见krb5安装与配置文档，此处略过（八）hadoop配置kerberos此处只描述大概的搭建步骤，无法详细叙原理及注意事项，强烈建议参照文档操作手册进行配置和搭建文档名称为：CDH3_Security_Guide_u2.pdf1）更改hdfs文件权限属性hdfsnode1 # ./hadoop fs chown mapred:hadoop /mapred/systemhdfsnode1 # ./hadoop fs chown hdfs:hadoop /hdfsnode1 # ./hadoop fs chmod -R 700 /mapred/systemhdfsnode1 # ./hadoop fs chmod 755 / 2)创建并生成用于hadoop进行kerberos验证的Principals 和 keytab 文件#登录kadmin#为hadoop集群中的每一台服务器添加pricipals#每台服务器不采用固定密码方式，随机生成key，参数-randkey#用实际的服务器名替换node1rootkerberos # kadmin.localkdmin: addprinc -randkey hdfs/node1EXAMPLE.COMkdmin: addprinc -randkey mapred/node1EXAMPLE.COMkdmin: addprinc -randkey host/node1EXAMPLE.COM#生成keytab文件，#hdfs.keytab 包含hdfs和host的principal#mapred.keytab 包含mapred和host的principal#为集群中每一台服务器重复以下步骤rootkerberos # kadmin.localkadmin: xst -k hdfs-unmerged.keytab hdfs/node1EXAMPLE.COMkadmin: xst -k mapred-unmerged.keytab mapred/node1EXAMPLE.COMkadmin: xst -k host.keytab host/node1EXAMPLE.COMrootkerberos # ktutilktutil: rkt hdfs-unmerged.keytab ktutil: rkt host.keytab ktutil: wkt hdfs.keytab ktutil: clear ktutil: rkt mapred-unmerged.keytab ktutil: rkt host.keytab ktutil: wkt mapred.keytab#显示keytab文件内容rootkerberos # klist e k t hdfs.keytab#检测文件是否有效rootkerberos # kinit -k -t hdfs.keytab hdfs/node1EXAMPLE.COMrootkerberos # kinit -k -t hdfs.keytab host/node1EXAMPLE.COM3）将生成的hdfs.keytab 和 mapred.keytab 文件拷贝到hadoop集群所有服务器rootkerberos #scp -rp hdfs.keytab node1:$HADOOP_HOME/conf/.rootkerberos #scp -rp mapred.keytab node1:$HADOOP_HOME/conf/. 4)修改keytab文件的权限hdfsnode1 # chown hdfs:hadoop $HADOOP_HOME/conf/hdfs.keytabhdfsnode1 # chown mapred:hadoop $HADOOP_HOME/conf/mapred.keytabhdfsnode1 # chmod 400 $HADOOP_HOME/conf/*.keytab5）如果haddop集群此时正在运行，停止6）为hadoop集群添加权限验证，修改配置文件core-site.xml hadoop.security.authentication kerberos hadoop.security.authorization true hdfs-site.xml注释: a)所有服务器的hdfs-site.xml文件必须保证一致 b)参数_HOST在运行时，会自动替换为本机hostnameNameNode: 由 决定其hostnameDataNode: 由DNS反向解析决定其hostnameJobTracker: 由mapred.job.tracker 决定其hostnameTaskTracker: 由DNS反向解析决定其hostname c)dfs.datanode.address 和 dfs.datanode.http.address 端口号必须低于1024 dfs.block.access.token.enable true dfs.https.address node1:50470 dfs.https.port 50470 node.keytab.file /usr/local/hadoop-0.20.2-cdh3u3/conf/hdfs.keytab node.kerberos.principal hdfs/_HOSTEXAMPLE.COM node.kerberos.https.principal host/_HOSTEXAMPLE.COM dfs.secondary.https.address node1:50495 dfs.secondary.https.port 50495 node.keytab.file /usr/local/hadoop-0.20.2-cdh3u3/conf/hdfs.keytab node.kerberos.principal hdfs/_HOSTEXAMPLE.COM node.kerberos.https.principal host/_HOSTEXAMPLE.COM dfs.datanode.data.dir.perm 700 dfs.datanode.address :1004 dfs.datanode.http.address :1006 dfs.datanode.keytab.file /usr/local/hadoop-0.20.2-cdh3u3/conf/hdfs.keytab dfs.datanode.kerberos.principal hdfs/_HOSTEXAMPLE.COM dfs.datanode.kerberos.https.principal host/_HOSTEXAMPLE.COM 7）用hdfs用户，启动namenode节点及datanode节点hdfsnode1 # ./hadoop namenode 或者hdfsnode1 # ./ hadoop-daemon.sh start namenode 8）启动datanode节点注意，datanode节点必须由root用户启动rootnode1 # ./hadoop datanode 或者rootnode1 # ./ hadoop-daemons.sh start datanode9）用hdfs用户，启动secondarynamenode 节点hdfsnode1 # ./hadoop secondarynamenode 或者hdfsnode1 # ./ hadoop-daemon.sh start secondarynamenode10）配置MapReducea)参数 mapred.local.dir 和 hadoop.log.dirs 在配置文件 mapred-site.xml 、taskcontroller.cfg 必须保持一直mapred-site.xml mapreduce.jobtracker.kerberos.principal mapred/_HOSTEXAMPLE.COM mapreduce.jobtracker.kerberos.https.principal host/_HOSTEXAMPLE.COM mapreduce.jobtracker.keytab.file /usr/local/hadoop-0.20.2-cdh3u3/conf/mapred.keytab mapreduce.tasktracker.kerberos.principal mapred/_HOSTEXAMPLE.COM mapreduce.tasktracker.kerberos.https.principal host/_HOSTEXAMPLE.COM mapreduce.tasktracker.keytab.file /usr/local/hadoop-0.20.2-cdh3u3/conf/mapred.keytab mapred.task.tracker.task-controller org.apache.hadoop.mapred.LinuxTaskController mapreduce.tasktracker.group mapred taskcontroller.cfg#configured value of mapred.local.dir. It can be a list of comma separated paths.mapred.local.dir=/data/mapred/local#configured value of hadoop.log.dir.hadoop.log.dir=$HADOOP_HOME/logs#sleep time before sig kill is to be sent to process group after sigterm is sent. Should be in secondsmapred.tasktracker.tasks.sleeptime-before-sigkill=#configured value of mapreduce.tasktracker.group.mapreduce.tasktracker.group=mapred banned.users=mapred,hdfs,bin min.user.id=50011)更改task-controol文件的权限rootnode1 # chown root:mapred task-controolrootnode1 # chmod 4754 task-controolrootnode1 # ls -l $HADOOP_HOME/sbin/Linux-amd64-64/task-controller-rwsr-xr- 1 root mapred 96944 Mar 21 04:26 task-controller12)用mapred用户启动 jobtrackermaprednode1 # ./hadoop jobtracker13) 用mapred用户启动 tasktrackermaprednode1 # ./hadoop jobtracker此时，将会报错，如下：2012-05-10 18:17:30,221 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform. using builtin-java classes where applicable2012-05-10 18:17:30,280 ERROR org.apache.