Greenplum详细安装.docx

杭州斯凯网络科技有限公司Greenplum安装一、 环境准备VMWARE6.5虚拟机上Redhat5(虚拟两个32位的redhatOS)每个虚拟OS的配置512M内存，8G硬盘容量二、 LINUX准备过程1. 查看OS的位数,确定GP的版本#getconf LONG_BIT#322. 官网上下载32位的Greenplum安装文件greenplum-db--build-1-RHEL5-i386.zip，大小44M，解压后也差不多44M，里面包含两个文件：一个是安装文件，一个是安装指导3. 解压命令：# unzip greenplum-db--build-1-RHEL5-i386.zip在各台GP主机上编辑/etc/sysctl.conf文件，替换或新增以下参数,重启：kernel.shmmax = 500000000kernel.shmmni = 4096kernel.shmall = 4000000000kernel.sem = 250 64000 100 512net.ipv4.tcp_tw_recycle=1 net.ipv4.tcp_max_syn_backlog=4096 dev_max_backlog=10000vm.overcommit_memory=2 net.ipv4.ip_local_port_range = 1025 65535 -文档上没有，新增的在各台主机上同样编辑/etc/security/limits.conf* soft nofile 65536 * hard nofile 65536 * soft nproc 131072* hard nproc 131072三、 软件安装及准备配置1. 将master主机改名为local_master，segment主机改名为local_segment;分别修改/etc/hosts为：Local_master: localhost.localdomain localhost28 local_masterLocal_segment: localhost.localdomain localhost31 local_segment/etc/sysconfig/network为：Local_master:NETWORKING=yesNETWORKING_IPV6=no#HOSTNAME=localhost.localdomainHOSTNAME=local_masterLocal_segment:NETWORKING=yesNETWORKING_IPV6=no#HOSTNAME=localhost.localdomainHOSTNAME=local_segment 改完后生效：#service network restart 改完后查看：# hostname 2. 在master安装GP软件：#/bin/bash greenplum-db--build-1-RHEL5-i386.bin然后lience accept yes,安装GP路径：/opt/ greenplum-db-安装好了文件夹greenplum-db-里面会出现以下文件：bin demo docs etc ext greenplum_path.sh include lib LICENSE sbin share 另外外部还会出现一个软链接：rootlocal_master opt# ls -ltotal 12lrwxrwxrwx 1 root root 22 Apr 19 18:52 greenplum-db - ./greenplum-db-drwxr-xr-x 11 root root 4096 Apr 19 18:52 greenplum-db-四、 Master主机配置1. 新增Greenplum用户并指定密码，在root用户下执行# useradd gpadmin# passwd gpadmin# New password: gpadmin# Retype new password: gpadmin2. 创建用户组,把gpadmin用户放入组gpadmin中# groupadd gpadmin# usermod -g gpadmin gpadmin3. 更改所有者,将创建的Greenplum文件用户更换为gpadmin chown -R gpadmin greenplum-db-chgrp -R gpadmin greenplum-db-chown -R gpadmin greenplum-dbchgrp -R gpadmin greenplum-db4. 配置环境变量greenplum_path.sh这个文件是用来配置环境变量和安装路径的,具体内容：# cat /opt/greenplum-db/greenplum_path.sh GPHOME=/opt/greenplum-db-# Replace with symlink path if it is present and correctif -h $GPHOME/./greenplum-db ; then GPHOME_BY_SYMLINK=(cd $GPHOME/./greenplum-db/ & pwd -P) if x$GPHOME_BY_SYMLINK = x$GPHOME ; then GPHOME=(cd $GPHOME/./greenplum-db/ & pwd -L)/. fi unset GPHOME_BY_SYMLINKfiPATH=$GPHOME/bin:$GPHOME/ext/python/bin:$PATHLD_LIBRARY_PATH=$GPHOME/lib:$GPHOME/ext/python/lib:$LD_LIBRARY_PATHPYTHONPATH=$GPHOME/lib/pythonPYTHONHOME=$GPHOME/ext/pythonexport GPHOMEexport PATHexport LD_LIBRARY_PATHexport PYTHONPATHexport PYTHONHOME切换到gpadmin用户下,修改.bash_profile文件为#su gpadmin#cat .bash_profile# .bash_profile# Get the aliases and functionsif -f /.bashrc ; then . /.bashrcfi# User specific environment and startup programsGPHOME=/opt/Greenplum# Replace with symlink path if it is present and correctif -h $GPHOME/./greenplum-db ; then GPHOME_BY_SYMLINK=(cd $GPHOME/./greenplum-db/ & pwd -P) if x$GPHOME_BY_SYMLINK = x$GPHOME ; then GPHOME=(cd $GPHOME/./greenplum-db/ & pwd -L)/. fi unset GPHOME_BY_SYMLINKfiPATH=$GPHOME/bin:$GPHOME/ext/python/bin:$PATHLD_LIBRARY_PATH=$GPHOME/lib:$GPHOME/ext/python/lib:$LD_LIBRARY_PATHPYTHONPATH=$GPHOME/lib/pythonPYTHONHOME=$GPHOME/ext/pythonexport GPHOMEexport PATHexport LD_LIBRARY_PATHexport PYTHONPATHexport PYTHONHOME初始化(gpadmin下)source ./bash_profile5. 创建master数据文件以root用户创建master的数据存放路径,该目录只存放系统数据，不会存放其他用户数据，并更改所有者#mkdir /opt/master_database#chown gpadmin /opt/ master_database#chgrp gpadmin /opt/ master_database五、 Segment主机配置1. 建立主机文件阵列以root用户先在master上建立一个包含所有(master,segment,backup master)主机名的文件如下：rootlocal_master # cat /etc/all_hosts local_masterlocal_segment2. 修改master的hosts文件分别查看虚拟机OS的ip地址，并加入master（选其中的一台）#ifconfig -ainet addr:28 -masterinet addr:31 -segment以128作为master，修改/etc/hosts文件为# Do not remove the following line, or various programs# that require network functionality will fail. localhost.localdomain localhost28 local_master31 local_segment建立信任的主机环境,在master上以root用户登录，初始化参数# source /opt/greenplum-db-/greenplum_path.sh测试master与segment的连接，采用Gp自带的包,测试过程如下：rootlocal_master # gpssh-exkeys -f /etc/all_hostsSTEP 1 of 5 create local ID and authorize on local host . /root/.ssh/id_rsa file exists . key generation skippedSTEP 2 of 5 keyscan all hosts and update known_hosts fileSTEP 3 of 5 authorize current user on remote hosts . send to local_segment * * Enter password for local_segment: STEP 4 of 5 determine common authentication file contentSTEP 5 of 5 copy authentication files to all remote hosts . finished key exchange with local_segmentINFO completed successfully期间会要求输入segment端root用户的密码，该过程主要是check远程的segment主机以及验证两个主机间的通信顺畅。3. 在mastert主机上创建segment主机名文件,在远程segment主机上上创建gpadmin用户，以root身份rootlocal_segment etc# cat /etc/all_segment_hosts local_segment以下是在master上远程创建gpadmin在segment上:gpssh -f /etc/all_segment_hosts /usr/sbin/useradd gpadmin -d /home/gpadmin -s /bin/bash以下是在master上远程修改segment上的gpadmin用户密码：gpadmin_passwordgpssh -f /etc/all_segment_hosts echo gpadmin_password | passwd gpadmin -stdin在master上切换到gpadmin用户,通过gpssh来连接segment主机端的gpadmin用户gpadminlocal_master $ cat /etc/all_hosts local_masterlocal_segmentgpadminlocal_master $ gpssh-exkeys -f /etc/all_hosts STEP 1 of 5 create local ID and authorize on local hostSTEP 2 of 5 keyscan all hosts and update known_hosts fileSTEP 3 of 5 authorize current user on remote hosts . send to local_segment * * Enter password for local_segment: STEP 4 of 5 determine common authentication file contentSTEP 5 of 5 copy authentication files to all remote hosts . finished key exchange with local_segmentINFO completed successfully期间需要输入远程segment的gpadmin用户密码(之前设置的是gpadmin_password).4. 拷贝GP软件并修改相关属性a. 创建gp.tar文件以root用户在master上创建gp.tar文件#su root#cd /opt#gtar -cvf /home/gpadmin/gp.tar greenplum-db-b. 将gp.tar文件copy到segment主机上的/opt路径下#source /opt/greenplum-db-/greenplum_path.sh#gpscp -f /etc/all_segment_hosts /home/gpadmin/gp.tar =:/optc. 开启一个交互的会话，远程操作segment解压segment上的tar文件到/opt目录下#gpssh -f /etc/all_segment_hosts=gtar -directory /opt -xvf /opt/gp.tar需要确保master与segment上装的GP软件的路径是一致的,并建立软链接：= ls -l /opt/greenplum-db-/=ln -s /opt/greenplum-db- /opt/greenplum-db 更改owner与group= chown -R gpadmin /opt/greenplum-db=chgrp -R gpadmin /opt/greenplum-db删除gp.tar=rm /opt/gp.tar5. 在segment主机上创建数据文件并更改属性#gpssh f /etc/all_segment_hosts= mkdir /opt/segment_database= chown -R gpadmin /opt/segment_database= chgrp -R gpadmin /opt/segment_database如果需要做mirror备份,则创建文件夹/opt/segment_database_m1,权限同上退出gpssh=exit;6. 同步系统时钟rootlocal_master opt# gpssh -f /etc/all_segment_hosts -v dateReset .INFO login local_segmentlocal_segment Mon Apr 19 22:08:15 PDT 2010INFO completed successfullyCleanup.也可以通过NTP时钟服务器来同步：$ gpssh -f /etc/all_segment_hosts -v ntpd7. 本地化设置主要是客户/服务端设置encoding,字符类等，略六、 初始化GreenPlum数据库1. 创建只有segment主机名的文件,可放在任意位置,我们可使用前面创建的文件rootlocal_master opt# cat /etc/all_segment_hosts local_segment rootlocal_master opt# chown -R gpadmin /etc/all_segment_hosts rootlocal_master opt# chgrp -R gpadmin /etc/all_segment_hosts注意该文件的路径，后续会使用到2. 从文档中拷贝例子$GPHOME/docs/cli_help/gp_init_config_examplecp $GPHOME/docs/cli_help/gp_init_config_example /home/gpadmin/gp_init_config修改初始化文件如下，如需要mirror还需新增参数：rootlocal_master gpadmin# cat gp_init_config ARRAY_NAME=Greenplum DatabaseMACHINE_LIST_FILE=/etc/all_segment_hostsSEG_PREFIX=gpPORT_BASE=50000declare -a DATA_DIRECTORY=(/opt)# declare -a DATA_DIRECTORY=(/gp_primary /gp_primary)MASTER_HOSTNAME=local_masterMASTER_DIRECTORY=/opt/master_databaseMASTER_PORT=5432TRUSTED_SHELL=sshCHECK_POINT_SEGMENTS=8ENCODING=UNICODE修改文件权限：#chown R gpadmin /home/gpadmin/gp_init_config#chgrp R gpadmin /home/gpadmin/gp_init_config2.4系统性能效验 1-运行gpcheckos 来检查刚配的segment机器的操作系统情况（如果是在master和segment同一台机器上，不需要执行这一步） gpcheckos -f