通信数据侦察站系统安装与部署手册V1 0_EN

Communication Data Reconnaissance Station System System Installation and Deployment Manual Communication Data Reconnaissance Station System Installation and Deployment Manual CEIECContents1 Preface12 Terms and definitions23 System network architecture33.1 Overall network structure34 System installation and configuration54.1 Basic service installation64.1.1 dos2unix64.1.2 Jdk64.2 GP installation and deployment64.2.1 Minimal installation and local yum configuration64.2.2 Swap partition74.2.3 Disable the firewall74.2.4 Disable the server end and client end of the monitoring program84.2.5 Change limits.conf84.2.6 Software required for the local YUM source installation94.2.7 Change Linux kernel parameters94.2.8 Greemplum installation124.2.9 Unzip the installation packet and create a new data directory124.2.10 Set environment variables under rhino users (if there are multiple sets, operation of each machine is required)134.2.11 Gp database configuration file change (executed on the Master node)134.2.12 Establish a server mutual trust mechanism (executed on the Master node)144.2.13 GPDB startup command and performance optimization (executed on the Master node)154.2.14 Add file space164.2.15 REDIS installation194.2.16 Install zlib204.2.17 Install ruby204.2.18 Install rubygems204.2.19 Install redis-3.0.0.gem214.2.20 Install redis214.2.21 Create a node folder214.2.22 Change the configuration file redis-3.2.3/redis.conf214.2.23 Copy redis.conf to the node folder214.2.24 Copy redis-server to the node folder224.2.25 Start each node224.2.26 Use redis-trib.rb for cluster creation224.3 RDT installation234.3.1 Change hosts234.3.2 User creation234.3.3 Zookeeper and kafka244.3.4 Rdt installation254.4 GPloader installation264.4.1 Install and unzip gpload274.4.2 Change configuration files274.4.3 Create file sharing284.4.4 Stop gpload284.5 Samba service configuration284.5.1 View whether samba is installed284.5.2 Change configuration files284.5.3 Set the samba password294.5.4 Execute the mount command294.5.5 Unmount command294.6 PRM installation294.7 PSM installation and unloading304.7.1 Preparation for software installation304.7.2 JDK&TOMCAT installation304.7.3 Application publication314.7.4 File configuration311 Preface About this Manual: This Manual aims to guide the engineering maintenance personnel to perform installation and deployment. Meanwhile, it may be used as the reference manual of the Chinese operation and maintenance engineers at the product maintenance stage. How to use: This Manual focuses on introduction of several aspects required by the engineering personnel in the beginning, including terms, definitions, preparations for system configuration, system installation and configuration, as well as maintenance experience. 2 Terms and definitions GP:Greenplum database, used for storage of several rules information of the system interface WEB platform: The system display platform of the Communication Data Renaissance Station System, for issuing various orders PRM:Control module, web issues commands to PRM and then issues rules to the special equipment to realize blockage RDT:Execute consumption warehousing after the interface issues rules G:Serial-connection equipment, for issuing rules to achieve blockage. D:Parallel-connection equipment, for traffic forwarding and backflow. Log Server:Generate the blockage record logs, statistics of the number of the hit records. HCA:Serial-connection DFX equipment ECA:Parallel-connection DFX equipment 43 System network architecture 3.1 Overall network structure Description: There are only two sites now, including algers and annaba. The network graphs of two sites are shown below Figure 3-1 Network topological graph of Algers Figure 3-2 Network topological graph of ANNABA 4 System installation and configuration Table 4-1 System deployment table Server NameIPWEB/Redis/Kafka(DataCenter)192.168.96.14RDT(DataCenter)192.168.96.13GP(DataCenter)192.168.96.11PRM(DataCenter)192.168.97.12ALG-G1192.168.97.203ALG-DFX1 192.168.97.201ALG-DFX2192.168.97.202ALG-LOG-SERVER1192.168.97.204ALG-LOG-SERVER2192.168.97.205ALG-D192.168.97.203ANB-G1192.168.97.211ANB-G2192.168.97.212ANB-G3192.168.97.213ANB-G4192.168.97.214ANB-G5192.168.97.215ANB-G6192.168.97.216ANB-G7192.168.97.217ANB-G8192.168.97.218ANB-G9192.168.97.219ANB-HCA1192.168.97.221ANB-HCA2192.168.97.222ANB-HCA3192.168.97.223ANB-HCA4192.168.97.224ANB-HCA5192.168.97.225ANB-HCA6192.168.97.226ANB-D1192.168.97.199ANB-ECA1192.168.97.231ANB-ECA2192.168.97.232HCA-LOG-SERVER1192.168.97.235HCA-LOG-SERVER2192.168.97.236HCA-LOG-SERVER3192.168.97.237MGMT-SW1192.168.97.243HCA-LOG-SW192.168.97.244HW Core Switch 192.168.97.2394.1 Basic service installation 4.1.1 dos2unixAfter installation of the operating system comes to an end, log in to the system with the root user, to acquire and unzip the version packet to enter the version packet directory, rootmachine:tar xvfZTFTS-V4.0-TX-2017XXXX.XXXXXX.tar.gzrootmachine:cd ZTFTS-V4.0rootmachine:./setup.sh dos2unix4.1.2 JdkAfter installation of the operating system comes to an end, log in to the system with the root user, to acquire and unzip the version packet to enter the version packet directory rootmachine:tar xvf ZTFTS-V4.0-TX-2017XXXX.XXXXXX.tar.gzrootmachine:cd ZTFTS-V4.0rootmachine:./setup.sh jdk4.2 GP installation and deployment 4.2.1 Minimal installation and local yum configuration Create the cdrom directory under the /mnt/ directory Enter the file path: /etc/yum.repos.d1.Create the bak directory and move all files to the bak directory 2.vi cdrom.repocdromname=cdrombaseurl=file:/mnt/cdromgpgcheck=0enabled=1After mount iso to /mnt/cdrom, the basic tool issues an installation command It is required to put a system mirror iso file on the system, find out the corresponding mirror iso file and execute it under the directory mount -o loop Centos-XXXX.iso /mnt/cdromAfter mounting comes to an end, view under cdrom and there shall be new files generated Then execute yum install net-tools4.2.2 Swap partition Recommended size configuration for swap partition Physical memory Swap partition =3; It is recommended that 1 set is deployed with the Master node; Others are deployed with the segment node; The Master node is deployed with gploader We just have one server at the site. It adopts one master and 3 segment nodes now 4.2.9 Unzip the installation packet and create a new data directory Create a new local directory (executed by Master and Standby machines) rhinorhino001 $ mkdir -p /gpdata/gpmasterrhinorhino001 $ chmod 700 /gpdata/gpmasterUnzip the installation packet daedb-2.0.5-el7.tar.gz to the rhino directory (each machine is required to unzip, not allowed to copy to other machines) rhinorhino001 $ tar -zxvf daedb-2.0.5-el7.tar.gzrhinorhino001 $ ln -s daedb-2.0.5 daedb4.2.10 Set environment variables under rhino users (if there are multiple sets, operation of each machine is required) 1.Change the GPHOME parameter of /daedb/greenplum_path.sh Figure 4-1 Change the GPHOME parameter 2.Change environment variables rhinorhino001 $ vi /.bashrcAdd the contents as below: source /home/rhino/daedb/greenplum_path.shexport MASTER_DATA_DIRECTORY=/home/rhino/gpdata/gpmaster/gpseg-1export PGUSER=rhinoexport PGDATABASE=postgresSource .bashrc takes effect 4.2.11 Gp database configuration file change (executed on the Master node) 1.Change seghosts rhinorhino001 $ vim /daedb/etc/ seghostsContents are as below: owls092.Change gpinitsystem_config, of which the standby machine is not in need. rhinorhino001 $ vim /daedb/etc/gpinitsystem_configThe change content is as below: ARRAY_NAME=OceanMindSEG_PREFIX=gpsegMASTER_MAX_CONNECT=1024PORT_BASE=4000declare -a DATA_DIRECTORY=(/data01/gpdata /data02/gpdata /data03/gpdata)MASTER_HOSTNAME=owls09MASTER_DIRECTORY=/home/rhino/gpdata/gpmasterMASTER_PORT=5432TRUSTED_SHELL=sshCHECK_POINT_SEGMENTS=256ENCODING=UNICODEMIRROR_PORT_BASE=6000REPLICATION_PORT_BASE=7000MIRROR_REPLICATION_PORT_BASE=8000declare -a MIRROR_DATA_DIRECTORY=(/data01/mirror /data02/mirror /data03/mirror)MACHINE_LIST_FILE=/home/rhino/daedb/etc/seghostsDATABASE_NAME=postgres4.2.12 Establish a server mutual trust mechanism (executed on the Master node) rhinorhino001 $gpssh-exkeys -f /daedb/etc/seghostsThe validation command is as below: rhinorhino001 $gpssh-exkeys -h owls09 -h owls10If Master node fails to execute, it could be executed on the standby machine gpssh-exkeys -h owls10 -h owls094.2.13 GPDB startup command and performance optimization (executed on the Master node) 1.Start the installation initialization command and designate the standby node when the Master node is installed (recommend to use the command) rhino002 is a standby node rhinorhino001$gpinitsystem-c/home/rhino/daedb/etc/gpinitsystem_config -a -B 6-Installation command; this step is easy to go wrong. Therefore, it is required to note if this step is not correctly executed, go to 3.2.19 to execute the re-installation command. Establish Standby nodes rhinorhino001 $ gpinitstandby -s owls102.Connect the database and verify whether startup succeeds: rhinorhino001 $ psqlrhinorhino002 $psql -h rhino0013.View whether standby nodes of the master are successfully installed; rhinorhino001 $ gpstate4.After startup succeeds, optimize the performance and execute: The gpconfig command could be synchronized to all nodes for configuration rhinorhino001 $ gpconfig -c gp_vmem_protect_limit -v 28000 -m 28000 rhinorhino001 $ gpconfig -c effective_cache_size -v 28000MB -m 28000MBrhinorhino001 $ gpconfig -c maintenance_work_mem -v 28000MBThe above is based on the setting of the field server. If it is the normal indoor installation, all 28,000 shall be changed to be 8,000. rhinorhino001 $ gpconfig -c shared_buffers -v 1024MB -m 1024MBrhinorhino001 $ gpconfig -c statement_mem -v 500MB -masteronlyrhinorhino001 $ gpconfig -c gp_max_tablespaces -v 1024# Change the time zone. If it is not configured, there will be problems for service function timing tasks rhinorhino001 $ gpconfig -c log_timezone -v Africa/Douala Africa/Doualarhinorhino001 $ gpconfig -c TimeZone -v Africa/DoualaIf you have no idea about the corresponding time zone, you could switch to the root user on the server, Execution timedatectl | grep Time zoneFigure 4-2 View the time zone 5# Disable the optimizer and just change the master node /home/rhino/gpdata/gpmaster/gpseg-1/postgresql.conf addition optimizer = off#Configuration validation - note: Configuration validation must be executed after the database is restarted every time rhinorhino001 $ gpstop -u4.2.14 Add file space Make preparations for the subsequent database refreshing. Create a new file space fs_bigdata and put all the new table spaces in the database refreshing script in this file space. In the following example, there are only two hosts (rhino118, rhino119) Each host has 6 segments: /data01/data064.2.14.1 Add paths There are 3 paths to be added: primary, mirror, master1.1Add the primary path Use gpssh and add /data*/bigdata path to all the hosts cd /home/rhino/daedb/etcgpssh -f seghosts = mkdir /data01/bigdata1.2Add the mirror path If there is mirror, then re-add the mirror path: gpssh -f seghosts = mkdir /data01/bigdata_mirrorCheck whether the path has been successfully generated at all nodes: gpssh -f seghosts = find /data* -type d -name bigdata*1.3Add the master path Add bigdata under the /home/rhino/gpdata path of the master host 4.2.14.2 Generate configuration files Operate gpfilespace -o gpfilespace_config to generate the configuration file gpfilespace_config. 4 parts of contents are required to be input in the operation process Name of file space Primary node path Mirror node path Master node path Figure 4-3 Generate configuration files 4.2.14.3 Create file space Operate gpfilespace -c gpfilespace_config to create file space Success mark: Filespace fs_bigdata successfully createdFigure 4-4 Create file space Enter psql and check whether the file space is successfully created select * from pg_filespace;Figure 4-5 Check whether creation succeeds The database start-stop command PG library start: gpstart aGentle PG library disablement: gpstop -a M fast;Rude PG library disablement, which is more common: gpstop -a -M immediate;PG library restart: gpstop -r -a -M immediate;4.2.15 REDIS installation Root user logs, and uploads redis_setup.tar.gz to the /root directory. Then unzip it Linux system shall be equ