ECstore企业版集群部署文档.docx

ECstore企业版集群部署文档2011.03目录1 文档历史32 目的33 系统架构图43.1. 架构图43.2. 应用部署图53.3. 部署说明54 硬件65 网络66 软件76.1. 底层环境76.1.1. 时间同步76.1.2. web服务器组. 安装nginx. php-fpm. 同步脚本. 远程命令236.1.3. KV数据库服务器组. flare节点286.1.4. mysql数据库服务器296.1.5. 图片服务器40. nginx40. NFS4. 文件同步4. 自启动466.1.6. 搜索服务器466.2. 应用部署496.2.1. 安装496.2.2. 代码更新506.2.3. 系统配置501 文档历史DateVersionDescriptionAuthor2011-030.5初始版本shopex2011-030.7结构完善shopex2011-030.8mysql配置完善shopex2011-031.0补充完善shopex2011-031.1补充完善shopex2 目的此文档为ECstore企业版服务器应用部署方案。 完成底层环境架构 搭建ECstore系统部署将为矩阵服务提供高可用服务器架构。3 系统架构图3.1. 架构图3.2. 应用部署图3.3. 部署说明将应用部署可以切分为以下几个部分: web服务器组 varnish反向代理服务器组 MySQL数据库服务器组 KV数据库服务器组 Memcached服务器组 图片服务器组 反向代理服务器 Sphinx搜索服务器组 备份服务器组下面就以上几组应用进行相应的说明: web服务器组 web服务器组为应用提供web服务，web服务器存放的是提供服务的源代码。 web服务器通过A10负载均衡器分发调度相应的应用。 web服务器通过连接数据库提供web服务。 每台web服务器都存放的是服务代码的副本，且保证部署文件一致性，又code push服务器提供分发和部署。 varnish反向代理服务器组 高效动态页缓存系统（未使用） MySQL数据库服务器组 数据库服务器组为MYSQL一主双从模式，为用户提供数据持久层服务。 KV数据库服务器组 为ecstore提供key-value存储服务，采用Flare+Tokyo Cabinet Memcached服务器组 通过缓存mysql数据持久层数据，减轻mysql的压力，存储mysql的数据。 图片服务器组 图片存储服务器组为存储为商品图片，采用NFS+sersync方案 反向代理服务器 使用squid为图片提供静态文件缓存服务。 Sphinx搜索服务器组 为前台搜索提供索引服务 备份服务器组 备份用户数据，数据包括mysql的切片备份，web应用的服务器和Flare的冷备数据。4 硬件总共12台服务器,详情见下表：品牌型号规格数量DELL未知硬盘134G cpu 8核 内存8G10DELL未知硬盘1T cpu 16核 内存24G25 网络ip列表内网ip外网IP应用说明OS主机名root帐号VIP5web服务器预留as5(64位)w018015.isvadminweb服务器as5(64位)w018002.isvadminweb服务器as5(64位)w018002.isvadminweb服务器as5(64位)w018003.isvadminweb服务器as5(64位)w018004.isvadmin图片服务器as5(64位)w018005.isvadmin图片服务器as5(64位)w018006.isvadminflare主as5(64位)w018007.isvadminflare从as5(64位)w018008.isvadmin备份服务器as5(64位)w018009.isvadmin0mysql从服务器as5(64位)w018010.isvadmin1mysql主服务器as5(64位)w018011.isvadmin2mysql从服务器as5(64位)w018012.isvadmin6 软件6.1. 底层环境6.1.1. 时间同步加入crontabcront -e*/5 * * * * ntpdate /dev/null6.1.2. web服务器组机器ip: . 安装nginx./configure -user=admin -group=admin -prefix=/home/admin/apps/nginx -with-http_stub_status_module -with-http_ssl_module -with-http_gzip_static_modulemake -j8主配置文件:/home/admin/apps/nginx/conf/nginx.confuser admin admin;worker_processes 4;error_log logs/nginx_error.log crit;worker_rlimit_nofile 65535;events use epoll; worker_connections 65535;http include mime.types; default_type application/octet-stream; server_names_hash_bucket_size 128; client_header_buffer_size 32k; large_client_header_buffers 4 32k; client_max_body_size 8m; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 0; fastcgi_connect_timeout 300; fastcgi_send_timeout 300; fastcgi_read_timeout 300; fastcgi_buffer_size 64k; fastcgi_buffers 4 64k; fastcgi_busy_buffers_size 128k; fastcgi_temp_file_write_size 128k; gzip on; gzip_min_length 1k; gzip_buffers 4 8k; gzip_http_version 1.1; gzip_types text/plain application/x-javascript text/css application/xml; gzip_disable MSIE 1-6.; log_format access $remote_addr - $remote_user $time_local $request $status $body_bytes_sent $http_referer $http_user_agent $http_x_forwarded_for; include site/*.conf;php fast cgi为了减少tcp连接，采用unix socket模式提供服务，配置文件:/home/admin/apps/nginx/conf/php_fcgi.conf#fastcgi_pass :9000;fastcgi_pass unix:/tmp/php_fcgi.sock;fastcgi_param GATEWAY_INTERFACE CGI/1.1;fastcgi_param SERVER_SOFTWARE nginx;fastcgi_param QUERY_STRING $query_string;fastcgi_param REQUEST_METHOD $request_method;fastcgi_param CONTENT_TYPE $content_type;fastcgi_param CONTENT_LENGTH $content_length;fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;fastcgi_param SCRIPT_NAME $fastcgi_script_name;fastcgi_param REQUEST_URI $request_uri;fastcgi_param DOCUMENT_URI $document_uri;fastcgi_param DOCUMENT_ROOT $document_root;fastcgi_param SERVER_PROTOCOL $server_protocol;fastcgi_param REMOTE_ADDR $remote_addr;fastcgi_param REMOTE_PORT $remote_port;fastcgi_param SERVER_ADDR $server_addr;fastcgi_param SERVER_PORT $server_port;fastcgi_param SERVER_NAME $server_name;# PHP only, required if PHP was built with -enable-force-cgi-redirectfastcgi_param REDIRECT_STATUS 200;pathinfo修正文件：/home/admin/apps/nginx/conf/pathinfo.confset $real_script_name $fastcgi_script_name;if ($fastcgi_script_name (.+?.php)(/.*) set $real_script_name $1;set $path_info $2;fastcgi_param SCRIPT_FILENAME $document_root$real_script_name;fastcgi_param SCRIPT_NAME $real_script_name;fastcgi_param PATH_INFO $path_info;站点配置文件server listen 8080 default; server_name _ ; index index.html index.htm index.php; root /home/admin/; #rewrite #location / # if (!-e $request_filename) # rewrite /(.+.(html|htm|xml|shopadmin)$ /index.php/$1 last; # # error_page 403 /themes/tmall/403.html; error_page 404 /themes/tmall/404.html; error_page 500 /themes/tmall/500.html; error_page 502 /themes/tmall/502.html; error_page 503 /themes/tmall/503.html; location .*.php/? include php_fcgi.conf; include pathinfo.conf; location .*.(gif|jpg|jpeg|png|bmp|swf) expires 30d; location .*.(js|css) expires 1h; access_log /home/admin/apps/nginx/logs/chaoshi_tmall_com_access.log;. php-fpm预安装包yum 安装包yum install libiconv libmcrypt nmhash mcrypt freetype freetype-devel libjpeg-devel libtool-ltdl libtool-ltdl-devel libjpeg-devel libpng-devel libmcrypt libmcrypt-devel -y加密库yum install libmcrypt编译tar zxvf libmcrypt-2.5.8.tar.gz cd libmcrypt-2.5.8/./configuremakemake install/sbin/ldconfigcd libltdl/./configure -enable-ltdl-installmakemake installcd ././ln -s /usr/local/lib/libmcrypt.la /usr/lib/libmcrypt.laln -s /usr/local/lib/libmcrypt.so /usr/lib/libmcrypt.soln -s /usr/local/lib/libmcrypt.so.4 /usr/lib/libmcrypt.so.4ln -s /usr/local/lib/libmcrypt.so.4.4.8 /usr/lib/libmcrypt.so.4.4.8hash函数库yumyum install mhash编译tar zxvf mhash-.tar.gz cd mhash-./configure makemake installln -s /usr/local/lib/libmhash.a /usr/lib/libmhash.aln -s /usr/local/lib/libmhash.la /usr/lib/libmhash.laln -s /usr/local/lib/libmhash.so /usr/lib/libmhash.soln -s /usr/local/lib/libmhash.so.2 /usr/lib/libmhash.so.2ln -s /usr/local/lib/libmhash.so.2.0.1 /usr/lib/libmhash.so.2.0.1cd .加密程序yumyum mcrypt编译tar zxvf mcrypt-2.6.8.tar.gzcd mcrypt-2.6.8/sbin/ldconfig./configuremakemake installcd ./异步事件库yumyum install libevent.x86_64 libevent-devel.x86_64编译MySQL库yumyum install mysql.x86_64 mysql-devel.x86_64给mysqlclient库建立一个符号链接ln -sv /usr/lib64/mysql/libmysqlclient.so /usr/lib/libmysqlclient.so编译安装phptar xzvf php-5.2.13.tar.gzgzip -cd php-5.2.13-fpm-0.5.13.diff.gz | patch -d php-5.2.10 -p1cd php-5.2.13./configure -prefix=/home/admin/apps/php -with-config-file-path=/home/admin/apps/php -with-gd -enable-gd-native-ttf -with-mysql=/home/mysql -with-iconv-dir -with-freetype-dir -with-png-dir -with-zlib -with-libxml-dir -enable-xml -disable-debug -disable-rpath -enable-discard-path -enable-safe-mode -enable-bcmath -enable-shmop -enable-sysvsem -enable-inline-optimization -with-curl -with-curlwrappers -enable-mbregex -enable-fastcgi -enable-fpm -with-iconv=/usr/local -enable-force-cgi-redirect -enable-mbstring make ZEND_EXTRA_LIBS=-liconvmake installcp php.ini-dist /etc/php.ini编译安装PHP5扩展模块1）memcache客户端tar zxvf memcache-2.2.5.tgzcd memcache-2.2.5/home/admin/php/bin/phpize./configure -with-php-config=/home/admin/php/bin/php-configmakemake installcd ./2）加速器eacceleratortar jxvf eaccelerator-.tar.bz2cd eaccelerator-/usr/local/webserver/php/bin/phpize./configure -enable-eaccelerator=shared -with-php-config=/usr/local/php/bin/php-configmakemake installcd ./3) PDOtar zxvf PDO_MYSQL-1.0.2.tgzcd PDO_MYSQL-1.0.2/usr/local/webserver/php/bin/phpize./configure -with-php-config=/usr/local/php/bin/php-config -with-pdo-mysql=/usr/local/mysqlmakemake installcd ./4) ImageMagicktar zxvf ImageMagick.tar.gzcd ImageMagick-6.5.1-2/./configuremakemake installcd ./tar zxvf imagick-2.2.2.tgzcd imagick-2.2.2/usr/local/php/bin/phpize./configure -with-php-config=/usr/local/php/bin/php-configmakemake installcd ./5) sphinxwget /downloads/sphinx-0.9.9.tar.gz安装libsphinxclient： 提取sphinx源码包api目录中的libsphinxclientvim sphinxclient.cset nu找到1216行 static void sock_close ( int sock )改为void sock_close ( int sock )./configure -prefix=/usr/local/libsphinxclientmake & make install/home/admin/apps/php/bin/pecl install sphinxphp.ini添加：sphinxextension=sphinx.so6) scwswget /scws/down/scws-1.1.3.tar.bz2wget /scws/down/scws-dict-chs-utf8.tar.bz2tar xjvf scws-1.1.3.tar.bz2cd scws-1.1.3./configure -prefix=/home/admin/apps/scwsmakemake installtar xjvf scws-dict-chs-utf8.tar.bz2 -C /home/admin/apps/scws/etccd /opt/shopex/scws-1.1.3/phpext/home/admin/apps/php/bin/phpize./configure -with-scws=/home/admin/apps/scws -with-php-config=/home/admin/apps/php/bin/php-configmakemake install添加php.iniscwsextension=scws.soscws.default.charset = utf-8scws.default.fpath = /home/admin/apps/scws/etcphp-fpm配置文件#vim /home/admin/apps/php/etc/php-fpm.conf All relative paths in this config are relative to phps install prefix Pid file /home/admin/apps/php/logs/php-fpm.pid Error log file /home/admin/apps/php/logs/php-fpm.log Log level error When this amount of php processes exited with SIGSEGV or SIGBUS . 30 . in a less than this interval of time, a graceful restart will be initiated. Useful to work around accidental curruptions in accelerators shared memory. 1m Time limit on waiting childs reaction on signals from master 10s Set to no to debug fpm yes Name of pool. Used in logs and stats. default Address to accept fastcgi requests on. Valid syntax is ip.ad.re.ss:port or just port or /path/to/unix/socket !- :9000 /- /tmp/php_fcgi.sock Set listen(2) backlog -1 Set permissions for unix socket, if one used. In Linux read/write permissions must be set in order to allow connections from web server. Many BSD-derrived systems allow connections regardless of permissions. 0666 Additional php.ini defines, specific to this pool of workers. !- /usr/sbin/sendmail -t -i - 1 Unix user of processes !- admin- Unix group of processes !- admin - Process manager settings Sets style of controling worker process count. Valid values are static and apache-like static Sets the limit on the number of simultaneous requests that will be served. Equivalent to Apache MaxClients directive. Equivalent to PHP_FCGI_CHILDREN environment in original php.fcgi Used with any pm_style. 300 Settings group for apache-like pm style Sets the number of server processes created on startup. Used only when apache-like pm_style is selected 20 Sets the desired minimum number of idle server processes. Used only when apache-like pm_style is selected 5 Sets the desired maximum number of idle server processes. Used only when apache-like pm_style is selected 35 The timeout (in seconds) for serving a single request after which the worker process will be terminated Should be used when max_execution_time ini option does not stop script execution for some reason 0s means off 300s The timeout (in seconds) for serving of single request after which a php backtrace will be dumped to slow.log file 0s means off 5s The log file for slow requests /home/admin/apps/php/logs/php-slow.log Set open file desc rlimit 65532 Set max core size rlimit 0 Chroot to this directory at the start, absolute path Chdir to this directory at the start, absolute path Redirect workers stdout and stderr into main error log. If not set, they will be redirected to /dev/null, according to FastCGI specs yes How much requests each process should execute before respawn. Useful to work around memory leaks in 3rd party libraries. For endless request processing please specify 0 Equivalent to PHP_FCGI_MAX_REQUESTS 1024 Comma separated list of ipv4 addresses of FastCGI clients that allowed to connect. Equivalent to FCGI_WEB_SERVER_ADDRS environment in original p