mpi实验 虚拟机环境下两节点的MPI集群搭建

信息学院实验报告学号:姓名:班级：网络工程实验时间：2015年5月26日学号:姓名:班级：网络工程实验时间：2015年5月26日实验地点：睿智4栋303开发环境：VMware-Ubunt14.04课程名称：云计算实验性质：验证性实验实验内容：虚拟机环境下MPI集群搭建题目来源：教师布置a）实验目的和任务了解建设高性能计算机集群系统的过程；熟悉利用MPICH搭建高性能计算机的方法;熟悉UbuntuLinux操作系统配置与管理。b）实验步骤2.集群局域网网络环境搭建与配置。a)Ubuntu中root用户和user用户切换及密码设置(node1和node2)。aibc@ubunXu:—$sudo:sutsuda]pas-swQrdfordbc:roct@ubun七ui/hone/abc#exvtex-Ltabic^ubuntu: ■b)网络搭建和ip地址配置obc@ubuntutfconftgethO Linkencapr太网硬件地址00i0ci29:df:d3if匚tnetifeilt：192,168.15,12S S5 1■Lnet6t也址：feSO::20c: f:fedf:d3fc/b4Scope:LinkUPBROADCASTRUNNINGMULTICASTMTUI1500畋点数：1接收数拒包：4错误沖丢弃汕过载沖帧数沖岌送数扌居包：135错误：0舌弃：◎过载曲载汝：0碰擢发送臥列长度Iiaaanode1接收字节(993,0B)发送字节I1S2E4(18,2KB)node1abc@uibuntu2: ifeonftg=七h® LinkLrtU0p;以衣网韦更彳牛t也址00:0c:29:2S:9f:84tnet±t!iy：：192r168*15*133 t192.108*15*255 ：255-255*255*□tnete土也址：fee®::20c：29ff:fe2&:?f84/6^1scope:LtnkUPBROAOCASTRUNNINGMULTICASTMTJ;1500族点数：1接收数扌居包：15萌稱误：0丟弃汕过载：3帧数：3躱送数扌居包：陀4错误池丢弃沖过载皿载菠池碰扌虽沖发送臥列长厘阿0node2接收宇节-334907(934”9KB)发送宇节Q94,&KB)node2测试网络的连通性nodelpingnode2tcnp_seq=ltcnp_seq=2tcnp_seq=3tempbytesof廿云七a_64ttne=9*584ms64tlme=©.443ms64ttme=G*490ms64ttme=0,S44ms192,168.15*133133:tcnp_seq=ltcnp_seq=2tcnp_seq=3tempbytesof廿云七a_64ttne=9*584ms64tlme=©.443ms64ttme=G*490ms64ttme=0,S44ms192,168.15*133133:192*168.15*133:192.168-15.133：PING33<192.168.61byxesfrom64bytesfrom64bytesfrom64bytesfrom——192*168*15.133ptngstatistics——4packetstransmtttzed*4recetved± packetloss±time3000ms「ttmln/avg/nmx/nMew=0*344/0*465/O+SS4/0+©S7msabc@ubu■修改计算机名/主机名、添加地址解析(两个节点nodel和node2均需配置)abc@nodel:sudogedit/etc/hostname[sudo]passwordforabc:abc^rodeli~$-匚日1z/et匚丿11口5七5192.168_15.128 nodel192.168.is.133 nodez(1)node1abcgnodez:sudogedLt/etc/hostname|2^udoJpasswordforabc:abc@n&dez: cat/etc/hosts.32node2192*16-8*15*1Z8nodel(2)node2修改完成，重启系统。步骤二创建SSH信任连接安装SSH服务，在终端窗口输入如下命令#sudoapt-getinstallssh升亦十V不僉茬包,新安装了6个软件包「要卸载由个软件包,有S5S个软件包未披升级。需要下载617kB的软件包。解压编后会消耗掉3,450kB的额外空间。径也望维绽执行吗？[V/n]y〒却【新】软件包将殺安装：libck-connector©ncurses-ternopenssh-serveropenssh-sftp-serversshssh-inport-id升级了&个软件包r新安装了6个软件包，要卸载0个软件包,有315个软件包未被升级。需要下载617kB的软件包◎解压缩后会消耗掉3,450kB的额外空间。您希望继续执行吗？E¥/n]yProcessingTriggersforufw(a.34-rc-0ubuntu2)正在设背ssh(1:6+6pl-2ubuntu2)…-ProcessingtriggersforItbc-bin(2,19-0uburtu6.5^abcgnodel:stdosuroot?rtodpi：/hopie/abcffshutdown-hnow来自at)c®nodel旳广播信忌(/dev/pts/o)于21:22.…现在f系统恪关闭(haltjroot^nodel:/horne/abc#4•测试登录本机(需要密码)sbc^nodel1—$sshnodelThe日1_1七hen七i_匚1七”ofhost'node1 (L92*168u15*12SJ1can'tbe巳：s■七曰bt-Lshed*ecds^key■fingerprintIs5a:7a：Z9：lb：4b:80:84:&7：g8 :&b:bs：ao:^ld:0©,Areyousureyouwanttocontinueconnecting(yes/no)?yesId3『ntng:FernaiftentLyadded'nodel#192»168a15.JLNE1(ECDSA)tothelist:ofknoi^nosts.-abcQnode1'spassword1WelcometaUbun"tu14*u2LTS(GNU/Li-nijx3J.16J0-3l!)-generi.匚只合&_凸4)*Oocumenfatton:https:///Theprogramsincludedu^l_ththeUbuntusystemarefreesoft:ware:the*exactdistributiontermsforeachprogramaredescribedInt:he■LndtwtduiaTTfLesLn/us厂/mhm厂E/dlo口*$copyiri_gih七亠Ubuntucomeswithabsolutelynowarrantytotheextentpermittedbyappltcabilelawabc@nodel|5•测试从nodel登录node2(互测：需要密码)abc@ne>del: sshnodesTheauthenticityofhost1nodes(192.168.15■丄33)'can1tbeestablished.ECDSAkeyfingerprintis2a:匚3:d日：41zbf:f亡：OS1：F】：号匸：db: ::丄5:fl:匸b:tb:5F・A厂皂yousureyouwanttocontinueconnecting(yes/no)?yesWarning:Permanentlyadded1node2^192168 fEC；代)totheListofkno-wntQS-tS*abc^node^'spassword:WelcometoUbuntu14*©4#2LFS(GNU/Linu^3+16+0-3©-generick86_64>*■Documentattain:https://help*ubuintu亠亡on/Theprogramsi-nclLidediwiththeUbuntusystemarefreesoftware;rheexactdtstribufLontermsforeachprogramaredescribedtnthetridtvidualfilestn/usr/share/doc/*/copyright.UbuntuconeswithABSOLUTELY1NOWARRANTY^totheextentp亡厂nittedbyappltcabtelaw.abc@node2:-$■

Node2登录nodelabc@node2$sshnodelThe*authenfLcltyofhost'nodel(192*168.15-128)'can't^stabltshcd.ECDSAkeyfingerprintis5a;7a;Z9zlb;4b;80：84;87;g8：96;9e;6bzb5;a0;4d：80*Areyousureyouwanttoconrttnjeconnecting(yes/no)?yeswarnting:Permamientl.yadded'modelj192.iL^a；・1孑・=£呂'(ECD5A)tothe"Listofknownfiosts*abcgnodel1spassword:WelcometoUbuntu14.04B2LTS(GNU/Linux3・Idgeneri匚x86_64)*Documentailztoni::https://help,ubuntu^con/345packagescanbeupdated*191updatesaresecurityupda七召£・Lastlogin:MonJun2921:38：532015fromnodel6•计算节点之间的无密码访问(nodel)abc@nodelpwd/home/abcisbcgnocie!- ssh-keygen-1rsaG&neraxtngpubLlc/prtvatersakeypair.passphras-eenp七yformoipassphrase)iEnt;亡厂filetnwhich七。save七h皀keyC/hone/abc/*ssh/td_rsaJpassphras-eenp七yformoipassphrase)iEntersanepassphraseagain:Your±dentT-fi_cati_onhasbeensavedi_n/home/abc/xS-sh/lcI_rsaYour-pubLLckeyhasbeensavedtn/home/abc^-ssh/Ldrsa,pub_Thmkeyri-ngerprtntts:Hr74:-fb:05j94:Sdre7:2dr4a:3S:7©:Sa:b2:b9:6frbfabcpnode1Thekey'srandomartImagets:*”-[RSA2048] -+Eo十- -M-abcgnodel:-5Is-al检杳公私钥对文件:3be®nodel:$Ts-atdrwxr-xr-x19abcabc□undirwxir-xir-xiroo七roott白合廿ApIT-irw 1日b匚aibcz47-4□'vnrrvj円厂■■「■■1abcabc220Apr-厂⑷一厂--r--1abcabcAprdrwx abcabc□undlirwx 曰t＞匸aibczApITJirwx Z2obc曰t＞匸J1vnjriftfx・・----3abcdbc^0915Apr-rw-r-・ir--1abcabcZ5AprJrwx 3abcabc□undlirwx; 2iroo七r0口七Jvn-厂冲-■・----1abcabc3800JunJriftjx・・----3abcabc^09*5AprJrwx 4abcabcApr-irw-ir it a三山uaibcz27^1Apir-irw-厂 r 1mb匸aibcz67SApIF1riftix・・----2abcabcJun-厂诃-----■・-1abc-abc10ilznurii-rw-rw-it--1abcabc132Apr-irw 1曰b匸abc:□un-IFW 1曰buabc:861□'vn1rnxr-xir-x2abcabc4①孕ISAprJrwxr-xr-x2abcabc4邸9疥AprJrwxr-xr-x2abcabc4。夕防Aprabc@nodel:+ssh$Is-al总用星.2.&drwx 2abcabc409©Jundrwxr'xr-x19abcobc4096□un-rw 1曰b匚曰be1675Jun-irw-r—r—1abcabc391Jun-rw-r--r--1abcabc88HJun&bc@node1：7*sshSI3S,27IT33_» q292122亠bas-h_hi-stzo厂¥27IT03)3■baiS-h Logout27IT03-ba&hrc3S鼻cache271835丄czonpi_x2总2牴■conf-Lg27丄72=4■dbu&Z7IT▼drnre35.#guemf2925•gv^s292133■ICEa<Jthor1ty27172=4■XocaTZ7IS3S27ISBQ亠pan_erbvLironmerTt271733亠pirolF-fLe29214<5.盂習nzm2ZL33.xaurhortty27IS30土xlriipsij七irc:29Z1MM亠xsessi_on-errors2P2122亠xsessi-on-errors-oTd27IS3S公■共的27Z7ieIS363S29zi:加■2921:38w■2921:■461d_rsa2921i4Ctd_r&a”pub2921:40known_hosts将该密钥用作认证，进行访问授权。按如下命令在nodel执行。（nodel）进入.ssh目录，生成authorizedkeys文件#cd.sshabc@nodel:〜/.ssh#cpid_rsa.pubauthorized_keysabc@nodelr~/*ssh$cpLdrsa*pubauthorizedkeysabc@nodelr~/*ssh$Isauthorlzed_keystd_rsatd_rsa.pubknown_hostsabc@node1:~/+ssh$pwrl/home/^bc/.ssh_7.在nodel安装/更新nfs服务器1） #sudoapt-getinstallnfs-kernel-servernfs-common建议妄装的软件包： 'open-iscsiwatchdog下列【新】软件包将襪安装：llbgssgluelltbnfstdnap2llbtirpclnfs-connorinfs-kernel-serverrpcbtnd升级了0亍软件包‘新安装了6个软件包，要卸载0亍软件包,有353个软件包未披升级。需要F载427kB的软件包。解压塑后会消耗掉1,900kB的额外空间"您希望继绽执行吗？[¥/n]y2） 创建共享文件夹在nodel（master或称server）的/home/abc目录下创建共享文件夹cluster，作为所有节点共享的目录（也可将MPICH2安装在该目录下，同时可以设置所有节点已启动便挂在该目录，便可只在主节点上安装一次MPICH即可。）abc@nodel:pwd/hone/abcabc^node1:-$Isrwd": 公共的樓板视频圈片文档下载育乐卓闻，亠 ahc^ntjde1: sudornkdtrclustermkdtri无法刨建目录"clusterHi文件已存在桌面abcOnode1.: ts桌面clusternodeZ:公共的樓扳视頻BB片文档下载育乐■3）修改cluster目录所属的用户及组，使用ls-al命令可以看到，刚才创建的cluster目录属于root用户，需要将它的用户和组改为abc。drwxr-xr-x21drwxr-xr-x21■ab匸asb匚4096Jundrwxr-xr-xSrootroot4096Apr-rw 1abcabc838Jun-rw-r--r——1abcabc220Ap厂-rw■r■*IT…1abcabc363-7Aprdrwx--17abcabc4096□undrwxr-XF-Xzrooxroot□undrwx--3abcabc^□9&Aprdrwx--2Zabcabc4096Junabc@nt>del:Ls-at总用量.12S22:1117:03-22:05・bashhi-stor17：asba^hl_ogou±17:0S亠bashrc22:09.cacheZZ：11dusker18：35■CORPUZ19:26-BconT"Lgi979779979222222ZZ2使用如下命令更改文件夹拥有者和群组:#sudochownabc:abc/home/abc/clusterz一爭虫udci■czHown-abui曰匕匸abc^nodiei4會=■-sT-S・1drwxr-xr-XNabc-abc4曰夕防dun29dirwM 4■at>u□'Un=20drwxr■=xr-X玄abcApr2L7drwxr-xr-X2aibc-abcApr27dirwxr-scr■-XN鼻t>U4C3召■&ApITN7dlirwxr-x:it-X=2mb匚日b匸却3/16ApIFN了drwxr■=xir-X玄abe-dbe吗13日＜5Apr2L7drwxr-xr-XNabc-abc4曰E专Apr27dirwxr->cit~M2mb匚mbu舛OQiSApITZ27dirwxr-x:if-X工mb匸日b匚ApIT=2了N2z已耳ISS3©185□tStL8:3SLBZ3右IS；3©X8二tL£3=Z3SJcXlJSXeFnodeZ*公捉的0H片立档下瞬音乐编辑/etc/exports文件（共享文件夹访问权限设置）：#Sudogedit/etc/exports或#sudovi/etc/exports▼ 保存旦4撇消立件苦）堀預（曰查召搜弟（S）H貝（T）立档（四帝助（H）■JexportxZhome/abc/cTusterrw^synCjnorootsqua^h^3） 重启服务nfs是一个RPC程序，使用前，需要映射好端口。重启portmap（新版本已替代为文件rpcbind）abc@nodel:/etc/lrblt;*d5sudoapt-gettnstaLLportmapnTs-commoniE在读取软件包列表*…完成[E在分析软件包的依赖关系树叵在谨取氷态信慝…完底生意,选取厂pcblnd而非portmap^fs-common己经是最新的版車了心pcbi-nd己经是最新的版本了口^pcbLnd被识置光手动安装。F列钦件包是自动安装的并且现在不需要了：•重启nfs-kernel-server服务root@nodel:#sudo/etc/init.d/nfs-kernel-serverrestartabc^nodelsudo/etc/inttxd/rpcbindres七日「七[sudo]passwordforabc;abc^nodelsudo/etc/intt^d/nfs-kernel-serverrestartoppingNFSkerneldaenon [OKLlnexportingdirectoriesforMFSkerneldaemon... [OKNotstartingNFSkerneldaemon:noexports*显示共享出的目录abc@nodel:~Sshownount-eExportlistfornodel:/home/abc/cLustsr七abc@nodel|4） 安装完毕，挂载共享目录：将node1共享目录/cluster挂载到各个节点的目录/cluster在node2安装/更新nfs服务器#sudoapt-getinstallnfs-kernel-servernfs-common在node2上创建共享目录/home/abc/clusterabt@nodel=-5^home/abcabc@nodel:~$mkdtr:无法刨ati匚^nodel:Ls:无法访冋1abc@nodel=-$总用量4Odrwxr-xr-xrlrwx drwxr-xr-xdrwxr-xr-xdrwxr-xr-xdrwxr-xr-xdrwxr-xr-xrlrwxr-xr-xdrwxr-xr-xdrwMr-xr-xhhEneHiTt-1■pwdnk-dvrclws七£「建目录"匚Iuster,F：Ls.1_:沒有那个忘f牛或文件己存在目录2422222222^bbbbbbbbbbamfo曰曰aBBccccccccccbbbbbbbbbb曰aa!曰m曰曰3曰曰4096Jun4395JUD4-&5S6Apr4096Apr439&Apr4-096Apr40^6Apr499©Apr斗。9石Af>r4096Apr997777777722:11Z2:3418:3S18:S318:33:3318:S318:斗3:LE:S818:38r■e的瞪共蠢片档載嗇"no公蠢旻下慕•挂载目录，并执行df命令检查文件系统的磁盘空间占用情况。（注意：系统重启后需重新挂载）abc@node2:~$sudomount-tnfsnode1:/home/abc/clustercluster（slave节点每次重启后需运行此挂载命令）abc@node2:~$df•并执行df命令检查文件系统的磁盘空间占用情况。abc@nodez:sudomounx-tnTsnodel:/home/abc/clusrerclusterabc^nodez:df文件系统lK-blocks已用可用己用离挂载点/devysda11947S2G4405243^144132^222^/none4■34e%/sys/Fs/匚groupudev426412目4264©a1%/devtzmpfs8742414728S9522%/runnone5120OS12®0%/run/Lockr»or>e-1371121524369601%/run/shnnone10240©Z81QZ3721%/run/usernodel:/home/abc/cluster—■zVx_ r=_19-aT8Z7243T7S5&1-1087936-2-1%/home/abc/cluster

8.安装MPICH3并行计算环境(1)安装前准备(nodel和node2)1)使用命令可以查看当前编辑器的版本：(nodel和node2)abcgnodel:gcc--versiongcc(Ubuntu4,8r2-19ubuntul)4*0*2Copyright(C)2013FreeSoftworeFoundB七ion，Inc,Thisisfreesoftware;seethesourceforcopyingconditions*ThereisNOwarranty;notevenforMERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE*abc@node1z --versiong+4-(ubuntu4,8,4-2ubun4.3.4Copyright(C)2013rreeSoftwareroundation,IncrThistsfreesoftware;seethesourceforcopyingconditions.ThereIsNOwarranty;notevenforMERCHANTABILITYorFITNESSFORAPARTICULARPURPOSE*abc^nodeli-$python--versionPythem2-7-6abcgnodel:~$从网上下载MPICH压缩包(mpich.tar.gz)：(nodel)2)图形模式：浏览器下载/downloadabc@node1:cdclusterabcgnodel:-/cluster?Is-1总用虽1122U-rwxrw-rw-1abcabc11487313Jun2923:soabc@node1:-/匚1口帝七巳「$■3)解压缩tar文件(node1)：abc@node1:〜/cluster$#tarxfzmpich-3.1.3tar.gz系统自动生成解压目录/文件夹:mpich-3.1.3mpich-3.1.3mpich-3.1.axar.gzmpich-3.1.3mpich-3.1.axar.gz创建mpich3.1安装目录(/home/abc/cluste):(nodel)abcfflnodel;~/cl_uster$nkdxrriptuhS”J.drwxrw^r-x2abcdrwxrw^r-x2abcabc409(5-drwxr-xr-x11abcabc-rwxrw-rw-1abcabc13.48733.33t*c@nodel: clustersLs-1总用量1122a□un30g:02mpi.Gh3■1OCT：BZO14Fip1_ch-3.1,3Jun2923:50mpxeh-3.1.3.itar.gz配置(node1)：CongureMPICH,specifyingtheinstallationdirectory,andrunningtheconfigurescriptinthesourcedirectory:cd命令进入mpich-3.1.3解压目录：指定安装配置：abc@node1:~/cluster/mpich-3.1.3$./configure-prefix=/home/abc/cluster/mpich3.1--disable-fortran--disable-fcdbcgrodei:-/cluster/mpich-3.1.3$./configure-prefix-Zhone/abc/cluster/mpich^*1--disable-fartrar--dtsable-fcconftgurLngMPICHverston3.1.3with'-preftx-/home/abc/clij£ter/mptchi.1--dtsabte-fortran' '--disable-fc'BuildMPICH(构建)(nodel)abc@node1:~/cluster/mpich-3.1.3$makembu电rmdlE].i—/cl.us七Eir/mpi-Czh-S亠 华mal<etestI-h*/s厂匚/*0匸1“£吕/mpiLQ#h;then\rm-f,/src/tnclude^mpto^h; \Ced./src/LncTude&.&Ln-s../mpl-/ronto/1-ncLLide/npl-och) ; \make a11-make a11-recursivemakefx]:正在进入曰灵'/hone/abcz/clijster/'mptch-3,1,3alti_n/home/abe/clusXer/mptch-3.1.3_/src/nplmakeL^l:IE在进入曰录'/hone/abc/dustier/mpi-ch-3.1.3/src/nplCC sirc/rnpTs"tirJ-l_oCC sre：/mplizirmem,1o匚匚 sre/-mplenvto

匸匚 src/env/mpt^arsCCLD src/env/mpivarscp-psrc/env/mptcc・Bai£hsrc/env/mptcccp-psrc/env/mptcxM^bash吕厂匚/env/mptcxxnnk亡[2]:正在商弁目承'/home/abc/cluster/mptch«3.1・3'Makingalttnexamplesmake[2]i正在琏入自呆“/home/abc/ctuster/nplch-台.1,3/eKampLes*CG Cpl.rOCCILD cp-Lm日k：E[2：]:正在蜃开目录⑴/home/abc/cLuster/mpich-3.1・3/exanples'make[1LJ*正在离亓目来u/hQme/abG/cluster/npT-ch-3+1.3'abc^nodel:-/cluster/mptch-3.1.3$H7） 安装（nodel）InstalltheMPICHcommands:abc@nodel:〜/cluster/mpich-3.1.3$#makeinstallmptehs.l/ltb/pkgcoriftg'make[3]|:正李睜幵目〔录"/home/abc/cluster/mpich-3^1,3'make[2]i正在离拜目录'/home/abc/cLuster/mptch-3*1*3'MakingInstalltnexamplesmake[2]|:正在进入目录x/home/abc/cluster/nptch-3亠13/examplesmake[3]:正在进入目I亲'/home/abc/cl_uster/nptch-3.1.3/examplesmake[3]i沒有什么可UX做的为'tnstall-exesz-am'4make[3]:程宥j十么討旦做的为'Install-dataam•omake|[3]iZE^E 目 '/home/aibc/c'Lus七&厂/冃卩1_匚h-3亠1*3/exampTes1make[2：]:正李商拜曰录/home/abc/clusrer/mptch-3.1.3/examplessmake|[1]|iTTF^F 目 '/home/abc/ulus■七er/mpi_ch-3*1*3'abc@nodel:-/cLuster/mplch-3.1»3$.检查安装：aDdgnodiel:—/dusrer/mpxcri-3.1.3S-cd...ab<@nodel:—/eXust:er$Tsrnp"LChi3.imp-Lch^S.±.3[PipVchi^-^. giat>iC@niOdel:—/elust;er$cdnptjcIti-3,1abc@niOdetL:«/dusxer/nHp"lch3#a$l&b^Lntnulmd豐 XXb名b目厂isabic®model.:«/dusxer/mp"lc：h3#a$8） 修改PATH环境变量，编辑用户主目录下的〜/.bashrc文件（node1和node2）Addthebinsubdirectoryoftheinstallationdirectorytoyourpath:将安装后MPICH3.1的bin目录添加到系统路径中，即添加到启动脚本中：abc@node1:〜$sudogedit.bashrca«bc@nodel:~5sudogedit,bashrc[sudo]passwordforabc:9）重新设置环境变量（node1和node2）abc@node1:~$source.bashrcmb匚gnodel:~$abc@no<del:^$mb匚gnodel:~$abc@no<del:^$source.bashrmptr^jnnp'Lv^arsmptr^jnnp'Lv^ars10）查看PATH变量变化（node1和node2）abc@node1:~$echo$PATHabc@irkodeJ.:—$echo$PATH/homeab匚fu1ui：stEi-#nip5LchM・1/btn:/usr/TocaT/sblnx/usr/Tocat/bi-ni/usr/sbi-n:/usr/btn;/sbi_n;/blnii/usr/ganes:/usr/local/ganesobc^node1:|abc@node2:echo$PATH/hone/abc/cluster/nptch^*1/bin:/usr/local/sbtn:/usr/tocal；btn:/usr；sbtn:fusr/btn:fshin:/btn:/usr/ganes:/usr/locol/ganes11）查看bin下的可执行程序（node1和node2）abc@node1:~$ls/home/abc/cluster/mpich3.1/binfibc@rhodle1:—STs/