《Hadoop大数据技术与应用》

車無爺七dooppq订碼谛《bOg*年霧祿*doop^H》

一、 实验目的1.安装Java环境2.搭建Hadoop单机模式3•在Hadoop单机基础上搭建伪分成式模式环境二、 实验环境Linux操作系统三、 实验内容与实验过程及分析（写出详细的实验步骤，并分析实验结果）实验内容：1.完成安装Java环境。wwp^ncdel:java-versionjavaversion,1*8*O_11,FJava(TH)SERuntimeEnvironnen七(build1*S*Oil-bl2)JavaHotSpot(TM)64-BttServerVM(build25*ll-b03.nixednode)wwperiodcl: I2.完成搭建Hadoop单机模式（1）在Linux终端用wget命令直接下载至Linux系统中（命令在同一行）$wget/dist/hadoop/common/hadoop-2.7.5/hadoop-2.7.5.tar.gz（2）解压安装包：解压安装包至~目录下，并创建软链接（或重命名mv）$tarzxvfhadoop-2.7.5.tar.gz-C~$ln-s~/hadoop-2.7.5~/hadoopwwwp@>node1:-wwp^nodel;-$Isdata,tsctexamples*desktuphiadoopwwp^nodel;-$Isdata,tsctexamples*desktuphiadoopha4ocp-3-2^0wwp^nodel:|inputjdkjdkloutputsoftwareVMwareT口口I盘-10・d・10-12斗巴dmsz・ta「・gt:vmware-tools-distrib公共的載乐面下音桌板罟档模视图文3）执行命令$./bin/hadoopversion查看hadoop版本信息：$cd~/hadoop/etc/hadoop$vihadoop-env.shMMp^nodei:/hone/wp/hadwop/bin/hadoopversionHadoopJr2,0Sourcecoderepositoryhttps://gtthub<con/apache/hadoop-git*「e97acb3bd8f3befd^741899&f:d5d4b59bf:2el7bfCompiledbysuntigon2019-91-0BT0608ZCompiledwithprotocFromE&jrcewithchecksund3f0795ed0d9dc37B&2c785d366Sf39Thisconnandwasrunustng/home/wwp/hadoop-3.2+EJ/sharEjh日doop/匸0Enon/hadoQp-c:onnon-3*Z+0.jarwwpQnodel:-$(4)找到exportJAVA_HOME，把#去掉，并修改为：#Thejavaimplementationtouse.exportJAVA_HOME=/home/hadoop/jdk1.8.0_181#Thejai/aimplenentmtiontouse,BydefauLt,thisenvironmentffvariableisREQUIREDonALLplatfornsexceptOSXIexportJAVA_HOME=yhone/wwp/jdk乎Locationofnadoop.Bydefault,HadoopwillattempttodeterntneMthislocationbaseduponitsexecutionpath,如亡HADQQPHOHE= (5)如果出现以下信息，说明安装成功：Hadoop2.7.5Subversion/repos/asf/hadoop.git-rbaa91f7c6bc9cb92be5982de4719c1c8af91ccffCompiledbyrooton2016-08-18T01:41ZCompiledwithprotoc2.5.0Fromsourcewithchecksum2e4ce5f957ea4db193bce3734ff29ff4Thiscommandwasrunusing~/hadoop/share/hadoop/common/hadoop-common-2.7.3.jar3.完成在Hadoop单机基础上搭建伪分成式模式环境单机模式测试：wponodei:vt hadoop/etc/hadoop/hadoop-env*shwwp^nodelcat~/ou七put/p吕rt-r-0080勺:LibreOFficeCalcWorld1uwpSnodel:~Scd~yhadoop/sbare/hadortp/napreduce/Mup^nodel:M/hadoop/share/ha{icap/napreduce$hsdoopjarhadoop-napreduce-exanples-3,2,0,jarwordcount~/input/data.txt~/outpuhadoop：未找到命令Mwp^nodel:~/hddoop/share/hadoQp/napreduce$-/hadoop/bin/hadoopjarhadoop-napreduce-ex3nples-3.2.0.jarwordcount-/input/data.txt~/outpu七/2019-0?'1713:51:34,601INFOclient.RMProxy:connectingtoResourceManageratnodel/19^,16S.Z01*128:0032伪分布式模式测试：wp@nodel^/hadoop/sharefhadoop/HapreduGe$ 目「t-寸LI,shNARNIMG:启七七emptingitostartallApacheHadoopdaenonsaswwpin10seconds.WARN!MG:ThisIsnotareconnendedproductiondeploynentconfigurationrWARNIMG:UseCFRLCtoabort.Startingnamenodggon[nodel]artingdatanud亡§Startingsecondarynamenades[nodel]StartingresourcenanagerStart;ingnodemanagewwp^nodel:~^adocp/share/hadoop/mapreduce$jps3797DataNode47SSJps3672NaneNode3977SecondaryNaneNode4396NodeMsnager4^69ResourceManagerWeb访问Hadoop：- □ X殆N^menodeInfwiriatloni x+«■ >CO© 192J6^ZO1J20:5OC7O/dfywatt^lrtml#tab-we™w*二1■v.£Z月 *TErtH©胡竝抿乍那T抚召深心岀MybaM— Q空理了几个幵JJ和.. 岀UML音相甜莒络碑..国*wMavenHeposiitary 捲HadOOpChi'pruiewCMtarxxies Dale?ncxleVolunieFailuresSnapshotStartupFrogre??Utilities▼Overview'nodel:8020h(active)Started:TueSep1719:52:444O8OO2019Versioiii：3.2.0,re97acb^bd8f3befcl27418gg&ta&d4b50bf2e17bfCompiled:TueJan0814.06.00+D80&2019bysunilqfrombfdrth-i.JjOOkist・F1氏CID-747bc&d5-e2f6-470l>-98ef-cf63ccd40e91PedId;BP-932278455-192.166201J3B-156B524165191SummarySecurity店off.S^femode鸣off.27fil&ianddirectories,13blocks(15r^plicaledblocks,&erasurecodedblockgroupi}=4Dlot^lfllpg胃於mobject(i).0AilAppli(ailorv$ x +□ Xe-CCr® 192.1S0.2O1.12B:8(J9e7clu5terS蜒了八苗嗥刃...山UMLSMiE.FS-gj...0[fFSa®]MESI...AllApplicationsOSKond^ryriamenodeiRform.料 卡- □ X「十TCDQ I92/I&B.201/12&SOWstatuSuhtml■・*|lit 1下ttf抽聒阪生6若宣iffl貳- 生孰般iMybaX-..[i整延了儿対瑚的… 占UML冉曲區忌虫-寵..站程】MESI...彥Mar.^nRepotltwy 川HadoopOverviewHdJacip.2019.:hdFsdf5-nkdir/inpu七nkdtr:/Input1:Fileexistshdfsdfs-Is/inputFound1i-tems-rw-r--r-- 1wwpsupergroup 252019-09-151318/tnput/dats*txtlniwp^nodel: cd-/hadoop/share/hado&p/mapredjce/wp^nodei:"/hadoop/share/hatioop/inapreduce^hadoopJsrhadoop-napreduce-examples-Is/output13;29/output/_SUCCES513:29/ou1tpu七-Is/output13;29/output/_SUCCES513:29/ou1tpu七-cat/output/pdrt-r-00000-rw-r--r--1wwpsupergroup 02019-09-15-rw-r--r--1岬岬p