Hadoop3大数据部署与数据分析实战实验指导书

2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载P1-部署Hadoop完全分布式与搭建开发环境2021-03-1916分钟阅读时⻓hadoopdv20250303任务1.1部署Hadoop完全分布式掌握搭建Hadoop完全分布模式环境。熟练掌握Linux常⽤命令如vi、ping、cat、ssh等。掌握VirtualBox、FinalShell等客户端的使⽤。操作系统:64位Windows系统。FinalShellCentOS7.9系统镜像VirtualBox7Hadoop3安装包https://heisun.xyz/docs/hadoop-e/hadoop-e01/1/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/2/412025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载（2）完成Hadoop完全分布模式的部署3台虚拟机来部署Hadoop完全分布式环境。开发常见的JDK+Maven+IDEA组合来进⾏部署和配置。安装VirtualBox1.安装VirtualBox，过程略。如果之前有安装旧版本的VirtualBox，请先卸载。2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/3/413.打开VirtualBox，并导⼊前⾯解压的“HadoopTmpl”虚拟机，由于接下来我们要从这4.导⼊成功以后可以在VirtualBox左侧看到HadoopTmpl虚拟机。https://heisun.xyz/docs/hadoop-e/hadoop-e01/4/415.打开VirtualBox的⽹络管理器进⾏配置。https://heisun.xyz/docs/hadoop-e/hadoop-e01/5/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/6/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/7/4110.安装FinalShell，过程略。11.使⽤FinalShell连接模板机，连接配置如下：https://heisun.xyz/docs/hadoop-e/hadoop-e01/8/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/9/4112.HadoopTmpl模板机有2个⽤户，通常只需要使⽤hadoop⽤户登录系统即可。https://heisun.xyz/docs/hadoop-e/hadoop-e01/10/4117.关闭HadoopTmpl模板机。依次从模板机复制出3台虚拟机，名称，主机名和IP地址如主机名（hostname）IP地址NodeAnodea+你学号后3位（例如nodea101）NodeBnodeb+你学号后3位（例如nodeb101）NodeCnodec+你学号后3位（例如nodec101）18.由于刚复制好的虚拟机都使⽤同⼀个IP地址，所以不能同时启动模板机、NodeA、NodeB或NodeC。需要依次启动，登录，并逐⼀修改为对应的hostname和IP。修改IP地址的⽅法请点击此链接查看19.同时启动NodeA、NodeB和NodeC3台虚拟机，配置FinalShell分别连接3台虚拟机，https://heisun.xyz/docs/hadoop-e/hadoop-e01/11/41client⽣成⼀对密钥（包括公钥和私钥然后将公钥传到服务器。当客户端通过ssh登录Hadoop的NameNode是通过SSH来启动和停⽌各个节点上的各种守护进程的，这就需注意此阶段命令如⽆特殊说明，均在NodeA的hadoop⽤户下执⾏！20.保证NodeA、NodeB和NodeC3台虚拟机都处于启动状态。使⽤hadoop⽤户登录NodeA节点。如果使⽤root登录的可以使⽤以下命令切换到hadoop⽤户。21.使⽤ping命令检查是否能够连通NodeB和NodeC。.如果没有看到以上返回消息，请检查/etc/hosts是否修改正确，参考Part1步骤14。22.配置免密登录。⾸先⽣成密钥对，运⾏以下命令。直接回车（Enter）3次。https://heisun.xyz/docs/hadoop-e/hadoop-e01/12/41在返回的对话⽂字中，直接回车（Enter）3次，输出内容类似以下。GeneratingGeneratingpublic/privatersakCreateddirectory'/home/hadoYouridentificationYourpublickeyhasbeensavedSHA256:MSUbr5VaCY4KSpsCM0l8uhYWkr5R9iNI05SFuF00jLAhadhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/13/41ssh-rsassh-rsaAAAAB3NzaC1yc2EAAAADAQABAAABAQC1Df9cM8NVGURMj3I86EJX9ISSXDZ9Qx+a5CCdoZJyySG3IlvAFBLv2Wnv60tDZ9xHEQ0WbkAV/IeDrdRk1OI51/bEGfic1eXsFC6luc7kbQYuxQRoeovl2UwHNgzAX/xTyUV0uAuvTeggyGWq05I9OiantybrumNUJOCA/zvNrJbuvVDKT9AAqQpn57jDsHkTiAlGoubKUcgAWy1EbYk7hVCL1gFkMcxDMvSOBoY23ouho2Cj2fNUinaDNDPPzoqbDwvU9IUCGhgfiNYb4Ub/hoabJRjlcNiEgoD+G79lNdhadoop26.修改authorized_keys的权限为444，让NodeA能够免密登录⾃⾝。ssh-copy-idssh-copy-id-i~/.ssh/id_rsssh-copy-id-i~/.ssh/id_rs.输⼊hadoop登录密码hadoophadoop@nodeb你学号后3位'spa28.使⽤以下⽅法测试免密登录是否配置成功，在Nodhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/14/41修改Hadoop配置⽂件注意此阶段命令如⽆特殊说明，均在NodeA的hadoop⽤户下执⾏！29.备份和编辑Hadoop的core-site.xml配置⽂件。在configuration标签内添加配置，注cpcp/opt/hadoop/etc/hadoop/corevim/opt/hadoop/etc/hadoop/cor<name>fs.defaultFS</name><value>hdfs://nodea+你学<name>hadoop.tmp.dir</name><name>erval</name><name>hadoop.http.stati30.备份和编辑Hadoop的hdfs-site.xml配置⽂件。请注意替换为你的学号。cpcp/opt/hadoop/etc/hadoop/hdfsvim/opt/hadoop/etc/hadoop/hdfhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/15/41<name>dfs.secondary.ht<name>dfs.replication</name>31.新建⼀个masters配置⽂件，写⼊SecondaryNameNode的主机名。vimvim/opt/hadoop/etc/hado32.备份和编辑Hadoop的mapred-site.xml配置⽂件。注意替换为你的学号后3位。cpcp/opt/hadoop/etc/hadoop/mapredvim/opt/hadoop/etc/hadoop/mapre<name>mapreduce.fram<name>mapreduce.jobhistor<name>mapreduce.applicationhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/16/41<value><value>$HADOOP_HOME/share/hadoop/mapreduce/*,$HADOOP_HOMpreduce/lib/*,$HADOOP_HOME/share/hadoop/common/*,$HADOOP_HOME/share/hadon/lib/*,$HADOOP_HOME/share/hadoop/yarn/*,$HADOOP_HOME/share/hadoop/y*,$HADOOP_HOME/share/hadoop/hdfs/*,$HADOOP_HOME/share/hadoop/hdfs/lib33.备份和编辑Hadoop的yarn-site.xml配置⽂件。注意替换为你的学号后3位。cpcp/opt/hadoop/etc/hadoop/yarnvim/opt/hadoop/etc/hadoop/yar<name>yarn.resourcemanager.hostname</name><name>yarn.nodemanager.au34.编辑workers，清除原来的所有内容，增加配置DataNode节点信息。注意替换为你的学vimvim/opt/hadoop/etc/hado从Hadoop3.0开始，slaves已经启⽤，改⽤workers来进⾏替代配置数据节点信息。35.修改hadoop-env.sh，在第1⾏加⼊以下代码。vimvim/opt/hadoop/etc/hadoop/hadoop-env.shhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/17/4136.把NodeA节点的Hadoop/opt/hadoop/etc/hadoop下所有配置⽂件发送到NodeB和scp-rhadoophadoop@nodeb+你学scp-rhadoophadoop@nodec+你学37.格式化HDFS。注意此命令请勿重复执⾏，因为会导致DataNode和NameNode的集群ID不一致，造成HDFS出错。2022-01-242022-01-2414:32:54,209INFOcommon.Stora38.创建Hadoop启动脚本，注意替换为你的学号后3位。vimvim/opt/hadoop/sbin/start-hdp.shmapred--daemonstarthistoryse39.创建Hadoop停⽌脚本，注意替换为你的学号后3位。vimvim/opt/hadoop/sbin/stop-hdp.shhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/18/41mapredmapred--daemonstophi40.创建Hadoop重启脚本，注意替换为你的学号后3位。vimvim/opt/hadoop/sbin/restart-hdp.shchmod744start-hdp.sh42.使⽤脚本启动Hadoop。https://heisun.xyz/docs/hadoop-e/hadoop-e01/19/41验证Hadoop是否正常启动46.在NodeA输⼊以下命令查看机架拓扑是否有NodeB和NodeC的信息hdfshdfsdfsadmin-print验证HDFS是否正常⼯作48.打开宿主机浏览器，访问HDFSWeb界⾯1:9870/49.查看NameNode是否Activehttps://heisun.xyz/docs/hadoop-e/hadoop-e01/20/4150.查看2个节点DataNode服务状态是否正常。52.把countryroad.txt从https://heisun.xyz/docs/hadoop-e/hadoop-e01/21/41hdfsdfs-put/home/hadoop/countryroad.tx验证MapReduce是否正常⼯作53.运⾏Hadoop⾃带的Wordcount程序，观察输出的内容。cdcd$HADOOP_HOME/share/hadoopjarhadoop-mapreduce-examples-3.3.1.jarwordcount/part2/country54.程序执⾏过程中，可以访问YarnWeb界⾯查看任务进展。1:8088/cluster/appshdfsdfs-cat/output/part任务1.2搭建Hadoop开发环境.掌握IDEA的安装和使⽤掌握Maven的安装、配置和使⽤命令https://heisun.xyz/docs/hadoop-e/hadoop-e01/22/412025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载Windows7以上64位操作系统JDK8-JavaDevelopmentKit是Oracle公司针对Java开发⼈员发布的免费软件开发⼯具包，是Java开发必备的开发⼯具。IntellijIDEA-业界简称IDEA，是jetbrains公司推出的和Eclipse⻬名的Java集成开发ApacheMaven-ApacheMaven，是⼀个软件项⽬管理及⾃动构建⼯具，由Apache软件基⾦会所提供。是Java构建打包最⼴泛使⽤的⼯具。为了能够使⽤编程的⽅式访问和调⽤Hado编程环境。本次任务主要是完成JDK、Maven和IDEA的安装和配置。完成JDK、IDEA和Maven的安装与配置安装JDK81.在Windows运⾏安装jdk-8u261-windows-x64.exe，安装过程略。此处以安装到2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/23/412.进⼊Windows的环境变量配置界⾯，配置以下环境变量。如果系统C盘会还原，每次重启https://heisun.xyz/docs/hadoop-e/hadoop-e01/24/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/25/41CLASSPATH=%JAVA_HOME%\lib\dt.jar;%JAVA_HOME%\lib\tools.3.打开Windows的命令⾏终端，运⾏以下命令，测试是否有JDK的版本输出。安装IDEAhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/26/41在⽂件末尾加上以下代码，让IDEA默认使⽤UTF8编码。安装和配置Maven36.解压apache-maven-3.6.3-bin.zip，这<localRepository>标签内内容注意修改为你的Maven的实际安装路径。<localRepository>D:/maven363/repos<<localRepository>D:/maven363/repos<注意：这个XML的标签，需要放在XML的注释外，放在https://heisun.xyz/docs/hadoop-e/hadoop-e01/27/41址。开发过程中依赖的Jar包可以通过配置从此地址下载。<name>Nexusaliyun</name><url>/nexus/content/gr注意：这个XML的标签，需要放在XML的注释外，放在9.进⼊Windows的环境变量配置界⾯，配置以下环境变量。可参考步骤2MAVEN_HOME=D:\maven以下截图仅供参考，请根据你⾃⼰的Maven路径设置。https://heisun.xyz/docs/hadoop-e/hadoop-e01/28/4110.打开Windows的命令⾏终端，运⾏以下命令，测试是否能够输出你的mvn脚本所在路.正常会输出你的mvn路径，以下截图仅供参考。安装和配置IDEAhttps://heisun.xyz/docs/hadoop-e/hadoop-e01/29/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/30/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/31/4116.修改IDEA的Maven配置，指向本地安装的Maven。这⾥注意替换为你的Maven的实https://heisun.xyz/docs/hadoop-e/hadoop-e01/32/4117.修改项⽬下的pom.xml⽂件，此⽂件是Maven项⽬的配置⽂件。2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/33/41<?xmlversion="1.0"e<projectxmlns="/POM/4.0.0"xmlns:xsi="/2001/XMLSchemaxsi:schemaLocation="/POM/4.0.0http:/<project.build.sourceEncoding>UTF-8</project.buil<piler.encoding>UTF-8</maven.c<piler.source>8</piler.source><piler.target>8</maven<hadoop.version>3.3.1<groupId>org.apache<artifactId>hadoop-co<version>${hadoop.v<groupId>org.apache<artifactId>hadoop-cl<version>${hadoop.v<groupId>org.apache<artifactId>hadoop-<version>${hadoop.v<artifactId>hbase-cl2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/34/41<artifactId>hbase-se<artifactId>hbase-se<artifactId>mockito-<scope>test</scope><groupId>org.apache.maven.<artifactId>maven-jar-pl<!--<manifestFile>${project.build.outp-INF/MANIFEST.MF</manifes<mainClass>hadoop.mapreduce.wc.Wor2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/35/41https://heisun.xyz/docs/hadoop-e/hadoop-e01/36/4121.JUnit是Java开发中最常⽤的单元测试框架，可以帮助测试我们编写的代码。JUnit⾮常容易上⼿。我们可以新建⼀个JUnit测试类学习使⽤它。https://heisun.xyz/docs/hadoop-e/hadoop-e01/37/41//assertEquals第1个参数时期待值，第2个参assertFalse(3==(1+1));//assertTrue(2==(1+1));////字符串的split方法是根据第一个参数字符来把字符串String[]strArr="唱,跳,RAP,篮球"}}https://heisun.xyz/docs/hadoop-e/hadoop-e01/38/411.NodeA节点NameNode或SecondaryNameNode⽆法启动。并且⽇志提⽰“/opt/hadoop/tmp/dfs/namesecondaryisinaninconsistentstate”。(1)⾸先把NodeA节点的Hadoop/opt/hadoop/etc/hadoop下所有配置⽂件发送到NodeBscp-rhadoophadoop@nodeb+你学scp-rhadoophadoop@nodec+你学.(2)删除/opt/hadoop/tmp下的所有内容。2.Hadoop执⾏MapReduce任务失败，并且⽇志伴有“Note:Systemtimesonmachinesmaybeoutofsync.Checksystemtimeandtimezones.”https://heisun.xyz/docs/hadoop-e/hadoop-e01/39/41.(2)重启Hadoop。3.YARN的ResourceManager进程找不到，⽽且⽇志报java.lang.NullPointerExceptionattocolrecords.impl.pributesMappingRequestPBImpl.initNodeAttributesMapping(NodesToAttributeattocolrecords.impl.pributesMappingRequestPBImpl.getNodesToAttributes(NodesToAttributesMappatorg.apache.hadoop.yarn.nodelabels.store.op.FSNodeStoreLooAttributesMap(FSNodeStoreLogOatorg.apache.hadoop.yarn.nodelabels.store.op.NodeAttributeover(NodeAttributeMirrorOatorg.apache.hadoop.yarn.nodelabels.store.op.NodeAttributeover(NodeAttributeMirrorOatorg.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStMirror(AbstractFSNodeStore.java:121)atorg.apache.hadoop.yarn.nodelabels.store.AbstractFSNodeStromStore(AbstractFSNodeStore.jaatorg.apache.hadoop.yarn.server.resourcemanager.nodelabelsodeAttributeStore.recover(FileSystemNodeAttributeStor答：在/tmp⽬录下找到以下yarn相关的⽬录进⾏删除4.HDFSWeb界⾯上传⽂件到HDFS失败，提⽰Couldn'tuploadthefile。https://heisun.xyz/docs/hadoop-e/hadoop-e01/40/41Windows⽆法识别主机名，则⽆法找到对应IP地址进⾏上传。只需要配置Hadoop的NameNode和DataNode的Hosts即可。.(1)使⽤⽂本编辑器打开Windows的C:\Windows\System32\drivers\etc\hosts⽂件。.(3)使⽤http://nodea999:9870/访问HDFSWeb界⾯。#hadoop#⼤数据2025/6/1816:55P1-部署Hadoop完全分布式与搭建开发环境|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e01/41/412025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e02/1/11P2-HDFS实战hadoopd任务2.1通过Shell命令访问和管理HDFS掌握WebConsole访问HDFS。掌握常⽤的Shell命令访问HDFS。操作系统:CentOS7.9FinalShellVirtualBoxHadoop完全分布式环境https://heisun.xyz/docs/hadoop-e/hadoop-e02/2/112025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载.hdfsdfs操作命令2025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e02/3/11HDFS安全模式操作HDFS开启关闭快照功能hdfsdfsadmin-allowSnapshot<snaphdfsdfsadmin-disallowSnapshot<snapHDFS操作快照功能hdfsdfs-deleteSnapshot<snapshotDir><snapshdfsdfs-renameSnapshot<snapshot.⽂件⽬录数量配额（Quota）setQuota指的是对HDFS中某个⽬录设置⽂件和⽬录数量之hdfshdfsdfsadmin-setQuota<quota><dirname>...<.空间配额（SpaceQuota）setSpaceQuota针对的是设置HDFS中某个⽬录可⽤存储空间⼤⼩，单位是byte。在使⽤该命令的时hdfshdfsdfsadmin-setSpaceQuota<quota><dirname>...<https://heisun.xyz/docs/hadoop-e/hadoop-e02/4/11hdfshdfsdfsadmin-print1.启动NodeA、NodeB、NodeC3个节点Hadoop。HDFS机架拓扑hdfshdfsdfsadmin-print3.查看put命令的使⽤⽅法和参数HDFS⽂件操作4.在⽤户⽬录下创建⼀个⽂本⽂件。注意替换为你的HDFS⽂件⽬录操作https://heisun.xyz/docs/hadoop-e/hadoop-e02/5/11HDFS⽂件内容操作HDFS安全模式操作HDFS快照操作https://heisun.xyz/docs/hadoop-e/hadoop-e02/6/1125.在该⽬录上开启快照（snapshot）功能，注意替换为你的学号。hdfsdfsadmin-allowSnapshot/snapshohdfsdfs-createSnapshot/snapshot+你学号后3diff<(hdfsdfs-cat/snapshot+你学号后3位/.snapshot/v1/haizi1-cat/snapshot+你学号后3位/haizi1HDFS容量操作hdfsdfsadmin-setQuota2/quot任务2.2通过编程接⼝访问和管理HDFS掌握如何使⽤IDEA创建Maven⼯程、运⾏Maven⼯程。掌握HDFS⽂件系统的Java编程接⼝的调⽤和编程.Windows7以上64位操作系统.JDK8.Maven3https://heisun.xyz/docs/hadoop-e/hadoop-e02/7/112025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载.IDEA社区版hadoop.dll按要求使⽤Java编程实现访问HDFS类org.apache.hadoop.fs.FileSystem的参考⽂档FileSystem.copyToLocalFile⽅法⽤于从HDFS复制⽂件到本地⽂件系统。详细参考⽂档FileSystem.copyFromLocalFile⽅法⽤于从本地⽂件系统复制⽂件到HDFS。详细参考publicvoidcopyFrom3.在包下⾯新建⼀个HdfsUtils的类。该类包含了⼀个在HDFS上创建⽂件并写⼊内容的⽅法2025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e02/8/11packagehadoop你学号**后3位importorg.apache.ha*@paramhdfsFilePathHD*@paramoverwritetrue表示覆盖原文件，publicstaticbooleancreateFile(Configurationconf,URIuri,StringPath,Stringcontent,booleanoverFileSystemfs=null;FSDataOutputStreamos=null;booleanrs=false;Pathdfs=newPath(hrs=true;}2025/6/1816:55P2-HDFS实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e02/9/11}}}}}5.在包下⾯新建⼀个HdfsUtilsTest的类，⽤于单元测试。代码如下，注意替换为你的学号后3位。此测试⽤例主要⽬的是在HDFS上创建⼀个⽂件，并把“hello”写⼊⽂件。packagehadoop替换为你学号后publicclassHdfsUtConfigurationconf=newConfigubooleanrs=false;StringfilePath="/替换为你学号后3位/test.txt";Stringcontent="hello";URIuri=newURI("hdfs://1:80rs=HdfsUtils.createFile(conf,uri,filePath,conten}System.out.println("CreatesuccSystem.out.println("Crehttps://heisun.xyz/docs/hadoop-e/hadoop-e02/10/11}}}}重要步骤，请勿忽略！7.启动虚拟机的Hadoop。注意一定要启动Hadoop，否则实验⽆法成功！8.运⾏HdfsUtilsTest。查看IDEA的控制台输出结果是否创建成功。hdfshdfsdfs-cat/替换为你学号后3位/test.1.报错：(null)entryincommandstring答：这是缺少⽂件hadoop.dll⽂件2.IDEA报错：.ConnectionException:CallFromXXXX/to1:8020failedonconnectionexception.答：这是连接NameNode进程出错。.2.确保能够ping到1.3.确保Hadoop启动成功，使⽤jps命令检查进程是否缺少，特别是namenode进程。https://heisun.xyz/docs/hadoop-e/hadoop-e02/11/11#hadoop#⼤数据#课程https://heisun.xyz/docs/hadoop-e/hadoop-e03/1/112025/6/1816:56P3-MapReduce实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载P3-MapReduce实战2021-04-115分钟阅读时⻓hadoopdv20250326任务3.1-编写MapReducewordcount程序掌握MapReduce程序编写要领Windows7以上64位操作系统JDK8IDEAHadoop3Maven3countryroad.txt.编写MapReducewordcount程序https://heisun.xyz/docs/hadoop-e/hadoop-e03/2/112025/6/1816:56P3-MapReduce实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载1.打开Part4的hadoopexp项⽬。packagepackagehadoop你的学号后4位.maprepublicclassWordCountMapperextendsMapper<LongWritable,Text,Text,IthrowsIOException,InteString[]words=datacontext.write(newText(w),new}}}packagehadoop你的学号后4位.maprepublicclassWordCountReducerextendsReducer<T2025/6/1816:56P3-MapReduce实战|Hadoop集群部署与开发V5|⿊隼个⼈技术记载https://heisun.xyz/docs/hadoop-e/hadoop-e03/3/11protectedvoidreduce(Textk3,Iterable<IntWritable}context.write(k3,newIntW}}packagehadoop你的学号后4位.mapreimportorg.apache.hadoop.mapreduceimportorg.apache.hadoop.mapreduce.limportorg.apache.hadooppublicclassWordCopublicstaticvoidmain(String[]args)throConfigurationconf=newConfiguString[]otherArgs=newGenericOptionsParser(conf,args).g}Jobjob=Job.getInstance(conf,"Worhttps://heisun.xyz/docs/hadoop-e/hadoop-e03/4/11job.setJarByClass(Worjob.setMapperClass(WordCjob.setReducerClass(WordCojob.setCombinerClass(WordCojob.setOutputValueClass(IFileInputFormat.addInputPath(job,newPath(otherArg}FileOutp