hadoop搭建与eclipse开发环境设置已验证通过

hadoopeclipseWindowseclipseeclipse的插 plugins下需要注意的是插件版本（jar包）hadoop一致，否则EOFException异常。重启eclipse，打开windows->open->other->map/reduce可以看map/reduce开发视图。后弹出的框（Generaltab）进行参数的添加：Locationname:map/reducemastermapred-site.xml里面mapred.job.trackerUsername:core-site.xml设置一致。mapred.job.trackermapred-site.xml里面mapred.job.tracker设置一致。dfs.replicationhdfs-site.xml里面的dfs.replication一致。hadoop.tmp.dircore-site.xml里hadoop.tmp.dir设置一致。hadoop.job.ugi：并不是设置用户名与。是用户与组名，所以这里填hadoop.job.ugi和dfs.replicationProjectExplorer中DFSLocations，应该可以年看到文件系统中的结构了。但是在/hadoop/mapred/system下却没有查看权限，如下图：的）进行hadoop系统操作，没有权限。此时再打开“Advancedparameters”设置面板，应该可以看到hadoop.job.ugi了，这了，将hadoop加在第一个，并用逗号分隔。如：OK。hadoophadoop安装包下面的所有jar包都导到eclipse文件内容读为字符串，将字符串写入文件。同时有个main函数，可以修改测试:packagepackageimportjava.io.IOException;importjava.io.InputStream;importimportorg.apache.hadoop.conf.Configuration;importorg.apache.hadoop.fs.FileSystem;import*Theutilitiestooperatefileonhadoop*@authorluolihui2011-07-publicclassDFSOperatorprivatestaticfinalStringROOT_PATH="hdfs:///";privatestaticfinalintBUFFER_SIZE=4096;*publicCreateafileonhdfs.Therootpathis thefilenameto@paramoverwriteifafilewiththisnamealreadyexists,theniftrue,thefilewill@returntrueifdeleteissuccessfulelsepublicstaticbooleancreateFile(Stringpath,booleanoverwrite)throws{//Stringuri=Configurationconf=newConfiguration();FileSystemfs=FileSystem.get(conf);Pathf=newPath(ROOT_PATH+path);fs.create(f,overwrite);returntrue;}Deleteafileonhdfs.Therootpathis/.@parampaththepathto@paramrecursive ifpathisadirectoryandsettotrue,thedirectoryisdeletedelsethrowsanexception.Incaseofafiletherecursivecanbesettoeithertrueorfalse.@returntrueifdeleteissuccessfulelsepublicstaticbooleandeleteFile(Stringpath,booleanrecursive)throws{//Stringuri=Configurationconf=newConfiguration();FileSystemfs=FileSystem.get(conf);Pathf=newPath(ROOT_PATH+path);fs.delete(f,recursive);returntrue;}Readafiletostringonhadoophdfs.Fromstreamtostring. @parampaththepathto@returntrueifreadissuccessfulelse{Configurationconf=newConfiguration();FileSystemfs=FileSystem.get(conf);InputStreamin=null;Stringstr=null;StringBuildersb=newStringBuilder(BUFFER_SIZE);if(fs.exists(f)){in=while((str=bf.readLine())!={}return}{return}}Writestringtoahadoophdfsfile.forexample:DFSOperator.writeStringToDFSFile("/lory/test1.txt","Youarea@parampaththefilewherethestringtowrite@paramstringthecontexttowriteina@returntrueifwriteissuccessfulelse{Configurationconf=newConfiguration();FileSystemfs=FileSystem.get(conf);FSDataOutputStreamos=null;Pathf=newPath(ROOT_PATH+path);os=fs.create(f,true);returntrue;}publicstaticvoidmain(String[]{try }}catch(IOExceptione)//TODOAuto-generatedcatchblock}}}RunAs→RunonHadoop→Chooseanexitsingserverfromthelist11/07/1611/07/1618:44:32WARNconf.Configuration:DEPRECATED:hadoop-site.xmlfoundintheclasspath.Usageofhadoop-site.xmlisdeprecated.Insteadusecore-site.xml,mapred-site.xmlandhdfs-site.xmltooverridepropertiesofcore-default.xml,mapred-default.xmlandhdfs-default.xmlYouareabadman.参数，然后同样“Runonhadoop每“Runonhadoopworkspace\.metadata\.plugins\org.apache.hadoop.eclipse下生成临时jarRunonhadoop，以后只需要点击那运行的绿色按钮了。eclipseDFSNameNode在安全模式中，其解决方案也一并给出。类似的运行hadoop程序时，有时候会报以下错误：Namenodeisinsafebin/hadoopdfsadmin-safemodedfsadmin-safemodevaluevalueenter-leave-NameNode离开安全模式get-返回安全模式是否开启的信息wait-等待，一直到安全模式结束。开发错Permissionorg.apache.hadoop.security.AccessControlException:Permissiondenied:user=Administrator,access=WRITE,atsun.reflect.NativeConstructorAccessorImpl.newInstance0(Nativeatatorg.apache.hadoop.fs.FileSystem.create(FileSystem.java:484)atorg.apache.hadoop.fs.FileSystem.create(FileSystem.java:465)atorg.apache.hadoop.fs.FileSystem.create(FileSystem.java:372)atcom.kingdee.hadoop.DFSOperato