播客2015就业班lucene-day02培训内容

1、一）索引库优化11索引库索引库是 Lucene 的重要的结构，它包括二部份：原始表，词汇表原始表：存放的是原始信息，Lucene 为存入的内容分配一个唯一的词汇表：存放的是经过分词器拆分出来的词汇和该词汇在原始12 为什么要将索引库进行优化表中的在默认情况下，向索引库中增加一个制压缩文件，如果向索引库中存会不断增加，影响索引库的大小。13 索引库优化方案对象时，索引库自动会添加一个扩展名叫*.cfs 的二进对象过多，那么*.cfs 也会不断增加，同时索引库的容量也131 合并cfs 文件，合并后的 cfs 文件是二进制压缩字符，能解决是的文件大小和数量132 设定合并因子，自动合并 cfs 文

2、件，默认 10 个 cfs 文件合并成一个 cfs 文件133 使用RAMDirectory，类似于内存索引库，能解决是的索引库文件的速度问题，它能以空换时，提高速度快，但不能持久保存，因此启动时加载硬盘中的索引库到内存中的索引库，退出时将内存中的索引库保存到硬盘中的索引库，且内容不能重复。Article article = new Article(1,培训,是一家Java培训机构);=LuceneUtil.javabean2(article);Directory fsDirectory = FSDirectory.open(new File(E:/indexDBDBDBDBDBDBDBDB)

3、;Directory ramDirectory = newRAMDirectory(fsDirectory);IndexWriter fsIndexWriter = new IndexWriter(fsDirectory,LuceneUtil.getyzer(),tr ue,LuceneUtil.getMaxFieldLength();IndexWriter ramIndexWriter = new IndexWriter(ramDirectory,LuceneUtil.getyzer(),L uceneUtil.getMaxFieldLength();ramIndexWriter.add()

4、; ramIndexWriter.close();indexWriter.add();indexWriter.setMergeFactor(3); indexWriter.close();indexWriter.add(); indexWriter.optimize();indexWriter.close();二）分词器21分词器采用一种算法，将中英文本中的字符拆分开来，形成词汇，以待用户输入22 为什么要分词器字后搜索因为用户输入的搜索的内容是一段文本中的一个字，和原始表中的内容有差别，但作为搜索引擎来讲，又得将相关的内容搜索出来，此时就得采用分词器来最大限度匹配原始表中的内容23 分词器工

5、作流程步一：按分词器拆分出词汇步二：去除停用词和禁用词步三：如果有英文，把英文字母转为小写，即搜索不分大小写24 分词器例子图解：“播客说的首都是呀 I AM zhaojun”25 演示常用分词器测试，只观查结果26 使用第IKyzer 分词器中文首选需求：过滤掉上面例子中的“说”，“的”，“呀”，且将“播客”看成一个整体字步一：导入 IKyzer 分词器jar 包，IKyzer3.2.0Stable.jar步二：将 IKyzer.cfg.xml 和 stopword.dic 和.dic 文件到MyEclipse 的 src 目录下，再进行配置，在配置时，首行需要一个空行三）搜索结果priva

6、te sic void testyzer(yzeryzer, String text) throws Exception System.out.prln(当前使用的分词器： +yzer.getClass(); TokenStream tokenStream =yzer.tokenStream(content,new StringReader(text); tokenStream.addAttribute(TermAttribute.class); while (tokenStream.incrementToken() TermAttribute termAttribute = tokenStr

7、eam.getribute(TermAttribute.class);System.out.prln(termAttribute.term();fsIndexWriter.addIndexesNoOptimize(ramDirectory);fsIndexWriter.close();31搜索结果在搜索结果中，将与字相同的字符用红色显示四）搜索结果摘要41搜索结果搞要String keywords = 培训;List articleList = new ArrayList(); QueryParser queryParser = newQueryParser(LuceneUtil.getVer

8、(),content,LuceneUtil.g etyzer();Query query = queryParsarse(keywords); IndexSearcher indexSearcher = newIndexSearcher(LuceneUtil.getDirectory(); TopDocs topDocs =indexSearcher.search(query,1000000);Formatter formatter = new SimpleHTMLFormatter(,);Scorer scorer = new QueryScorer(query); Highlighter

9、highlighter = newHighlighter(formatter,scorer);for(i=0;itopDocs.scoreDocs.length;i+) ScoreDoc scoreDoc = topDocs.scoreDocsi;no = scoreDoc.doc;= indexSearcher.doc(no);String highlighterContent = highlighter.getBestFragment(LuceneUtil.getyzer(),cont ent,.get(content);.getField(content).setValue(highli

10、ghterContent);Article article = (Article)LuceneUtil.2javabean(,Article.class); articleList.add(article);for(Article article : articleList) System.out.prln(article);如果搜索结果内容太多，只想显示前几个字符，必须与一起使用五）搜索结果排序String keywords = 培训;List articleList = new ArrayList(); QueryParser queryParser = newQueryParser(Lu

11、ceneUtil.getVer(),content,LuceneUtil.g etyzer();Query query = queryParsarse(keywords); IndexSearcher indexSearcher = newIndexSearcher(LuceneUtil.getDirectory(); TopDocs topDocs =indexSearcher.search(query,1000000);Formatter formatter = new SimpleHTMLFormatter(,);Scorer scorer = new QueryScorer(query

12、); Highlighter highlighter = newHighlighter(formatter,scorer);Fragmenter fragmenter = new SimpleFragmenter(4); highlighter.setTextFragmenter(fragmenter);for(i=0;itopDocs.scoreDocs.length;i+) ScoreDoc scoreDoc = topDocs.scoreDocsi;no = scoreDoc.doc;= indexSearcher.doc(no);String highlighterContent =

13、highlighter.getBestFragment(LuceneUtil.getyzer(),cont ent,.get(content);.getField(content).setValue(highlighterContent);Article article = (Article)LuceneUtil.2javabean(,Article.class); articleList.add(article);for(Article article : articleList) System.out.prln(article);51搜索结果排序搜索结果是按某个或某些字段高低排序来显示的结

14、果52 影响head/meta/网页的的先后的有多种整洁网页执行速度采用 div+css。53Lucene 中的显示结果次序与相关度得分有关ScoreDoc.score;默认情况下，Lucene 是按相关度得分排序的，得分高排，得分低排在后如果相关度得分相同，按54Lucene 中的设置相关度得分索引库的先后次序排序55Lucene 中按单个字段排序56Lucene 中按多个字段排序在多字段排序中，只有第一个字段排序结果相同时，第二个字段排序才有作用提倡用数值型排序六）条件搜索61用条件搜索字与指定的单列或多例进行匹配的搜索62 单字段条件搜索63 多字段条件搜索，项目中提倡多字段搜索Quer

15、yParser queryParser = new MultiFieldQueryParser(LuceneUtil.getVer(),new Stringcontent,titlQueryParser queryParser = new QueryParser(LuceneUtil.getVer(),content,LuceneSort sort = new Sort(new SortField(count,SortField.,true),new SortField(id,SortField.,true);TopDocs topDocs = indexSearcher.search(que

16、ry,null,1000000,sort);Sort sort = new Sort(new SortField(id,SortField.,true);TopDocs topDocs =indexSearcher.search(query,null,1000000,sort);IndexWriter indexWriter = new IndexWriter(LuceneUtil.getDirectory(),LuceneUtil.getyz er(),LuceneUtil.getMaxFieldLength();.setBoost(20F);indexWriter.add(); index

17、Writer.close();七）用第导入第工具类，将 JavaBean、List、Map转成 JSON 文本jar 包：commons-beanutils-1.7.0.jarcommons-collections-3.1.jarcommons-lang-2.5.jarcommons-logging-1.1.1.jarezmorph-1.0.3.jarjson-lib-2.1-jdk15.jarJavaBean-JSONJSONArray jsonArray = JSONArray.fromObject(city);String jsonJAVA = jsonArray.toString();

18、List-JSONJSONArray jsonArray = JSONArray.fromObject(cityList);String jsonJAVA = jsonArray.toString();List-JSONJSONArray jsonArray = JSONArray.fromObject(stringList);String jsonJAVA = jsonArray.toString();Map-JSON【重点】八）用 JSON 文本动态创建 DataGrid$(#dg).datagrid(url : data/datagrid_data.json,List userList

19、= new ArrayList(); userList.add(new User(100,哈哈,1000); userList.add(new User(200,呵呵,2000); userList.add(new User(300,嘻嘻,3000);Map map = newLinkedHashMap(); map.put(total,userList.size(); map.put(rows,userList);JSONArray jsonArray = JSONArray.fromObject(map); String jsonJAVA = jsonArray.toString(); S

20、ystem.out.prln(jsonJAVA);jsonJAVA = jsonJAVA.substring(1,jsonJAVA.length()-1);System.out.prln(jsonJAVA);九）用 Servlet 返回 JSON 文本动态创建 DataGrid$(#dg).datagrid(url : /lucene-day02/JsonServlet, columns:field:code,title:,width:100,field:name,title:,width:100,field:price,title:薪水,width:100);Servlet：public v

21、oid dot(HttpServletRequest request, HttpServletResponse response) request.setCharacterEncoding(UTF-8);eger currPageNO = null; try /DateGrid 会向服务端传入 page 参数，表示第几页currPageNO =egarse(request.getParameter(page); catch (Exception e) currPageNO = 1;/DateGrid 会向服务端传入 rows 参数，表示几条/eger rows =egarse(request.

22、getParameter(rows);/System.out.prln(currPageNO+:+rows);UserService userService = new UserService(); PageBean pageBean = userService.fy(currPageNO);Map map = new LinkedHashMap(); map.put(total,pageBean.getAllRecordNO(); map.put(rows,pageBean.getUserList();JSONArray jsonArray = JSONArray.fromObject(ma

23、p); String jsonJAVA = jsonArray.toString();jsonJAVA = jsonJAVA.substring(1,jsonJAVA.length()-1);columns:field:code,title:,width:100,field:name,title:,width:100,field:price,title:薪水,width:100);十）使用 Jsp +Js + Jquery + EasyUI + Servlet + Lucene，完成分页步一：创建 Article.java 类public class Articlepubliceger get

24、AllObjectNum(String keywords)throws ExceptionQueryParser queryParser = new QueryParser(LuceneUtil.getVer(),content,LuceneU til.getyzer();Query query = queryParsarse(keywords); IndexSearcher indexSearcher = newIndexSearcher(LuceneUtil.getDirectory();TopDocs topDocs = indexSearcher.search(query,3);ret

25、urn topDocs.totalHits;public List findAllObjectWithFY(String keywords,eger start,eger size) throws ExceptionList articleList = newArrayList();QueryParser queryParser = new QueryParser(LuceneUtil.getVer(),content,LuceneU til.getyzer();Query query = queryParsarse(keywords); IndexSearcher indexSearcher

26、 = newIndexSearcher(LuceneUtil.getDirectory(); TopDocs topDocs =indexSearcher.search(query,100000000); middle =Math.min(start+size,topDocs.totalHits);for(i=start;imiddle;i+)ScoreDoc scoreDoc = topDocs.scoreDocsi; no = scoreDoc.doc;= indexSearcher.doc(no);Article article = (Article)System.out.prln(js

27、onJAVA); response.setContentType(text/html;charset=UTF-8); response.getWriter().write(jsonJAVA); response.getWriter().flush(); response.getWriter().close();步二：创建 PageBean.java 类public class PageBean privateeger allObjectNum; privateeger allPageNum; privateeger currPageNum; privateegageNum = 2;privai

28、st articleList = newArrayList();public PageBean()publiceger getAllObjectNum() return allObjectNum;public void setAllObjectNum(eger allObjectNum) this.allObjectNum = allObjectNum; if(this.allObjectNum % this.pageNum = 0)this.allPageNum = this.allObjectNum /this.pageNum;elsethis.allPageNum = this.allO

29、bjectNum /this.pageNum + 1;publiceger getAllPageNum() return allPageNum;public void setAllPageNum(eger allPageNum) this.allPageNum = allPageNum;publiceger getCurrPageNum() return currPageNum;public void setCurrPageNum(eger currPageNum) this.currPageNum = currPageNum;publiceger getPageNum() return pa

30、geNum;LuceneUtil.2javabean(,Article.class); articleList.add(article);return articleList;步三：创建 ArticleService.java 类步四：创建 ArticleServlet.java 类public claserServlet extends HttpServlet public void dot(HttpServletRequest request, HttpServletResponse response)throws ServletException, IOException try /获取

31、当前页号，默认1 String strCurrPageNO =request.getParameter(page);if(strCurrPageNO = null) strCurrPageNO = 1;public class ArticleService private Articlearticle= new Article();public PageBean fy(String keywords,eger currPageNum) throws ExceptionPageBean pageBean = negeBean(); pageBean.setCurrPageNum(currPage

32、Num);eger allObjectNum =article.getAllObjectNum(keywords); pageBean.setAllObjectNum(allObjectNum);eger size = pageBean.getPageNum();eger start = (pageBean.getCurrPageNum()-1) *size;List articleList =article.findAllObjectWithFY(keywords,start,size); pageBean.setArticleList(articleList);return pageBean;public void setPageNum(egageNum) this.pageNum = pageNum;public List getArticleList() return articleList;public void setArticleList(List articleList) this.articleList = articleList;步五：导入EasyUI 相关的js 包的目录eger currPageNO = egarse(strCurrPageNO);/获取字String keywords = request.getParameter