RoadRunner代码学习笔记.doc

ROADRUNNER 代代码码学学习习笔笔记记之之一一 PACKAGE ROADRUNNER PARSER http blog seforge org uid 244 action viewspace itemid 93 1 包结构 roadrunner parser AmbiguityMismatchFactory java Binder java BindingException java Extractor java FreeTextHunter java Lexer java Parser java ParserListener java ParserListenerAdapter java Token java TokenList java TokenListRegion java 2 Parser 类的学习 public class Parser extends Walker implements Visitor private ParserListener listener Parser 的监听器 public interface Visitor public boolean visit NodeAdapter node public boolean visit ASTToken node public boolean visit ASTAnd node public boolean visit ASTPlus node public boolean visit ASTHook node public boolean visit ASTVariant node public boolean visit ASTSubtree node 延伸学习 访问者模式 访问者模式在 HtmlParser 中也经常用到 一个接口 Visitor 访问者 在这个接口中 有 一些访问操作 这些访问操作是专门访问对象集合 Collection 中可能的所有类 访问者模式好的学习文章 http blog seforge org uid 244 action viewspace itemid 93 监听器模式 public class Preferences 对 XML 文件的解析 public void decode Reader in String base throws IOException Create a DOM builder and parse the fragment try decode DOMLoader parseXML in base catch SAXException saxe log severe Cannot decode preferences from xml source s axe getMessage System exit 1 public void decode Document doc decode doc PREFERENCES 文件 Reader HtmlDocument 文档 将配置文件的信息读进 HashMap public void decode Document doc String base Element prefNode Element doc getElementsByTagName base it em 0 if prefNode null log severe Cannot find a root element node named base System exit 1 setName prefNode NodeList children prefNode getChildNodes int len children getLength for int i 0 i Reader HtmlDocument 文档 将配置文件的信息读进 HashMap public void decode Document doc String base Element prefNode Element doc getElementsByTagName base item 0 if prefNode null log severe Cannot find a root element node named base System exit 1 setName prefNode NodeList children prefNode getChildNodes int len children getLength for int i 0 i len i Node node children item i if node getNodeType Document ELEMENT NODE continue skip spurious no des String key node getNodeName trim String value node getAttributes getNamedItem VALUE getNodeValue prefs put key value ROADRUNNER 代代码码学学习习笔笔记记之之三三 如如何何将将 HTML XML 文文档档转转换换成成 DOM 和和节节 点点集集合合 DomLoader 将 HTML 和 XML 转换为 DOM 对象 1public class DOMLoader 2 3 static final private Logger log Logger getLogger DOMLoader class getName 4 5 static public Document parseXML InputStream in throws SAXEx ception IOException 6 return parseXML new InputSource in 7 8 9 static public Document parseXML Reader in throws SAXExcepti on IOException 10 return parseXML new InputSource in 11 12 13 static public Document parseXML InputSource in throws SAXEx ception IOException 14 return getDocumentBuilder parse in 15 16 17 static public Document parseHTML Reader in throws SAXExcept ion IOException 18 DOMParser parser new DOMParser 19 parser setFeature http xml org sax features namespaces false 20 parser setFeature http cyberneko org html features bal ance tags true 21 XMLDocumentFilter filters new Purifier 净化的过程 将 HTML 转换为标准的 XML 22 parser setProperty http cyberneko org html properties filters filters 23 parser parse new InputSource in 24 return parser getDocument 25 26 27 static public DocumentBuilder getDocumentBuilder 28 try 29 DocumentBuilderFactory factory DocumentBuilderFacto ry newInstance 30 factory setIgnoringComments false 31 factory setExpandEntityReferences true 32 factory setValidating false 33 factory setIgnoringElementContentWhitespace true 34 DocumentBuilder parser factory newDocumentBuilder 35 指定使用 EntityResolver 解析要解析的 XML 文档中存在的实体 36 parser setEntityResolver new EntityResolver 37 38 public InputSource resolveEntity String publicId String systemId 39 if systemId endsWith dtd 40 this deactivates all DTDs by giving em pty XML docs 41 return new InputSource new ByteArrayInput Stream getBytes 42 else return null 43 44 45 return parser 46 47 catch ParserConfigurationException pce 48 log severe DOM Parser Misconfiguration pce 49 System exit 1 50 return null 51 52 53 54 Lexer 类 HTML 的词法解析器 根据配置参数进一步解析 DOM 文档 1public class Lexer implements Constants 2 3 得到词法分析器分解得到的节点列表 4 5 public List getTokens throws SAXException IOException 6 7 this tokens new ArrayList 8 Document doc DOMLoader parseHTML this in 得到文档对象 模型 9 if freetextEnabled 10 11 doc new FreeTextHunter doc this prefs getCollapse dDocument 12 13 getTokens doc 0 14 return this tokens 15 16 17 DOM Document Token List 18 19 20 private void getTokens Node node int depth 21 22 int type node getNodeType 23 switch type 24 25 case Node DOCUMENT NODE 26 Document document Document node 27 getTokens document getDocumentElement depth 28 break 29 case Node ELEMENT NODE 30 if isSubtreeToSkip node break 31 boolean toSkip isTagToSkip node 32 Token token null 33 open tag 34 if toSkip 35 36 token Token factory createOpenTagToken nod e depth this prefs 37 System out println 开始标签 38 System out println token toString 39 System out println 40 this tokens add token 41 42 visit subtrees 43 NodeList children node getChildNodes 44 if children null 45 46 int len children getLength 47 for int i 0 i len i 48 49 getTokens children item i toSkip dept h depth 1 50 51 52 close tag 53 if toSkip 54 55 Token token1 factory createCloseTagToken t oken 56 System out println 结束标签 57 System out println token1 toString 58 System out println 59 this tokens add factory createCloseTagToken token 60 61 break 62 63 case Node TEXT NODE 64 skip strings of all white spaces 65 if Util isAllWhiteSpaceChars getText node 66 Token text factory createTextToken node dep th 67 this