分布异构数据源的集成方法研究.doc

分布异构数据源的集成方法研究摘 要：随着计算机网络的飞速发展，特别是最近十年来，各种网络中可供利用的信息总量以惊人的速率增长。然而，这些为不同应用服务的信息都存储在许多不同的数据源之中，其管理系统也各不相同。为更有效地利用这些信息，需要从多个分布、异构和自治的数据源中集成数据，同时还需要保持数据在不同系统上的完整性和一致性。另外，必须向用户隐藏这些差异，提供给用户一个统一和透明的数据访问接口。研究的重点即在于确立一种具有普遍意义的、可操作性强的分布异构数据源的集成方法。多数据库系统是在不同成员系统之间实现数据共享和互操作的理想途径。这些成员系统通常是传统的数据库系统，随着可共享数据源种类和规模的增长，目前传统的数据库技术正向能集成文件系统等非数据库成员系统的方向扩展。在数据库系统中，数据都是结构化的；而在文件系统中，存在大量如Web网页的半结构化数据。在充分吸收国内外数据集成领域研究成果的基础上，通过分析比较各类主流分布式对象技术、Agent技术和XML技术，提出了一种以多数据库系统为主要技术路线的分布异构数据集成方法“CAX”方法。该方法提出采用CORBA中间件作为集成系统的物理模型框架，以便解决多数据库系统中成员系统的分布性和环境异构性问题。采用基于Agent的逻辑模型框架，以解决多数据库系统中成员系统的自治性问题。采用基于XML的公共数据模型，以解决多数据库系统中成员系统的数据异构性问题。该方法通过融合CORBA和Agent技术，实现对各类数据源的封装，使集成系统的结构更适合分布式环境，增加业务流程的灵活性，同时也简化了集成系统的设计，为解决数据集成中的新问题提供了有效途径。公共数据模型一直是多数据库系统研究的重点，由于数据源在数据格式和表达方式上的差异，使得各自的数据难于集成。通过比较各类数据模型特点，在XML标准的基础上，给出了一种面向XML Schema的扩展多数据库的公共数据模型XIDM，该模型是基于图结构的模型，适合表现结构化和半结构化数据，同时有丰富的数据类型和约束定义，满足了同其他数据模型转换的需要。各类数据源的描述方法一般可以用代数工具进行表达，这些描述方法彼此之间往往存在很大差异，当集成这些数据源时，这些差异可能会造成数据语义的丢失。针对异构数据源都需要提供对存储数据集合的查询和应答这一共同特性，借助泛代数和代数逻辑等数学工具，提出了一种对分布异构数据源代数结构的统形式化方法。该方法不仅适用于关系数据库系统，而且也适用于图结构表达的半结构化数据源。基于上述方法，设计实现了一个名为“Panorama”的扩展多数据库原型系统，该系统具有模式集成、查询处理和事务处理功能，能够提供对Oracle、Sybase、和DM2等成员数据库系统的透明互操作。关键词：分布异构数据源，数据集成，多数据库，文件系统，多种类代数Research on Integrating Method for Distributed and Heterogeneous Data SourcesABSTRACT: With the rapid development of computer networks, the amount of information available on-line is proliferating at a tremendous rate during the last decade. However, all those information for many different applications is independently stored in a great variety of data sources and managed by different systems. In order to utilize this information more effectively, there is a need to integrate data from multiple distributed, heterogeneous and autonomous sources, and furthermore, preserve data integrity and consistency over different systems. In addition, it is necessary to make differences invisible and providing a user uniform and transparent access to the data. This research focuses on how to establish a universal and well usable methodology on information integration from distributed and heterogeneous data sources.A multidatabase system (MDBS) is a perfect approach to implement data sharing and interoperability among various component systems. MDBSs are usually composed of traditional database systems with structured data. While shared data sources is rapidly increasing in kind and expanding in scope, MDBSs have to extend their range so as to enable them to integrate file system (FS) that manage large amounts of semi-structured data such as web pages.Based on summarizing series of important MDBS-related research works and analyzing developments of several main distributed object technologies, agent technology and XML, this research presents an integration method for information integration from distributed and heterogeneous data sources, named “CAX ”, which is founded on multidatabase technology. The CAX method proposes to adopt CORBA specification as framework of physics model of integration system so as to solve problems about distribution and environment heterogeneity of local component systems, use agent technology to construct framework of logic model of integration system in order to solve problems about autonomy of component systems and apply an XML-oriented common data model to solve problems about data heterogeneity. By means of combining CORBA with agent technology, this method encapsulates more different data sources, forms more adaptive construction of integration system for distributed environment and brings more flexibility of operation flows. CAX method simplifies design of integration system, and in the mean while, it provides an effective approach to solve newly emerging problems on data integration. Common data model has been the keystone of MDBS. On account of differences in data structure and representation, it is difficult to integrate data from diverse sources. By comparing features of different data models, we provide a data model, called XIDM, which bases on XML schema and serves as a common data model for an extended MDBS. On one hand, XIDM is founded on graph structure so that it is enabled to represent structure data and semi-structure data as well. On the other hand, owing to its rich datatypes and constraints rules, XIDM is convenient to achieve transformation with other data models.Every data source is always use different algebra tool to describe its data. These differences are likely to result in loss of semantic information. Focusing on the common feature that every information system is able to receive query and send response, we provides a formal method to describe algebraic structure of distributed heterogonous data sources by universal algebra and algebraic logic. The description method suit not only relational database system but also semi-structure data source represented by a graph structure model.Finally, an extended MDBS prototype that designed and implemented on above methods and technologies, named Panorama, is introduced. Panorama can completely perform schema integration, query processing