基于OAIPMH协议数字图书馆中数据提供者的研究

1、随着网络、计算机和通信技术的快速发展，数字图书馆已经成为人们取得信息的重要来源，然而对于网络上众多的分布式数字资源1?2，人们希望通过检索获得所需的信息。但由于早期在建立数字图书馆时没有统一的标准，其资源的元数据格式并不相同，因而要制定出一套能够描述不同数字图书馆资源的统一元数据格式相当困难；同时，早期建立的数字图书馆数据提供者与服务提供者间多是点对点的协议服务，协议外的数据提供者与服务提供者间无法进行元数据交互3?4。针对上述问题，设计了数字图书馆中广泛使用的元数据格式MARC和DC的格式转换器，并重新设计了基于OAI?PMH协议互操作平台的命令动词，对数字图书馆间资源的共享具有重要意义。1

2、 OAI?PMH协议概述数据提供者、服务提供者、注册服务器构成了数字图书馆领域5?7中OAI?PMH协议的框架。数据提供者是框架中存储大量元数据的信息库，服务提供者可从多个数据提供者中提取元数据并提供检索、浏览等增值服务，服务提供者对数据提供者发出的是到UDP的请求，也就是基于OAI?PMH的命令动词，数据提供者对服务提供者的请求是通过HTTP协议以XML的形式响应的，注册服务器中可以进行数据提供者和服务提供者的注册。其具体框架如图1所示。2 MARC到DC格式转换器的设计与实现目前数字图书管中使用的元数据格式主要为MARC，而基于OAI?PMH协议的数据提供者只能支持DC格式的元数据8。所以

3、要实现基于OAI?PMH协议的元数据互操作平台，就需要把MARC格式的元数据转换为DC格式的元数据。2.1 MARC与DC格式的域对照关系DC格式包含的元素描述共有15个，按照其描述内容的范围和类别可分为三组，具体如表1所示。本系统提供了一个项批输入接口，所以设计的MarcDc模块应在项批输入者之前进行处理。如果输入的是MARC格式，则先经过MarcDc格式转换器进行数据转换，然后通过项批处理模块进行处理；如果输入的就是DC格式，则直接应用项批处理模块进行处理。3 OAI?PMH互操作平台的实现3.1 基于OAI?PMH协议的命令动词分析基于OAI?PMH协议的数字图书馆中，为实现多个数据提供

4、者与服务提供者间之间的元数据交互，重新设计了基于OAI?PMH协议互操作平台的命令动词，命令动词的主要功能如下：（1） GetRecord：此动词可从仓储中搜索元数据记录。如果从仓储中和指定的项中无法得到由metadataPrefix指定的元数据格式，其可以返回状态属性的头部信息值为“删除”，此功能的实现依赖于仓储所跟踪的删除级别；（2） Identify：此动词用于检索仓储的有关信息。利用此动词，仓储也可以返回余下的描述性信息，在返回的信息中，一部分对于基于OAI?PMH协议的互操作平台是需要的；（3） ListIdentifiers：此动词不返回记录本身，而仅返回头部的记录，通过基于集合成

5、员和时间戳的头部可获取选择的参数。基于仓储对删除的支持特性，如过请求中删除了与指定参数匹配的记录，则返回的记录中将含有值为“删除”的状态属性；（4） ListMetadataFormats：此动词可从仓储中检索获得所需的元数据格式，同时可以通过参数限制该请求对指定项元数据格式的获取； （5） ListRecords：此动词用于从仓储中获取记录。基于仓储对删除的支持特性，如果删除了请求中与参数匹配的记录，则返回的记录中将含有值为“删除”的状态属性，但返回有“删除”状态的记录是不能被表达为元数据格式的；（6） ListSets：此动词用于返回仓储的集合结构，对于选择性获取有益。3.2 数据提供者与

6、服务提供者间命令动词使用方式分析数据提供者与数据服务者之间的请求和应答是通过上述六个核心动词来实现的。使用标准的Web服务器是一个典型请求的实现方式，通过配置该Web服务器，使其可以向能够处理OAI?PMH请求的软件分发OAI?PMH请求，具体交互过程如图5所示。数据提供者与数据服务者之间的交互步骤具体如下：（1） 服务提供者首先找到其所需元数据的数据提供者，然后获取其惟一标识；（2） 服务提供者向数据提供者发出ListSets请求和ListMetaFormats请求。数据提供者根据服务提供者的请求返回其所能提供的满足一定条件（如某种格式，某种主题，某个时间段）的元数据；服务提供者取得元数据后

7、向用户提供服务。3.3 OAI?PMH命令动词的请求和应答格式分析（1） 请求格式3.4 OAI六个动词的设计（1） 概要设计当数据提供者接收到OAI请求后，需对该请求进行解析。首先判断该请求类型是否合法，若不合法，则向服务提供者发送一个错误的信息；若合法，则判断该请求属于六个有效请求的哪个类型。由于参数metadataPrifix对于请求类型 ListIdentifiers是强制的，所以如果数据提供者接收到的请求类型是ListIdentifiers，那么解析器可以直接检查这个请求的第二个参数。但如果请求中没有相关参数，那么要确认这个请求有效，则需确认请求中必须包含resumptionToke

8、n （恢复标志参数），并且数据提供者知道这个参数。假设数据提供者在unqualified DC模式下只能发送元数据集合，那么metadataPrifix参数惟一有效的值只能是oai_dc。在正常情况下，请求中可选取的参数是必须被解析的，但可以简单化，以一种非正式的形式进行描述。然后，根据接收到的请求参数，数据提供者运用SQL语句在仓储中进行查询，如果产生了记录大于传递标识符一次所能传递的最大值，那么数据提供者则会新生成一个resumptionToken标志，并将查询参数与指针信息存储在一起，具体实现流程如图6所示。（2） 详细设计 利用Protocol数据库包进行六个动词的请求、响应及信息显示

9、，其中Date类使用java.util.Date，Calendar或String方法实现年、月、日等各种格式间的相互转换和输出；Set类可通过一个已命名的集合创建一个新的集合，并用XML形式描述。 处理客户端源代码Client，可以利用一个URL或一个集合的性质创建HarvesterItinerary类用于描述harvester类的状态，HarvesterItinerary类中可以对Harvester类的状态进行保存。OAIConnection类可以描述一个到OAI的连接，用于接收单一的请求，该连接是基于用doReques或基于仓储的URL所返回的Response。 互操作平台中的Server

10、包可以提供服务器端的OAI源代码，并把源代码转换为文档服务。OAI服务的目标编程接口通过Target类实现，互操作平台通过TargetAdapter类定义了一个不做任何事的执行，GenericTarget类可以实现由一个非常简单的机制以创建小型的OAI收集器，并在存储器中寻找与服务，利用集合规程和给定的元数据前缀得到所有的记录； JDBCServer类是OAI servlet提供的一般服务器，OAIServerIfc类为OAI提供了一个框架，元数据前缀用在DC元数据核心记录，处理GetRecord 请求、Identify 请求、ListRecords请求、 ListSets请求、ListMet

11、adataFormats请求和ListIdentifiers请求。ResumableResultSet类描述了一个结果，该结果通过与重用标记相结合，可以被客户作为一系列局部结果而重用。3.5 数据提供者的功能实现（1） 项的订购及提交用户可以使用e?mail订购自己所需的项，这样可以共享数字图书馆的资源。用户首先提交项的要素描述元数据，然后上传源文件，在通过系统的验证后可把文件提交到互操作平台上供大家浏览及下载。（2） 工作流的实现三个小组负责人负责实现社团的工作流，每个小组负责人完成不同的工作流步骤。工作流的次序如下：当社团收到一个递呈时，如果社团中有小组负责人，那么他将会选择接受或拒绝，如

12、果社团中没有小组负责人，这一步工作流将会被直接省略。第二步和第三步的递呈也是按照此步骤处理。当调用工作流的第一步时，完成工作流步骤的任务把相关的递呈放到“任务箱”中，如果组中的一个成员接受任务箱中的任务，则任务被从任务箱中移出。如果一个递呈被拒绝，则系统会通过e?mail把原因发送给递呈提交者，提交者可以修改后重新提交；如果一个递呈被“接受”，将转到工作流的下一步。（3） 搜索和浏览功能实现终端用户可以使用多种方法发现内容，具体如下：使用关键字进行搜索；使用外部的ID号进行搜索；使用标题、数据进行浏览。在数字图书馆系统中，发现内容最基本的方法是搜索。基于OAI?PMH协议的互操作平台的搜索和索

13、引模块是一个简单的API接口，它能够在全部的社区、社团中完成新内容的索引、再生索引，系统的具体搜索界面如图7所示。 4 结 论本文设计了MARC格式到DC格式的格式转换器及基于OAI?PMH协议互操作平台的命令动词，解决了数字图书馆中数据提供者与服务提供者间的元数据交互问题。给出了不同格式间的对照关系及格式转换器的实现方法，同时完成了命令动词的实现代码，对推动OAI?PMH协议在数字图书馆领域的应用具有重要意义。With the rapid development of network, computer and communication technology, digital librar

14、y has become the important sources of information for people to obtain information, but for many distributed digital resources on the network 1? 2, it is hoped that through to retrieve the required information. But since early in the establishment of digital library, without unified standard, its re

15、source metadata format is not the same, and thus to develop a set of can describe the unity of the different digital library resources metadata format is very difficult; Early at the same time, set up the digital library of data between the provider and the service provider is a point-to-point proto

16、col service, outside of the agreement between data providers and service providers to metadata interaction 3? 4.According to the above problem, design is widely used in the digital library metadata format format of MARC and DC converter, and redesigned based on OAI? PMH protocol interoperability pla

17、tform command verbs, between the digital library resources sharing is of great significance.1 OAI? PMH protocol overviewData providers and service providers, registered server constitutes the OAI in digital library field ? 5 7? PMH the framework of the agreement. Data providers are stored in the fra

18、mework of a large number of metadata repository, the service provider can extract meta data from multiple data provider and provide searching, browsing and other value-added services, the service provider is sent to the data provider to UDP request, which is based on OAI? PMH command verbs, a data p

19、rovider to the service provider's request is through HTTP protocol in the form of XML response, registered in the server can be the registration data providers and service providers. The specific framework is shown in figure 1.2, MARC format to DC converter design and implementationCurrently use

20、d in the digital library metadata format to MARC, and based on OAI? PMH protocol data providers can only support DC format metadata 8. So in order to realize based on OAI? PMH protocol metadata interoperability platform, you need to put the metadata of MARC format is converted to DC metadata format.

21、2.1 comparison relationship of MARC and DC format domainDC format contains element describes a total of 15, according to the scope and categories describe content can be divided into three groups, as shown in table 1.This system provides a number of input interface, so the design of MarcDc module sh

22、ould be performed before the item number of input processing. If the input is MARC format, after first MarcDc format converter to convert the data, then by a batch processing module; If the input is a DC format, the direct application of a batch processing module for processing.3 OAI? The realizatio

23、n of PMH interoperable platform3.1 based on OAI? PMH protocol analysis of the command verbsBased on the OAI? PMH agreement in the digital library, in order to realize multiple data interaction between metadata between providers and service providers, redesigned based on OAI? PMH protocol interoperab

24、ility platform command verbs, the main functions of the command verbs are as follows:GetRecord (1) : this verb can search metadata record from the warehouse. If the specified item from the warehouse and can't get appointed by metadataPrefix metadata format, the head of the state can return prope

25、rty information value for the "delete", the realization of the function of this depends on the storage by tracking the deletion of level;(2) Identify: this verb is used to retrieve information about the warehouse. Using the verb, storage can also return to the rest of the descriptive infor

26、mation, in the information returned part based on OAI? PMH protocol interoperability platform is needed;(3) ListIdentifiers: this verb does not return the record itself, and records of return only the head, by head based on the set members and timestamp available choice of parameters. Based on wareh

27、ouse to remove the support features, such as a request to delete a record with the specified parameters matching, it returns the record will contain values for the "delete" state property;(4) ListMetadataFormats: this verb can be retrieved from the storage to obtain the required metadata f

28、ormat, at the same time can through parameter limits the request to specify item metadata format for; (5) ListRecords: extracted from the verbs used in the warehouse record. Support feature based on warehouse to delete, if you remove the request and parameter matching of records, it returns the reco

29、rd will contain values for the state of the "delete" attribute, but returned to a state of "delete" record cannot be expressed as metadata format;(6) ListSets: the collection structure of verbs used to return to the warehouse, to get good selectivity.3.2 between data providers an

30、d service providers command verb usage analysisBetween data providers and data service request and response is done by the above six core verb. Use the standard Web server implementation approach, is a typical request by configuring the Web server, allows it to be able to handle OAI? PMH request sof

31、tware distribution of OAI? PMH requests, the specific interaction process as shown in figure 5.Interaction between data providers and data server steps specific as follows:(1) the service provider first find the metadata for the data provider, and then get its unique identity;(2) the service provide

32、r to the data provider ListSets requests and ListMetaFormats request. Data provider according to the service provider's request to return to its can provide satisfy certain conditions (such as some format, a theme, a certain period of time) metadata; After the service provider to obtain metadata

33、 to provide service to the users.3.3 OAI? PMH command request and response format analysis of verbs(1) request format3.4 the design of the OAI six verbs(1) the profile designWhen the data provider receives the OAI request, need to parse the request. First determines whether the request type is legal

34、, if not legal, then send a wrong message to the service provider; If legally, it concludes that the request which belong to six effective request type. As the parameter metadataPrifix for request types ListIdentifiers is mandatory, so if the data provider receives the request type is ListIdentifier

35、s, so the parser can directly check the request of the second parameter. But if there is no related parameters in the request, so want to confirm that the request is valid, will be expected to confirm the request must contain resumptionToken (recovery mark parameters), and the data provider know thi

36、s parameter.The assumption that data provider in unqualified DC mode can only send the metadata collection, so the only effective value can only be oai_dc metadataPrifix parameters. Under normal circumstances, can select the parameters in the request must be resolved, but can be simple, in the form

37、of an informal description. Then, according to the received request parameters, the data provider query using SQL statements in the warehouse, if the record is more than a passing identifier can pass the maximum, then the data provider will generate a new resumptionToken logo, and the query paramete

38、r with pointer information is stored in the concrete implementation process is shown in figure 6.(2) the detailed design(1) using Protocol data packet six verbs, according to the request, response, and the information of the Date class using Java. Util. The Date, year, month, day Calendar or String

39、method between the various formats, such as mutual conversion and output; The Set class by a named Set to create a new Set of formal description using XML.(2) to deal with the Client source code, the Client, can use a URL or a collection of properties to create HarvesterItinerary class used to descr

40、ibe the state of the harvester class, HarvesterItinerary class the harvester state of a class can be saved. OAIConnection class can describe a connection to the OAI, to accept a single request, the connection is based on using doReques or URL returned by the Response based on warehouse.(3) interoper

41、ability platform of the OAI source code package can provide the Server Server, and convert the source code to document service. OAI service Target programming interface on a Target class implements the interoperability platform through TargetAdapter class defines a don't do anything, GenericTarg

42、et class can implement by a very simple mechanism to create a small OAI collector, and are looking for and service, in the memory collection procedures and the given metadata prefixes are used to get the all records; JDBCServer class is to provide general server of OAI servlet OAIServerIfc such as O

43、AI provides a framework, core metadata prefix used in DC metadata records, processing GetRecord request, the Identify, ListRecords request, ListSets, ListMetadataFormats request with ListIdentifiers request. ResumableResultSet class describes a result, the result by combined with reuse marker, can b

44、e reused. As a result of the series of local customer3.5 the realization of the function of the data providerItem (1) of the order and submitThe user can use the e? Mail you need to order the items that we can share the resources of the digital library. User submitted item first describes the elemen

45、ts of metadata, then upload the source file, through the system after the validation of documents submitted to interoperable platform for everyone to browse and download.(2) the realization of the workflowThree team leader is responsible for implementing corporate workflow, each team leader to compl

46、ete different workflow steps. The order of the workflow is as follows: when the club received a presented, if a team leader in the community, so he will choose to accept or reject, if there is no team leader in the community, will be directly omit this step workflow. The second and third steps presented is also in accordance with this step.When calli