DataStage基础培训教程ppt课件

1、DataStageDataStage基础培训基础培训 JerryJerry 2006.032006.032议程议程Hello WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash File、TransformerTransformer、AggregatorAggregatorDirector & MonitorDirector & MonitorAdministrator &

2、ManagerAdministrator & ManagerRoutine & ControlRoutine & Control3演示：演示：Hello WorldExtractTransformLoan4Hello WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameters & TablesDefine Parameters & TablesHash FileHash File、TransformerTransformer、AggregatorAggregatorDi

3、rector & MonitorDirector & MonitorAdministrator & ManagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程5DataStage ArchitectureTarget(Database or File)ODBC/NativeDataStage Connect APIDataStage Server (WinNT, Win2000 or UNIX)ODBC/NativeData Sources (Database or F

4、ile)DataStage Connect APIDataStage Connect APIDataStage Connect APIData flowData flow6DataStage ComponentsManager DesignerDirectorMetadata collection and management Design process flowRun jobs, check logs and set schedulesDataStageAdministratorCreate, Edit projects7Hello WorldHello WorldDataStageDat

5、aStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash File、TransformerTransformer、AggregatorAggregatorDirector & MonitorDirector & MonitorAdministrator & ManagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程8全局变量与全局变量与

6、Job变量变量全局变量全局变量- - 生命周期：整个生命周期：整个ProjectProject- - 在在AdministratorAdministrator中定义中定义JobJob变量变量- - 生命周期：一生命周期：一个个JobJob- - 在在DesignerDesigner、ManagerManager中定中定义义9演示：定义一个演示：定义一个Job变量变量在在DesignerDesigner中定义参数中定义参数10Meta data definition元数据管理的重要组成部分元数据管理的重要组成部分在在MangerManger或或DesignDesign中中定义定义演示：演示：-

7、import from a flat .txt format- import from a flat .txt format- import from an DBMS table- import from an DBMS table11演示演示Table Definition在在ManageManage中定义中定义TableTable12Hello WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash

8、 File、TransformerTransformer、AggregatorAggregatorDirector & MonitorDirector & MonitorAdministrator & ManagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程13演示：生成事实表演示：生成事实表明细表事实表关联聚合14Hash File用途：用途：- - 左连接时用作副表左连接时用作副表- - 多次被访问的数据集多次被访问的数据集- - 存储其他临时数据存储其他临时数据

9、关键点：关键点：- - 必须指定必须指定keykey- output- output的的positionposition必须与必须与inputinput一致一致15Transformer用途：用途：- - 提供丰富的运算符和函数提供丰富的运算符和函数- - 数据清洗、转换数据清洗、转换- - 关联多个数据源关联多个数据源关键点：关键点：- - 副表的副表的keykey必须被主表的某个字段关联必须被主表的某个字段关联- - 尽量避免两个尽量避免两个TransformerTransformer直接相连直接相连16Aggregator用途：用途：- - Sum, Max, Min, AverageS

10、um, Max, Min, Average等聚合函数等聚合函数- - 一般用于生成事实表一般用于生成事实表17Hello WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash File、TransformerTransformer、AggregatorAggregatorDirector & MonitorDirector & MonitorAdministrator & Ma

11、nagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程18Debug and TuningView Status and LogsView Status and Logs- status, log, detail- status, log, detail等多等多种视图种视图- - 配合配合MonitorMonitor来查错来查错、调优调优19Job StatusNot CompiledNot CompiledCompiledCompiledResetResetRunningRunningFin

12、ishedFinishedFinished (with warning)Finished (with warning)AbortAbort20ScheduleJob Job Add to ScheduleAdd to Schedule21Hello WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash File、TransformerTransformer、AggregatorAggregatorDi

13、rector & MonitorDirector & MonitorAdministrator & ManagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程22AdministratorAdd a new projectAdd a new projectModify project propertiesModify project properties- - 字符集字符集- - 日志保留天数日志保留天数- hash write catch - hash write c

14、atch Define environment viableDefine environment viable23ManagerImport and export projects or jobsImport and export projects or jobs- - 两种两种文件格式：文件格式：.dsx .xml.dsx .xml- - 整整个个project, project, 根据根据categorycategoryTable definitionTable definitionManage RoutineManage Routine24演示：备份演示：备份project25Hello

15、 WorldHello WorldDataStageDataStage ComponentsComponentsDefine Parameter & TableDefine Parameter & TableHash FileHash File、TransformerTransformer、AggregatorAggregatorDirector & MonitorDirector & MonitorAdministrator & ManagerAdministrator & ManagerRoutine & ControlRoutine & Control议程议程26Routine一种自定义函数，使用一种自定义函数，使用VBVB语法语