Hive编程入门-weibo-大数据文档资料

集团技术平台部-核心系统-海量数据天照1–OpenSourceMapReduceframework–支持SQL语义的大规模数据分析工具–离线/数据仓库应用•为超大数据集设计的计算/扩展能力•统一的元数据管理整体架构•Client端应用程序•元数据•编程接口•环境变量–HIVE_HOME–HIVE_CONF_DIR–HADOOP_HOME/HADOOP_CONF_DIR–hive-default.xml（默认不做修改）/hive-site.xml（用户的配置）–perties•选择一个元数据服务器–int/bigint/smallint/tinyint–boolean–double/float–string•没有精度/长度设定(col_namedata_type,...)等同于：…–INSERTOVERWRITET–INSERTOVERWRITETABLE[WHEREwhere_condition]–join_table:table_referenceJOINtable_factor[{LEFT|RIGHT|FULL}OUTER|LEFTSEMI]JOINtable_referencejoin_condition–table_factor:tbl_name[alias]|table–join_condition:ONequality_expression(ANDequality_expression)*equality_expression:-e'quotedquerystring'Sqlfromcommandline$HIVE_HOME/bin/hive-e'selecta.colfromtab1a‘$HIVE_HOME/bin/hive-e'selecta.colfromtab1a'-hiveconfmapred.reduce.tasks=32$HIVE_HOME/bin/hive-f/home/my/hive-script.sql$HIVE_HOME/bin/hive-i/home/my/hive-init.sqlquitorexittocomeoutofinteractiveshell.Usethistosetvalueofparticularconfigurationvariable.Onethingtonotehereisthatifyoumisspellthevariablename,cliwillnotshowanerror.Thiswillgiveallpossiblehadoop/hiveconfigurationvariables.JARresourcesarealsoaddedtotheJavaclasspath.ThisisrequiredinordertoreferenceobjectstheycontainsuchasUDF's.executedfscommandcommandfromhiveshell•<querystring>executeshivequeryandprintsresultstostdoutexecutessqlfromfileandprintsresultstostdouthive>setmapred.reduce.tasks=32;//设置值hive>setmapred.reduce.tasks;//显示值hive>selecta.*fromtab1;hive>addjar/home/tianzhao/hive/hive-1.1.2/lib/hive_contrib.jar;hive>CREATETEMPORARYFUNCTIONdate_formatAS'com.taobao.hive.udf.UDFDateFormat';hive>source/home/tianzhao.blj/test_others_hive/union.sql;显示partition的信息•Java语言编写的用户自定义函数–UDF-1:1–UDAF-N:1–UDTF-1:N–CREATETEMPORARYFUNCTIONfunction_nameASclass_name–implementevaluate()function–extendsUDAF/GenericUDAF–implementUSING'my_map_script'(AScolName(','colName)*)?(clusterBy?|distribute)REDUCEUSING'my_reduce_script'(AS