1.下载Hive安装包:
官网下载:
百度云分享:
2.上传Hive的tar包,并解压:
解压:tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /usr/local/src/
修改解压后的文件名称:mv apache-hive-1.2.1-bin hive-1.2.1
3.安装MySql:
MySQL用于存储Hive的元数据,不建议使用Hive自带Derby作为Hive的元数据库,因为它的数据文件默认保存在运行
目录下面,下次换一个目录启动就看不见之前的数据了
4.修改配置文件:主要是配置metastore(元数据存储)存储方式
4.1. vi /usr/local/src/hive-1.2.1/conf/hive-site.xml(存储方式:内嵌Derby方式、本地mysql、远端mysql)
4.2 粘贴如下内容:
javax.jdo.option.ConnectionURL jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName root username to use against metastore database javax.jdo.option.ConnectionPassword root //mysql密码不正确, 查看hive-site.xml配置与mysql的密码是否一致password to use against metastore database
5.拷贝jar包:
拷贝mysql驱动jar包到Hive的lib目录下面去,下载路径:
6.启动Hive:启动诸多问题参考:
启动Hive之前需要先把Hadoop集群启动起来。然后使用下面的命令来启动Hive:
启动命令:/usr/local/src/hive-1.2.1/bin/hive
出现如下表示启动成功:
验证Hive运行正常:启动Hive以后输入下面的命令:
输出为:
创建数据库,输出结果如下:数据库的数据文件被存放在HDFS的/user/hive/warehouse/test_db.db下面
创建表:表的数据文件被存放在HDFS的/user/hive/warehouse/test_db.db/t_test下面
插入数据
准备下面的数据文件,sz.data,文件内容如下:
1,张三2,李四3,风凌4,三少5,月关6,abc
上传到HDFS上:hadoop fs -put sz.data /user/hive/warehouse/test_db.db/t_test/sz.data
然后使用下面的语句尝试进行一下查询:
输出的结果如下:
数据没有被成功识别,这是因为没有指定数据的分隔符。使用下面的命令清空表数据:
count查询:select count(1) from t_test;
能看到下面的运行提示:
Query ID = root_20170325234306_1aaf3dcf-e758-4bbd-9ae5-e649190d8417Total jobs = 1Launching Job 1 out of 1Number of reduce tasks determined at compile time: 1In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=In order to limit the maximum number of reducers: set hive.exec.reducers.max= In order to set a constant number of reducers: set mapreduce.job.reduces= Starting Job = job_1490454340487_0001, Tracking URL = http://amaster:8088/proxy/application_1490454340487_0001/Kill Command = /root/apps/hadoop-2.7.3/bin/hadoop job -kill job_1490454340487_0001Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 12017-03-25 23:43:23,084 Stage-1 map = 0%, reduce = 0%2017-03-25 23:43:36,869 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 1.71 sec2017-03-25 23:43:48,392 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 3.4 secMapReduce Total cumulative CPU time: 3 seconds 400 msecEnded Job = job_1490454340487_0001MapReduce Jobs Launched: Stage-Stage-1: Map: 1 Reduce: 1 Cumulative CPU: 3.4 sec HDFS Read: 6526 HDFS Write: 2 SUCCESSTotal MapReduce CPU Time Spent: 3 seconds 400 msec
发现Hive的速度确实很慢,不适合用于在线业务支撑,同时,在YARN集群里面也可以看到任务信息
输出结果如下:
Hive的元数据
接下来,我们来看一下在MySQL里面保存的Hive元数据
可以看到有很多表。其中DBS表记录了数据库的记录: