好不容易搞出了hbase和hive的外链表组合应用。却在 hive端执行 select count(*) from cdprojectdetail; 出现如下错误:
Query ID = root_20220421173314_ade31c99-1ec5-4419-88c7-997bf8e82577
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=
In order to set a constant number of reducers:
set mapreduce.job.reduces=
2022-04-21 17:33:15 INFO client.AHSProxy: Connecting to Application History server at udp02/10.12.3.12:10201
2022-04-21 17:33:15 INFO client.AHSProxy: Connecting to Application History server at udp02/10.12.3.12:10201
Starting Job = job_1649670158454_0012, Tracking URL = http://udp01:8088/proxy/application_1649670158454_0012/
Kill Command = /opt/usdp-srv/srv/udp/2.0.0.0/yarn/bin/mapred job -kill job_1649670158454_0012
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2022-04-21 17:33:48,903 Stage-1 map = 0%, reduce = 0%
Ended Job = job_1649670158454_0012 with errors
Error during job, obtaining debugging information...
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
尝试了多种方法,甚至怀疑hive是否支持 count(*)
终于见网友说:
namenode内存空间不够,JVM剩余内存空间不够新job运行所致
于是在hive shell 里将hive设置成本地模式来执行任务
set hive.exec.mode.local.auto=true;
测试,成功!偶也。。。
为了感谢大家的相互分享,本人即将启动10停了10多年的csdn进行心路历程分享。
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)