Hive端执行 select count

Hive端执行 select count,第1张

好不容易搞出了hbasehive的外链表组合应用。却在 hive端执行  select count(*) from cdprojectdetail; 出现如下错误:

Query ID = root_20220421173314_ade31c99-1ec5-4419-88c7-997bf8e82577
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=
In order to set a constant number of reducers:
  set mapreduce.job.reduces=
2022-04-21 17:33:15 INFO client.AHSProxy: Connecting to Application History server at udp02/10.12.3.12:10201
2022-04-21 17:33:15 INFO client.AHSProxy: Connecting to Application History server at udp02/10.12.3.12:10201
Starting Job = job_1649670158454_0012, Tracking URL = http://udp01:8088/proxy/application_1649670158454_0012/
Kill Command = /opt/usdp-srv/srv/udp/2.0.0.0/yarn/bin/mapred job  -kill job_1649670158454_0012
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0
2022-04-21 17:33:48,903 Stage-1 map = 0%,  reduce = 0%
Ended Job = job_1649670158454_0012 with errors
Error during job, obtaining debugging information...
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched: 
Stage-Stage-1:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

尝试了多种方法,甚至怀疑hive是否支持 count(*) 

终于见网友说:

namenode内存空间不够,JVM剩余内存空间不够新job运行所致

于是在hive shell 里将hive设置成本地模式来执行任务

set hive.exec.mode.local.auto=true;

测试,成功!偶也。。。

为了感谢大家的相互分享,本人即将启动10停了10多年的csdn进行心路历程分享。

欢迎分享,转载请注明来源:内存溢出

原文地址: http://outofmemory.cn/langs/725541.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-04-26
下一篇 2022-04-26

发表评论

登录后才能评论

评论列表(0条)

保存