21-01-2022 20:27:56 CST dmJob INFO - FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
21-01-2022 20:27:56 CST dmJob INFO - Process with id 62758 completed unsuccessfully in 136 seconds.
21-01-2022 20:27:56 CST dmJob ERROR - Job run failed!
java.lang.RuntimeException: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 3
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:312)
at azkaban.execapp.JobRunner.runJob(JobRunner.java:823)
at azkaban.execapp.JobRunner.doRun(JobRunner.java:602)
at azkaban.execapp.JobRunner.run(JobRunner.java:563)
at java.util.concurrent.Executors
R
u
n
n
a
b
l
e
A
d
a
p
t
e
r
.
c
a
l
l
(
E
x
e
c
u
t
o
r
s
.
j
a
v
a
:
511
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
F
u
t
u
r
e
T
a
s
k
.
r
u
n
(
F
u
t
u
r
e
T
a
s
k
.
j
a
v
a
:
266
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
r
u
n
W
o
r
k
e
r
(
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
j
a
v
a
:
1142
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor
RunnableAdapter.call(Executors.java:511)atjava.util.concurrent.FutureTask.run(FutureTask.java:266)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)atjava.util.concurrent.ThreadPoolExecutorWorker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 3
at azkaban.jobExecutor.utils.process.AzkabanProcess.run(AzkabanProcess.java:125)
at azkaban.jobExecutor.ProcessJob.run(ProcessJob.java:304)
… 8 more
21-01-2022 20:27:56 CST dmJob ERROR - azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 3 cause: azkaban.jobExecutor.utils.process.ProcessFailureException: Process exited with code 3
21-01-2022 20:27:56 CST dmJob INFO - Finishing job dmJob at 1642768076810 with status FAILED
解决方法：
set hive.mapjoin.localtask.max.memory.usage=0.99
hive.mapjoin.localtask.max.memory.usage
说明：本地任务可以使用内存的百分比
默认值0.90，如果你的localtask mapjoin 表很小可以试试，但彻底解决需要
set hive.auto.convert.join=false;关闭自动mapjoin 但这个参数用的时候一定要注意，
如果你的sql 很长join会常多，关闭mapjoin任务数会成10倍激增，contener满了任务同样会非常之慢
set hive.auto.convert.join=false;一定要用在localtask级别这种超轻量及的job上。
欢迎分享，转载请注明来源：内存溢出
微信扫一扫
支付宝扫一扫
评论列表（0条）