线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“

线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“,第1张

线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“

线上Flink任务重启时,提交任务失败,具体报错如下:

2021-12-31 18:09:19
java.lang.NumberFormatException: For input string: "30s"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1441)
	at org.apache.hadoop.hdfs.client.impl.DfsClientConf.(DfsClientConf.java:250)
	at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:303)
	at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:287)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3242)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:121)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3291)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3259)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:470)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
	at org.apache.iceberg.hadoop.Util.getFs(Util.java:51)
	at org.apache.iceberg.hadoop.HadoopOutputFile.fromPath(HadoopOutputFile.java:53)
	at org.apache.iceberg.hadoop.HadoopFileIO.newOutputFile(HadoopFileIO.java:64)
	at org.apache.iceberg.io.OutputFileFactory.newOutputFile(OutputFileFactory.java:120)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.openCurrent(baseTaskWriter.java:264)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.(baseTaskWriter.java:231)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.(baseTaskWriter.java:221)
	at org.apache.iceberg.io.baseTaskWriter$RollingFileWriter.(baseTaskWriter.java:303)
	at org.apache.iceberg.io.UnpartitionedWriter.(UnpartitionedWriter.java:33)
	at org.apache.iceberg.flink.sink.RowDataTaskWriterFactory.create(RowDataTaskWriterFactory.java:89)
	at org.apache.iceberg.flink.sink.IcebergStreamWriter.open(IcebergStreamWriter.java:59)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:433)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:545)
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:535)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
	at java.lang.Thread.run(Thread.java:748)

解决方案:
在hdfs-site.xml中新增配置后


    dfs.client.datanode-restart.timeout
    30

问题解决

欢迎分享,转载请注明来源:内存溢出

原文地址: https://outofmemory.cn/zaji/5697835.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-12-17
下一篇 2022-12-17

发表评论

登录后才能评论

评论列表(0条)

保存