hadoop 3.x服役 | 退役数据节点

hadoop 3.x服役 | 退役数据节点,第1张

hadoop 3.x 服役 | 退役数据节点

在服役前要配置好新增主机的环境变量,ssh等信息,个人环境介绍

hadoop002(namenode),hadoop003(resourcemanager),hadoop004(secondarynamenode),准备新增hadoop005

一.服役数据节点

1.在namenode节点主机下的${HADOOP_HOME}/etc/hadoop/下创建dfs.hosts文件添加你要新增的主机名

hadoop002
hadoop003
hadoop004
hadoop005

2.打开hfds-site,xml,添加以下配置

 <property>
<name>dfs.hosts</name>
<value>/opt/module/hadoop-3.1./etc/hadoop/dfs.hosts</value>
<description>Names a file that contains a list of hosts that are
permitted to connect to the namenode. The full pathname of the file
must be specified. If the value is empty, all hosts are
permitted.</description>
</property>

3.刷新namenode与yarn

hdfs dfsadmin -refreshNodes

yarn rmadmin -refreshNodes

此时在页面查看hadoop005状态,发现是dead

4.在hadoop005上启动datanode与nodemanager

hdfs --daemon start datanode

yarn --daemon start nodemanager

再次查看状态

到这基本完成,但还需要在workers文件中添加主机名,以便下次执行start-dfs命令时能够直接启动新增的hadoop005节点

 hadoop002
hadoop003
hadoop004
hadoop005

如果需要平衡数据的话,在执行下start-balancer.sh即可(记得修改分发脚本)

二.退役旧节点

1.在namenode节点主机下的${HADOOP_HOME}/etc/hadoop/下创建dfs.hosts.exclude文件添加你要退役的主机名

 hadoop005

2.打开hfds-site,xml,添加以下配置

 <property>
<name>dfs.hosts.exclude</name>
<value>/opt/module/hadoop-3.1./etc/hadoop/dfs.hosts.exclude</value>
<description>Names a file that contains a list of hosts that are
not permitted to connect to the namenode. The full pathname of the
file must be specified. If the value is empty, no hosts are
excluded.</description>
</property>

3.刷新namenode与yarn

hdfs dfsadmin -refreshNodes

yarn rmadmin -refreshNodes

此时在页面查看hadoop005状态,发现正在退役

继续等待,直到had005变成退役状态

4.在hadoop005上停止namenode与nodemanager

hdfs --daemon stop datanode

yarn --daemon stop nodemanager

5.从dfs.hosts中删除hadoop005

6.再次刷新namenode与yarn后查看页面

7.从workers文件中删除hadoop005,这样执行start-dfs时就不会启动hadoop005上的datanode了

8.数据平衡start-balancer.sh

9.修改分发脚本

欢迎分享,转载请注明来源:内存溢出

原文地址: http://outofmemory.cn/zaji/588113.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-04-12
下一篇 2022-04-12

发表评论

登录后才能评论

评论列表(0条)

保存