kafka中的partition迁移

kafka中的partition迁移 一、kafka节点内磁盘使用不均衡 1、原因：

  每次新建partition时，会选择partition少的目录，而不会考虑磁盘的使用空间

2、解决 1）停止broker 2）对目录下的文件进行移动

 kafka-logs每个dir下会有两个文件
 recovery-point-offset-checkpoint
 replication-offset-checkpoint
 移动partition后，需要修改对应目录中的这两个文件内容

二、kafka中新扩容broker需要进行partition迁移时 1、创建move.json

多个topic，则写每个topic需要指定的分区，多副本情况，指定每个partition的存储节点
move.json如下

{"version":1,"partitions":[
{"topic":"wanghaha_ceshi","partition":3,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":4,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":5,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":6,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":7,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":8,"replicas":[2,1]},
{"topic":"wanghaha_ceshi","partition":9,"replicas":[2]}]}

2、分区迁移

使用命令迁移partition
./kafka-reassign-partitions.sh --zookeeper 10.24.102.1:24002/kafka --reassignment-json-file move.json --execute