kafka安装和集群的搭建linux

1、kafka安装 需要先安装zookeeper ，可以看：https://blog.csdn.net/weixin_44130094/article/details/122807986

1、下载地址 ：https://kafka.apache.org/downloads

2、上传 kafka_2.11-2.4.1.tgz
解压：tar -zxvf kafka_2.11-2.4.1.tgz
3、修改配置文件 server.properties

log.dirs=/usr/local/kafka_2.11-2.4.1/data/kafka-logs
#listeners=PLAINTEXT://内网ip:9092
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://外网ip:9092

zookeeper.connect=localhost:2181

内网ip就用

listeners=PLAINTEXT://内网ip:9092

外网访问就用：

listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://外网ip:9092

server.properties

############################# Server Basics #############################
# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0
############################# Socket Server Settings 
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://外网ip:9092

num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
############################# Log Basics #############################
# 日志
log.dirs=/usr/local/kafka_2.11-2.4.1/data/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1

############################# Internal Topic Settings 
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
# The minimum age of a log file to be eligible for deletion due to age
# kafka消息默认保存时间
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000

############################# Zookeeper #############################
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
#zookeeper的地址
zookeeper.connect=localhost:2181

# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000

############################# Group Coordinator Settings 
group.initial.rebalance.delay.ms=0

4、启动kafka
进入bin目录

# -daemon 后台启动
 ./kafka-server-start.sh -daemon ../config/server.properties
 # 或者
 ./kafka-server-start.sh ../config/server.properties &

5、查看启动日志
tail -f /usr/local/kafka_2.11-2.4.1/logs/server.log

云服务记得开启 9092端口 2、集群搭建

1、修改另外2台机器的配置文件

#修改broker
broker.id=1
log.dirs=/usr/local/kafka_2.11-2.4.1/data/kafka-logs
# 3个kafka的zoookeeper要配置同一个zoookeeper地址
zookeeper.connect=ip:2181
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://外网ip:9092

#修改broker
broker.id=2
log.dirs=/usr/local/kafka_2.11-2.4.1/data/kafka-logs
# 3个kafka的zoookeeper要配置同一个zoookeeper地址
zookeeper.connect=ip:2181
listeners=PLAINTEXT://:9092
advertised.listeners=PLAINTEXT://外网ip:9092

2、启动2台机器
3、在zookeeper客户端看到如下信息就代表搭建成功