zookeeper和kafka的启动和关闭顺序及注意事项_python

首先：一定要先启动ZooKeeper 再启动Kafka 顺序不可以改变。
先关闭kafka ，再关闭zookeeper。

启动ZooKeeper后不要暴力关闭命令行，否则接下来启动kafka时会连接不上，显示Timeout错误

两个都不能暴力关闭

kafka关闭时：
现象：在windows的命令行里启动kafka之后，当关闭命令行窗口时，就会强制关闭kafka。这种关闭方式为暴力关闭，很可能会导致kafka无法完成对日志文件的解锁。届时，再次启动kafka的时候，就会提示日志文件被锁，无法成功启动。
方案：将kafka的日志文件全部删除，再次启动即可。
建议：不要暴力关闭kafka，建议通过在命令行执行kafka-server-stop命令来关闭它。
其他：将来在Linux上部署kafka之后，采用后台运行的方式，就会避免这样的问题。

启动失败：

[2022-05-04 08:28:34,013] INFO [ZooKeeperClient Kafka server] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)
[2022-05-04 08:28:34,017] INFO Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2022-05-04 08:28:34,021] INFO Socket connection established to localhost/127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2022-05-04 08:28:40,025] WARN Client session timed out, have not heard from server in 6005ms for sessionid 0x0 (org.apache.zookeeper.ClientCnxn)
[2022-05-04 08:28:40,026] INFO Client session timed out, have not heard from server in 6005ms for sessionid 0x0, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
[2022-05-04 08:28:40,026] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
[2022-05-04 08:28:40,140] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2022-05-04 08:28:40,144] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn)
[2022-05-04 08:28:40,146] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2022-05-04 08:28:40,155] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
        at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:258)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
        at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:254)
        at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:112)
        at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1826)
        at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:364)
        at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:387)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:207)
        at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
        at kafka.Kafka$.main(Kafka.scala:84)
        at kafka.Kafka.main(Kafka.scala)
[2022-05-04 08:28:40,162] INFO shutting down (kafka.server.KafkaServer)
[2022-05-04 08:28:40,175] INFO shut down completed (kafka.server.KafkaServer)
[2022-05-04 08:28:40,177] ERROR Exiting Kafka. (kafka.server.KafkaServerStartable)
[2022-05-04 08:28:40,198] INFO shutting down (kafka.server.KafkaServer)

成功启动：

[2022-05-03 22:20:39,636] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2022-05-03 22:20:39,636] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2022-05-03 22:20:39,639] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2022-05-03 22:20:39,687] INFO Log directory d:\work\data\kafka-logs not found, creating it. (kafka.log.LogManager)
[2022-05-03 22:20:39,714] INFO Loading logs. (kafka.log.LogManager)
[2022-05-03 22:20:39,732] INFO Logs loading complete in 18 ms. (kafka.log.LogManager)
[2022-05-03 22:20:39,748] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2022-05-03 22:20:39,767] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
log4j:ERROR Failed to rename [D:\kafka\kafka_2.12-2.3.0/logs/log-cleaner.log] to [D:\kafka\kafka_2.12-2.3.0/logs/log-cleaner.log.2022-05-03-21].
[2022-05-03 22:20:40,356] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2022-05-03 22:20:40,403] INFO [SocketServer brokerId=0] Created data-plane acceptor and processors for endpoint : EndPoint(null,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.network.SocketServer)
[2022-05-03 22:20:40,406] INFO [SocketServer brokerId=0] Started 1 acceptor threads for data-plane (kafka.network.SocketServer)
[2022-05-03 22:20:40,447] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,447] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,447] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,452] INFO [ExpirationReaper-0-ElectPreferredLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,486] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
[2022-05-03 22:20:40,529] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)
[2022-05-03 22:20:40,561] INFO Stat of the created znode at /brokers/ids/0 is: 24,24,1651587640550,1651587640550,1,0,0,72060518031163392,200,0,24
 (kafka.zk.KafkaZkClient)
[2022-05-03 22:20:40,562] INFO Registered broker 0 at path /brokers/ids/0 with addresses: ArrayBuffer(EndPoint(LAPTOP-5S4ELDN5,9092,ListenerName(PLAINTEXT),PLAINTEXT)), czxid (broker epoch): 24 (kafka.zk.KafkaZkClient)
[2022-05-03 22:20:40,566] WARN No meta.properties file under dir d:\work\data\kafka-logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2022-05-03 22:20:40,642] INFO [ExpirationReaper-0-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,642] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,642] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2022-05-03 22:20:40,670] INFO Successfully created /controller_epoch with initial epoch 0 (kafka.zk.KafkaZkClient)
[2022-05-03 22:20:40,687] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.group.GroupCoordinator)
[2022-05-03 22:20:40,689] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.group.GroupCoordinator)
[2022-05-03 22:20:40,696] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 7 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
[2022-05-03 22:20:40,716] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)
[2022-05-03 22:20:40,756] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2022-05-03 22:20:40,759] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2022-05-03 22:20:40,759] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2022-05-03 22:20:40,829] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2022-05-03 22:20:40,841] INFO [SocketServer brokerId=0] Started data-plane processors for 1 acceptors (kafka.network.SocketServer)
[2022-05-03 22:20:40,855] INFO Kafka version: 2.3.0 (org.apache.kafka.common.utils.AppInfoParser)
[2022-05-03 22:20:40,856] INFO Kafka commitId: fc1aaa116b661c8a (org.apache.kafka.common.utils.AppInfoParser)
[2022-05-03 22:20:40,857] INFO Kafka startTimeMs: 1651587640843 (org.apache.kafka.common.utils.AppInfoParser)
[2022-05-03 22:20:40,865] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

有时候kafka启动成功后，zookeeper会出现如下信息ERROR

这不是一个报错故障，只是一个user-level KeeperException。可以忽略不做处理的
kafka安装好后，第一次启动。zookeeper日志Error:KeeperErrorCode = NoNode for /config/topics/test，是因为kafka请求访问这个路径，但是这个路径还不存在，zookeeper就抛了这个error。kafka会创建这个topic，然后访问zookeeper里topic对应的路径，zookeeper日志抛出error NodeExists for /config/topics（kafka已经把topic创建好了），路径已经存在了。
综上所述，这些error是正常的日志信息，可以忽略。

[2022-05-04 08:46:42,340] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2022-05-04 08:47:07,313] INFO Accepted socket connection from /127.0.0.1:4333 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2022-05-04 08:47:07,328] INFO Client attempting to establish new session at /127.0.0.1:4333 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-05-04 08:47:07,333] INFO Creating new log file: log.1 (org.apache.zookeeper.server.persistence.FileTxnLog)
[2022-05-04 08:47:07,353] INFO Established session 0x1000036e0580000 with negotiated timeout 6000 for client /127.0.0.1:4333 (org.apache.zookeeper.server.ZooKeeperServer)
[2022-05-04 08:47:07,503] INFO Got user-level KeeperException when processing sessionid:0x1000036e0580000 type:create cxid:0x2 zxid:0x3 txntype:-1 reqpath:n/a Error Path:/brokers Error:KeeperErrorCode = NoNode for /brokers (org.apache.zookeeper.server.PrepRequestProcessor)
[2022-05-04 08:47:07,519] INFO Got user-level KeeperException when processing sessionid:0x1000036e0580000 type:create cxid:0x6 zxid:0x7 txntype:-1 reqpath:n/a Error Path:/config Error:KeeperErrorCode = NoNode for /config (org.apache.zookeeper.server.PrepRequestProcessor)
[2022-05-04 08:47:07,529] INFO Got user-level KeeperException when processing sessionid:0x1000036e0580000 type:create cxid:0x9 zxid:0xa txntype:-1 reqpath:n/a Error Path:/admin Error:KeeperErrorCode = NoNode for /admin (org.apache.zookeeper.server.PrepRequestProcessor)
[2022-05-04 08:47:07,980] INFO Got user-level KeeperException when processing sessionid:0x1000036e0580000 type:create cxid:0x15 zxid:0x15 txntype:-1 reqpath:n/a Error Path:/cluster Error:KeeperErrorCode = NoNode for /cluster (org.apache.zookeeper.server.PrepRequestProcessor)
[2022-05-04 08:47:09,567] INFO Got user-level KeeperException when processing sessionid:0x1000036e0580000 type:multi cxid:0x38 zxid:0x1c txntype:-1 reqpath:n/a aborting remaining multi ops. Error Path:/admin/preferred_replica_election Error:KeeperErrorCode = NoNode for /admin/preferred_replica_election (org.apache.zookeeper.server.PrepRequestProcessor)