canal小试牛刀第二篇：简单运行

canal小试牛刀第二篇：简单运行

github项目直达

1. 启动docker

进入根目录执行 docker-compose up。
当看到如下相关信息表示相应服务已经成功启动：

mysql

canal-db     | 2022-01-19T07:29:51.996555Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
canal-db     | 2022-01-19T07:29:52.033543Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
canal-db     | 2022-01-19T07:29:52.699673Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
canal-db     | 2022-01-19T07:29:52.708262Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.27'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

zookeeper

canal-zk     | 2022-01-19 07:37:57,891 [myid:1] - INFO  [main:RequestThrottler@75] - zookeeper.request_throttler.shutdownTimeout = 10000
canal-zk     | 2022-01-19 07:37:57,900 [myid:1] - INFO  [ProcessThread(sid:0 cport:2181)::PrepRequestProcessor@137] - PrepRequestProcessor (sid:0) started, reconfigEnabled=false
canal-zk     | 2022-01-19 07:37:58,296 [myid:1] - INFO  [main:ContainerManager@84] - Using checkIntervalMs=60000 maxPerMinute=10000 maxNeverUsedIntervalMs=0
canal-zk     | 2022-01-19 07:37:58,314 [myid:1] - INFO  [main:ZKAuditProvider@42] - ZooKeeper audit is disabled.

kafka

canal-kafka  | [2022-01-19 07:38:42,082] INFO Kafka commitId: 8cb0a5e9d3441962 (org.apache.kafka.common.utils.AppInfoParser)
canal-kafka  | [2022-01-19 07:38:42,082] INFO Kafka startTimeMs: 1642577921993 (org.apache.kafka.common.utils.AppInfoParser)
canal-kafka  | [2022-01-19 07:38:42,165] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)
canal-kafka  | [2022-01-19 07:38:42,891] INFO [BrokerToControllerChannelManager broker=1 name=alterIsr]: Recorded new controller, from now on will use broker canal-kafka:9092 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)
canal-kafka  | [2022-01-19 07:38:42,893] INFO [BrokerToControllerChannelManager broker=1 name=forwarding]: Recorded new controller, from now on will use broker canal-kafka:9092 (id: 1 rack: null) (kafka.server.BrokerToControllerRequestThread)

canal-server

canal-server | start canal successful
canal-server | ==> START SUCCESSFUL ...

2. 查看容器状态

➜  ~ docker ps | grep canal
b2cad4259440   canal/canal-server:latest                "/alidata/bin/main.s…"   2 minutes ago    Up 2 minutes    9100/tcp, 11110-11112/tcp                              canal-server
cecdea07b9a1   bitnami/kafka:latest                     "/opt/bitnami/script…"   2 minutes ago    Up 2 minutes    0.0.0.0:9098->9092/tcp                                 canal-kafka
9bbbbe433645   mysql:latest                             "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes    33060/tcp, 0.0.0.0:3310->3306/tcp                      canal-db
6fbd7bbd09b0   redis:alpine                             "docker-entrypoint.s…"   2 minutes ago    Up 2 minutes    0.0.0.0:6375->6379/tcp                                 canal-redis
ae0bd9157f32   bitnami/zookeeper:latest                 "/opt/bitnami/script…"   2 minutes ago    Up 2 minutes    2888/tcp, 3888/tcp, 8080/tcp, 0.0.0.0:2188->2181/tcp   canal-zk

注意：3、 4 同时进行，且当查询topic列表中有example时再消费 3. 进入canal-db容器中

➜  ~ docker exec -it 9bbbbe433645 /bin/bash
root@9bbbbe433645:/# mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or g.
Your MySQL connection id is 13
Server version: 8.0.27 MySQL Community Server - GPL

Copyright (c) 2000, 2021, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

mysql>

# 创建数据库
- 略

# 创建表
- 略

# 执行插入语句

mysql> INSERT INTO `test_user`(`id`, `name`, `phone`) VALUES (null, 'name11', '18890901211');
Query OK, 1 row affected (0.17 sec)
# 更新语句

# 删除语句

4. 进入 canal-kafka容器中

➜  ~ docker exec -it cecdea07b9a1 /bin/bash
I have no name!@cecdea07b9a1:/$

查看topic 列表, example为定义的topic

I have no name!@cecdea07b9a1:/$ kafka-topics.sh --bootstrap-server canal-kafka:9092 --list

example

订阅 消费，此时所有在canal-db中执行的sql命令都会被同步消费

I have no name!@cecdea07b9a1:/$ kafka-console-consumer.sh --bootstrap-server canal-kafka:9092 --from-beginning --topic example
{"data":[{"id":"9","name":"name11","phone":"18890901211"}],"database":"test","es":1642578664000,"id":1,"isDdl":false,"mysqlType":{"id":"int","name":"varchar(255)","phone":"varchar(255)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":4,"name":12,"phone":12},"table":"test_user","ts":1642578666254,"type":"INSERT"}
{"data":[{"id":"2","name":"name02-1","phone":"188012340002"}],"database":"test_base","es":1642587435000,"id":4,"isDdl":false,"mysqlType":{"id":"int(11)","name":"varchar(255)","phone":"varchar(255)"},"old":[{"name":"name02"}],"pkNames":["id"],"sql":"","sqlType":{"id":4,"name":12,"phone":12},"table":"base_user","ts":1642587435286,"type":"UPDATE"}