canal整合rocketmq实现数据库redis双写一致

canal整合rocketmq实现数据库redis双写一致

视频参考:`

https://www.bilibili.com/video/BV1tb4y1676p

1.开启mysql中binlog模式

修改mysql配置文件

[mysqld]
log-bin=mysql-bin # 开启 binlog
binlog-format=ROW # 选择 ROW 模式
server_id=1 # 配置 MySQL replaction 需要定义，不要和 canal 的 slaveId 重复

进入etc目录下:


加入以下内容:

log-bin=mysql-bin
binlog-format=ROW
server_id=12345

2.安装rocketmq

拉取镜像

docker pull rocketmqinc/rocketmq

启动namesrv服务

docker run -d -p 9876:9876 
-v /export/server/mq/data/namesrv/logs:/root/logs 
-v /export/server/mq/data/namesrv/store:/root/store 
--name rmqnamesrv 
-e "JAVA_OPTS=-Duser.home=/opt" 
-e "JAVA_OPT_EXT=-server -Xms512m -Xmx512m" rocketmqinc/rocketmq sh mqnamesrv

配置broker文件

brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
#换成网络IP
namesrvAddr = 192.168.3.36:9876
#换成网络IP
brokerIP1 = 192.168.3.36
#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭
autoCreateSubscriptionGroup=true
#消费端重试次数与间隔
messageDelayLevel=5s 10s 15s 35s 55s 105s 205s 305s 405s 505s 605s 705s 805s 905s 1005s 1015s 1025s 1355s

启动broker服务

docker run -d -p 10911:10911 -p 10909:10909 
-v /export/server/mq/data/broker/logs:/root/logs 
-v /export/server/mq/rocketmq/data/broker/store:/root/store 
-v /export/server/mq/conf/broker.conf:/opt/rocketmq/conf/broker.conf 
--name rmqbroker 
--link rmqnamesrv:namesrv 
-e "NAMESRV_ADDR=namesrv:9876" 
-e "JAVA_OPTS=-Duser.home=/opt" 
-e "JAVA_OPT_EXT=-server -Xms512m -Xmx512m" rocketmqinc/rocketmq sh mqbroker 
-c /opt/rocketmq/conf/broker.conf

安装控制台

docker pull styletang/rocketmq-console-ng

docker run -d -e "JAVA_OPTS=-Drocketmq.config.namesrvAddr=192.168.3.36:9876 
-Drocketmq.config.isVIPChannel=false" -p 8003:8080 -t styletang/rocketmq-console-ng

访问控制台:

http://192.168.3.36:8003/#/message

3.安装canal

链接：https://pan.baidu.com/s/1ym8gkS7QKpam3wyv52bQag 
提取码：b2a0

解压文件:

[root@localhost canal]# tar zxvf canal.deployer-1.1.5.tar.gz

修改配置文件 vi /export/server/canal/conf/example/instance.properties
instance.properties

修改配置文件:vi /export/server/canal/conf/canal.properties


启动canal

sh bin/startup.sh

停止

sh bin/stop.sh

代码测试

链接：https://pan.baidu.com/s/1Zt_kWgreEqbdNyNMV-Wj1A 
提取码：hdjh