参考:pulsar-4:单机部署pulsar单节点集群 - 云+社区 - 腾讯云
1、安装包下载wget https://archive.apache.org/dist/pulsar/pulsar-2.8.0/apache-pulsar-2.8.0-bin.tar.gz
解压
$ tar -xzvf apache-pulsar-2.8.0-bin.tar.gz -C servers/ $ cd $ mv apache-pulsar-2.8.1 pulsar $ cd pulsar2、zookeeper部署启动
进入到pulsar主目录下,修改con/zookeeper.conf配置文件
增加配置:
server.1=10.50.30.178:2888:3888 修改zk metric port:因为bookie也用原值8000,在同一台部署bookie会失败。 metricsProvider.httpPort=8001 #查看zookeeper配置 $ cat zookeeper.conf |grep -vE "^$|^#" #创建zookeeper的data保存路径 mkdir -p data/zookeeper echo 1 > data/zookeeper/myid #启动zookeeper bin/pulsar-daemon start zookeeper
3、初始化集群元数据$ bin/pulsar initialize-cluster-metadata --cluster pulsar-cluster --zookeeper 10.50.30.178:2181 --configuration-store 10.50.30.178:2181 --web-service-url http://10.50.30.178:8008 --web-service-url-tls https://10.50.30.177:8043 --broker-service-url pulsar://10.50.30.177:6650 --broker-service-url-tls pulsar+ssl://10.50.30.177:6651 #对应的删除元数据命令是: $ bin/pulsar delete-cluster-metadata --cluster pulsar-cluster --zookeeper 10.50.30.178:2181
4、部署bookeeper单节点
修改 conf/bookkeeper.conf文件
修改zkServers的ip:zkServers=ip:2181
关闭autorecovry:autoRecoveryDaemonEnabled=false
否则报错:BKNotEnoughBookiesException: Not enough non-faulty bookies available
配置如下
bookiePort=3181 journalDirectory=data/bookkeeper/journal minUsableSizeForIndexFileCreation=1073741824 advertisedAddress= allowLoopback=false bookieDeathWatchInterval=1000 flushInterval=60000 useHostNameAsBookieID=false gcWaitTime=900000 gcOverreplicatedLedgerWaitTime=86400000 numAddWorkerThreads=0 numReadWorkerThreads=8 numHighPriorityWorkerThreads=8 maxPendingReadRequestsPerThread=2500 maxPendingAddRequestsPerThread=10000 enableBusyWait=false auditorPeriodicBookieCheckInterval=86400 rereplicationEntryBatchSize=100 openLedgerRereplicationGracePeriod=30000 autoRecoveryDaemonEnabled=false lostBookieRecoveryDelay=0 serverTcpNoDelay=true nettyMaxframeSizeBytes=5253120 journalFormatVersionToWrite=5 journalMaxSizeMB=2048 journalMaxBackups=5 journalPreAllocSizeMB=16 journalWriteBufferSizeKB=64 journalRemoveFromPageCache=true journalSyncData=true journalAdaptiveGroupWrites=true journalMaxGroupWaitMSec=1 journalBufferedWritesThreshold=524288 numJournalCallbackThreads=8 journalAlignmentSize=4096 journalFlushWhenQueueEmpty=false ledgerStorageClass=org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage ledgerDirectories=data/bookkeeper/ledgers auditorPeriodicCheckInterval=604800 openFileLimit=0 fileInfoFormatVersionToWrite=0 pageLimit=0 zkLedgersRootPath=/ledgers logSizeLimit=1073741824 entryLogFilePreallocationEnabled=true flushEntrylogBytes=268435456 readBufferSizeBytes=4096 writeBufferSizeBytes=65536 compactionRate=1000 minorCompactionThreshold=0.2 minorCompactionInterval=3600 compactionMaxOutstandingRequests=100000 majorCompactionThreshold=0.5 majorCompactionInterval=86400 isThrottleByBytes=false compactionRateByEntries=1000 compactionRateByBytes=1000000 statsProviderClass=org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider prometheusStatsHttpPort=8000 readonlyModeEnabled=true diskUsageThreshold=0.95 diskCheckInterval=10000 zkServers=localhost:2181 zkTimeout=30000 zkEnableSecurity=false httpServerEnabled=false httpServerPort=8000 httpServerClass=org.apache.bookkeeper.http.vertx.VertxHttpServer dbStorage_writeCacheMaxSizeMb= dbStorage_readAheadCacheMaxSizeMb= dbStorage_readAheadCacheBatchSize=1000 dbStorage_rocksDB_blockCacheSize= dbStorage_rocksDB_writeBufferSizeMB=64 dbStorage_rocksDB_sstSizeInMB=64 dbStorage_rocksDB_blockSize=65536 dbStorage_rocksDB_bloomFilterBitsPerKey=10 dbStorage_rocksDB_numLevels=-1 dbStorage_rocksDB_numFilesInLevel0=4 dbStorage_rocksDB_maxSizeInLevel1MB=256 zkServers=10.50.30.178:2181 advertisedAddress=10.50.30.177
后台启动bookie:
$ bin/pulsar-daemon start bookie # 你可以通过运行BookKeeper shell上的bookiesanity命令验证 bookie 是否正常工作 这个命令会在本地的 bookie 创建一个临时的 BookKeeper ledger,往里面写一些条目,然后读取它,最后删除这个 ledger。 $ bin/bookkeeper shell bookiesanity5、部署broker单节点
Broker 配置中有一些非常重要的参数,这些参数可以确保每个Broker 连接到已部署的 ZooKeeper 集群。需要确认 zookeeperServers 和 configurationStoreServers 配置项的值是正确的。在当前情况下,由于只有一个集群,没有单独用来存储配置的 Zookeeper 集群,那么配置项configurationStoreServers和zookeeperServers 是一样的值。
修改broker.conf配置文件里的:
zookeeperServers=10.50.30.178:2181
configurationStoreServers=10.50.30.178:2181
你必须配置集群的名字( 初始化集群元数据 提供的集群名字必须和这个配置项匹配):
clusterName=pulsar-cluster
如果你部署的是单节点的 Pulsar 集群,你需要把配置文件conf/broker.conf的副本数量配置为1.
managedLedgerDefaultEnsembleSize=1
managedLedgerDefaultWriteQuorum=1
managedLedgerDefaultAckQuorum=1
启用 Pulsar Function(可选)
If you want to enable Pulsar Functions, you can follow the instructions as below:
将conf/broker.conf文件的配置项functionsWorkerEnabled设置为true,启用函数 worker,
functionsWorkerEnabled=true
修改conf/functions_worker.yml的配置项pulsarFunctionsCluster 为集群的名称,该名称和初始化集群元数据使用的集群名字是同一个。
pulsarFunctionsCluster: pulsar-cluster
后台启动broker:
bin/pulsar-daemon start broker
6、pulsar-cient连接集群vim conf/client.conf
如果你没有 DNS 服务器,你能够使用多个域名组成的格式的服务的 URL:
webServiceUrl=http://10.50.30.178:8008
brokerServiceUrl=pulsar://10.50.30.178:6650
启动consumer监听:
bin/pulsar-client consume persistent://public/default/test -n 100 -s "consumer-test" -t "Exclusive"
启动producer发送消息:
bin/pulsar-client produce persistent://public/default/test -n 1 -m "Hello Pulsar"
docker部署pulsar-manager 1、下载docker的conf克隆gitee的pulsar-manager到本地
$git clone https://gitee.com/mirrors_apache/pulsar-manager.git # 新建docker映射目录 $ mkdir -p /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf $ cd /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf #下载bkvm.conf文件到指定目录:
修改bkvm.conf:
docker run -it -d -p 9527:9527 -p 7750:7750 -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties -v /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf/bkvm.conf:/pulsar-manager/pulsar-manager/bkvm.conf -v /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf/application.properties:/pulsar-manager/pulsar-manager/application.properties --name pulsar-manager apachepulsar/pulsar-manager docker run -it -d -p 9527:9527 -p 7750:7750 -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties -v /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf/bkvm.conf:/pulsar-manager/pulsar-manager/bkvm.conf -v /home/dingzepeng/servers/pulsar-manager/docker-local/docker-local-pulsar-manager/conf/application.properties:/pulsar-manager/pulsar-manager/application.properties --name pulsar-manager apachepulsar/pulsar-manager curl -H 'X-XSRF-TOKEN: $CSRF_TOKEN' -H 'cookie: XSRF-TOKEN=$CSRF_TOKEN;' -H "Content-Type: application/json" -X PUT http://localhost:7750/pulsar-manager/users/superuser -d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}' 注: 1、8080端口占用
若8080端口占用需要修改
broker.conf里的webServicePort=8008
client.conf里的webServiceUrl=http://10.50.30.178:8008
欢迎分享,转载请注明来源:内存溢出
评论列表(0条)