prometheus、influxdb安装及flume

prometheus、influxdb安装及flume,第1张

目录 🥇prometheus🥈创建用户🥈下载解压部署🥉下载🥉解压🥉部署启动 🥈登录验证🥈数据存储 🥇influxdb🥈创建用户🥈下载解压部署🥉下载🥉解压部署 🥈登录验证🥉初始化用户信息,全都是必填项🥉查看数据库信息 🥈prometheus指定数据库配置🥉prometheus.yml🥉重新加载🥉查看 🥇flume_exporter🥈创建目录🥈下载🥈编译🥉安装go环境🥉编译flume_exporter项目🏅设置变量,初始化flume目录🏅安装依赖包🏅指定编译目录,执行安装 🥈启动🥉简单部署flume 🥈promethues配置flume_exporter🥉加载验证🥉influx查看flume指标数据🏅 安装telegrafs🏅 prometheus修改db配置🏅 再次查看flume指标数据 🥇其它🥈查看系统及处理器型号🥈lsb_release: 未找到命令🥈 解压报错🥈 influxdb2.0 写入数据库必须要telegrafs嘛

🥇prometheus

一款监控告警软件,详见官方资料

🥈创建用户

创建部署用户,赋予sudo权限

[root@host1 ~]# useradd prometheus
[root@host1 ~]# passwd prometheus
更改用户 prometheus 的密码 。
新的 密码:
无效的密码: 密码少于 8 个字符
重新输入新的 密码:
passwd:所有的身份验证令牌已经成功更新。
[root@host1 ~]# echo 'prometheus ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
[root@host1 ~]# su - prometheus
[prometheus@host1 ~]$ sudo -v
[prometheus@host1 ~]$ 
🥈下载解压部署 🥉下载

prometheus下载地址

选择最新版amd64(网速差,可以先下载到本地,再上传到服务器)

wget https://github.com/prometheus/prometheus/releases/tag/v2.35.0/prometheus-2.35.0.linux-amd64.tar.gz  --no-check-certificate

🥉解压

解压修改配置文件,默认localhost替换为具体地址

[prometheus@host1 ~]$ tar xf prometheus-2.35.0-rc0.linux-amd64.tar.gz 
[prometheus@host1 ~]$ ll
总用量 78748
drwxr-xr-x. 4 prometheus prometheus      132 4月   8 21:41 prometheus-2.35.0-rc0.linux-amd64
-rw-r--r--. 1 prometheus prometheus 80636489 5月   9 10:50 prometheus-2.35.0-rc0.linux-amd64.tar.gz
[prometheus@host1 ~]$ mv prometheus-2.35.0-rc0.linux-amd64 prometheus
[prometheus@host1 ~]$ cd prometheus
[prometheus@host1 prometheus]$ ll
总用量 198240
drwxr-xr-x. 2 prometheus prometheus        38 4月   8 21:38 console_libraries
drwxr-xr-x. 2 prometheus prometheus       173 4月   8 21:38 consoles
-rw-r--r--. 1 prometheus prometheus     11357 4月   8 21:38 LICENSE
-rw-r--r--. 1 prometheus prometheus      3773 4月   8 21:38 NOTICE
-rwxr-xr-x. 1 prometheus prometheus 105568448 4月   8 21:29 prometheus
-rw-r--r--. 1 prometheus prometheus       934 4月   8 21:38 prometheus.yml
-rwxr-xr-x. 1 prometheus prometheus  97405202 4月   8 21:33 promtool
[prometheus@host1 prometheus]$ sed -i 's/localhost/192.168.56.10/g' prometheus.yml 
[prometheus@host1 prometheus]$ pwd
/home/prometheus/prometheus
🥉部署启动
nohup /home/prometheus/prometheus/prometheus --config.file=/home/prometheus/prometheus/prometheus.yml --web.enable-lifecycle  > prometheus.log 2>&1 &

参数说明:

nohup “xxxx” & :后端启动prometheus :执行命令文件–config.file :指定配置文件–web.enable-lifecycle:支持通过http请求重载配置(热加载)

prometheus.log 2>&1:>-覆盖写入日志、prometheus.log-日志名称、2>&1-写入全部日志,不管对错
查看日志,对应监听端口为9090

🥈登录验证

登录页面查看

🥈数据存储

默认情况下prometheus会将采集的数据存放到本地磁盘data下面(二进制文件),为了方便查看和扩展,接下来使用influxdb来存储数据。

🥇influxdb 🥈创建用户

创建部署用户,赋予sudo权限

[root@host1 ~]# useradd influx
[root@host1 ~]# passwd influx
更改用户 influx 的密码 。
新的 密码:
无效的密码: 密码少于 8 个字符
重新输入新的 密码:
passwd:所有的身份验证令牌已经成功更新。
[root@host1 ~]# echo 'influx ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
[root@host1 ~]# su -  influx
[influx@host1 ~]$ sudo -v
[influx@host1 ~]$ 
🥈下载解压部署 🥉下载

官方资料,目前最新版本2.2

wget https://dl.influxdata.com/influxdb/releases/influxdb2-2.2.0-linux-amd64.tar.gz

🥉解压部署
[influx@host1 ~]$ tar xf influxdb2-2.2.0-linux-amd64.tar.gz 
[influx@host1 ~]$ mv influxdb2-2.2.0-linux-amd64 influxdb
[influx@host1 ~]$ cd influxdb
[influx@host1 influxdb]$ ll
总用量 169660
-rwxr-xr-x. 1 influx influx 173712472 4月   7 01:39 influxd
-rw-rw-r--. 1 influx influx      1067 4月   7 01:40 LICENSE
-rw-rw-r--. 1 influx influx      9834 4月   7 01:40 README.md
[influx@host1 influxdb]$ nohup ./influxd &

查看日志,默认启动端口8086

🥈登录验证

登录界面

🥉初始化用户信息,全都是必填项


🥉查看数据库信息


🥈prometheus指定数据库配置 🥉prometheus.yml

添加如下配置

# set db
remote_write:
  - url: "http://192.168.56.10:8086/api/v1/prom/write?db=prometheus&u=prometheus&p=prometheus"
remote_read:
  - url: "http://192.168.56.10:8086/api/v1/prom/read?db=prometheus&u=prometheus&p=prometheus"
🥉重新加载

直接刷新页面没反应,需执行如下命令

 curl -X POST http://192.168.56.10:9090/-/reload
🥉查看

🥇flume_exporter 🥈创建目录

直接在prometheus用户下面建立exporter目录

[prometheus@host1 ~]$ ll
总用量 78748
drwxr-xr-x. 5 prometheus prometheus      166 5月  15 16:36 prometheus
-rw-r--r--. 1 prometheus prometheus 80636489 5月   9 10:50 prometheus-2.35.0-rc0.linux-amd64.tar.gz
[prometheus@host1 ~]$ mkdir exporter
[prometheus@host1 ~]$ cd exporter/
[prometheus@host1 exporter]$ ll
总用量 0
[prometheus@host1 exporter]$ 
[prometheus@host1 exporter]$ 
🥈下载

下载地址,下载下来的是源文件,还要进行编译(网上没找到编译好的压缩包)

🥈编译 🥉安装go环境

下载地址

解压配置环境变量

[prometheus@host1 ~]$ tar xf go1.18.1.linux-amd64.tar.gz 
[prometheus@host1 ~]$ ll
总用量 217128
drwxrwxr-x.  3 prometheus prometheus       106 5月  15 17:45 exporter
drwxr-xr-x. 10 prometheus prometheus       257 4月  12 23:42 go
-rw-r--r--.  1 prometheus prometheus 141699677 5月   9 15:22 go1.18.1.linux-amd64.tar.gz
drwxr-xr-x.  5 prometheus prometheus       166 5月  15 16:36 prometheus
-rw-r--r--.  1 prometheus prometheus  80636489 5月   9 10:50 prometheus-2.35.0-rc0.linux-amd64.tar.gz
[prometheus@host1 ~]$ vi .bash_profile 
[prometheus@host1 ~]$ . .bash_profile 
[prometheus@host1 ~]$ go version
go version go1.18.1 linux/amd64
[prometheus@host1 ~]$ 

.bash_profile 文件配置如下

export GOROOT=$HOME/go
export PATH=$PATH:$GOROOT/bin
🥉编译flume_exporter项目 🏅设置变量,初始化flume目录
[prometheus@host1 exporter]$ ll
总用量 67908
-rw-r--r--. 1 prometheus prometheus 67938106 5月   9 16:45 apache-flume-1.9.0-bin.tar.gz
drwxrwxr-x. 6 prometheus prometheus      267 2月  20 2019 flume_exporter-0.0.2
-rw-r--r--. 1 prometheus prometheus  1594290 5月  15 17:05 flume_exporter-0.0.2.tar.gz
[prometheus@host1 exporter]$ go env -w GO111MODULE=on
[prometheus@host1 exporter]$ go mod init flume_exporter-0.0.2
go: creating new go.mod: module flume_exporter-0.0.2
go: to add module requirements and sums:
	go mod tidy
[prometheus@host1 exporter]$ go env -w GOPROXY=https://goproxy.cn
[prometheus@host1 exporter]$ 

进入flume项目,执行安装

🏅安装依赖包

下载logrus

[prometheus@host1 flume_exporter-0.0.2]$ go get github.com/Sirupsen/logrus
go: github.com/Sirupsen/[email protected]: parsing go.mod:
	module declares its path as: github.com/sirupsen/logrus
	        but was required as: github.com/Sirupsen/logrus

下载余下依赖

[prometheus@host1 flume_exporter-0.0.2]$ go get github.com/prometheus/client_golang/prometheus &&go get github.com/prometheus/client_golang/prometheus/promhttp && go get github.com/prometheus/common/version && go get github.com/woozhijun/flume_exporter/exporter && go get gopkg.in/alecthomas/kingpin.v2

🏅指定编译目录,执行安装
[prometheus@host1 flume_exporter-0.0.2]$  export GOPATH=/home/prometheus/exporter/flume_exporter-0.0.2
[prometheus@host1 flume_exporter-0.0.2]$ go install 
go: downloading github.com/Sirupsen/logrus v1.0.6
go: downloading github.com/prometheus/client_golang v1.12.2
go: downloading github.com/prometheus/common v0.34.0
go: downloading github.com/woozhijun/flume_exporter v0.0.2
go: downloading gopkg.in/alecthomas/kingpin.v2 v2.2.6
go: downloading golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
go: downloading golang.org/x/sys v0.0.0-20220114195835-da31bd327af9
go: downloading github.com/beorn7/perks v1.0.1
go: downloading github.com/cespare/xxhash/v2 v2.1.2
go: downloading github.com/golang/protobuf v1.5.2
go: downloading github.com/prometheus/client_model v0.2.0
go: downloading github.com/prometheus/procfs v0.7.3
go: downloading google.golang.org/protobuf v1.26.0
go: downloading github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
go: downloading github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.1
go: downloading github.com/bitly/go-simplejson v0.5.0
go: downloading gopkg.in/yaml.v2 v2.4.0

查看生成的文件,统一管理

🥈启动
nohup /home/prometheus/exporter/flume_exporter/flume_exporter-0.0.2 --config-file=/home/prometheus/exporter/flume_exporter/config.yml --metric-file=/home/prometheus/exporter/flume_exporter/metrics.yml >flumexport.log 2>&1 &

config.yml 配置flume机器IP信息,为了查看效果,只能再搭建一个flume单机版了

🥉简单部署flume

下载地址
下载解压

配置java变量

[prometheus@host1 flume]$ cd conf/
[prometheus@host1 conf]$ ls
flume-conf.properties.template  flume-env.ps1.template  flume-env.sh.template  log4j.properties
[prometheus@host1 conf]$ cp flume-env.sh.template flume-env.sh
[prometheus@host1 conf]$ cp flume-conf.properties.template flume-conf.properties
[prometheus@host1 conf]$ vi flume-env.sh


定义相关参数

[prometheus@host1 conf]$ vi flume-conf.properties
[prometheus@host1 conf]$ cat flume-conf.properties
#Agent: 名称定义为 a1
#Source:可以理解为输入端,定义名称为 r1
#channel:传输频道,定义为 c1,设置为内存模式
#sinks:可以理解为输出端,定义为 s1

#命名此代理上的组件别名
a1.sources = r1
a1.channels = c1
a1.sinks = s1

#描述配置源
a1.sources.r1.type = netcat 
a1.sources.r1.bind = 192.168.56.10
a1.sources.r1.port = 36001

#描述接收器
#设置 Sink 为 logger 模式 
a1.sinks.s1.type = logger 

#设置 channel 信息 
a1.channels.c1.type = memory 
#内存模式 
a1.channels.c1.capacity = 1000 
#通道中最大的缓冲事件,要大于或等于batchSize的数
a1.channels.c1.transactionCapacity = 100   

#将源和接收器绑定到通道
a1.sources.r1.channels  =  c1 
a1.sinks.s1.channel  =  c1
[prometheus@host1 conf]$ 

启动flume

nohup /home/prometheus/flume/bin/flume-ng agent -n a1 -c conf -f /home/prometheus/flume/conf/flume-conf.properties -Dflume.monitoring.type=http -Dflume.monitoring.port=36001 -Dflume.root.logger=INFO,console &

🥈promethues配置flume_exporter

prometheus.yml 添加配置信息

# set flume export
  - job_name: flume
    file_sd_configs:
      - files:
        - '/home/prometheus/exporter/flume_exporter/flume-nodes.json'
    metric_relabel_configs:
      - source_labels: [192.168.56.10]
        regex: '192.168.56.10:36001'
        replacement: flume-92.168.56.10-36001
        target_label: logs
        action: replace

创建json配置文件

[prometheus@host1 prometheus]$ cd /home/prometheus/exporter/flume_exporter
[prometheus@host1 flume_exporter]$ vi flume-nodes.json
[prometheus@host1 flume_exporter]$ cat flume-nodes.json 
[{
    "targets": ["192.168.56.10:9360"],
    "labels": {
        "alias": "flume1",
        "job": "flume"
    }
}]
[prometheus@host1 flume_exporter]$ 
🥉加载验证
[prometheus@host1 prometheus]$ curl -X POST http://192.168.56.10:9090/-/reload
[prometheus@host1 prometheus]$ 

监控目标


🥉influx查看flume指标数据

数据并未写入influxdb,看页面,还需要创建telegraf,而且上面配置的api应该也不对,里面有v1,实际上安装的是v2

🏅 安装telegrafs

下载

wget https://dl.influxdata.com/telegraf/releases/telegraf-1.22.3_linux_amd64.tar.gz

登录influxdb,拷贝前面定义的telegrafs的内容,添加到telegraf.conf配置文件,同时配置环境变量

export INFLUX_TOKEN=-jExfGZAinmMYdY98xQLuKK3xtuJdjvFasLBq_X8wb_gtnkgdHozcb0ph-gW-8C8xFtx3JFIM7irEwKe2kjkxw==



telegraf.conf 内容(增加了inputs.prometheus和inputs.http_listener_v2)

[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  hostname = ""
  omit_hostname = false
[[outputs.influxdb_v2]]
  urls = ["http://192.168.56.10:8086"]
  token = "$INFLUX_TOKEN"
  organization = "prometheus"
  bucket = "prometheus"
[[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false
[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs", "iso9660", "overlay", "aufs", "squashfs"]
[[inputs.diskio]]
[[inputs.mem]]
[[inputs.net]]
[[inputs.processes]]
[[inputs.swap]]
[[inputs.system]]
[[inputs.prometheus]]
  urls = ["http://192.168.56.10:9090/metrics","http://192.168.56.10:9360/metrics"]
[[inputs.http_listener_v2]]
  service_address = ":8087"
  paths = ["/receive"]
  data_format = "influx"
[influx@host1 telegraf]$ 

启动telegraf

nohup /home/influx/telegraf-1.22.3/usr/bin/telegraf --config /home/influx/telegraf-1.22.3/etc/telegraf/telegraf.conf  &
🏅 prometheus修改db配置

修改为remote_write

remote_write:
   - url: "http://192.168.56.10:8087/receive"


重新加载

[prometheus@host1 prometheus]$ curl -X POST http://192.168.56.10:9090/-/reload
🏅 再次查看flume指标数据


注意:[[inputs.prometheus]]对应下面targets,配置几个采集几个!!!

[[inputs.prometheus]]
  urls = ["http://192.168.56.10:9090/metrics","http://192.168.56.10:9360/metrics"]

🥇其它 🥈查看系统及处理器型号
[prometheus@host1 ~]$ uname -a
Linux host1 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[prometheus@host1 ~]$ 
[prometheus@host1 ~]$ lsb_release -a
LSB Version:	:core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID:	CentOS
Description:	CentOS Linux release 7.4.1708 (Core) 
Release:	7.4.1708
Codename:	Core
[prometheus@host1 ~]$ 

🥈lsb_release: 未找到命令
[prometheus@host1 ~]$ lsb_release -a
-bash: lsb_release: 未找到命令
[prometheus@host1 ~]$ yum provides lsb_release
已加载插件:fastestmirror
base                                                                                                           | 3.6 kB  00:00:00     
extras                                                                                                         | 2.9 kB  00:00:00     
mysql-connectors-community                                                                                     | 2.6 kB  00:00:00     
mysql-tools-community                                                                                          | 2.6 kB  00:00:00     
mysql80-community                                                                                              | 2.6 kB  00:00:00     
updates                                                                                                        | 2.9 kB  00:00:00     
(1/4): mysql-tools-community/x86_64/primary_db                                                                 |  86 kB  00:00:01     
(2/4): mysql-connectors-community/x86_64/primary_db                                                            |  87 kB  00:00:01     
(3/4): updates/7/x86_64/primary_db                                                                             |  15 MB  00:00:01     
(4/4): mysql80-community/x86_64/primary_db                                                                     | 211 kB  00:00:02     
Determining fastest mirrors
 * base: mirrors.bfsu.edu.cn
 * extras: mirrors.bfsu.edu.cn
 * updates: mirrors.bfsu.edu.cn
base/7/x86_64/filelists_db                                                                                     | 7.2 MB  00:00:01     
extras/7/x86_64/filelists_db                                                                                   | 277 kB  00:00:00     
mysql-connectors-community/x86_64/filelists_db                                                                 |  35 kB  00:00:00     
mysql-tools-community/x86_64/filelists_db                                                                      | 325 kB  00:00:01     
mysql80-community/x86_64/filelists_db                                                                          | 1.9 MB  00:00:01     
updates/7/x86_64/filelists_db                                                                                  | 8.4 MB  00:00:01     
redhat-lsb-core-4.1-27.el7.centos.1.i686 : LSB Core module support
源    :base
匹配来源:
文件名    :/usr/bin/lsb_release



redhat-lsb-core-4.1-27.el7.centos.1.x86_64 : LSB Core module support
源    :base
匹配来源:
文件名    :/usr/bin/lsb_release



[prometheus@host1 ~]$ sudo yum install -y redhat-lsb
🥈 解压报错

包未下载完整,网速太差,可以浏览器下载之后,丢到服务器

🥈 influxdb2.0 写入数据库必须要telegrafs嘛

看2.0API官网说明不用好像也行,问题是不知道prometheus该怎么配。Java开发、curl模拟都ok

欢迎分享,转载请注明来源:内存溢出

原文地址: https://outofmemory.cn/yw/926294.html

(0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
上一篇 2022-05-16
下一篇 2022-05-16

发表评论

登录后才能评论

评论列表(0条)

保存