4. ELK整合

4. ELK整合

 祝您身体健康，前程似锦，小弟期待文章对您有帮助，也期待您的打赏:

目录

一、配置

二、查看

三. 使用一个配置文件收集多个日志

---

前提: elasticserrch、logstash、kibana组件都已经安全并单独功能测试没问题了.

一、配置

1.  配置logstash 去搬运日志的配置

vim /etc/logstash/conf.d/samanager.conf

input {

file {

path => "/usr/local/sa-managerd/log/sa-manager.log"

type => "samanager"

start_position => "beginning"

stat_interval => "2"

}

}

# logstash-6.0.0版本，配置这个filter, 这样timestamp就不会相差8个小时

filter {

date {

match => ["message","UNIX_MS"]

target => "@timestamp"

}

ruby {

code => "event.set('timestamp', event.get('@timestamp').time.localtime + 8*60*60)"

}

ruby {

code => "event.set('@timestamp',event.get('timestamp'))"

}

mutate {

remove_field => ["timestamp"]

}

}

output {

elasticsearch {

hosts => ["192.168.161.131:9200"]

index => "logstash-samanager-%{+YYYY.MM.dd}"

}

}

systemctl restart logstash

前提需要logstash用户对被收集的日志文件有读的权限并对写入的文件有写权限

2.  使用logstash配置文件收集messages日志

1. 添加如下配置:

vim /etc/logstash/conf.d/system.conf

input {

file {

path => "/var/log/messages" #日志路径

type => "systemlog" #类型,自定义，在进行多个日志收集存储时可以通过该项进行判断输出

start_position => "beginning" #logstash 从什么位置开始读取文件数据，默认是结束位置（end），也就是说 logstash 进程会以类似 tail -F 的形式运行。如果你是要导入原有数据，把这个设定改成"beginning"，logstash 进程就从头开始读取>，类似 less +F 的形式运行。

stat_interval => "2" #logstash 每隔多久检查一次被监听文件状态（是否有更新） ，默认是 1 秒。

}

}

output {

elasticsearch {

hosts => ["192.168.161.131:9200"] #指定hosts

index => "logstash-systemlog-%{+YYYY.MM.dd}" #索引名称

}

}

3. 检测配置文件是否有语法错误

[root@localhost conf.d]# vim /etc/logstash/conf.d/system.conf

[root@localhost conf.d]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/system.conf -t

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults

Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console

Configuration OK

[root@localhost conf.d]#

4. 对要监控的日志文件一定要增加如下权限，不然监控不了:

chmod 644 /var/log/messages

5. systemctl restart logstash

二、查看

三. 使用一个配置文件收集多个日志

修改logstash的配置文件，这里增加收集数据库mariadb的日志： [root@linux-node1 ~]# vim /etc/logstash/conf.d/system.conf input { file { path => "/var/log/messages" type => "systemlog" start_position => "beginning" stat_interval => "2" } file { path => "/var/log/mariadb/mariadb.log" type => "mariadblog" start_position => "beginning" stat_interval => "2" } } output { if [type] == "systemlog" { #使用if来判断类型，并输出到elasticsearch和file，展示一个out可以作多样输出 elasticsearch { hosts => ["192.168.56.11:9200"] index => "logstash-systemlog-%{+YYYY.MM.dd}" } file { path => "/tmp/logstash-systemlog-%{+YYYY.MM.dd}" }} if [type] == "mariadblog" { elasticsearch { hosts => ["192.168.56.11:9200"] index => "logstash-mariadblog-%{+YYYY.MM.dd}" } file { path => "/tmp/logstash-mariadblog-%{+YYYY.MM.dd}" }} }

配置文件检测语法是否正常：

/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/system.conf -t

重启logstash： [root@linux-node1 ~]# systemctl restart logstash

修改mariadb的日志权限： [root@linux-node1 ~]# ll /var/log/mariadb/ -d drwxr-x--- 2 mysql mysql 24 12月 4 17:43 /var/log/mariadb/ [root@linux-node1 ~]# chmod 755 /var/log/mariadb/ [root@linux-node1 ~]# ll /var/log/mariadb/mariadb.log -rw-r----- 1 mysql mysql 114993 12月 27 14:23 /var/log/mariadb/mariadb.log [root@linux-node1 ~]# chmod 644 /var/log/mariadb/mariadb.log

此时head上查看

删除数据

 （期待您上面二维码打赏，也祝您前程似锦，步步高升）