投稿
  1. 首页
  2. 随笔

ELK7.15.1 部署LINUX

nes游戏 • 2022-12-17 • 随笔 • 阅读 15

ELK7.15.1 部署LINUX,第1张

ELK7.15.1 部署LINUX

1、安装包下载

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.1-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/logstash/logstash-7.15.1-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-7.15.1-linux-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.15.1-linux-x86_64.tar.gz

2.解压安装包

tar -xzf elasticsearch-7.15.1-linux-x86_64.tar.gz
tar -xzf logstash-7.15.1-linux-x86_64.tar.gz
tar -xzf kibana-7.15.1-linux-x86_64.tar.gz
tar -xzf filebeat-7.15.1-linux-x86_64.tar.gz

3、创建用户
因为es和kibana必须用非root用户登录

groupadd es
useradd es_user -g es
chown -R es_user:es elasticsearch-7.15.1 
chown -R es_user:es kibana-7.15.1

4、安装配置elasticsearch

cd elasticsearch-7.15.1 
vim config/elasticsearch.yml

插入以下内容

#更换存储文件位置以及logs位置
path.data: /mnt/elasticsearch/data 
path.logs: /mnt/elasticsearch/logs
#禁止交换分区
bootstrap.memory_lock: false

network.host: 0.0.0.0

http.port: 9200
#为了让elasticsearch-head插件可以访问es
http.cors.enabled: true
http.cors.allow-origin: "*"
cluster.initial_master_nodes: ["elk"]
node.name: elk

修改配置 如果你有JAVA jdk 是11或以上可忽略此步

vim elasticsearch-7.15.1/bin/elasticsearch-env
# 在 set -e -o pipefail 下一行插入如下内容
export JAVA_PATH=/www/server/elasticsearch-7.15.1/jdk

修改系统文件

vim /etc/security/limits.conf 
# 在文件中插入如下内容 给 es_user 赋予更多 *** 作空间（否则它施展不开拳脚）
es_user soft nofile 65535
es_user hard nofile 65535
es_user soft nproc 4096
es_user hard nproc 4096

vi /etc/sysctl.conf
# 文件中写入如下内容 增加最大运行内存
vm.max_map_count=262144

刷新内存

sysctl -p

开启ES对外默认端口 9200

firewall-cmd --zone=public --add-port=9200/tcp --permanent
#重启防火墙
firewall-cmd --reload

配置注册es服务

cd /usr/lib/systemd/system
vim elasticsearch.service
//插入以下内容
[Unit]
Description=elasticsearch
After=network.target

[Service]
Type=forking
User=es_user
#自己更改es位置
ExecStart=/www/server/elasticsearch-7.15.1/bin/elasticsearch -d
PrivateTmp=true
# 指定此进程可以打开的最大文件数
LimitNOFILE=65535
# 指定此进程可以打开的最大进程数
LimitNPROC=65535
# 最大虚拟内存
LimitAS=infinity
# 最大文件大小
LimitFSIZE=infinity
# 超时设置 0-永不超时
TimeoutStopSec=0
# SIGTERM是停止java进程的信号
KillSignal=SIGTERM
# 信号只发送给给JVM
KillMode=process
# java进程不会被杀掉
SendSIGKILL=no
# 正常退出状态
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

设置自启

systemctl enable elasticsearch.service

启动es

# 启动服务
service elasticsearch start

测试，如下证明启动成功

curl 127.0.0.1:9200
{
  "name" : "elk",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "EVntMHqzS2WEsFSsUoQDMQ",
  "version" : {
    "number" : "7.15.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "83c34f456ae29d60e94d886e455e6a3409bba9ed",
    "build_date" : "2021-10-07T21:56:19.031608185Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

5.kibana安装
修改kibana.yml配置文件

vim /www/server/kibana-7.15.1/config/kibana.yml

修改以下参数

# 端口
server.port: 5601
# 服务地址
server.host: "0.0.0.0"
# 服务名称
server.name: "bp-kibana"
# elasticsearch服务地址
elasticsearch.hosts: ["http://localhost:9200"]
# 设置为简体中文
i18n.locale: "zh-CN"

打开5601端口

firewall-cmd --permanent --zone=public --add-port=5601/tcp
#重启防火墙
systemctl reload firewalld

注册服务

vim /usr/lib/systemd/system/kibana.service

填入以下内容

[Unit]
Description=Kibana
[Service]
LimitNOFILE=100000
LimitNPROC=100000
ExecStart=/www/server/kibana/bin/kibana
User=es_user
Group=es
[Install]
WantedBy=multi-user.target

设置开机启动

systemctl enable kibana.service

相关命令

# 启动服务
systemctl start kibana.service
# 重启服务
systemctl restart kibana.service
# 停止服务
systemctl stop kibana.service
# 禁止开机启动
systemctl disable kibana.service 
# 启用开机启动
systemctl enable kibana.service

测试
访问你自己本机外网ip:5601 例如192.168.1.255:5061看到以下界面

6 logstash安装
logstash不用创建用户直接用root用户安装就好

cd logstash-7.15.1/
vim config/logstash.yml
编辑以下内容
node.name: cpy04.dev.xjh.com    #设置节点名称，一般写主机名
path.data: /mnt/logstash    #创建logstash 和插件使用的持久化目录
config.reload.automatic: true    #开启配置文件自动加载
config.reload.interval: 10    #定义配置文件重载时间周期
http.host: "XXX.com"    #定义访问主机名，一般为域名或IP

配置logstash input 段
在config下编写最后以.conf结尾

input {
  beats {
    port => "5044"
  }
}
filter {
   if [fields][doc_type] == 'order' {
    grok {
			match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{JAVALOGMESSAGE:msg}" }
		}
   }

   if [fields][doc_type] == 'customer' { 
    grok {
			match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{JAVALOGMESSAGE:msg}" }
		}
   }
}

output {
  stdout { codec => rubydebug }
  elasticsearch {
        hosts => [ "127.0.0.1:9200" ]
        index => "%{[fields][doc_type]}-%{+YYYY.MM.dd}"
    }
}

配置服务启动
1、修改config目录下的 startup.optins

################################################################################
# These settings are onLY used by $LS_HOME/bin/system-install to create a custom
# startup script for Logstash and is not used by Logstash itself. It should
# automagically use the init system (systemd, upstart, sysv, etc.) that your
# Linux distribution uses.
#
# After changing anything here, you need to re-run $LS_HOME/bin/system-install
# as root to push the changes to the init script.
################################################################################

# Override Java location
#JAVACMD=/usr/bin/java

# Set a home directory
LS_HOME=/www/server/logstash

# logstash settings directory, the path which contains logstash.yml
LS_SETTINGS_DIR=/www/server/logstash/config

# Arguments to pass to logstash
LS_OPTS="--path.settings ${LS_SETTINGS_DIR} -f ${LS_HOME}/conf.d"

# Arguments to pass to java
LS_JAVA_OPTS=""

# pidfiles aren't used the same way for upstart and systemd; this is for sysv users.
LS_PIDFILE=/var/run/logstash.pid

# user and group id to be invoked as
LS_USER=root
LS_GROUP=root

# Enable GC logging by uncommenting the appropriate lines in the GC logging
# section in jvm.options
LS_GC_LOG_FILE=/var/log/logstash/gc.log

# Open file limit
LS_OPEN_FILES=16384

# Nice level
LS_NICE=19

# Change these to have the init script named and described differently
# This is useful when running multiple instances of Logstash on the same
# physical box or vm
SERVICE_NAME="logstash"
SERVICE_DEscriptION="logstash"

# If you need to run a command or script before launching Logstash, put it
# between the lines beginning with `read` and `EOM`, and uncomment those lines.
###
## read -r -d '' PRESTART << EOM
## EOM

执行

 bin/system-install

显示如图

启动Logstash服务
设置服务自启动：systemctl enable logstash
启动服务：systemctl start logstash
停止服务：systemctl stop logstash
重启服务：systemctl restart logstash
查看服务状态：systemctl status logstash

7 filebeat安装配置
配置filebeat.yml文件

filebeat.inputs:
- paths:
    - /mnt/logs/order.log (自己找到要记录的log文件)
  multiline:
      pattern: ^d{4}
      negate: true
      match: after
  fields:
    doc_type: order
- paths:
    - /mnt/logs/customer.log
  multiline:
      pattern: ^d{4}
      negate: true
      match: after
  fields:
    doc_type: customer
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
output.logstash:
  hosts: ["127.0.0.1:5044"]

配置自启动

vi /usr/lib/systemd/system/filebeat.service

[Unit]
Description=filebeat
Wants=network-online.target
After=network-online.target
[Service]
User=root
ExecStart=/www/server/filebeat/filebeat -e -c /www/server/filebeat/filebeat.yml
Restart=always
[Install]
WantedBy=multi-user.target

注意：如果不知道filebeat的路径可以用find查找
例：# find / -name filebeat.yml
就会找到：/www/server/filebeat/filebeat.yml
启动

systemctl daemon-reload              #加载配置    
systemctl start filebeat             #启动filebeat服务   
systemctl enable filebeat            #设置开机自启   
systemctl list-units --type=service        #查看所有已启动的服务   
filebeat.service        loaded active running filebeat          #如果有这一行就证明设置成功


恭喜您看到这里就成功配置完ELK服务环境了.现在我们登录kibana看下吧!
浏览器打卡127.0.0.1:5601地址有密码的输入密码没有的就接着看

1登录成功后点击设置=>索引管理
可以看到logstash上传过来的文件

2点击 设置=>kibana=>索引模式

这里需要创建索引才能做到可视化处理哦!!!


创建完成后点击 首页=>discover

很好终于大功告成了,赶紧去程序里嵌入相关log配置吧!!!困了睡觉!!!

欢迎分享，转载请注明来源：内存溢出

原文地址: http://outofmemory.cn/zaji/5700148.html

赞 (0)
打赏 微信扫一扫 微信扫一扫 支付宝扫一扫 支付宝扫一扫
nes游戏 nes游戏 一级用户组
0 0
上一篇 2022-12-17
下一篇 2022-12-17

发表评论

登录后才能评论

评论列表（0条）

保存