ELK--搜集Tomcat日志（实战）

2023年12月7日 1144点热度 0人点赞 0条评论

通过EKL收集 tomcat 日志：

收集两个服务器的tomcat 服务日志

收集 Tomcat 服务器的访问日志以及 Tomcat 错误日志进行实时统计，在 kibana页面进行搜索展现，每台 Tomcat 服务器要安装 logstash 负责收集日志，然后将日志转发给 elasticsearch 进行分析，在通过 kibana 在前端展现，配置过程如下

服务器部署 tomcat 服务：

需要安装 java 环境，并自定义一个 web 界面进行测试。

配置 java 环境并部署 tomcat

两个tomcat服务器都要这么配置我只演示一个

[root@web1 ~]# yum -y install java-1.8.0-openjdk.x86_64
[root@web1 ~]# mkdir /apps/
[root@web1 ~]# cd /apps/
`上传文件`
[root@web1 apps]# ll
total 10280
-rw-r--r-- 1 root root 10523269 Mar 30  2021 apache-tomcat-8.5.65.tar.gz
[root@web1 apps]# tar xf apache-tomcat-8.5.65.tar.gz
[root@web1 apps]# chown -R root:root /apps/apache-tomcat-8.5.65/
[root@web1 apps]# ln -sv /apps/apache-tomcat-8.5.65/ /apps/tomcat
[root@web1 apps]# sed -i.bak '$a export TOMCAT_HOME=/apps/tomcat\nexport PATH=$TOMCAT_HOME/bin:$PATH' /etc/profile
[root@web1 apps]# source /etc/profile
`修改配置文件将tomcat的日志转为json格式`
[root@web1 apps]# vim tomcat/conf/server.xml 
#文件最后几行修改下面内容
        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
               prefix="tomcat_access_log" suffix=".log"
               prefix="tomcat_access_log" suffix=".log"
               pattern="{&quot;clientip&quot;:&quot;%h&quot;,&quot;ClientUser&quot;:&quot;%l&quot;,&quot;authenticated&quot;:&quot;%u&quot;,&quot;AccessTime&quot;:&quot;%t&quot;,&quot;method&quot;:&quot;%r&quot;,&quot;status&quot;:&quot;%s&quot;,&quot;SendBytes&quot;:&quot;%b&quot;,&quot;Query?string&quot;:&quot;%q&quot;,&quot;partner&quot;:&quot;%{Referer}i&quot;,&quot;AgentVersion&quot;:&quot;%{User-Agent}i&quot;}"/>

`启动tomcat`
[root@web2 apps]# catalina.sh start
`访问tomcat页面`
http://192.168.139.36:8080/
http://192.168.139.231:8080/
`查看日志类型`
[root@web1 apps]# tail -f tomcat/logs/tomcat_access_log.2023-11-11.log
{"clientip":"192.168.139.1","ClientUser":"-","authenticated":"-","AccessTime":"[11/Nov/2023:23:02:55 +0800]","method":"GET / HTTP/1.1","status":"200","SendBytes":"11156","Query?string":"","partner":"-","AgentVersion":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"}

[root@web2 apps]# tail -f tomcat/logs/tomcat_access_log.2023-11-11.log 
{"clientip":"192.168.139.1","ClientUser":"-","authenticated":"-","AccessTime":"[11/Nov/2023:23:03:04 +0800]","method":"GET / HTTP/1.1","status":"200","SendBytes":"11156","Query?string":"","partner":"-","AgentVersion":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"}
{"clientip":"192.168.139.1","ClientUser":"-","authenticated":"-","AccessTime":"[11/Nov/2023:23:03:04 +0800]","method":"GET /tomcat.svg HTTP/1.1","status":"200","SendBytes":"67795","Query?string":"","partner":"http://192.168.139.231:8080/","AgentVersion":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36"}

4.2.1.2）验证日志是否是 json 格式

在 tomcat 服务器安装 logstash 收集 tomcat 和系统日志：

需要部署 tomcat 并安装配置 logstash

两个tomcat服务器都要这么配置我只演示一个

安装logstash

`上传rpm包`
[root@web1 apps]# ll
total 371308
drwxr-xr-x 9 root root       220 Nov 11 22:50 apache-tomcat-8.5.65
-rw-r--r-- 1 root root  10523269 Mar 30  2021 apache-tomcat-8.5.65.tar.gz
-rw-r--r-- 1 root root 369691684 Nov 11 18:08 logstash-7.12.1-x86_64.rpm
lrwxrwxrwx 1 root root        27 Nov 11 22:50 tomcat -> /apps/apache-tomcat-8.5.65/
[root@web1 apps]# yum -y localinstall logstash-7.12.1-x86_64.rpm 
`修改配置文件`
[root@web1 apps]# vim /etc/logstash/conf.d/tomcat.conf
input {
  #stdin{} 
  file {
    path => "/apps/tomcat/logs/tomcat_access_log.*.log"
    type => "logstash-stangj-tomcat-access-log"
    start_position => "beginning"
    stat_interval => "3"
  }
  file {
    path => "/var/log/messages"
    type => "logstash-stangj-messages-log"
    start_position => "beginning"
    stat_interval => "3"
  }

} 
output { 
  #stdout{} 
  if [type] == "logstash-stangj-tomcat-access-log" {  
    elasticsearch {
      hosts => ["192.168.139.31:9200","192.168.139.32:9200","192.168.139.33:9200"]
      index => "logstash-stangj-tomcat-access-%{+YYYY.MM.dd}"
    }
  }
  if [type] == "logstash-stangj-messages-log" {
    elasticsearch {
      hosts => ["192.168.139.31:9200","192.168.139.32:9200","192.168.139.33:9200"]
      index => "logstash-stangj-messages-log-%{+YYYY.MM.dd}"
    }
  }
}
`检查配置`
[root@web1 apps]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/tomcat.conf -t
`修改service文件以root用户身份启动`
[root@web1 apps]#  vim /etc/systemd/system/logstash.service
......
User=root
Group=root
......
`启动服务`
[root@web1 apps]# systemctl start logstash.service 
`刷新tomcat页面`
http://192.168.139.36:8080/
http://192.168.139.231:8080/

在elasticsearsh的head插件查看索引情况

在kibana创建索引模式

创建logstash-stangj-messages-log索引模式

创建logstash-stangj-tomcat-access-log索引模式

查看索引日志信息

修改日志信息看kibana是否可以搜索捕获

[root@web1 ~]# vim /apps/tomcat/logs/tomcat_access_log.2023-11-12.log

成功捕获