Why
The company purchased a third-party service and deployed it on the company's machine in the form of a jar package.This service is on a critical data link, and the concurrency is very high. The service http request appears 500, which is what we cannot receive.
Even without the source code, we also need to monitor HTTP requests to provide detailed information to the provider.
Ideas to Solve the Problem
My first thought was to use java agent to monitor.Java agents are a special type of class which, by using the Java Instrumentation API, can intercept applications running on the JVM, modifying their bytecode.
Another solution is to capture network traffic. My core requirement is to monitor requests with a status code of 5xx, so capturing traffic is more suitable.
Capture network traffic
Tcpdump can capture traffic but needs parsing to be more suitable for reading.
httpry is a tool designed for displaying and logging HTTP traffic.
Nice!This is the tool I was looking for.
Install httpry on Centos
sudo yum install epel-release
sudo yum install httpry
Filter Response with a Status Code other than 200
sudo httpry -m post -f timestamp,source-ip,dest-ip,direction,status-code -o test.log 'tcp dst port 80 and tcp[((tcp[12:1] & 0xf0) >> 2)+8:4] != 0x20323030'
The output in test.log like this:
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 403
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 403
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
2022-09-01 19:22:34 192.168.0.1 192.168.0.2 < 500
Manager Large Data Volume Output File
All the information is typed into one file, this file will be very large.
You need another tool:logrotate.
I want to organize the files by date and I create the file logrotate.conf
/xxxx/xxxxx/http.log {
notifempty
copytruncate
dateext
dateformat .%Y%m%d
olddir /xxx/backup
}
The following command can debug the process
logrotate -d xxx/lograte.conf
Then add timed tasks through ctontab，crontab will execute the rotate according to the config and We will get a file named http.log.20220815.