nginx 日志改为json格式

---

场景描述

正常使用nginx时，使用默认的日志输出格式，对于后续日志接入其他第三方日志收集、清洗环节，因分隔符问题可能不是很友好。

xxxx - - [19/Feb/2024:11:16:48 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
xxxx - - [19/Feb/2024:11:16:49 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
xxxx - - [19/Feb/2024:11:16:50 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
xxxx - - [19/Feb/2024:11:16:51 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
xxxx - - [19/Feb/2024:11:16:52 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
xxxx - - [19/Feb/2024:11:16:53 +0800] "GET /time_feed HTTP/1.1" 200 42 "http://sre-support.xxxx-inc.top/back" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"
1
2
3
4
5
6

nginx同时支持将日志输出为json格式，修改较小，下面为效果。

效果

{"@timestamp":"2024-02-19T11:23:09+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"}
{"@timestamp":"2024-02-19T11:23:10+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.001,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"}
{"@timestamp":"2024-02-19T11:23:11+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"}
{"@timestamp":"2024-02-19T11:23:12+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"}
{"@timestamp":"2024-02-19T11:23:13+08:00","server_addr":"10.0.0.251","remote_addr":"xxxx","host":"sre-support.xxxx-inc.top","uri":"/time_feed","body_bytes_sent":42,"upstream_response_time":0.000,"request":"GET /time_feed HTTP/1.1","request_length":349,"request_time":0.001,"status":"200","http_referer":"http://sre-support.xxxx-inc.top/max_data","http_x_forwarded_for":"","http_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36"}
1
2
3
4
5

变更

旧样式

nginx一般默认会打开日志格式化格式，我这里为默认的格式，可能有的同学使用的是自定义格式，但是都是用“ - ” 分割的。

样式为：

新样式

因本身nginx支持json格式日志输出，下面为我使用的示例，我这里增加了获取源站响应时间upstream_response_timer，所以下面先增加了upstream_response_timer取值方式。
下面的log_format展示了当前获取的字段，各位可根据自己需求进行自定义。

http {
        map $upstream_response_time $upstream_response_timer {
        default $upstream_response_time;
        ""        0;
    }
    log_format log_json escape=json '{"@timestamp":"$time_iso8601",'
                '"server_addr":"$server_addr",'
                '"remote_addr":"$remote_addr",'
                '"host":"$host",'
                '"uri":"$uri",'
                '"body_bytes_sent":$body_bytes_sent,'
                '"upstream_response_time":$upstream_response_timer,'
                '"request":"$request",'
                '"request_length":$request_length,'
                '"request_time":$request_time,'
                '"status":"$status",'
                '"http_referer":"$http_referer",'
                '"http_x_forwarded_for":"$http_x_forwarded_for",'
                '"http_user_agent":"$http_user_agent"'
                '}';
        access_log  /var/log/nginx/access.log  log_json;
        error_log /var/log/nginx/error.log error;
        ........
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

样式为：