Jahed Ahmed

Formatting Nginx Access Logs as JSON

While Nginx and Apache's log format has become pretty standard in the industry, it'd be nice if I could use the same tools for processing them as all my other logs, which are in JSON.

Luckily, Nginx has support for it using the escape=json parameter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
log_format  main  escape=json
'{'
'"time_iso8601":"$time_iso8601",'
'"remote_addr":"$remote_addr",'
'"server_protocol":"$server_protocol",'
'"status": "$status",'
'"body_bytes_sent":"$body_bytes_sent",'
'"request_time":"$request_time",'
'"request_method":"$request_method",'
'"request_uri":"$request_uri",'
'"request_filename":"$request_filename",'
'"http_referrer":"$http_referer",'
'"http_user_agent":"$http_user_agent",'
'"http_x_forwarded_for":"$http_x_forwarded_for",'
'"host":"$host"'
'}';

The most obvious downside to this approach is that there are no null values. Every key will always exist with a string value, and empty values are typically denoted with a "-". So when processing, you'll need to make sure to exclude them.

Also, escape=json only escapes characters for use in JSON strings (for example, / becomes \/). The actual JSON formatting is done manually. So, when you're modifying the log_format to add or remove keys, make sure it's still outputting valid JSON. Specifically ensure commas (,) and speech marks (") are in the right place.

Thanks for reading.

Support My Work
Related Tags