Jahed Ahmed

Formatting Nginx Access Logs as JSON

While Nginx and Apache's log format has become pretty standard in the industry, it'd be nice if I could use the same tools for processing them as all my other logs, which are in JSON.

Luckily, Nginx has support for it using the escape=json parameter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
log_format  main  escape=json
'{'
'"time_iso8601":"$time_iso8601",'
'"remote_addr":"$remote_addr",'
'"server_protocol":"$server_protocol",'
'"status": "$status",'
'"body_bytes_sent":"$body_bytes_sent",'
'"request_time":"$request_time",'
'"request_method":"$request_method",'
'"request_uri":"$request_uri",'
'"request_filename":"$request_filename",'
'"http_referrer":"$http_referer",'
'"http_user_agent":"$http_user_agent",'
'"http_x_forwarded_for":"$http_x_forwarded_for",'
'"host":"$host"'
'}';

The most obvious downside to this approach is that there are no null values. Every key will always exist with a string value, and empty values are typically denoted with a "-". So when processing, you'll need to make sure to exclude them.

Also, escape=json only escapes characters for use in JSON strings (for example, / becomes \/). The actual JSON formatting is done manually. So, when you're modifying the log_format to add or remove keys, make sure it's still outputting valid JSON. Specifically ensure commas (,) and speech marks (") are in the right place.

Thanks for reading.