Using Nginx to access logs and metrics in Humio lets you follow what is happening in Nginx in great detail
To ship Nginx access logs to Humio, use Filebeat.
On Linux, the access log is in /var/log/nginx/access.log
filebeat.inputs:
- paths:
- /var/log/nginx/access.log
fields:
"@type": accesslog
output.elasticsearch:
hosts: ["$YOUR_HUMIO_URL/api/v1/ingest/elastic-bulk"]
username: my-organization
password: $INGEST_TOKEN
The $YOUR_HUMIO_URL
variable is the base URL of your Humio server, either (https://cloud.humio.com:443
or http://localhost:8080
). The $INGEST_TOKEN
is the ingest token for your repository, (i.e., a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii
). See the page on Filebeat for further details.
The above Filebeat configuration uses the built-in parser accesslog
. The parser can parse logs formatted in the default Nginx log configuration. If your log Nginx configuration is modified, create a custom parser, by copying the accesslog parser and modifying it. Then connect the parser to the ingest token or put its name as the value of the @type field in the Filebeat configuration.
Response time: By default Nginx does not include response time in the log. Response time can be added by editing the nginx logging configuration (nginx.conf). Add the field $request_time
to the log_format. Read more about logging responsetime and other performance metrics here
#type=accesslog | groupby(statuscode) | sort()
#type=accesslog statuscode >= 400 | timechart(statuscode)
#type=accesslog | timechart(function=percentile(responsetime, percentiles=[50, 75, 90, 99, 100]))
Unfortunately responsetime for each request is not part of the default Nginx logging. See the tip above on how to add it.
Show top 5 referring web sites
#type=accesslog | regex("https?://(?<domain>[^:/]+)", field=referrer) | groupBy(domain) | sort(limit=10)
Field extraction at search time: The regex( ) function extracts a new field domain
and captures the domain part of the referrer URL. The field is then used later in the query pipeline.
To get connection-related metrics from Nginx, use Metricbeat. It includes an Nginx module that uses the http_stub_status_module
module in Nginx to collect metrics.
You can check if the http_stub_status_module
module is enabled by running
this command:
$ nginx -V 2>&1 | grep -o
with-http_stub_status_module
If the command produces output, then the module is enabled.
Ensure that the http_stub_status_module
module is exposed by adding the following
configuration to Nginx:
server {
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
}
This ensures that the http_stub_status_module
module is only accessible from localhost.
metricbeat.modules:
- module: nginx
metricsets: ["stubstatus"]
enabled: true
period: 10s
hosts: ["http://127.0.0.1/nginx_status"] # Nginx hosts
- module: system
enabled: true
period: 10s
metricsets: ["process"]
processes: ['.*nginx.*']
output.elasticsearch:
hosts: ["$YOUR_HUMIO_URL/api/v1/ingest/elastic-bulk"]
username: my-organization
password: $INGEST_TOKEN
The $YOUR_HUMIO_URL
variable is the base URL of your Humio server, either (https://cloud.humio.com:443
or http://localhost:8080
). The $INGEST_TOKEN
is the ingest token for your repository, (i.e., a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii
).
See also the page on Metricbeat for more information.