Docker

In this guide, we assume that you use Docker in the standard way, where logs are captured from stdout and stderr. Looking for how to run Humio in a Docker container? Try the Docker installation guide instead.

Humio has full support for the Docker Splunk logging driver. Getting logs from a Docker container is as simple as setting the logging driver and adding the splunk-url and splunk-token logging options to the container

docker run --rm -it \
  --log-driver=splunk \
  --log-opt splunk-url=$YOUR_HUMIO_URL \
  --log-opt splunk-token=$INGEST_TOKEN \
  alpine ping 8.8.8.8

The $YOUR_HUMIO_URL variable is the base URL of your Humio server, either Humio Cloud or self-hosted. The $INGEST_TOKEN is the ingest token for your repository, (i.e., a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

Parsing Logs

Since Docker handles log lines from stdout as text blobs, you must parse the lines to get the full value from them. To do this, you can either use a built-in parser, or create new ones for your log types. For more details on creating parsers, see Parsers.

In terms of log management, Docker is a transport layer. Before writing a custom parser, see Built-in Parsers to see if Humio already supports your log type.

Configuring Docker Daemon

To configure the Docker daemon to forward all logs for all containers by default you’ll have to update the daemon.json configuration file with the following parameters

{
  "log-driver" : "splunk",
  "log-opts" : {
    "splunk-token" : "$INGEST_TOKEN",
    "splunk-url" : "$YOUR_HUMIO_URL"
  }
}

When finished, restart the Docker daemon.

To exclude from log forwarding, you can run your container with the default json-file logging driver

docker run --log-driver=json-file --rm alpine whoami

By default, Docker logging drivers are blocking, meaning that they will prevent the process from printing to stdout and stderr while logs are being handled. This can, and should be, controlled by the mode log-opt.

In addition to the mode, the Splunk logging driver has it’s own buffer, which will postpone the process pausing somewhat. Also, Docker will discard the oldest logs in non-blocking mode when the buffer is full.

Docker Daemon Metrics

To get standard host level metrics for your docker containers, use Metricbeat. It includes a docker module.

Below is an example configuration of Metricbeat:

metricbeat.modules:
  - module: docker
    metricsets: ["cpu", "info", "memory", "network", "diskio", "container"]
    hosts: ["unix:///var/run/docker.sock"]
    enabled: true
    period: 10s

output.elasticsearch:
  hosts: ["$YOUR_HUMIO_URL/api/v1/ingest/elastic-bulk"]
  username: my-organization
  password: $INGEST_TOKEN

The $YOUR_HUMIO_URL variable is the base URL of your Humio server, either (https://cloud.humio.com:443 or http://localhost:8080). The $INGEST_TOKEN is the ingest token for your repository, (i.e., a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

See also Beats for more information.