Docker

In this guide, we assume that you use Docker in the standard way, where logs are captured from stdout and stderr.

Looking for how to run Humio in a Docker container? Try the Docker installation guide instead.

Container Logs

As of Humio version 1.2.6 we now have full support for the Docker Splunk logging driver.

Getting logs from a Docker container is as simple as setting the logging driver and adding the splunk-url and splunk-token logging options to the container, i.e.

docker run --rm -it \
  --log-driver=splunk \
  --log-opt splunk-url=$BASEURL \
  --log-opt splunk-token=$INGEST_TOKEN \
  alpine ping 8.8.8.8

Where:

  • $BASEURL - is the base URL of your Humio server (e.g. https://cloud.humio.com or http://localhost:8080)
  • $INGEST_TOKEN - is the ingest token for your repository, (e.g. a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

Parsing the logs

Since Docker just handles log lines from stdout as text blobs, you must parse the lines to get the full value from them.

To do this, you can either use a built-in parser, or create new ones for your log types. For more details on creating parsers, see the parsing page.

In terms of log management, Docker is just a transport layer. Before writing a custom parser, see the built in parsers page to see if Humio already supports your log type.

Configuring Docker daemon

To configure the Docker daemon to forward all logs for all containers by default you’ll have to update the daemon.json configuration file with the following parameters

{
  "log-driver" : "splunk",
  "log-opts" : {
    "splunk-token" : "$INGEST_TOKEN",
    "splunk-url" : "$BASEURL"
  }
}

Don’t forget to restart Docker daemon.

To excluding from log forwarding you can run your container with the default json-file logging driver, i.e.

docker run --log-driver=json-file --rm alpine whoami

Notes on blocking behaviour

By default Docker logging drivers are blocking, meaning that it will prevent the process from printing to stdout and stderr while logs are being handled. This can, and should be, controlled by the mode log-opt.

In addition to the mode, the Splunk logging driver has it’s own buffer, which will postpone the process pausing somewhat.

Finally it should be noted that Docker will throw away the oldest logs in non-blocking mode when the buffer runs full.

Docker daemon Metrics

To get standard host level metrics for your docker containers, use Metricbeat. It includes a docker module.

Example Metricbeat Configuration

metricbeat.modules:
  - module: docker
    metricsets: ["cpu", "info", "memory", "network", "diskio", "container"]
    hosts: ["unix:///var/run/docker.sock"]
    enabled: true
    period: 10s

output.elasticsearch:
  hosts: ["$BASEURL/api/v1/ingest/elastic-bulk"]
  username: $INGEST_TOKEN
Where:

  • $BASEURL - is the base URL of your Humio server (e.g. https://cloud.humio.com:443 or http://localhost:8080)
  • $INGEST_TOKEN - is the ingest token for your repository, (e.g. a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii).

See also the page on Beats for more information.