This guide describes how to send logs from your Kubernetes cluster to your Humio cluster. If you’re looking to install Humio on Kubernetes, see our Installation Guide.
When it comes to managing micro-services in a Kubernetes cluster, Humio is a great way to get insights into your applications. While other data shippers are supported, we mainly focus on using Fluent Bit for forwarding log messages to Humio.
Take advantage of Humio with your Kubernetes setup!
We’ll start with Helm, the Kubernetes package manager. Directions for installing Helm for your particular OS can be found on the Helm Github page.
Once that is done you need to update the main Helm chart repository. This main repository contains subcharts for Humio.
helm repo add humio https://humio.github.io/humio-helm-charts
helm repo update
Next, create a file named humio-agent.yaml
with the following content:
humio-fluentbit:
enabled: true
humioHostname: $HOST
es:
tls: true
Replace $HOST
with the hostname of your Humio installation (this defaults to cloud.humio.com
).
For on-site installation, don’t forget to enable the ELASTIC_PORT
property.
Take your ingest token from your Humio Repository page…
…and replace $INGEST_TOKEN
with the ingest token when running the install command below. We recommend running in a
separate namespace, in our example the logging
namespace.
helm install -f humio-agent.yaml humio/humio-helm-charts --set humio-fluentbit.token=$INGEST_TOKEN --name humio --namespace logging
Once this is in place, the Kubernetes logs should be flowing into the Humio cluster.
Humio will look for the pod
label
humio-parser
and use that parser on the log line. This way you can
control how data is parsed by just configuring your pods. If the
humio-parser
label is not set, no parser will be applied to the log
lines.
In some cases you might want to make some changes to the Fluent Bit configuration. The easiest way to do that is by updating the values.yaml with the filter and updating the Helm chart.
First, update the values.yaml by adding a customFluentBitConfig
section containing custom
Fluent Bit configurations:
humio-fluentbit:
enabled: true
humioHostname: $HOST
es:
tls: true
customFluentBitConfig:
custom-filter-kubernetes.conf: |-
[FILTER]
Name Modify
Match *
Rename log rawstring
And then upgrade the helm chart:
helm upgrade -f humio-agent.yaml humio humio/humio-helm-charts
helm delete --purge humio
And optionally clean up the namespace:
kubectl delete namespace logging --cascade=true