In Fluent Bit 1.4, there is a filter called rewrite_tag
that allows you to change tags based on various conditions and enables message routing to different outputs. It can also act as a filter for which logs actually need to be forwarded to Humio.
Suppose you have some container logs that you want in the Humio logs
repository. Suppose further that some logs you consider metrics and you want them in the Humio metrics
repository. With this new filter you can do the following:
[FILTER]
Name rewrite_tag
Match kube.*
Rule $kubernetes['labels']['humio'] ^(logs)$ logs.container false
Emitter_Name logs_rewrite
[OUTPUT]
Name es
Match logs.container
Index logs
HTTP_User {{ .Values.logs.token }}
[FILTER]
Name rewrite_tag
Match kube.*
Rule $kubernetes['labels']['humio'] ^(metrics)$ metrics.container false
Emitter_Name metrics_rewrite
[OUTPUT]
Name es
Match metrics.*
Index metrics
HTTP_User {{ .Values.metrics.token }}
As a result, if a pod has a label humio: logs
it goes to the logs
repo and if the label is humio: metrics
, it goes to the metrics repository. If a pod doesn’t have any of those labels, its output is ignored.
For more information, see the Kubernetes Integration for Humio documentation page.
This use case is a community submission from Evgeni Sladkovski Trifork Amsterdam