The Elastic Beats are a great group of data shippers. They are cross-platform, lightweight, and can ship data to a number of tools including Humio.
All Beats are built using the libbeat library. Along with the official Beats, there are a growing number of community Beats.
There are currently five official Beats. The Elastic documentation site and Humio’s documentation offer resources that describe how to use each of them:
Filebeat - Ships regular log files.
Metricbeat - Ships metrics from your OS and common services.
Packetbeat - Analyzes network packets and common protocols like HTTP
Winlogbeat - Ships Windows event logs
Heartbeat - Checks system status and availability
Community Beats
In addition, the Elastic community has created many other Beats that you can download and use.
These Community Beats cover many less common use cases.
All beats are built using the libbeat library and share output configuration. Humio supports parts of the ElasticSearch ingest API, so to send data from Beats to Humio, you just use the ElasticSearch output (the documentation is identical for all Beats).
You can use the following elasticsearch
output configuration template:
output:
elasticsearch:
hosts: ["$BASEURL/api/v1/ingest/elastic-bulk"]
username: $INGEST_TOKEN
Where:
$BASEURL
- is the base URL of your Humio server (e.g. https://cloud.humio.com:443
or http://localhost:8080
)$INGEST_TOKEN
- is the ingest token for your repository, (e.g. a string such as fS6Kdlb0clqe0UwPcc4slvNFP3Qn1COzG9DEVLw7v0Ii
).To optimize performance for the data volumes you want to send, and to keep shipping latency down, change the default settings for compression_level
, bulk_max_size
and flush_interval
.
All Beats also have a fields
section in their configuration. You can add fields to all events by specifying them in the fields
section:
fields:
service: user-service
datacenter: dc-a