Humio can run as distributed system in a cluster.
Root users can access the Cluster Node Administration Page in the UI from the account menu.
All nodes in the cluster run the same software that can be configured to assume a combination of different logical roles:
A node can have any combination of the three roles, and all play a part in Humio’s data ingest flow.
Below is a short overview of the node types. For more detailed explanation refer to the data ingest flow documentation page.
It can be beneficial to specialize to only assume one role since that allows you to better tune cluster performance.
An arrival node is a node that is responsible for servicing:
The arrival node role is a logical role in the sense that there is no special configuration option that “flags” node as “Arrival Node”, it simply means that incoming traffic is directed at this node (maybe by means of a load balancer, DNS, or similar), and that it is “a node that is not configured to store data or execute queries”.
Arrival nodes are usually the only nodes exposed to external systems and are placed behind a load-balancer.
A digest node is a node responsible for:
Once a segment file is completed it is passed on to storage nodes.
Digest nodes are designated by adding them to the cluster’s Digest Rules. Any node that appears in the digest rules is a Digest Node.
A storage node is a node that saves data to disk. They are responsible for:
The the data directory of a storage node is used to store the segment files. Segment files make up for the bulk of all data in Humio.
Storage nodes are configured using the cluster’s Storage Rules. Any node that appears in the storage rules is considered a Storage Node. A node that was previusly in a storage rule can still contain segment files that are used for querying.
The Storage Rules are used to configure data replication.
A cluster node is identified in the cluster by it’s UUID. The UUID is
automatically generated the first time a node is started. The UUID is stored
$HUMIO_DATA_DIR/cluster_membership.uuid. When moving/replacing a node you
can use this file to ensure a node rejoins the cluster with the same identity.