Backup

Humio has a built-in backup facility. It requires a separate directory for Humio to write to, preferably on another disk or network drive from the data directory. When configured, Humio keeps a full backup of the current state in this directory.

A Humio node can start with no data and restore the previous state from such a backup.

Backup Encryption

The files written on the backup drive are encrypted using a secret provided during Humio’s configuration, and allow you to store the backup on a network drive where others may have read access.

Configuring Backup

You can back up your Humio installation by adding a special mounted directory when you run the Docker container. Humio writes its backup files to this directory.
Each Humio node can have its own backup directory that is not shared with the other nodes. This is possible since it is not always feasible to create one backup directory that can hold all the data in a Humio cluster. In the backup directory each Humio node will create a subdirectory with its node ID as name and use this directory.

  1. Create an empty directory on the host machine to store data for Humio:

    mkdir /humio-backups-on-host
    

    We recommend creating the backup directory on a different disk from the main Humio data directory. Make the directory a mount point for a network drive or other similar separation from the main data drive.

  2. Edit the Humio configuration file to set the backup parameters. Add the following lines:

    BACKUP_NAME=humio-backup
    BACKUP_KEY=mysecretkey-myhost-+R+q(AB9QG86xZMCKGyj
    

    Humio encrypts all backups with a secret key that you provide. This means that you can safely store backups on an unencrypted disk, or send them over the Internet. Keep the secret key safe and store it in another place. You cannot recover the backup if you lose access to it! If you lose the secret, delete all the files in the backup, or provide a new location to back up to, and start over. Humio will then write a fresh backup.

  3. Run Humio using the Docker run command. Add the following argument to the command, which maps the backups directory on the host (here, /humio-backups-on-host) to the /backup directory in the container:

    -v /humio-backups-on-host:/backup
    

    Humio will start backing up data to the specified directory.

Not running Docker?

The procedure is mostly the same. Instead of mounting the directory using “-v”, you specify the location using “BACKUP_DIR”. A full example configuration is then:

BACKUP_NAME=humio-backup
BACKUP_KEY=mysecretkey-myhost-+R+q(AB9QG86xZMCKGyj
BACKUP_DIR=/mnt/my-net-server/humio-backup01

Deleting data in backup

When is data deleted in backup? Right when retention kicks in and deletes data in Humio. It is possible to configure a delay so that data is not deleted in the backup until some time has elapsed since it was deleted in Humio. This is configured using “DELETE_BACKUP_AFTER_MILLIS”. By default Humio will not delete data in the backup until seven days after the data was deleted in Humio.

DELETE_BACKUP_AFTER_MILLIS=604800000

Restoring From Backup

Humio can restore all events that were stored in segment files for a Humio node in the humio-data dir from the backup. It can also help a node claim the nodeID of a lost node in a cluster.

If you have lost an entire node with the humio-data dir

The node is lost, but you have reinstalled the OS on the existing hardware, or perhaps found a spare server that should now take on the role of the lost Humio node. To do so, create humio-data and place a copy of the uuid file from the lost node — this file is present in the folder

/backup/BACKUP-NAME/globaldata/cluster_membership-NODEID-UUID.uuid

copy this file to:

/data/humio-data/cluster_membership.uuid

Restoring lost segment files

Humio restores missing segment files when it discovers they are missing, and are present in the backup folder.

Third Party Backup Software

Humio is designed to support other strategies like Amazon S3 Backup. You can do backup using any backup software that is able to back up all the files in the Humio data directory. Note that the software need to support “sparse files” to be efficient.