Integrating Humio with Grafana For A Comprehensive Dashboarding Experience

Using the Humio Web UI, you can create dashboards that display the results of your queries in real time and in a way that makes them easy to understand. This is important, as having a comprehensive dashboard can mean the difference between spotting an important development in your infrastructure and letting it pass by unnoticed.

While we are proud of our UI, it is not your sole option for creating dashboards. We know that some of our customers like to create their dashboards in Grafana, a popular open source project that has been around since 2014. The main benefit of using Grafana as a Humio user is that Grafana allows you to combine many different data sources, including Prometheus, Sensu, Splunk, Elasticsearch, and now Humio, into the same dashboard.

While the Humio plugin for Grafana has been available as an open source project since 2017, it has recently had a major overhaul thanks to our integration team as well as contributions from our community. This update includes a lot of refactored code, bug fixes, but also some much-needed features such as support for queries that return tabular data.

Panels

A Grafana dashboard may contain many panels, each showcasing data from a different type of data source. After installing the Humio plugin, you will be able to create a Humio data source within Grafana, and start adding panels to your dashboards. These panels can then be populated with results from Humio queries, which you can type directly into the Grafana UI.

Here is a showcase of some the panels that you’ll be able to display your Humio data in.

Time Series Graphs

When you use the timechart()function in one of your queries, it creates one or more time-series based on the event data fed into the function. Time-series data can easily be plotted as graphs in Grafana by using the Graph panel type, which can be seen below.

This specific panel showcases the number of incoming connections over time across different hosts in a network. It was created by querying a Humio repository filled with Windows logs, coming from different Windows PCs on the same network. The query used can be seen below. It filters for all events that indicate a network connection, and then pipes those into timechart(), generating a time series for each unique hostname discovered in the filtered events.

event.action = "Network connection detected (rule: NetworkConnect)" |
timechart(series=host.name)

Like in the Humio UI, you are able to zoom into specific parts of your graph in Grafana, and you may plot multiple time-series from multiple different queries on the same panel.

Bar Gauges

Humio supports the use of the groupby() function in its queries, following the same concept as the GROUP BY found in SQL. The grouped data can be visualized using Grafana’s Bar Gauge panel type, as shown below.

This specific panel is generated from a HaProxy server, which sits in front of a Humio instance. It shows the average number of bytes read per Humio query job, created on that particular Humio instance. The query used to create the panel can be seen below. As the logs from HaProxy only leave details about web requests, we have to use Humio’s regex feature to filter for the events, which describe a web requesting polling from a query job. These events are then grouped by query job, summing up the number of bytes read from the job. Finally the data is grouped across the different repos on the Humio instance.

method=GET |
"status_code"=200 |
regex("/api/v1/dataspaces/(?<refdomain>.+?)/queryjobs/1-(?<jobId>.{24})",field=path) |
groupby([refdomain, jobId],  function=sum(bytes_read)) |
groupby(refdomain, function=avg(_sum))

As with all other Grafana panels, the Bar Gauge panel is visually customizable, and you can change its look and feel by editing the settings of the panel. In this example we have chosen a retro LCD look for each of our gauges.

Tables

Humio generates tabular data, when the table() function is used within a query. Naturally, this type of data can be displayed in Grafana’s Table panel type, as shown below. As a neat feature, if you do not want to use the header names generated by the query, Grafana allows you to change them through the Grafana UI.

This particular panel was created from the query below, which looks at Windows logs in a Humio repository. First, we use filters to get ahold of all the unsuccessful login attempts. Afterwards we group the logons by username before finally piping the grouped data into the aforementioned table() function.

winlog.task = Logon |
winlog.keywords[0] = "Audit Failure" |
groupby(winlog.event_data.TargetUserName) |
table([_count, winlog.event_data.TargetUserName], sortby=_count, limit=5)

Single Stats

While some queries produce very simple results, they can still be very useful. As an example, take queries that use the count() function, which only returns a single number. When displaying this in a Singlestat panel, as shown below, you can add thresholds values. Now each time the value breaks a threshold the panel will change color.

Again, this panel has been populated by querying a Humio repo filled with Windows logs. It uses a few filters to get all unsuccessful login attempts, and then pipes these into count().

winlog.task = Logon  |
winlog.keywords[0] = "Audit Failure"  |
count()

Worlmap

Humio has a great feature in the worldmap() function, which can take input from log events, such as IP addresses, and map them into a data format that can be visualized on a map.

While Grafana doesn’t have a native Worldmap panel, there is a third party Worlmap plugin. This can be combined with the Humio plugin to showcase locational data on your Grafana dashboard, as shown below.

The query to create this panel was very simple. It only required us to filter on http events and then pipe those into the worldmap() function.

type=http | worldmap(ip=client_ip)

Please note that in order to get the results shown on the panel above, you will need to configure the panel a bit. This is however covered in our documentation.

As you can see, there are many options for displaying Humio query results within Grafana. While this was not a comprehensive listing, we hope this inspired you to try out the plugin and see what it can do.

To help you we’ve written some new documentation for the plugin. The code is still being improved upon, so if you find any issues or would like to contribute, please get in touch with us through the Github repository.

Happy Dashboarding!

For more information