Humio provides an alert feature that can be configured to send messages to external systems or internally to another Humio repository about certain parameters being exceeded, or about errors or other events that occur. Every repository has its own set of alerts.
Basically, alerts are standard Live Queries that run continuously, and can trigger whenever there is at least one row in the search result. Since triggering can be excessive, you can throttle the triggering not to be annoying.
To create an alert, go to the Alerts page and click on the + New Alert button at the top right. A box will appear asking you to give the alert a name. Enter text there and click Create.
In the right margin you’ll see the properties of the alert. You can see an example of this in Figure 1 here, with it cropped in Figure 3 further down this page. We’ve highlighted some of the labels to make it easier to reference them in this document.
The first input box is the name we gave already to the alert. You can change that here, though. This screen is the page to edit new and existing alerts. You can also add a description to the alert.
Under the name and description, there’s a section for Actions, which will be changed soon to Actions. This is where you would tell Humio what to do to alert you of some occurence you set here. This is covered on this documentation page, but further down under Actions. The next section in the right margin is called, Throttling. This is covered under Throttling below. Both Actions and Throttling will make more sense after looking at the next section on searching.
Although you will still be on the Alerts page when creating an alert, the screen will look similar to the Search page of the User Interface. This is because an alert is primarily based on a search of the repository data.
One you’re on the screen shown in Figure 1, you would enter a search query in the main, wide input box at the top left. In the screenshot here, that text is highlighted in violet. The query entered there is used to for discovering invalid users, attempts by someone to log into the server that failed. There was one event found. We clicked on it to enlarge it, so we could see the detail of the event. As you can see from the text highlighted, that it was found in the Testeroo repository. We’ve highlighted the message, which provides the user name and port all all. The parser also extracted the user name — you can see it at the bottom. It also shows that it was an attempt to log into the server through sshd.
Notice that there’s a severity field with a value of
info. This means that the message is to inform us of the attempt and that the hacker did not get in. We might adjust our query to show only events with a higher severity value before alerting us.
When you’re satisfied with the query and your settings, click on + Create Alert button to save the alert. That button will change to a grayed Save Changes. If you make any changes, it will become an active, purple button saying Save Changes still.
After you create an alert, in the right margin you will see a section about Actions — which was previously labeled, Notifiers. Initially, you’ll have no actions. To create one, you could click on the link it provides to create one. The other way is to click on the Alerts tab at the top. There you’ll see a page that has a tab on the left for Alerts and another on the left for Actions. From there you would click on + New Action.
On the Edit Action screen, the first thing you’ll be asked is to choose the type of action. There are a few choices available, which are described in the Actions section of the Humio Documentation. For this introduction to the User Interface, we chose Email. See the screenshot in Figure 2 here, where we highlighted some of the text.
For the email type, you have to give the action a name and an email address for the recipient. We chose to use a custom email subject and message. This allows us to use the variables presented at the bottom of that screen. These choices are available for several types of actions. You can see that we set it to put the name of the alert in the subject field. We set it to put the name of the repository, the value of the
severity field, and the number of occurences in the message of the email.
When you’re satisfied with the settings for an Action, click on the large purple button at the bottom left to save it. You will see it in the list of Actions under the _Alerts_tab. There you can edit actions, duplicate or delete them, or add new ones.
As soon as the search query for the alert returns a row, it can trigger an action. Some queries may find multiple events, return multiple rows in a short period of time. Unless you want the alerts to trigger an action for each event and row it finds, you should throttle the alert. By throttling, you can tell it to alert you only one time per set period. Otherwise, you might get notified of a problem dozens of times within a couple of minutes, depending on the situation.
The first radio button choice says to throttle all alert actions. Once an alert is triggered, no more actions will be taken during select in the throttle period. In Figure 3 above, it’s set for one hour. In Figure 4 here you can see the choices available.
The other radio button says to throttle only alert actions with identical field values. If you select this, an input box will become visible. In it you would enter the field and its value on which to determine the limit of the throttle. Field names are found in the left margin. For the example used in these screenshots, we might enter,
severity. On this basis, it will notify us when anything triggers it the first time, but not again during the throttle period (i.e., for an hour) unless some event meeting the search criteria of a different severity occurs.
Incidentally, once an alert has notified you, if resolve the problem about which its notified you may take longer than the throttle period, you could edit the alert and disable it by unchecking the box under Alert Enabled (see Figure 3 above).