This guide will introduce you to how to ingest data and the basics of working with it in Humio.
Before we get started, make sure you have a running installation of Humio. Humio is easy to deploy. You can deploy Humio on your own servers or cloud infrastructure, or you can let Humio take care of the hosting and management in our cloud.
Once you have a running instance, go through the interactive in-app tutorial.
If you want to skip to working with your own data, go ahead to step 2
It is a 101-course that teaches you the basics of searching. You can find the tutorial in the Help menu at the top of the UI:
While the tutorial panel is open it will continuously stream simulated log data from a web server and step you through searching and visualizing the logs.
Once you have completed the tutorial move on to step 2.
You can use the UI’s built-in function documentation by hitting alt+enter on Mac/ctr+space on Windows or Linux while focusing the search field for making a query. We also have documentation and examples for all functions in the query function reference.
It is time to get your own data into Humio.
Before you can send data you need a repository to store the data in.
You can use either your sandbox repository or if you are running Humio locally you can create a new dedicated repository (make sure to pick a “Repository” and not “View” since views cannot be used for ingest).
The first step in getting your data into Humio is getting what is known as an ingest token. Ingest tokens are used to authorize clients and route incoming data to the right repository where it should be stored.
Humio will automatically create an ingest token called “default” when a new repository is created and you can find your repository’s ingest tokens by going to:
You can either use the default token or create a new one for the purpose of this tutorial.
Choose which token to use and copy the token by clicking the icon marked with number 1 in the figure.
Before you can configure your system to send data to Humio, you need to know the format of the data you want to send and assign a parser to your ingest token. Assigning a parser to a token will make any data sent using that token use the assigned parser to ingest the data.
When data arrives at Humio it needs to be parsed and you have to specify which parser should be used to interpret your data. The one exception to this rule is if you use Humio’s structured ingest API, in which case all fields and timestamp are specified as part of the data sent to Humio. For the purpose of this guide we will assume you are using something else.
A parser is a script that takes a text message (your data in any textual format) as input and produces an event as output. In Humio an Event is an object stored in it’s database. Events can contain any number of fields (key-value pairs) and are always associated with a timestamp.
You will need to pick a parser based on the format of your data. Humio has a set of built-in parsers that you can choose from, or you can define your own - but that is outside the scope of this guide. It is a good idea to stick with one of the built-in parsers for your first experiments with Humio.
A good parser to start with is the parser named
kv (short for Key-Value). It looks at incoming messages and finds
key=value pairs in the text - producing a field
"key" with the value
"value" on the stored event.
You can go to the “Parsers” menu item in the top menu of the UI to explore the parsers that are available. A similar list can be seen by clicking on the ‘Assigned Parser’ dropdown marked in the figure above with number 2, which is also where you select which parser to use.
Having created a repository, copied your ingest token, and selected a parser, you are now ready to read one of the following guides on how to ship data to Humio:
You can read more about these different ways of sending data in the section on ingesting data.
Tip: If you are already using ElasticSearch ELK you can also take a look at how easy it is to migrate from an Elastic Stack to Humio.
While Humio has built-in support for the most popular logging formats (AccessLog, JSON),
and can extract almost anything with the
kv parser, you
most likely want to create a parser before long. Fortunately Humio allows you to
create your own custom parsers as documented here: create a custom parser.
Once your data is streaming into your repository you can start visualizing data and creating dashboards.
Use alerts to monitor your systems and create incidents in systems like PagerDuty and VictorOps, or simply send a Slack message or Email when things go wrong.
Need some help? Reach out to Humio Support.