New fields can be created in two ways:
You can extract new fields from your text data using regular expressions and then test their values. This lets you access data that Humio did not parse when it indexed the data.
For example, if your log entries contain text such as … disk_free=2000 …, then you can use a query like the following to find the entries that have less than 1000 free disk space
regex("disk_free=(?<space>[0-9]+)") | space < 1000
Named capturing groups are used to extract fields in regular expressions. The field
space is extracted and is then available after the regex function. The same can be written using a regex literal
/disk_free=(?<space>[0-9]+)/ | space < 1000
You can apply
repeat to field extration to yield one event for each match of the regex. This allows processing multiple values for a named field, or a field name that matches a pattern, as in this example:
regex("value[^=]*=(?<someBar>\\S+)", repeat=true) | groupby(someBar)
On an input event with a field value of
type=foo value=bar1 valueExtra=bar2 value=bar3 the groupby sees all three “bar” values
In order to use field-extraction this way, the regex must be a top-level expression, that is,
| between bars
|. The following doesn’t work
// DON'T DO THIS - THIS DOES NOT WORK type=FOO or /disk_free=(?<space>[0-9]+)/ | space < 1000
Since regular expressions do need some computing power, it is best to do as much simple filtering as possible earlier in the query chain before applying the regex function. (For version 1.13+: If the field name is known it also helps to specify the field to search to avoid searching all fields.)
Fields can also be added by functions.
Most functions set their result in a field that has the function name prefixed
with a ‘_’ by default. For example the count( ) puts its result in a field
Most functions that produce fields have a parameter called
as. By setting this parameter you
can specify the name of the output field, for example
Assigns the result of the count( ) to the field named
cnt (instead of the default
See also the assignment operator for shorthand syntax for assigning results to a field.
The function eval( ) can assign fields while doing numeric computations on the input.
:= syntax is short for eval. Use
| between assignments.
... | foo := a + b | bar := a / b | ...
is short for
... | eval(foo = a + b) | eval(bar = a / b) | ...
You can use the operator
:= with functions that take an
as-parameter. When what’s on the right hand side of the assignment is a function call, the assignment is rewritten to specify the
as= argument which, by convention, is the output field name. For example
... | foo := min(x) | bar := max(x) | ...
is short for
... | min(x, as=foo) | max(x, as=bar) | ...
You can use attr =~ fun() with any function that has a parameter named
field. It designates the field=attr argument and lets you write:
... | ip_addr =~ cidr(subnet="127.0.0.1/24") | ...
... | cidr(subnet="127.0.0.1/24", field=ip_addr) | ...