Adding New Fields

New fields can be created in two ways:

Regex Field Extraction

You can extract new fields from your text data using regular expressions and then test their values. This lets you access data that Humio did not parse when it indexed the data.

For example, if your log entries contain text such as … disk_free=2000 …, then you can use a query like the following to find the entries that have less than 1000 free disk space

regex("disk_free=(?<space>[0-9]+)") | space < 1000

Named capturing groups are used to extract fields in regular expressions. The field space is extracted and is then available after the regex function. The same can be written using a regex literal

/disk_free=(?<space>[0-9]+)/ | space < 1000

You can apply repeat to field extration to yield one event for each match of the regex. This allows processing multiple values for a named field, or a field name that matches a pattern, as in this example:

regex("value[^=]*=(?<someBar>\\S+)", repeat=true) | groupby(someBar)

On an input event with a field value of type=foo value=bar1 valueExtra=bar2 value=bar3 the groupby sees all three “bar” values

In order to use field-extraction this way, the regex must be a top-level expression, that is, | between bars |. The following doesn’t work

// DON'T DO THIS - THIS DOES NOT WORK
type=FOO or /disk_free=(?<space>[0-9]+)/ | space < 1000

Since regular expressions do need some computing power, it is best to do as much simple filtering as possible earlier in the query chain before applying the regex function. (For version 1.13+: If the field name is known it also helps to specify the field to search to avoid searching all fields.)

Fields Produced by Functions (as-parameters)

Fields can also be added by functions. Most functions set their result in a field that has the function name prefixed with a ‘_’ by default. For example the count( ) puts its result in a field _count.

Most functions that produce fields have a parameter called as. By setting this parameter you can specify the name of the output field, for example

count(as=cnt)

Assigns the result of the count( ) to the field named cnt (instead of the default _count).

See also the assignment operator for shorthand syntax for assigning results to a field.

Eval Syntax

The function eval( ) can assign fields while doing numeric computations on the input.

The := syntax is short for eval. Use | between assignments.

... | foo := a + b | bar := a / b |  ...

is short for

... | eval(foo = a + b) | eval(bar = a / b) | ...

Assignment Operator

You can use the operator := with functions that take an as-parameter. When what’s on the right hand side of the assignment is a function call, the assignment is rewritten to specify the as= argument which, by convention, is the output field name. For example

... | foo := min(x) | bar := max(x) |  ...

is short for

... | min(x, as=foo) | max(x, as=bar) | ...

Field Operator

You can use attr =~ fun() with any function that has a parameter named field. It designates the field=attr argument and lets you write:

... | ip_addr =~ cidr(subnet="127.0.0.1/24") | ...

rather than

... | cidr(subnet="127.0.0.1/24", field=ip_addr) | ...

This also works well with regex and replace . It’s a shorthand, and very convenient.