Preview Releases

Below is a list of all of the Preview releases of Humio, in reverse order of their release — the latest at the top and the oldest at the bottom. If you want to see all of the Stable releases go to the Stable Releases page for a list of them. For a list of all releases, Preview and Stable, go to the All Releases page.

Remember, the way to distinguish between Preview and Stable releases is based on the secondary number for the release. If it’s an even number, it indicates it’s a stable release; if the secondary number is an odd number, it indicates it’s a preview of the stable release that will follow it. See the main Humio Releases page for a more detailed explanation of this numbering system.

1.23.1

Humio activity log and GraphQL API policy

Release Date: 2021-03-24
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.23.1 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.23.1. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

Changelog

  • Allow users group to be represented as a json string and not only array when logging in with oauth.
  • New config: S3_ARCHIVING_IBM_COMPAT for compatility with S3 archiving to IBM Cloud Object Storage.

1.23.0

Humio activity log and GraphQL API policy

Release Date: 2021-03-18
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.23.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.23.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

New humio-activity repository

Humio will make some internal logs available in a new repository called humio-activity. This is meant for logs that are relevant to users of Humio, as compared to logs that are only relevant for operators. The latter logs are still put into the humio repository. For this release, only new log events will be put into humio-activity, but in later releases, some existing log events that are relevant for users, will be put into the humio-activity repository instead of the humio repository.

For cloud users, the logs for your organization can be accessed through the humio-organization-activity view.

For on-prem users, the logs can be accessed directly through the humio-activity repository. They are also output into a new log file named humio-activity.log which can be ingested into the humio repository, if you want it available there as well. If you do and you are using the Humio Insights Application, you should upgrade that to version 0.0.4. For more information, see docs.

GraphQL API policy

Humio has decided to adopt an evoutionary approach to its GraphQL API, meaning that we will strive to do only backwards compatible changes. Instead of making non-backwards compatible changes to existing fields, we will instead add new fields alongside the existing fields. The existing fields will be deprecated and might be removed in some later release. We reserve the right to still do non-backwards compatible changes, for instance to fix security issues.

For new experimental features, we will mark the corresponding GraphQL fields as PREVIEW. There will be no guarantees on backwards compatibility on fields marked as PREVIEW.

Deprecations and preview

Deprecated and preview fields and enum values will be marked as such in the GraphQL schema and will be shown as deprecated or preview in the API Explorer. Apart from that, the result of running a GraphQL query using a deprecated or preview field will contain a new field extensions, which contains a field deprecated with a list of all deprecated fields used in the query and a field preview with a list of all preview fields used in the query. Example:

{
  "data": ...
  "extensions": {
    "deprecated": [
      {
        "name": "alert",
        "reason": "[DEPRECATED: Since 2020-11-26. Deprecated since 1.19.0. Will be removed March 2021. Use 'searchDomain.alert' instead]"
      }
    ]
  }
}

Deprecated fields and enum values will also be noted in the release note, when they are first deprecated. All use of deprecated fields and enum values will also be logged in the Humio repository humio-activity. They will have #category=GraphQL, subCategory=Deprecation and #severity=Warning. If you are using the API, consider creating an alert for such logs.

Removed support for CIDR shorthand

Previous version of Humio supported a shorthand for IPv4 CIDR expressions.

For example 127.1/16 would be equivalent to 127.1.0.0/16. This was contrary to other implementations like the Linux function inet_aton, where 127.1 expands to 127.0.0.1. Support for this shorthand has been removed and the complete address must now be written instead.

Changelog

  • Added IP Filter for readonly dashboard links, and started to audit log readonly dashboard access. In this initial version. The readonly ip filter can be configured with the graphql mutation: mutation { updateReadonlyDashboardIPFilter(ipFilter: “FILTER”) } The FILTER is expected in this format: ip-filter From Humio 1.25 this can be configured in the configuration UI.
  • Added GraphQL queries and mutations for alerts and actions, which correspond to the deprecated REST endpoints for those entities.
  • Added GraphQL mutations addAlertLabelV2, removeAlertLabelV2, addStarToAlertV2 and removeStarFromAlertV2.
  • Added an option to make it easier to diagnose problems by detecting inconsistencies between globals in different Humio instances. Each Humio instance has its own copy of the global state which must all be identical. It has happened that they were not, so now we check and if there is a difference we report an error and dump the global state into a file.
  • Added a new introduction message to empty repositories.
  • Added mode parameter to match(), allowing different ways to match the key.
  • Added support for CIDR matching on match() using mode=cidr.
  • Improved performance when using match() with mode=cidr compared to using cidr() with file().
  • Negated, non-strict match() or lookup is no longer allowed.
  • Deprecated glob parameter on match(), use mode=glob instead.
  • Deprecated file and column parameter on cidr(). Use match() with mode=cidr instead.
  • Deprecated the ReadEvents enum variant from the ViewAction enum in GraphQL. Use the ReadContents variant instead, which has the same semantics, but a more accurate name. ReadEvents will be removed in a future release.
  • Deprecated the REST endpoints for alerts and actions.
  • Deprecated GraphQL mutations addAlertLabel, removeAlertLabel, addStarToAlert and removeStarFromAlert as they did not follow the standard for mutation input.
  • The GraphiQL API Explorer is now available from inside Humio. You can access it using the Help->API Explorer menu.
  • The GraphiQL API Explorer has been upgraded to a newer version. The new version includes a history of the queries that have been run.
  • The SearchDomain.queries GraphQL field has been deprecated, and will be removed in a future release. Use SearchDomain.savedQueries instead.
  • The SearchDomain.viewerCanChangeConnections GraphQL field has been deprecated, and will be removed in a future release. Use SearchDomain.isActionAllowed instead.
  • Allow turning encryption of files stored in bucket storage off by explicitly setting S3_STORAGE_ENCRYPTION_KEY=off (similar for GCP_ )
  • Removed the deprecated Repository.isFreemium GraphQL field.
  • Removed support for shorthand IPv4 CIDR notation in cidr(). See section “Removed support for CIDR shorthand”.
  • Changed the URL of the Kafka cluster page in the settings.
  • Implemented toggle button for dark mode.
  • UI enhancements for the new repository Access Permissions page.
  • Mark required fields on the Accept Terms and Conditions page.
  • Enforce accepting terms and conditions.
  • Enforce permissions to enter Organization Settings page.
  • Enforce permissions to enter creating new repository page.
  • Refactor All Organizations page.
  • Refactor Organization Overview page.
  • Refactor client side action cache of allowed permissions.
  • Refactor how the width of the repository name in the main navigation bar is calculated.
  • It is again possible to sort the events on the test parser page.
  • Made the S3 archiving save button work again.
  • Restyled the alert dialogue.
  • Improved memory use for certain numerical aggregrating functions
  • Improved performance of free-text search using regular expressions
  • Fixed the requirement condition for the time retention on a repository.
  • Fixed an issue which caused ingest-to-multiple-repos() to break, when the parser used copyEvent to duplicate the input events into multiple repositories
  • Fixed an issue causing undersized segment merging to repeatedly fetch the same segments, in cases where the merger job took too long to finish.
  • Fixed an issue where regular expressions too large to handle would sometimes cause the query to hang. Now we report an error.
  • Fixed an issue with the Missing Segments API that caused missing segments to not appear in the missing segments list if they had a replacement segment.
  • Fixed an issue where changes to files would not propagate to parsers or event forwarders.
  • Fixed an issue where Prometheus metrics always reported 0.0 for humio_primary_disk_usage
  • Fixed a bug in event forwarding that made start(), end() and now() return the time at which the event forwarding rule was cached. Instead, now() will return the time at which the event forwarding rule was run. start() and end() were never meant to be used in an event forwarding rule and will return 0, which means Unix Epoch.
  • Fixes a bug where events deleted with the delete-event API would appear deleted at first, but then resurface again after 24h. If user applying delete did not have permission to search the events being deleted.
  • Fixed a bug which caused in() with values=[] to give incorrect results
  • Fixed a bug which caused match() to give incorrect results in certain cases due to incorrect caching
  • Fixed a bug which caused glob-patterns in match() to not match newline characters
  • Fixed a bug which caused tag-filters in anonymous functions to not work in certain cases (causing to many events to be let through)
  • Fixed a bug where the same regex pattern occurring multiple times in a query could cause incorrect results

1.21.1

Minor bug fix

Release Date: 2021-02-23
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.21.1 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.21.1. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

Changelog

  • Improve performance of “decrypt step” in downloads from bucket storage
  • New “prefetch from bucket” job. When a node starts with an empty disk it will download a relevant subset of segment files from the bucket in order to have them present locally for queries.
  • Server: header in responses from from Humio HTTP server now includes (Vhost, NodeRole) after the version string.

1.21.0

Complete UI revamp

Release Date: 2021-02-22
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.21.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.21.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

Changelog

  • The default parser kv has been changed from using the parseTimestamp() function to use the findTimestamp() function. This will make it able to parse more timestamp formats. It will still only parse timestamps with a timezone. It also no longer adds a timezone field with the extracted timestamp string. This was only done for parsing the timestamp and not meant for storing on the event. To keep the old functionality, clone the kv parser in the relevant repositories and store the cloned parser with the name kv. This can be done before upgrading to this release. See docs.
  • kvParse() now unescapes backslashes when they’re inside (' or ") quotes.
  • kvParse() now only unescapes quotes and backslashes that are inside a quoted string.
  • kvParse() now also unescapes single quotes. (')
  • The findTimestamp() function has been changed, so that it no longer has a default value for the timezone parameter. Previously, the default was UTC. If no timezone argument is supplied to the function, it will not parse timestamps that do not contain a timezone. To get the old functionality, simply add timezone=UTC to the function. This can be done before upgrading to this release.
  • The deprecated built-in parser json-for-notifier has been deleted. It has been replaced by the parser json-for-action.
  • The deprecated built-in parser bro-json has been deleted. It has been replaced by the parser zeek-json.
  • Split() no longer adds a @display field to the event it outputs.
  • Make the thread dump job run on a dedicated thread, rather than running on the thread pool shared with other jobs.
  • Added support for disaster recovery of a cluster where all nodes including Kafka has been lost, restoring the state present in bucket storage as a fresh cluster using the old bucket as read-only, and forming a fresh cluster from that. New Configs: S3_RECOVER_FROM_REPLACE_REGION and S3_RECOVER_FROM_REPLACE_BUCKET to allow modifying names of region/bucket while recovering to allow running on a replica, specifying read-only source using S3_RECOVER_FROM_* for all the bucket storage target parameters otherwise named S3_STORAGE_*
  • Improve hit rate of query state cache by allowing similar but not identical queries to share cache when the entry in the cache can form the basis for both. The cache format is incompatible with previous versions, this is handled internally by handling incompatible cache entries as cache misses.
  • Improve performance of writeJson() a bit.
  • Improve number formatting in certain places by being better at removing trailing zeros.
  • Change handling of groupBy() in live-queries which should in many cases reduce memory cost.
  • The experimental function moment has been removed.
  • subnet() now reports an error if its argument bits is outside the range 0 to 32.
  • The transpose function now reports an error if the arguments header or column is provided together with the argument pivot.
  • The replace() function now reports an error if the arguments replacement and with are provided at the same time.
  • The replace() function now reports an error if an unsupported flag is provided in the flags argument.
  • The functions worldMap() and geohash() now errors if requested precision is greater than 12.
  • When on ephemeral disks, nodes being replaced with new ones on empty disks no longer download most of the segments they had before being replaced, but instead schedule downloads based on is being searhed.
  • The Auth0 login page will no longer load a local version of the Auth-Lock library, but instead load a login script hosted on Auth0’s CDN. This may require opening access to https://cdn.auth0.com/ if hosting Humio behind a firewall.
  • Lowered the severity level for some loggings for running alerts.
  • Made loggings for running alerts more consistent and more structured. All loggings regarding a specific alert will contain the keys alertId, alertName and viewId. Loggings regarding the alert query will always contain the key externalQueryId and sometimes also the keys queryId with the internal id and query with the actual query string. If there are problems with the run-as-user, the id of that user is logged with the key user.
  • Made loggings for running scheduled searches more consistent and more structured. All loggings regarding a specific alert will contain the keys scheduledSearchId, scheduledSearchName and viewId. Loggings regarding the alert query will always contain the key externalQueryId and sometimes also the keys queryId with the internal id and query with the actual query string. If there are problems with the run-as-user, the id of that user is logged with the key user.
  • Prevent Humio from booting when Zookeeper has been reset but Kafka has not.
  • Create, update and delete of an alert, scheduled search or action is now recorded in the audit log.
  • Running test of a parser is no longer recorded in the audit log, and irrelevant fields are no longer recorded upon parser deletion.
  • When using filters on dashboards, you can now easily reset the filter, either removing it completely, or using the default filter if one is present.
  • Made sure the humio view humio default parser is only installed when missing, instead of overwriting it every time humio starts.
  • When exporting a package, you now get a preview of the icon you’ve added for the package.
  • Packages can now be updated with the same version but new content. This makes iterating over a package before finalizing it easier.
  • Humio insights package installed if missing on the humio view when humio is started.
  • Raised the parser test character length limit to 20000.
  • Raised the note widget text length limit to 20000.
  • Fixed a performance and a robustness problem with the function unit:convert(). The formatting of the numbers in its output may in some cases be different now.
  • Fixed a number of potential concurrency issues.
  • Fixed a memory leak in rdns() in cases where many different name servers are used.
  • Fixed a bug in parseJson which resulted in failed JSON parsing if an object contained an empty key ("").
  • Fixed a bug which caused eventInternals() to crash if used late in the pipeline.
  • Fixed a bug which caused validation to miss rejecting window() inside window() and session().
  • Fixed a bug which could cause saving of query state cache to take a rather long time.
  • Fixed a bug which could potentially cause a query state cache file to be read in an incomplete state.
  • Fixed a bug in upper() and lower() which could cause its output to be corrupted (in cases where no characters had been changed).
  • Fixed a bug where analysis of a regex could consume extreme amounts of memory.
  • Fixed a bug in lowercase() which caused the case lowercase(field=”*", include="values”) to not process all fields but only the field named “*".
  • Fixed a bug where referenced saved queries were not referenced correctly after exporting them as part of a package.
  • Fixed bugs in format() which caused output from ‘%e’/'%g’ to be incorrect in certain cases.
  • Fixed an issue causing Humio to crash when attempting to delete an idle empty datasource right as the datasource receives new data.
  • Fixed an issue with the validation of the query prefix set on a view for each repository within the view: Invoking macros is not allowed and was correctly rejected when creating a view, but was not rejected when editing an existing connection.
  • Fixed an issue where merge of segments were reported as failed due to input files being deleted while merging. This is not an error, and is no longer reported as such.
  • Fixed an issue where the segment mover might schedule too many segments for transfer at a time.
  • Fixed an issue with lack of escaping in filename when downloading.
  • Fixed an issue causing segment tombstones to potentially be deleted too early if bucket storage is enabled, causing an error log.
  • Fixed an issue causing event redirection to break when using copyEvent to get the same events ingested into multiple repositories.
  • Fixed an issue where repeating queries would not validate in alerts
  • Fixed an issue where cancelled queries could be cached
  • Bumped minimum supported versions of Chrome and Chromium from 60 to 69 due to updated dependencies - note added late

1.19.2

Bug fix release

Release Date: 2021-01-25
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.19.2 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.19.2. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to an earlier release can result in data loss.

Changelog

  • Fixed an issue for on-prem users not on multitenant setup by reverted a metric change introduced in 1.18.0, jmx and Slf4j included an OrgId in all metrics for repositories.
  • Fixed automatic installation of Humio Insights package to the humio repository

1.19.1

Bug fix release

Release Date: 2021-01-19
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: false

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.19.1 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.19.1. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to an earlier release can result in data loss.

Changelog

  • Fixed bug where the format() function produced wrong output for some floating-point numbers
  • Update dependencies with known vulnerabilities
  • Do not retry a query when getting a HTTP 400 error
  • Do not cache cancelled queries.
  • Fixed bug in a saved query in the Humio Insights package
  • Fixed an issue - Do not delete datasource before the segments have been deleted also in bucket storage if present there.

1.19.0

New query editor, Packages (in beta)

Release Date: 2021-01-14
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: true

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.19.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.19.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

Packages

This version introduces Humio packages - a way of bundling and sharing assets such as dashboards and parsers. You can create your own packages to keep your Humio assets in Git or create utility packages that can be installed in multiple repositories. All assets can be serialized to YAML files (like what has been possible for dashboards for a while). With tight integration with Humio’s CLI humioctl you can install packages from local disk, URL, or directly from a Github repository. Packages are still in beta, but we encourage you do start creating packages yourself, and sharing them with the community. At Humio we are also very interested in talking with package authors about getting your packages on our upcoming marketplace. Read more about packages on our package documentation.

Humio Insights application

With the introduction of Humio packages we have created the application Humio Insights. The application is a collection of dashboards and saved searches making it possible to monitor and observe a Humio cluster.

New Query Editor

The new query editor has a much better integration with Humio’s query language. It will give you suggestions as you type, and gives you inline errors if you make a mistake. We will continue to improve the capabilities of the query editor to be aware of fields, saved queries, and other contextual information.

If Humio is running behind a reverse proxy, it must be configured to permit WebSockets to pass through. Given this scenario, if when using the query editor in Humio, if you’re not seeing syntax coloring, it’s an indication that this is not yet configured and the reverse proxy is blocking the sockets. (edited)

New Test Function

A new function called test() has been added for convenience. What used to be done like: tmp := <expression> | tmp=true can now be done using: test( <expression> ). Inside <expression> field names appearing on the right hand side of an equality test, such as field1==field2 compares the values of the two fields. When comparing using = at top-level field1=field2 compares the value of field1 against the string "field2". This distinction is a cause of confusion for some users, and using test() simplifies that.

Changes to Humio’s internal logging

We have made small changes to how Humio logs internally. We did this to better support the new Humio Insights Application. We have tried to keep the changes as small and compatible as possible, but we have made some changes that can break existing searches in the humio repository (or other repositories receiving Humio logs). We made these changes as we think they are important in order to improve things moving forward. One of the benefits is the new Humio Insights App Read more about the details here

Changelog

  • Introduced humio insights package that is installed per default on startup on the humio repository
  • Raised the limit for note widget text length to 20000
  • Raised the parser test character length to 20000.
  • No longer overwrite the humio parser in the humio repository on startup.
  • Improved app loading logic.
  • Kafka client inside Humio has been bumped from 2.4.1 to 2.6.0.
  • Improve handling of broken local cache files
  • New config QUERY_QUOTA_EXCEEDED_PENALTY with value 50 by default. When set >= 1.0 then this throttles queries from users that are over their quota by this factor rather than stopping their queries. Set to 0 to disable and revert to stopping queries.
  • For ingest using a URL with a repository name in it, Humio now fails ingest if the repository in the URL does not match the repository of the ingest token. Previously, it would just use the repository of the ingest token.
  • “Notifiers” have been renamed to “actions” throughout the UI and in log statements. The REST APIs have not been changed and all message templates can still be used.
  • In the GraphQL API, on the Alert type, the notifiers field has been deprecated and will be removed in a later release. It has been replaced by the actions field.
  • In the GraphQL API, the value ChangeAlertsAndNotifiers on the Permission enum has been deprecated and will be removed in a later release. It has been replaced by the ChangeTriggersAndActions value. The same is true for the ViewAction enum. On the ViewPermissionsType type, the administerAlertsfield has been deprecated and will be removed in a later release. It has been replaced by the administerTriggersAndActions field.
  • The built-in json-for-notifier parser used by the Humio Repository action (formerly notifier) is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name json-for-action, see docs.
  • The built-in bro-json parser is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name zeek-json, see docs.
  • The configuration option IP_FILTER_NOTIFIERS has been renamed to IP_FILTER_ACTIONS. The old name will continue to work.
  • The configuration option HTTP_PROXY_ALLOW_NOTIFIERS_NOT_USE has been renamed to HTTP_PROXY_ALLOW_ACTIONS_NOT_USE. The old name will continue to work.
  • New feature “Scheduled Searches” making it possible to run queries on a schedule and trigger actions (formerly notifiers) upon query results. See docs.
  • New feature “Event forwarding” making it possible to forward events during ingest out of Humio to a Kafka server. See docs. Currently only available for on-prem customers.
  • The Humio Repository action (formerly notifier) now replaces a prefix ‘#’ character in field names with @tag. so that #source becomes @tag.source. This is done to make them searchable in Humio. You can change the name by creating a custom parser. See docs for more details.
  • The names of the metadata fields added by the Humio Repository action (formerly notifier) has been changed to accomodate that it can now also be used from scheduled searches. See docs for more details.
  • New filter function test( <boolean expression> ) makes it convenient to test complex expressions.
  • New ingest endpoint /api/v1/ingest/raw for ingesting singular webcalls as events. See docs.
  • API Changes (Non-Documented API): getFileContent has been moved to a field on the SearchDomain type.
  • API Changes (Non-Documented API): Queries and Mutations for Parser now expects an id field in place of a name field, when fetching and updating parsers.
  • API Changes (Non-Documented API): Getting Alert by ID has been moved to a field on the SearchDomain type.
  • New feature: “stateless Ingest-only nodes”: A node that the rest of the cluster does not know exists, but is capable of ingesting events into the ingest queue. Enable using NODE_ROLES=ingestonly.
  • unit on timechart (and bucket) now works also when the function within uses nesting and anonymous pipielines.
  • New feature “Custom ingest tokens” making it possible for root users to create ingest tokens with a custom string.
  • The function parseCEF() now deals with extension fields with labels, i.e. cs1=Value cs1Label=Key becomes cef.label.Key=Value.
  • Added asn function for retrieving the ASN number for a given IP address, see docs.
  • New config AUTO_UPDATE_MAXMIND for enabling/disabling updating of all maxmind databases. Deprecates AUTO_UPDATE_IP_LOCATION_DB, but old config will continue to work.
  • New config MAXMIND_IP_LOCATION_EDITION_ID for selecting the maxmind edition of the IP location database. Deprecates MAXMIND_EDITION_ID, but old config will continue to work.
  • New function hash for computing hashes of fields. See docs.
  • Renamed LOG4J_CONFIGURATION environment variable to HUMIO_LOG4J_CONFIGURATION. See Docs. The old variable will no longer work.
  • New validation when creating an ingest token using the API that the parser, if specified, actually exists in the repository.
  • Humio no longer deletes an existing humio-search-all view if the CREATE_HUMIO_SEARCH_ALL environment variable is false. The view instead becomes deleteable via the admin page.
  • When a host dies and Humio reassigns digest, it will warn if a fallback host is picked that is in the same zone as existing replicas. Eliminate warning if falling back to a host in the null zone.
  • Reduced the number of writes to global on restart, due to merge targets not being properly reused.
  • Cluster management stats now shows segments as underreplicated if they are replicated to enough hosts, but are not present on all configured hosts.
  • Add an error message to the event if the user is trying to redirect it to another repo using #repo, and the target repo is invalid.
  • Make the query functions window and series be enabled by default. They can be disabled by seting the configuration options WINDOW_ENABLED and SERIES_ENABLED to false, respectively.
  • The transfer job will delete primary copies shortly after transferring the segments to secondary storage. The copies would previously only be deleted once a full bulk had been moved.
  • Added config option for Auth0 based sign on method: AUTH_ALLOW_SIGNUP defaults to true. The config is forwarded to the auth0 configuration for the lock widget setting: allowSignUp
  • Reduce contention on the query scheduler input queue. It was previously possible for large queries to prevent each other from starting, leading to timeouts.
  • Made cluster nodes log their own version as well as the versions of all other nodes. This makes it easier to tell which versions are running in the cluster.
  • Introduction of the new log file humio-requests.log. Also the log format for the files humio-metrics.log and humio-nonsensitive.log has changed as described above. The guide for sending Humio logs to another Humio cluster has been updated.
  • Custom made saved queries, alerts and dashboards in the humio repository searching for events of the kinds metrics, requests or nonsensitive may need to be modified. This is described in more detail here.
  • Upgraded Log4j2 from 2.13.3 to 2.14.0.
  • Removed config IDLE_POLL_TIME_BEFORE_DASHBOARD_QUERY_IS_CANCELLED_MINUTES. Queries on dashboards now have the same life cycle as other queries.
  • Updated the permission checks when polling queries. This will results in dashboard links “created by users who are either deleted or lost permissions to the view” to get unauthorized. To list all dashboard links, run this graphql query as root: query { searchDomains {dashboards { readOnlyTokens { createdBy name token } } } }
  • Added mutation to update the runAsUser for a read only dashboard token.
  • Added timeout for TCP ingest listeners. By default the connection is closed if no data is received after 5 minutes. This can be changed by setting TCP_INGEST_MAX_TIMEOUT_SECONDS. See docs.
  • Humio will only allow using Zookeeper for node id assignment (ZOOKEEPER_URL_FOR_NODE_UUID) when configured for ephemeral disks (USING_EPHEMERAL_DISKS). When using persistent disks, there is no need for the extra complexity added by Zookeeper.
  • Fixed a rare issue where a node that was previously assigned digest could write a segment to global, even though it was no longer assigned the associated partition.
  • Fixed an rare issue where the digest coordinator would assign digest fewer hosts than configured.
  • Fixed an issue which caused free-text-search to not work correctly for large (>64KB) events.
  • Fixed crash in CleanupDatasourceFilesJob when examining a file size fails due to that file being deleted concurrently.
  • Fixed an issue where the segment rewrite job handling event deletion might rewrite segments sooner than configured.
  • Fixed an issue where cancelling queries could produce a spurious error log.
  • Fixed an issue with the cidr function that would make some IPv4 subnets accept IPv6 addresses and some strings that were not valid IP addresses.
  • Fixed an issue that could cause node id assignment to fail when running on ephemeral disks and using Zookeeper for node id assignment. Nodes in this configuration will now try to pick a new id if their old id has been acquired by another node.
  • Fixed an issue causing Humio to retain deleted minisegments in global for longer than expected.
  • Fixed an issue where unit-convertion (by timechart) did not take effect through groupBy() and window().
  • Fixed an issue causing queries using kvParse() to filter out too much in specific circumstances when filtering on a field assigned before kvParse().
  • Fixed an issue causing queries using kvParse() to be executed incorrectly in certain circumstances when kvParse() assigned fields starting with a non-alphanumeric character.
  • Fixed an issue causing the secondary storage transfer job to select and queue too many segments for transfer at once. The job will now stop and recalculate the bulk to transfer periodically.
  • Fixed an issue with updating user profile, in some situations save failed.
  • Fixed an issue where the filter and groupBy buttons on the search page would not restart the search automatically
  • Fixed logic for when the organization owner panel should be shown in the User’s Danger zone.
  • Fixed an issue where segment merge occasionally reported BrokenSegmentException when merging, while the segments where not broken.
  • Fixed timeout issue in S3 Archving
  • Fixed bug where repeating queries would not validate in alerts.
  • Fixed a bug where fullscreen mode could end up blank

1.17.0

Repeating Queries, search result caching and new query functions.

Release Date: 2020-11-18
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: true

Repeating Queries, search result caching and new query functions.

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.17.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.17.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to earlier release can result in data loss.

Repeating Queries

Humio can now run repeating queries using the beta:repeating() function. These are live queries that are implemented by repeatedly making a query. This allows using functions in alerts and dashboards that typically do not work in live queries, such as selfJoin() or selfJoinFilter(). See the function docs for more information.

Improved Security for Alerts

In order to prevent alert notifiers being used to probe services on the internal network (eg. Zookeeper or the AWS metadata service), Humio now has an IP filter on alert notifiers. The default is to block access to all link-local addresses and any addresses on the internal network; however, you can opt-in to the old behavior by setting the configuration option IP_FILTER_NOTIFIERS to allow all. See IP Filter documentation.

New experimental query function series()

A new experimental query function called series() has been added. It needs to be explicitly enabled on the cluster using the config option SERIES_ENABLED=true.

series() improves upon session() and collect() for grouping events into transactions.

What used to be done with:

groupby(id, function=session(function=collect([fields, ...])))

can now be done using:

groupby(id, function=series([fields, ..])).

See docs for more details.

Search result caching

This new feature stores a copy of live search results to the local disk in the server nodes, and reuses the relevant parts of that cached result when an identical live search is later started. Caching is controlled with the config option QUERY_CACHE_MIN_COST, which has a default value of 1000. To disable caching, set the config option to a very high number, such as 9223372036854775807 (max long value).

Changelog

  • New config option IP_FILTER_NOTIFIERS, to set up IP filters for Alert Notifications, see docs.
  • New config option ENABLE_ALERTS makes it possible to disable alerts from running (enabled by default), see docs.
  • New config option ALERT_DESPITE_WARNINGS makes it possible to trigger alerts even when warnings occur, see docs.
  • New config option DEFAULT_MAX_NUMBER_OF_GLOBALDATA_DUMPS_TO_KEEP, see docs.
  • New query function parameter to parseJson, removePrefixes, see docs.
  • New query function concatArray, see docs.
  • New query function parseCEF used to parse events which are formatted according to the Common Event Format(CEF), see docs.
  • New experimental query function beta:repeating(), see docs.
  • New experimental query function series(), enabled by config option SERIES_ENABLED=true, see docs.
  • New experimental query function window(), enabled by config option WINDOW_ENABLED=true, see docs.
  • The {events_html} notifier template will now respect the field order from the query, see docs.
  • It is again possible to override a built-in parser in a repository by creating a parser with the same name.
  • Periodically release object pools used by mapper pipeline, to avoid a possible source of memory leaks.
  • Fix negating join expressions.
  • Fixes a bug where join function in some circumstances would fetch subquery results from other cluster nodes more than once.
  • Setting the default query for a view in the UI has been moved from the “Save as Query” to the View’s “Settings” tab.
  • The notifier list is sorted when selecting notifiers for an alert.
  • Improved wording of diagnostics regarding function arguments.
  • Tweaked location of diagnostics regarding missing function arguments.
  • API Changes (Non-Documented API): Saved Query REST API has been replaced by GraphQL.
  • API Changes (Non-Documented API): View Settings REST API has been replaced by GraphQL.
  • Allow running Humio on JDK-14 and JDK-15 to allow testing these new builds.
  • Free-text search has been fixed to behave more in line with the specification.
  • Refuse to boot if the global topic in Kafka does not contain the expected starting offset.
  • Crash the node if an exception occurs while reading from the global Kafka topic, rather than trying to recover.
  • Reduce the max fetch size for Kafka requests, as the previous size would sometimes lead to request timeouts.
  • Improve logic attempting to ensure other live nodes can act as substitutes in case the preferred digest nodes are not available when writing new segments.
  • Fixes the case where datasources receiving data might not be marked idle, causing Humio to retain too much ingest data in Kafka.
  • Fixes the case where Humio would consider local node state when deciding which ingest data was safe to delete from Kafka.
  • Fix several cases where Humio might attempt to write a message to Kafka larger than what Kafka will allow.
  • Fixes the issue where Humio could behave incompatibly with Kafka versions prior to 2.3.0 if KAFKA_MANAGED_BY_HUMIO was true.
  • Refuse to boot if the booting node would cause violations of the “Minimum previous Humio version” as listed in the release notes.
  • Fixes an issue which caused free-text-search to not work correctly for large (>64KB) events.
  • Fixes a bug where unit:convert couldn’t handle numbers in scientific notation.
  • Rename a few scheduler threads so they reflect whether they’re associated with streaming queries (“streaming-scheduler”) or not (“normal-scheduler”)
  • Fixes an issue where Humio might try to get admin access to Kafka when KAFKA_MANAGED_BY_HUMIO was false.
  • If KAFKA_MANAGED_BY_HUMIO is true, Humio will ensure unclean leader election is disabled on the global-events topic.
  • The Humio-search-all view will no longer be removed if CREATE_HUMIO_SEARCH_ALL is set to false. The view will instead become possible to delete manually via the admin UI.
  • Reduce the number of merge target updates Humio will write to global on digest leader reassignment or reboot.
  • Fixes a bug causing join() to not work after an aggregating function.
  • Fixes a bug causing sort()/head()/tail() to work incorrectly after other aggregating functions.
  • Fixes a bug causing the sub-queries of join() etc. to not see events with an @ingesttimestamp occurring later than the search time interval.
  • Fixes an issue causing Humio to fail to upload files to bucket storage in rare cases.
  • Switch from JDK to BouncyCastle provider for AES decrypt to reduce memory usage.
  • Changed default TLS ciphers and protocols accepted by Humio, see docs.

1.15.2

Bugfixes and stability enhancements.

Release Date: 2020-09-29
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bugfixes and stability enhancements.

Changelog

  • Generate ingest tokens in UUID format, replacing the current format for any new tokens being created.
  • In the dialog for saving a search as an alert, the save button is no longer always grey and boring, but can actually save alerts again.
  • Fixed a problem where the login link did not work in Safari and Firefox.
  • Fixed a problem with scrolling on the login page on screens with low resolution.
  • Made the login and sign up pages responsive to the device.
  • Fixed a bug in the partition table optimizer that lead to unbalanced layouts.
  • Fixed a bug causing an authentication error when trying to download a file when authenticating by proxy.
  • Fixed an issue showing duplicate entries when searching for users.
  • Fixed a memory leak when authenticating in AWS setups.
  • Avoid overloading kafka with updates for the global database by collecting operations in bulk.
  • Only consider fully replicated data when calculating which offsets can be pruned from Kafka.
  • Improved naming of threads to get more usable thread dumps.
  • Changed the query scheduling to account for the work of the overall query, rather than per job started. This allows fairer scheduling of queries hitting many dataspaces e.g. when using search-all.
  • Changed priorities when fetching segments to a node which have been offline for a longer period. This avoids waiting too long before the cluster becomes fully synced.
  • Improved handling of sub-queries polling state from the main query when using join().
  • Fixed an issue where a slow data stream could cause Humio to retain more data in Kafka than necessary, as well as cause a restarted Humio node to reprocess too much data.
  • Added logging to detect issues when truncating finished files.

1.15.1

Bugfixes and IP location DB HTTP_PROXY support.

Release Date: 2020-09-22
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bugfixes and IP location DB HTTP_PROXY support.

Changelog

  • The job for updating the IP location database now uses the configured HTTP proxy, if present. See docs.
  • Fixed a problem with AWS, where STS tokens would fail to authenticate.
  • Fixed a problem in the UI, where the wrong timestamp was displayed as @ingesttimestamp.
  • Revert login UI to same behavior as before 1.15.0.

1.15.0

Improved alerts and addition of ingest timestamps

Release Date: 2020-09-15
Minimum previous Humio version: 1.12.0
Requires data migrations: true
Has changes to configuration: true

Ingest timestamps

Humio will set ingest timestamps on all events. This is set in the field named @ingesttimestamp. In later versions, Humio will also support specifying the search time interval using @ingesttimestamp when searching. This will support use cases where data is backfilled etc.

Alerts and notifiers

Field based throttling

It is now possible to make an alert throttle based on a field, so that new values for the field trigger the alert, but already seen values do not until the throttle period has elapsed. See docs.

Notifier logging to a Humio repository

It is now possible to configure an alert notifier that will log all events to a Humio repository. See docs.

Slack notifier upgrade to notify multiple Slack channels

It is now possible to use the Slack notifier to notify multiple slack channels at once. See docs.

Events as HTML table

In an email notifier, it is now possible to format the events as an HTML table using the new message template {events_html}. See docs.

Configure notifier to not use the internet proxy

It is now possible to configure an alert notifier to not use the HTTP proxy configured in Humio. See docs.

User signup/login flow

Signup & Login

We introduce new signup/login pages for social login and have split the behavior so users have to explicitly either login or signup.

Invite flow

When adding a user to Humio they will now by default get an email telling them that they have been invited to use Humio. See docs.

AWS Java SDK V2

The AWS SDK Humio uses has been upgraded to v2. When configuring Humio bucket storage with Java system properties, the access key must now be in the aws.secretAccessKey property instead of the aws.secretKey property.

Configure Humio to not use the internet proxy for S3

It is now possible to configure Humio to not use the globally configured HTTP proxy for communcation with S3. See docs.

Auto-balanced partition table suggestions

When changing digest and storage partitions it is now possible to get auto-balanced suggestions based on node zone and replication factor settings (via ZONE, DIGEST_REPLICATION_FACTOR, STORAGE_REPLICATION_FACTOR configurations). See docs.

Changelog

  • Bugfix: CSV files can no longer contain unnamed columns and also trailing commas are disallowed. Queries based on such files will now fail with an error.
  • Improved error handling when a parser cannot be loaded. Before, this resulted in Humio returning an error to the data shipper. Now, data is ingested without being parsed, but marked with an error as described in Parser Errors.
  • If automatically creating users upon login and syncing their groups from the authentication mechanims, the configuration ONLY_CREATE_USER_IF_SYNCED_GROUPS_HAVE_ACCESS now controls whether users should only be created if the synced groups have access to a repository or view. The default is false.
  • Humio will set the field @ingesttimestamp on all events.
  • S3 communication can be configured to not use an HTTP proxy. See docs.
  • Upgrade to AWS SDK v2. When using Java system properties for configuring Humio bucket storage, use aws.secretAccessKey instead of aws.secretKey.
  • New explicit signup and login pages for social login.
  • Newly added users will by default get an email. See docs.
  • Alert notifiers can be configured to not use an HTTP proxy. See docs.
  • Field based throttling on alerts. See docs.
  • New alert notifier type logging to a Humio repository. See docs.
  • New alert notifier template {events_html} formatting events as an HTML table. See docs.
  • Auto-balanced partition table suggestions. See ZONE, DIGEST_REPLICATION_FACTOR, STORAGE_REPLICATION_FACTOR in configuration. See docs.

1.13.5

Security and bugfixes

Release Date: 2020-08-12
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfixes

Changelog

  • This release fixes a security issue. More information will follow when Humio customers have had time to upgrade. See: Humio Security Disclosures.
  • Bugfix: export to file now allows for fieldnames with special characters.
  • Bugfix: export to file can now include query parameters
  • Bugfix: missing migration of non-default groups would result in alerts failing until the user backing the alert logs in again.

1.13.4

Security and bugfixes

Release Date: 2020-08-05
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfixes

Changelog

  • This release fixes a security issue. For more information see: Humio Security Disclosures.
  • Fix issue where a query could fail to search all segments if digest reassignment was occurring at the same time as the query.
  • Fix issue where a node with no digest assignment could fail to delete local segment copies in some cases.

1.13.3

Security and bugfix

Release Date: 2020-08-04
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfix

Changelog

  • This release fixes a security issue. For more information see: Humio Security Disclosures.
  • Bugfix: avoid forbidden access error on shared dashboard links by ensuring correct use of time stamps

1.13.2

Bug fixes

Release Date: 2020-08-03
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Bug fixes

Changelog

  • Bugfix: all ingest methods now support the ALLOW_CHANGE_REPO_ON_EVENTS configuration parameter
  • Bugfix: avoid saving invalid bucket storage configurations
  • Bugfix: export to file no longer fails/timeouts on heavy sub queries
  • Bugfix: joins will now propagate limit warnings from sub queries to the main query
  • Bugfix: make sure join-subqueries gets canceled when the main query is canceled
  • Default groups added

1.13.1

Bug fixes and improved search speeds for many-core systems

Release Date: 2020-07-03
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bug fixes and improved search speeds for many-core systems

Changelog

  • Improved query scheduling on machines with many cores. This can improve search speeds significantly.
  • Support for a new storage format for segment files that will be introduced in a later release (to support rollback)
  • Bugfix: S3Archiving could write events twice in a special case (When a merge happens where all inputs have been archived, write in global that the merge-result was archived too).
  • Bugfix: Bucket storage in GCP could did not clean up all tmp files

1.13.0

Release Date: 2020-06-24
Minimum previous Humio version: 1.12.0
Requires data migrations: true
Has changes to configuration: true

Free text search now searches all fields rather than only @rawstring.

Load balancing of queries

Humio can now balance and reuse existing queries internally in the cluster. Load balancer configuration to achieve this is no longer needed. See Humio Configuration and Reverse proxy configuration.

TLS support

TLS Encrypt communication using TLS to/from Zookeeper, Kafka, and other Humio nodes.

Iplocation database management changed

The database used as data source for the ipLocation() query function must be updated within 30 days when a new version of the database is made public by MaxMind. To comply with this, the databased is no longer shipped as part of the humio artifacts but will either:

  • Be fetched automatically by Humio provided that Humio is allowed to connect to the db updata service hosted by Humio. This is the default behaviour.
  • Have to be updated manually (See IP location)

If the database cannot be automatically updated and no database is provided manually, the ipLocation() query function will no longer work.

Configuration change: Controlling what nodes to use as query coordinators

Due to the load balancing in Humio, customers that previously relied on load balancing to control which nodes are query coordinators now need to set QUERY_COORDINATOR to false on nodes they do not want to become query coordinators. See Humio Configuration and Reverse proxy configuration.

Changelog

  • Free text search now searches all fields rather than only @rawstring.
  • Humio can now balance and reuse existing queries internally in the cluster. See Humio Configuration
  • Internal communication in a Humio installation can now be encrypted using TLS. See TLS Configuration
  • Added support for WebIdentityTokenCredentialsProvider on AWS.
  • The data source for the ipLocation() query function is no longer shipped with humio but installed/updated separately.
  • Introduced a new ChangeViewOrRepositoryDescription permission for editing the description of a view or repository. This was previously tied to ConnectView and any user with that permission will now have the new permission as well.

1.11.1

Bug fixes and memory optimizations

Release Date: 2020-05-28
Minimum previous Humio version: 1.10.0
Requires data migrations: false
Has changes to configuration: false

Bug fixes and memory optimizations

Changelog

Elastic Bulk API change - Fluent Bit users might need to change the Fluent Bit configuration To ensure compatability with the newest Beats clients, the Elastic Bulk API has been changed to always return the full set of status information for all operations, as it is done in the official Elastic API

This can however cause problems when using Fluent Bit to ingest data into Humio.

Fluent Bit in default configuration uses a small buffer (4KB) for responses from the Elastic Bulk API, which causes problems when enough operations are bulked together. The response will then be larger than the response buffer as it contains the status for each individual operation. Make sure the response buffer is large enough, otherwise Fluent Bit will stop shipping data. See: https://github.com/fluent/fluent-bit/issues/2156 and https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch

Other changes

  • Several improvements to memory handling
  • Several improvements to query error handling
  • Dashboard widgets now display an error if data is not compatible with the widget

1.11.0

Export to bucket, findTimestamp, selfjoin, Emergency user subsystem

Release Date: 2020-05-19
Minimum previous Humio version: 1.10.0
Requires data migrations: true
Has changes to configuration: true

SelfJoin

selfJoin query function allows selecting log lines that share an identifier; for which there exists (separate) log lines that match a certain filtering criteria; such as “all log lines with a given userid for which there exists a successful and an unsuccessful login”.

findTimestamp

findTimestamp query function will try to find and parse timestamps in incoming data. The function should be used in parsers and support automatic detection of timestamps. It can be used instead of making regular expressions specifying where to find the timestamp and parsing it with parseTimestamp. Checkout the documentation for details.

Export to bucket storage/S3

As an alternative to downloading streaming queries directly, Humio can now upload them to an S3 or GCS bucket from which the user can download the data. See docs

Emergency user subsystem

If there are issues with the identity provider that Humio is configured to use, it might not be possible to log in to Humio. To mitigate this, Humio now provides emergency users that can be created locally within the Humio cluster. See docs

Changelog

1.9.3

Security fixes, bug fixes, and timechart improvements

Release Date: 2020-04-22
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security Fixes

A few security vulnerabilities have been discovered as part of a proactive penetration test. None are known to have been exploited. More information will be forthcoming.

New Time Charts Features

Dealing with missing data points in timecharts

This release improves the handling of missing data points in time charts. Previously you could either interpolate missing data points based on the surrounding data, or leave gaps in the charts. With the introduction of the new charts in 1.9.0 the gaps became more apparent than previously, and we have added new options to deal with missing data points. These replace the previous option “Allow Gaps”, with four new options:

  • Do Nothing - This will leave gaps in your data
  • Linear Interpolation - Impute values using linear interpolation based on the nearest known data points.
  • Replace by Mean Value - Replace missing values with the mean value of the series.
  • Replace by Zero - Replace missing values with zeros.

New line interpolation options

The release also introduces new options for line interpolation.

  • Monotone
  • Natural
  • Cardinal
  • Catmull-Rom
  • Bundle

The latter three are impacted by the ‘tension’ setting in the timechart Style editor.

Changelog

  • Security: [critical] Fixed more security vulnerabilities discovered through proactive penetration testing (more information will be forthcoming).
  • New query function fieldstats.
  • More efficient “collect” function implementation.
  • Allow more concurrent processing to take place in “export” query processing.
  • Styling improvements in the “Style” panel for widgets.
  • New Time Chart interpolation options.
  • New options for dealing with missing data in Time Charts.
  • Improves responsiveness of the recent queries dropdown, and limits the number of stored recent queries to 100 per user per repository.
  • Allow dots in tagged field names.
  • If at startup the global-snapshot.json file is missing, then try loading the “.1” backup copy.
  • Bug fix: The segment queue length metric was not correct when segments got fetched from bucket storage by a query.
  • Bug fix: the query metric only measured time for streaming queries, now it includes non-streaming as well.
  • Bug fix: api-explorer not working due to CSP inline script.
  • Improve disk space monitoring when using bucket storage.

1.9.2

Security fix and bug fixes

Release Date: 2020-03-25
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security fix and bug fixes

Changelog

  • Added API to the list and deleted missing segments from global. See reference here.
  • Security: [critical] Fixed a security vulnerability discovered through proactive penetration testing (more information will be forthcoming).

1.9.1

Security fix and bug fixes

Release Date: 2020-03-23
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security fix and bug fixes

Changelog

  • This is a critical update. Self-hosted systems with access for non-trusted users should upgrade immediately. We will follow up with more details when this update has been rolled out.
  • Bug fix: the health-check failed-http-status-check would get stuff in warn state, this has now been fixed

1.9.0

UI for Role Based Access Control (RBAC), Health Check API, Kafka version update, Vega charts

Release Date: 2020-03-12
Minimum previous Humio version: 1.8.5
Requires data migrations: true
Has changes to configuration: true

RBAC, Health Check API, Kafka, Vega

Changelog

Role Based Access Control

Role Based Access Control (RBAC) through the UI is now the only permission model in Humio. Please see the Authorization documentation for more information.

Kafka version update

  • Updated Humio to use Kafka 2.4. Humio can still use versions of Kafka down through 1.1.
  • Be aware that updating Kafka also requires you to update Zookeeper to 3.5.6. There is a migration involved in updating Zookeeper. See Zookeeper’s migration FAQ here. Use the migration approach using an empty snapshot. The other proposed solution can loose data.
  • Updated Kafka and Zookeeper Docker images to use Kafka 2.4. Updating to Kafka 2.4 should be straightforward using Humio’s Kafka/Zookeeper Docker images. Zookeper image will handle migration. Stop all Kafka nodes. Stop all Zookeeper nodes. Start all Zookeeper nodes on the new version. Start all Kafka nodes on the new version. Before updating Kafka/Zookeeper, we recommend backing up the Zookeeper data directory. Then, add the Zookeeper properties described below. If you are deploying Kafka/Zookeeper using other tools, for example Ansible scripts, be aware there is a migration involved in updating Zookeeper.
  • When updating Kafka/Zookeeper we recommend setting these Zookeeper properties
    # Do not start the new admin server. Default port 8080 conflicts with Humio port.admin.enableServer=false
    # purge old snapshot files autopurge.purgeInterval=1
    # Allow 4 letter commands. Used by Humio to get info about the Zookeeper cluster 4lw.commands.whitelist=*
    

Query Function updates

  • New caseSensitive option added to the parseTimestamp query function.
  • Queries involving join can now be with ‘used export to file’ and the /query HTTP endpoint.
  • New selectLast function, which is like select but aggregate.
  • Improved (reduced) memory consumption for live groupby, and for groupby’s involving many distinct keys.

Health Check APIs

The overall health of a Humio system is determined by a set of individual health checks. For more information about individual checks see the Health Check page and the Health Check API page.

IPFIX

Humio’s NetFlow support has been extended to also support IPFIX. See Humio’s docs for IPFIX.

Vega + chart series colors

This version replaces our chart library with Vega. The goal is to create a better, customizable, and more interactive charting experience in Humio. This first iteration is largely just a feature replacement for the existing functionality, with a few exceptions

Support for controlling color and title in widgets

Each chart type now supports assigning colors to specific series. This will allow you to, for instance, assign red to errors and green to non-errors.

You can find the series configuration controls in the Style tab of the Search page.

Control widget styling directly from dashboards

Now, you can click Edit Styling in the widget’s menu and modify styling directly from the dashboard view.

Time Chart series roll-up

To prevent the charts from getting cluttered, you can adjust the maximum number of series that should be shown in the chart. Any series that are not part of the top-most series will be summed together and added to a new series called Other.

Interpolation types

Linear interpolation is now the default, and we have added a new type of interpolation: Basis.

Bar Chart styling support

You can now style your bar charts to control things like label position and colors.

Pie Chart styling support

You can now style your pie charts, and they will default to having a center radius (actually making them donuts!).

Disabling Vega

Since charts are such a central feature, we allow disabling the new implementation of widgets if you are experiencing issues with them. You can disable Vega charts globally using the ENABLE_VEGA_CHARTS=false flag.

1.7.4

Bug fixes

Release Date: 2020-01-27
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • Add Chromium to the list of compatible browsers
  • Allow webhook notifiers to optionally not validate certificates.
  • Bug fix: join now accepts absolute timestamps in millis in start and end parameters.
  • Bug fix: Stabilized sync of uploaded files within a cluster in combination with bucket storage.
  • Bug fix: Allows “Force remove” of a node from a cluster.

1.7.3

Bug fixes

Release Date: 2020-01-17
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • New log output option for the LOG4J_CONFIGURATION configuration now allows the built-in: log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
  • ERROR logs get output to stderr instead of stdout to avoid breaking the potential stdout format.
  • top() function allows limit up to 20,000 by default now. Used to be 1,000.

1.7.2

Bug fixes

Release Date: 2020-01-16
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • Change: When the system starts with no users at all, the first user to log get root priviledges inside the system.
  • The “query monitor” and “query quota” new share the definition of “cost points”. The definition has changed in such a way that quotas saved by version up to 1.7.1 and earlier are disregarded by this (and later) versions.
  • New config: LIVEQUERY_STALE_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_STALE_CANCEL_COST_PERCENTAGE controls discard of live queries that have not been polled by a client for a while when the system experiences digest latency of more than the delay.
  • New config: LIVEQUERY_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_CANCEL_COST_PERCENTAGE controls cancelling of live queries that have been consuming the most cost for the previous 30s when the system experiences digest latency of more than the delay. New metrics: livequeries-canceled-due-to-digest-delay, livequeries-rate-canceled-due-to-digest-delay and livequeries-rate
  • New config: USING_EPHEMERAL_DISKS allows running a cluster on disks that may be lost when the system restarts by assuming that only copies in Bucket Storage and the events in Kafka are preserved across restarts. If the filesystem remains during restart this is also okay in this mode and more efficient then fetching the files from the bucket.
  • New config: LOG4J_CONFIGURATION allows a custom log4j file. Or set to one of the built-in: log4j2-stdout.xml to get the log in plain text dumped on stdout, or log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
  • New Utility inside the jar. Usage java -cp humio.jar com.humio.main.DecryptAESBucketStorageFile <secret string> <encrypted file> <decrypted file>. Allows decrypting a file that was uploaded using bucket storage outside the system.
  • Bug fix: The Zookeeper status page now shows a warning when the commands it needs for the status page to work are not whitelisted on the ZK server.
  • Bug fix: Restart of queries using lookup/match/cidr when the uploaded file changes only worked for top-level functions, not when nested inside another function.
  • Bug fix: Query of segments only present in a bucket now works even if disabling further uploads to bucket storage.
  • Bug fix: Bucket storage, GCP variant: Remove temporary files after download from GCP. Previous versions left a copy in the tmp dir.
  • Bug fix: Top(x, sum=y) now also support non-integer values of y (even though the internal state is still an integer value)
  • Bug fix: #repo=* never matched but should always match.
  • Bug fix: Retention could in fail to delete obsolete files in certain cases.
  • Bucket storage: Support download after switching provider from S3 to GCP or vice versa.
  • Bucket storage: Continue cleaning the old buckets after switching provider from S3 to GCP or vice versa.
  • Bucket storage: Also keep copies of the “metadata files” that you use for lookup and match functions in the bucket and restore from there when needed.

1.7.1

Bug fixes

Release Date: 2020-01-06
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: true

Bug fixes and removal of limitations.

Changelog

  • Bug fix: Handle large global snapshot files (larger than 2 G).
  • Allow explicit auto as argument to the span parameter in bucket and timechart. This makes it easier to set span from a macro argument.
  • Remove 64 K restriction on individual fields to be parsed by parsers.
  • Bug fix: Reuse of live dashboard queries on the humio-search-all repository did not work correctly. As an effect the number of live queries could keep increasing.
  • Bug fix: Saved Queries/macros was not expanded when checking if a live dashboard query could reuse an existing query.
  • Bug fix: The Postmark integration was always assuming a humio.com from address. This has been fixed by introducing a new POSTMARK_FROM configuration parameter.

1.7.0

Join, Bucket Storage Backend, Query Quotas, UI Improvements

Release Date: 2019-12-17
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: true

Joins

Humio now supports joins in the query language; the functionality is largely similar to what could previously be done by running a query, exporting it as a .csv, uploading said .csv file, and then using the match() function to filter/amend a query result. See docs.

Bucket Storage

Humio now supports storing segment files on Amazon S3 (and Google cloud storage) and compatible services to allow keeping more segment files than the local disks have room for and managing the local disk as a cache of these files. See docs.

New stable/preview release versioning

Stable release will have an even Minor version. If Minor is an odd number (like in this release), it is a preview release. Critical fixes will be back ported to the most recent stable release. More details can be found here.

Dashboard Improvements

To make it easier to integrate with external systems Humio dashboards, can now be passed URL parameters to set the dashboard’s global time interval. By passing query parameters ?time=<unix ms timestamp>&window=5m the dashboard will be opened with a 10m time window (5m before and after the the origin specified by time). The feature is not available for shared dashboards - since they do not support changing time intervals.

You can now also disable shared dashboards completely using the SHARED_DASHBOARDS_ENABLED=false configuration setting.

See the changelog for a more complete list of changes.

Changelog

  • Upgrading: After installling this version, it is not possible to roll back to a version lower than 1.6.10. Be on version 1.6.10 before upgrading to this version.
  • Top Feature: Bucket Storage with support for S3 and Google cloud storage, see description.
  • Top Feature: Joins allowing subqueries and joining data from multiple repositories, see description.
  • Top Feature: Query errors will now be highlighted as-you-type in on the search page.
  • Top Feature: The “Queries” page has been replaced with a dropdown on the Search page, that allows searching saved and recent queries.
  • Top Feature: Query quotas allowing limiting how many resources users can use when searching, see description
  • UI: Improved Query Monitor in the administration section, making it much easier to find expensive queries. See description
  • UI: Queries page removed, and delete and edit saved query functionality moved into “Queries” dropdown on search page.
  • UI: Add support for loading a specific time window when launching a dashboard, by setting time= and window= in the URL.
  • UI: Improve word-wrap and allow columns in the event list to be marked as ‘autosize’. Autosizing columns will adapt to the screen size when word-wrap is enabled.
  • UI: Word-wrap and event list orientation is now sticky in a session, meaning revisiting the search page will keep the previous selected options.
  • UI: Allow disabling automatically searching when entering a repository search page, on a per-repo basis.
  • UI: The time selector on dashboards now allow panning and zooming - like the one on the search page.
  • UI: Bugfix - Don’t show “unexpected error” screen when Auth Token expires.
  • UI: Bugfix - Ensure counts of fields and value occurrences on the event list are reliable.
  • Function: New function json:prettyPrint()
  • Function: New function xml:prettyPrint()
  • Function: New function callFunction, allows you to call a humio function by name. This is useful if you for instance want a dashboard where you can control what statistics your widgets show based on a parameter, e.g. timechart(function=callFunction(?statistic, field=response_time))
  • Function: The function top has a new max=field argument, that can be used to make it work as a more efficient alias a groupby/sort combination, like top(field, max=value, limit=5) is equivalent (and much faster than) groupby(field, function=max(value)) | sort(limit=5).
  • Function: The implementation of the percentile function has been updated to be more precise (and faster).
  • Config: COMPRESSION_TYPE=high is now the default compression type. Clusters running with default configuration, wil change to high compression unless the configuration COMPRESSION_TYPE=fast is set.
  • Config: Add SHARED_DASHBOARDS_ENABLED configuration setting which allows disabling access to the shared dashboards feature - if e.g. your organization has strict security policies.
  • Config: Autosharding can now bet set “sticky” which means fixed as set by user on a specific (input) datasource. The API also allows listing all autosharding rules, both system-manages and sticky.
  • New stable/preview release versioning scheme. See description.
  • Use case-insensitive comparison of usernames (historically an email address) when logging into Humio.
  • Java 13 is the recommended Java version. Docker images are now running Java 13.