Preview Releases

Below is a list of all of the Preview releases of Humio, in reverse order of their release — the latest at the top and the oldest at the bottom. If you want to see all of the Stable releases go to the Stable Releases page for a list of them. For a list of all releases, Preview and Stable, go to the All Releases page.

Remember, the way to distinguish between Preview and Stable releases is based on the secondary number for the release. If it’s an even number, it indicates it’s a stable release; if the secondary number is an odd number, it indicates it’s a preview of the stable release that will follow it. See the main Humio Releases page for a more detailed explanation of this numbering system.

1.19.0

New query editor, Packages (in beta)

Release Date: 2021-01-14
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: true

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.19.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded at least to 1.16.0 before trying to upgrade to 1.19.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer. Rolling directly back to an earlier release can result in data loss.

Packages

This version introduces Humio packages - a way of bundling and sharing assets such as dashboards and parsers. You can create your own packages to keep your Humio assets in Git or create utility packages that can be installed in multiple repositories. All assets can be serialized to YAML files (like what has been possible for dashboards for a while). With tight integration with Humio’s CLI humioctl you can install packages from local disk, URL, or directly from a Github repository. Packages are still in beta, but we encourage you do start creating packages yourself, and sharing them with the community. At Humio we are also very interested in talking with package authors about getting your packages on our upcoming marketplace. Read more about packages on our package documentation.

Humio Insights application

With the introcution of Humio packages we have created the application Humio Insights. The application is a collection of dashboards and saved searches making it possible to monitor and observe a Humio cluster.

New Query Editor

The new query editor has a much better integration with Humio’s query language. It will give you suggestions as you type, and gives you inline errors if you make a mistake. We will continue to improve the capabilities of the query editor to be aware of fields, saved queries, and other contextual information.

New Test Function

A new function called test() has been added for convenience. What used to be done like: tmp := <expression> | tmp=true can now be done using: test( <expression> ). Inside <expression> field names appearing on the right hand side of an equality test, such as field1==field2 compares the values of the two fields. When comparing using = at top-level field1=field2 compares the value of field1 against the string "field2". This distinction is a cause of confusion for some users, and using test() simplifies that.

Changes to Humio’s internal logging

We have made small changes to how Humio logs internally. We did this to better support the new Humio Insights Application. We have tried to keep the changes as small and compatible as possible, but we have made some changes that can break existing searches in the humio repository (or other repositories receiving Humio logs). We made these changes as we think they are important in order to improve things moving forward. One of the benefits is the new Humio Insights App Read more about the details here

Changelog

  • Introduced humio insights package that is installed per default on startup on the humio repository
  • Raised the limit for note widget text length to 20000
  • Raised the parser test character length to 20000.
  • No longer overwrite the humio parser in the humio repository on startup.
  • Improved app loading logic.
  • Kafka client inside Humio has been bumped from 2.4.1 to 2.6.0.
  • Improve handling of broken local cache files
  • New config QUERY_QUOTA_EXCEEDED_PENALTY with value 50 by default. When set >= 1.0 then this throttles queries from users that are over their quota by this factor rather than stopping their queries. Set to 0 to disable and revert to stopping queries.
  • For ingest using a URL with a repository name in it, Humio now fails ingest if the repository in the URL does not match the repository of the ingest token. Previously, it would just use the repository of the ingest token.
  • “Notifiers” have been renamed to “actions” throughout the UI and in log statements. The REST APIs have not been changed and all message templates can still be used.
  • In the GraphQL API, on the Alert type, the notifiers field has been deprecated and will be removed in a later release. It has been replaced by the actions field.
  • In the GraphQL API, the value ChangeAlertsAndNotifiers on the Permission enum has been deprecated and will be removed in a later release. It has been replaced by the ChangeTriggersAndActions value. The same is true for the ViewAction enum. On the ViewPermissionsType type, the administerAlertsfield has been deprecated and will be removed in a later release. It has been replaced by the administerTriggersAndActions field.
  • The built-in json-for-notifier parser used by the Humio Repository action (formerly notifier) is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name json-for-action, see docs.
  • The built-in bro-json parser is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name zeek-json, see docs.
  • The configuration option IP_FILTER_NOTIFIERS has been renamed to IP_FILTER_ACTIONS. The old name will continue to work.
  • The configuration option HTTP_PROXY_ALLOW_NOTIFIERS_NOT_USE has been renamed to HTTP_PROXY_ALLOW_ACTIONS_NOT_USE. The old name will continue to work.
  • New feature “Scheduled Searches” making it possible to run queries on a schedule and trigger actions (formerly notifiers) upon query results. See docs.
  • New feature “Event forwarding” making it possible to forward events during ingest out of Humio to a Kafka server. See docs. Currently only available for on-prem customers.
  • The Humio Repository action (formerly notifier) now replaces a prefix ‘#’ character in field names with @tag. so that #source becomes @tag.source. This is done to make them searchable in Humio. You can change the name by creating a custom parser. See docs for more details.
  • The names of the metadata fields added by the Humio Repository action (formerly notifier) has been changed to accomodate that it can now also be used from scheduled searches. See docs for more details.
  • New filter function test( <boolean expression> ) makes it convenient to test complex expressions.
  • New ingest endpoint /api/v1/ingest/raw for ingesting singular webcalls as events. See docs.
  • API Changes (Non-Documented API): getFileContent has been moved to a field on the SearchDomain type.
  • API Changes (Non-Documented API): Queries and Mutations for Parser now expects an id field in place of a name field, when fetching and updating parsers.
  • API Changes (Non-Documented API): Getting Alert by ID has been moved to a field on the SearchDomain type.
  • New feature: “stateless Ingest-only nodes”: A node that the rest of the cluster does not know exists, but is capable of ingesting events into the ingest queue. Enable using NODE_ROLES=ingestonly.
  • unit on timechart (and bucket) now works also when the function within uses nesting and anonymous pipielines.
  • New feature “Custom ingest tokens” making it possible for root users to create ingest tokens with a custom string.
  • The function parseCEF() now deals with extension fields with labels, i.e. cs1=Value cs1Label=Key becomes cef.label.Key=Value.
  • Added asn function for retrieving the ASN number for a given IP address, see docs.
  • New config AUTO_UPDATE_MAXMIND for enabling/disabling updating of all maxmind databases. Deprecates AUTO_UPDATE_IP_LOCATION_DB, but old config will continue to work.
  • New config MAXMIND_IP_LOCATION_EDITION_ID for selecting the maxmind edition of the IP location database. Deprecates MAXMIND_EDITION_ID, but old config will continue to work.
  • New function hash for computing hashes of fields. See docs.
  • Renamed LOG4J_CONFIGURATION environment variable to HUMIO_LOG4J_CONFIGURATION. See Docs. The old variable will no longer work.
  • New validation when creating an ingest token using the API that the parser, if specified, actually exists in the repository.
  • Humio no longer deletes an existing humio-search-all view if the CREATE_HUMIO_SEARCH_ALL environment variable is false. The view instead becomes deleteable via the admin page.
  • When a host dies and Humio reassigns digest, it will warn if a fallback host is picked that is in the same zone as existing replicas. Eliminate warning if falling back to a host in the null zone.
  • Reduced the number of writes to global on restart, due to merge targets not being properly reused.
  • Cluster management stats now shows segments as underreplicated if they are replicated to enough hosts, but are not present on all configured hosts.
  • Add an error message to the event if the user is trying to redirect it to another repo using #repo, and the target repo is invalid.
  • Make the query functions window and series be enabled by default. They can be disabled by seting the configuration options WINDOW_ENABLED and SERIES_ENABLED to false, respectively.
  • The transfer job will delete primary copies shortly after transferring the segments to secondary storage. The copies would previously only be deleted once a full bulk had been moved.
  • Added config option for Auth0 based sign on method: AUTH_ALLOW_SIGNUP defaults to true. The config is forwarded to the auth0 configuration for the lock widget setting: allowSignUp
  • Reduce contention on the query scheduler input queue. It was previously possible for large queries to prevent each other from starting, leading to timeouts.
  • Made cluster nodes log their own version as well as the versions of all other nodes. This makes it easier to tell which versions are running in the cluster.
  • Introduction of the new log file humio-requests.log. Also the log format for the files humio-metrics.log and humio-nonsensitive.log has changed as described above. The guide for sending Humio logs to another Humio cluster has been updated.
  • Custom made saved queries, alerts and dashboards in the humio repository searching for events of the kinds metrics, requests or nonsensitive may need to be modified. This is described in more detail here.
  • Upgraded Log4j2 from 2.13.3 to 2.14.0.
  • Removed config IDLE_POLL_TIME_BEFORE_DASHBOARD_QUERY_IS_CANCELLED_MINUTES. Queries on dashboards now have the same life cycle as other queries.
  • Updated the permission checks when polling queries. This will results in dashboard links “created by users who are either deleted or lost permissions to the view” to get unauthorized. To list all dashboard links, run this graphql query as root: query { searchDomains {dashboards { readOnlyTokens { createdBy name token } } } }
  • Added mutation to update the runAsUser for a read only dashboard token.
  • Added timeout for TCP ingest listeners. By default the connection is closed if no data is received after 5 minutes. This can be changed by setting TCP_INGEST_MAX_TIMEOUT_SECONDS. See docs.
  • Humio will only allow using Zookeeper for node id assignment (ZOOKEEPER_URL_FOR_NODE_UUID) when configured for ephemeral disks (USING_EPHEMERAL_DISKS). When using persistent disks, there is no need for the extra complexity added by Zookeeper.
  • Fixed a rare issue where a node that was previously assigned digest could write a segment to global, even though it was no longer assigned the associated partition.
  • Fixed an rare issue where the digest coordinator would assign digest fewer hosts than configured.
  • Fixed an issue which caused free-text-search to not work correctly for large (>64KB) events.
  • Fixed crash in CleanupDatasourceFilesJob when examining a file size fails due to that file being deleted concurrently.
  • Fixed an issue where the segment rewrite job handling event deletion might rewrite segments sooner than configured.
  • Fixed an issue where cancelling queries could produce a spurious error log.
  • Fixed an issue with the cidr function that would make some IPv4 subnets accept IPv6 addresses and some strings that were not valid IP addresses.
  • Fixed an issue that could cause node id assignment to fail when running on ephemeral disks and using Zookeeper for node id assignment. Nodes in this configuration will now try to pick a new id if their old id has been acquired by another node.
  • Fixed an issue causing Humio to retain deleted minisegments in global for longer than expected.
  • Fixed an issue where unit-convertion (by timechart) did not take effect through groupBy() and window().
  • Fixed an issue causing queries using kvParse() to filter out too much in specific circumstances when filtering on a field assigned before kvParse().
  • Fixed an issue causing queries using kvParse() to be executed incorrectly in certain circumstances when kvParse() assigned fields starting with a non-alphanumeric character.
  • Fixed an issue causing the secondary storage transfer job to select and queue too many segments for transfer at once. The job will now stop and recalculate the bulk to transfer periodically.
  • Fixed an issue with updating user profile, in some situations save failed.
  • Fixed issue where the filter and groupBy buttons on the search page would not restart the search automatically
  • Fixed logic for when the organization owner panel should be shown in the User’s Danger zone.
  • Fixed an issue where segment merge occasionally reported BrokenSegmentException when merging, while the segments where not broken.
  • Fixed timeout issue in S3 Archving
  • Fixed bug where repeating queries would not validate in alerts.
  • Fixed a bug where fullscreen mode could end up blank

1.17.0

Repeating Queries, search result caching and new query functions.

Release Date: 2020-11-18
Minimum previous Humio version: 1.16.0
Requires data migrations: false
Has changes to configuration: true

Repeating Queries, search result caching and new query functions.

Important information about upgrading

Beginning with version 1.17.0, if your current version of Humio is not directly able to upgrade to the new version, you will get an error if you attempt to start up the incompatible version. The 1.17.0 release is only compatible with Humio release 1.16.0 and newer. This means that you will have to ensure that you have upgraded to minimum 1.16.0 before trying to upgrade to 1.17.0. In case you need to do a rollback, this can also ONLY happen back to 1.16.0 or newer, rolling directly back to earlier release can result in data loss.

Repeating Queries

Humio can now run repeating queries using the beta:repeating() function. These are live queries that are implemented by repeatedly making a query. This allows using functions in alerts and dashboards that typically do not work in live queries, such as selfJoin() or selfJoinFilter(). See the function docs for more information.

Improved Security for Alerts

In order to prevent alert notifiers being used to probe services on the internal network (eg. Zookeeper or the AWS metadata service), Humio now has an IP filter on alert notifiers. The default is to block access to all link-local addresses and any addresses on the internal network; however, you can opt-in to the old behavior by setting the configuration option IP_FILTER_NOTIFIERS to allow all. See IP Filter documentation.

New experimental query function series()

A new experimental query function called series() has been added. It needs to be explicitly enabled on the cluster using the config option SERIES_ENABLED=true.

series() improves upon session() and collect() for grouping events into transactions.

What used to be done with:

groupby(id, function=session(function=collect([fields, ...])))

can now be done using:

groupby(id, function=series([fields, ..])).

See docs for more details.

Search result caching

This new feature stores a copy of live search results to the local disk in the server nodes, and reuses the relevant parts of that cached result when an identical live search is later started. Caching is controlled with the config option QUERY_CACHE_MIN_COST, which has a default value of 1000. To disable caching, set the config option to a very high number, such as 9223372036854775807 (max long value).

Changelog

  • New config option IP_FILTER_NOTIFIERS, to set up IP filters for Alert Notifications, see docs.
  • New config option ENABLE_ALERTS makes it possible to disable alerts from running (enabled by default), see docs.
  • New config option ALERT_DESPITE_WARNINGS makes it possible to trigger alerts even when warnings occur, see docs.
  • New config option DEFAULT_MAX_NUMBER_OF_GLOBALDATA_DUMPS_TO_KEEP, see docs.
  • New query function parameter to parseJson, removePrefixes, see docs.
  • New query function concatArray, see docs.
  • New query function parseCEF used to parse events which are formatted according to the Common Event Format(CEF), see docs.
  • New experimental query function beta:repeating(), see docs.
  • New experimental query function series(), enabled by config option SERIES_ENABLED=true, see docs.
  • New experimental query function window(), enabled by config option WINDOW_ENABLED=true, see docs.
  • The {events_html} notifier template will now respect the field order from the query, see docs.
  • It is again possible to override a built-in parser in a repository by creating a parser with the same name.
  • Periodically release object pools used by mapper pipeline, to avoid a possible source of memory leaks.
  • Fix negating join expressions.
  • Fixes a bug where join function in some circumstances would fetch subquery results from other cluster nodes more than once.
  • Setting the default query for a view in the UI has been moved from the “Save as Query” to the View’s “Settings” tab.
  • The notifier list is sorted when selecting notifiers for an alert.
  • Improved wording of diagnostics regarding function arguments.
  • Tweaked location of diagnostics regarding missing function arguments.
  • API Changes (Non-Documented API): Saved Query REST API has been replaced by GraphQL.
  • API Changes (Non-Documented API): View Settings REST API has been replaced by GraphQL.
  • Allow running Humio on JDK-14 and JDK-15 to allow testing these new builds.
  • Free-text search has been fixed to behave more in line with the specification.
  • Refuse to boot if the global topic in Kafka does not contain the expected starting offset.
  • Crash the node if an exception occurs while reading from the global Kafka topic, rather than trying to recover.
  • Reduce the max fetch size for Kafka requests, as the previous size would sometimes lead to request timeouts.
  • Improve logic attempting to ensure other live nodes can act as substitutes in case the preferred digest nodes are not available when writing new segments.
  • Fixes the case where datasources receiving data might not be marked idle, causing Humio to retain too much ingest data in Kafka.
  • Fixes the case where Humio would consider local node state when deciding which ingest data was safe to delete from Kafka.
  • Fix several cases where Humio might attempt to write a message to Kafka larger than what Kafka will allow.
  • Fixes the issue where Humio could behave incompatibly with Kafka versions prior to 2.3.0 if KAFKA_MANAGED_BY_HUMIO was true.
  • Refuse to boot if the booting node would cause violations of the “Minimum previous Humio version” as listed in the release notes.
  • Fixes an issue which caused free-text-search to not work correctly for large (>64KB) events.
  • Fixes a bug where unit:convert couldn’t handle numbers in scientific notation.
  • Rename a few scheduler threads so they reflect whether they’re associated with streaming queries (“streaming-scheduler”) or not (“normal-scheduler”)
  • Fixes an issue where Humio might try to get admin access to Kafka when KAFKA_MANAGED_BY_HUMIO was false.
  • If KAFKA_MANAGED_BY_HUMIO is true, Humio will ensure unclean leader election is disabled on the global-events topic.
  • The Humio-search-all view will no longer be removed if CREATE_HUMIO_SEARCH_ALL is set to false. The view will instead become possible to delete manually via the admin UI.
  • Reduce the number of merge target updates Humio will write to global on digest leader reassignment or reboot.
  • Fixes a bug causing join() to not work after an aggregating function.
  • Fixes a bug causing sort()/head()/tail() to work incorrectly after other aggregating functions.
  • Fixes a bug causing the sub-queries of join() etc. to not see events with an @ingesttimestamp occurring later than the search time interval.
  • Fixes an issue causing Humio to fail to upload files to bucket storage in rare cases.
  • Switch from JDK to BouncyCastle provider for AES decrypt to reduce memory usage.
  • Changed default TLS ciphers and protocols accepted by Humio, see docs.

1.15.2

Bugfixes and stability enhancements.

Release Date: 2020-09-29
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bugfixes and stability enhancements.

Changelog

  • Generate ingest tokens in UUID format, replacing the current format for any new tokens being created.
  • In the dialog for saving a search as an alert, the save button is no longer always grey and boring, but can actually save alerts again.
  • Fixed a problem where the login link did not work in Safari and Firefox.
  • Fixed a problem with scrolling on the login page on screens with low resolution.
  • Made the login and sign up pages responsive to the device.
  • Fixed a bug in the partition table optimizer that lead to unbalanced layouts.
  • Fixed a bug causing an authentication error when trying to download a file when authenticating by proxy.
  • Fixed an issue showing duplicate entries when searching for users.
  • Fixed a memory leak when authenticating in AWS setups.
  • Avoid overloading kafka with updates for the global database by collecting operations in bulk.
  • Only consider fully replicated data when calculating which offsets can be pruned from Kafka.
  • Improved naming of threads to get more usable thread dumps.
  • Changed the query scheduling to account for the work of the overall query, rather than per job started. This allows fairer scheduling of queries hitting many dataspaces e.g. when using search-all.
  • Changed priorities when fetching segments to a node which have been offline for a longer period. This avoids waiting too long before the cluster becomes fully synced.
  • Improved handling of sub-queries polling state from the main query when using join().
  • Fixed an issue where a slow data stream could cause Humio to retain more data in Kafka than necessary, as well as cause a restarted Humio node to reprocess too much data.
  • Added logging to detect issues when truncating finished files.

1.15.1

Bugfixes and IP location DB HTTP_PROXY support.

Release Date: 2020-09-22
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bugfixes and IP location DB HTTP_PROXY support.

Changelog

  • The job for updating the IP location database now uses the configured HTTP proxy, if present. See docs.
  • Fixed a problem with AWS, where STS tokens would fail to authenticate.
  • Fixed a problem in the UI, where the wrong timestamp was displayed as @ingesttimestamp.
  • Revert login UI to same behavior as before 1.15.0.

1.15.0

Improved alerts and addition of ingest timestamps

Release Date: 2020-09-15
Minimum previous Humio version: 1.12.0
Requires data migrations: true
Has changes to configuration: true

Ingest timestamps

Humio will set ingest timestamps on all events. This is set in the field named @ingesttimestamp. In later versions, Humio will also support specifying the search time interval using @ingesttimestamp when searching. This will support use cases where data is backfilled etc.

Alerts and notifiers

Field based throttling

It is now possible to make an alert throttle based on a field, so that new values for the field trigger the alert, but already seen values do not until the throttle period has elapsed. See docs.

Notifier logging to a Humio repository

It is now possible to configure an alert notifier that will log all events to a Humio repository. See docs.

Slack notifier upgrade to notify multiple Slack channels

It is now possible to use the Slack notifier to notify multiple slack channels at once. See docs.

Events as HTML table

In an email notifier, it is now possible to format the events as an HTML table using the new message template {events_html}. See docs.

Configure notifier to not use the internet proxy

It is now possible to configure an alert notifier to not use the HTTP proxy configured in Humio. See docs.

User signup/login flow

Signup & Login

We introduce new signup/login pages for social login and have split the behavior so users have to explicitly either login or signup.

Invite flow

When adding a user to Humio they will now by default get an email telling them that they have been invited to use Humio. See docs.

AWS Java SDK V2

The AWS SDK Humio uses has been upgraded to v2. When configuring Humio bucket storage with Java system properties, the access key must now be in the aws.secretAccessKey property instead of the aws.secretKey property.

Configure Humio to not use the internet proxy for S3

It is now possible to configure Humio to not use the globally configured HTTP proxy for communcation with S3. See docs.

Auto-balanced partition table suggestions

When changing digest and storage partitions it is now possible to get auto-balanced suggestions based on node zone and replication factor settings (via ZONE, DIGEST_REPLICATION_FACTOR, STORAGE_REPLICATION_FACTOR configurations). See docs.

Changelog

  • Bugfix: CSV files can no longer contain unnamed columns and also trailing commas are disallowed. Queries based on such files will now fail with an error.
  • Improved error handling when a parser cannot be loaded. Before, this resulted in Humio returning an error to the data shipper. Now, data is ingested without being parsed, but marked with an error as described in Parser Errors.
  • If automatically creating users upon login and syncing their groups from the authentication mechanims, the configuration ONLY_CREATE_USER_IF_SYNCED_GROUPS_HAVE_ACCESS now controls whether users should only be created if the synced groups have access to a repository or view. The default is false.
  • Humio will set the field @ingesttimestamp on all events.
  • S3 communication can be configured to not use an HTTP proxy. See docs.
  • Upgrade to AWS SDK v2. When using Java system properties for configuring Humio bucket storage, use aws.secretAccessKey instead of aws.secretKey.
  • New explicit signup and login pages for social login.
  • Newly added users will by default get an email. See docs.
  • Alert notifiers can be configured to not use an HTTP proxy. See docs.
  • Field based throttling on alerts. See docs.
  • New alert notifier type logging to a Humio repository. See docs.
  • New alert notifier template {events_html} formatting events as an HTML table. See docs.
  • Auto-balanced partition table suggestions. See ZONE, DIGEST_REPLICATION_FACTOR, STORAGE_REPLICATION_FACTOR in configuration. See docs.

1.13.5

Security and bugfixes

Release Date: 2020-08-12
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfixes

Changelog

  • This release fixes a security issue. More information will follow when Humio customers have had time to upgrade. See: Humio Security Disclosures.
  • Bugfix: export to file now allows for fieldnames with special characters.
  • Bugfix: export to file can now include query parameters
  • Bugfix: missing migration of non-default groups would result in alerts failing until the user backing the alert logs in again.

1.13.4

Security and bugfixes

Release Date: 2020-08-05
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfixes

Changelog

  • This release fixes a security issue. For more information see: Humio Security Disclosures.
  • Fix issue where a query could fail to search all segments if digest reassignment was occurring at the same time as the query.
  • Fix issue where a node with no digest assignment could fail to delete local segment copies in some cases.

1.13.3

Security and bugfix

Release Date: 2020-08-04
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Security and bugfix

Changelog

  • This release fixes a security issue. For more information see: Humio Security Disclosures.
  • Bugfix: avoid forbidden access error on shared dashboard links by ensuring correct use of time stamps

1.13.2

Bug fixes

Release Date: 2020-08-03
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: true

Bug fixes

Changelog

  • Bugfix: all ingest methods now support the ALLOW_CHANGE_REPO_ON_EVENTS configuration parameter
  • Bugfix: avoid saving invalid bucket storage configurations
  • Bugfix: export to file no longer fails/timeouts on heavy sub queries
  • Bugfix: joins will now propagate limit warnings from sub queries to the main query
  • Bugfix: make sure join-subqueries gets canceled when the main query is canceled
  • Default groups added

1.13.1

Bug fixes and improved search speeds for many-core systems

Release Date: 2020-07-03
Minimum previous Humio version: 1.12.0
Requires data migrations: false
Has changes to configuration: false

Bug fixes and improved search speeds for many-core systems

Changelog

  • Improved query scheduling on machines with many cores. This can improve search speeds significantly.
  • Support for a new storage format for segment files that will be introduced in a later release (to support rollback)
  • Bugfix: S3Archiving could write events twice in a special case (When a merge happens where all inputs have been archived, write in global that the merge-result was archived too).
  • Bugfix: Bucket storage in GCP could did not clean up all tmp files

1.13.0

Release Date: 2020-06-24
Minimum previous Humio version: 1.12.0
Requires data migrations: true
Has changes to configuration: true

Free text search now searches all fields rather than only @rawstring.

Load balancing of queries

Humio can now balance and reuse existing queries internally in the cluster. Load balancer configuration to achieve this is no longer needed. See Humio Configuration and Reverse proxy configuration.

TLS support

TLS Encrypt communication using TLS to/from Zookeeper, Kafka, and other Humio nodes.

Iplocation database management changed

The database used as data source for the ipLocation() query function must be updated within 30 days when a new version of the database is made public by MaxMind. To comply with this, the databased is no longer shipped as part of the humio artifacts but will either:

  • Be fetched automatically by Humio provided that Humio is allowed to connect to the db updata service hosted by Humio. This is the default behaviour.
  • Have to be updated manually (See IP location)

If the database cannot be automatically updated and no database is provided manually, the ipLocation() query function will no longer work.

Configuration change: Controlling what nodes to use as query coordinators

Due to the load balancing in Humio, customers that previously relied on load balancing to control which nodes are query coordinators now need to set QUERY_COORDINATOR to false on nodes they do not want to become query coordinators. See Humio Configuration and Reverse proxy configuration.

Changelog

  • Free text search now searches all fields rather than only @rawstring.
  • Humio can now balance and reuse existing queries internally in the cluster. See Humio Configuration
  • Internal communication in a Humio installation can now be encrypted using TLS. See TLS Configuration
  • Added support for WebIdentityTokenCredentialsProvider on AWS.
  • The data source for the ipLocation() query function is no longer shipped with humio but installed/updated separately.
  • Introduced a new ChangeViewOrRepositoryDescription permission for editing the description of a view or repository. This was previously tied to ConnectView and any user with that permission will now have the new permission as well.

1.11.1

Bug fixes and memory optimizations

Release Date: 2020-05-28
Minimum previous Humio version: 1.10.0
Requires data migrations: false
Has changes to configuration: false

Bug fixes and memory optimizations

Changelog

Elastic Bulk API change - Fluent Bit users might need to change the Fluent Bit configuration To ensure compatability with the newest Beats clients, the Elastic Bulk API has been changed to always return the full set of status information for all operations, as it is done in the official Elastic API

This can however cause problems when using Fluent Bit to ingest data into Humio.

Fluent Bit in default configuration uses a small buffer (4KB) for responses from the Elastic Bulk API, which causes problems when enough operations are bulked together. The response will then be larger than the response buffer as it contains the status for each individual operation. Make sure the response buffer is large enough, otherwise Fluent Bit will stop shipping data. See: https://github.com/fluent/fluent-bit/issues/2156 and https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch

Other changes

  • Several improvements to memory handling
  • Several improvements to query error handling
  • Dashboard widgets now display an error if data is not compatible with the widget

1.11.0

Export to bucket, findTimestamp, selfjoin, Emergency user subsystem

Release Date: 2020-05-19
Minimum previous Humio version: 1.10.0
Requires data migrations: true
Has changes to configuration: true

SelfJoin

selfJoin query function allows selecting log lines that share an identifier; for which there exists (separate) log lines that match a certain filtering criteria; such as “all log lines with a given userid for which there exists a successful and an unsuccessful login”.

findTimestamp

findTimestamp query function will try to find and parse timestamps in incoming data. The function should be used in parsers and support automatic detection of timestamps. It can be used instead of making regular expressions specifying where to find the timestamp and parsing it with parseTimestamp. Checkout the documentation for details.

Export to bucket storage/S3

As an alternative to downloading streaming queries directly, Humio can now upload them to an S3 or GCS bucket from which the user can download the data. See docs

Emergency user subsystem

If there are issues with the identity provider that Humio is configured to use, it might not be possible to log in to Humio. To mitigate this, Humio now provides emergency users that can be created locally within the Humio cluster. See docs

Changelog

1.9.3

Security fixes, bug fixes, and timechart improvements

Release Date: 2020-04-22
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security Fixes

A few security vulnerabilities have been discovered as part of a proactive penetration test. None are known to have been exploited. More information will be forthcoming.

New Time Charts Features

Dealing with missing data points in timecharts

This release improves the handling of missing data points in time charts. Previously you could either interpolate missing data points based on the surrounding data, or leave gaps in the charts. With the introduction of the new charts in 1.9.0 the gaps became more apparent than previously, and we have added new options to deal with missing data points. These replace the previous option “Allow Gaps”, with four new options:

  • Do Nothing - This will leave gaps in your data
  • Linear Interpolation - Impute values using linear interpolation based on the nearest known data points.
  • Replace by Mean Value - Replace missing values with the mean value of the series.
  • Replace by Zero - Replace missing values with zeros.

New line interpolation options

The release also introduces new options for line interpolation.

  • Monotone
  • Natural
  • Cardinal
  • Catmull-Rom
  • Bundle

The latter three are impacted by the ‘tension’ setting in the timechart Style editor.

Changelog

  • Security: [critical] Fixed more security vulnerabilities discovered through proactive penetration testing (more information will be forthcoming).
  • New query function fieldstats.
  • More efficient “collect” function implementation.
  • Allow more concurrent processing to take place in “export” query processing.
  • Styling improvements in the “Style” panel for widgets.
  • New Time Chart interpolation options.
  • New options for dealing with missing data in Time Charts.
  • Improves responsiveness of the recent queries dropdown, and limits the number of stored recent queries to 100 per user per repository.
  • Allow dots in tagged field names.
  • If at startup the global-snapshot.json file is missing, then try loading the “.1” backup copy.
  • Bug fix: The segment queue length metric was not correct when segments got fetched from bucket storage by a query.
  • Bug fix: the query metric only measured time for streaming queries, now it includes non-streaming as well.
  • Bug fix: api-explorer not working due to CSP inline script.
  • Improve disk space monitoring when using bucket storage.

1.9.2

Security fix and bug fixes

Release Date: 2020-03-25
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security fix and bug fixes

Changelog

  • Added API to the list and deleted missing segments from global. See reference here.
  • Security: [critical] Fixed a security vulnerability discovered through proactive penetration testing (more information will be forthcoming).

1.9.1

Security fix and bug fixes

Release Date: 2020-03-23
Minimum previous Humio version: 1.8.5
Requires data migrations: false
Has changes to configuration: false

Security fix and bug fixes

Changelog

  • This is a critical update. Self-hosted systems with access for non-trusted users should upgrade immediately. We will follow up with more details when this update has been rolled out.
  • Bug fix: the health-check failed-http-status-check would get stuff in warn state, this has now been fixed

1.9.0

UI for Role Based Access Control (RBAC), Health Check API, Kafka version update, Vega charts

Release Date: 2020-03-12
Minimum previous Humio version: 1.8.5
Requires data migrations: true
Has changes to configuration: true

RBAC, Health Check API, Kafka, Vega

Changelog

Role Based Access Control

Role Based Access Control (RBAC) through the UI is now the only permission model in Humio. Please see the Authorization documentation for more information.

Kafka version update

  • Updated Humio to use Kafka 2.4. Humio can still use versions of Kafka down through 1.1.
  • Be aware that updating Kafka also requires you to update Zookeeper to 3.5.6. There is a migration involved in updating Zookeeper. See Zookeeper’s migration FAQ here. Use the migration approach using an empty snapshot. The other proposed solution can loose data.
  • Updated Kafka and Zookeeper Docker images to use Kafka 2.4. Updating to Kafka 2.4 should be straightforward using Humio’s Kafka/Zookeeper Docker images. Zookeper image will handle migration. Stop all Kafka nodes. Stop all Zookeeper nodes. Start all Zookeeper nodes on the new version. Start all Kafka nodes on the new version. Before updating Kafka/Zookeeper, we recommend backing up the Zookeeper data directory. Then, add the Zookeeper properties described below. If you are deploying Kafka/Zookeeper using other tools, for example Ansible scripts, be aware there is a migration involved in updating Zookeeper.
  • When updating Kafka/Zookeeper we recommend setting these Zookeeper properties
    # Do not start the new admin server. Default port 8080 conflicts with Humio port.admin.enableServer=false
    # purge old snapshot files autopurge.purgeInterval=1
    # Allow 4 letter commands. Used by Humio to get info about the Zookeeper cluster 4lw.commands.whitelist=*
    

Query Function updates

  • New caseSensitive option added to the parseTimestamp query function.
  • Queries involving join can now be with ‘used export to file’ and the /query HTTP endpoint.
  • New selectLast function, which is like select but aggregate.
  • Improved (reduced) memory consumption for live groupby, and for groupby’s involving many distinct keys.

Health Check APIs

The overall health of a Humio system is determined by a set of individual health checks. For more information about individual checks see the Health Check page and the Health Check API page.

IPFIX

Humio’s NetFlow support has been extended to also support IPFIX. See Humio’s docs for IPFIX.

Vega + chart series colors

This version replaces our chart library with Vega. The goal is to create a better, customizable, and more interactive charting experience in Humio. This first iteration is largely just a feature replacement for the existing functionality, with a few exceptions

Support for controlling color and title in widgets

Each chart type now supports assigning colors to specific series. This will allow you to, for instance, assign red to errors and green to non-errors.

You can find the series configuration controls in the Style tab of the Search page.

Control widget styling directly from dashboards

Now, you can click Edit Styling in the widget’s menu and modify styling directly from the dashboard view.

Time Chart series roll-up

To prevent the charts from getting cluttered, you can adjust the maximum number of series that should be shown in the chart. Any series that are not part of the top-most series will be summed together and added to a new series called Other.

Interpolation types

Linear interpolation is now the default, and we have added a new type of interpolation: Basis.

Bar Chart styling support

You can now style your bar charts to control things like label position and colors.

Pie Chart styling support

You can now style your pie charts, and they will default to having a center radius (actually making them donuts!).

Disabling Vega

Since charts are such a central feature, we allow disabling the new implementation of widgets if you are experiencing issues with them. You can disable Vega charts globally using the ENABLE_VEGA_CHARTS=false flag.

1.7.4

Bug fixes

Release Date: 2020-01-27
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • Add Chromium to the list of compatible browsers
  • Allow webhook notifiers to optionally not validate certificates.
  • Bug fix: join now accepts absolute timestamps in millis in start and end parameters.
  • Bug fix: Stabilized sync of uploaded files within a cluster in combination with bucket storage.
  • Bug fix: Allows “Force remove” of a node from a cluster.

1.7.3

Bug fixes

Release Date: 2020-01-17
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • New log output option for the LOG4J_CONFIGURATION configuration now allows the built-in: log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
  • ERROR logs get output to stderr instead of stdout to avoid breaking the potential stdout format.
  • top() function allows limit up to 20,000 by default now. Used to be 1,000.

1.7.2

Bug fixes

Release Date: 2020-01-16
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: false

Bug fixes

Changelog

  • Change: When the system starts with no users at all, the first user to log get root priviledges inside the system.
  • The “query monitor” and “query quota” new share the definition of “cost points”. The definition has changed in such a way that quotas saved by version up to 1.7.1 and earlier are disregarded by this (and later) versions.
  • New config: LIVEQUERY_STALE_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_STALE_CANCEL_COST_PERCENTAGE controls discard of live queries that have not been polled by a client for a while when the system experiences digest latency of more than the delay.
  • New config: LIVEQUERY_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_CANCEL_COST_PERCENTAGE controls cancelling of live queries that have been consuming the most cost for the previous 30s when the system experiences digest latency of more than the delay. New metrics: livequeries-canceled-due-to-digest-delay, livequeries-rate-canceled-due-to-digest-delay and livequeries-rate
  • New config: USING_EPHEMERAL_DISKS allows running a cluster on disks that may be lost when the system restarts by assuming that only copies in Bucket Storage and the events in Kafka are preserved across restarts. If the filesystem remains during restart this is also okay in this mode and more efficient then fetching the files from the bucket.
  • New config: LOG4J_CONFIGURATION allows a custom log4j file. Or set to one of the built-in: log4j2-stdout.xml to get the log in plain text dumped on stdout, or log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
  • New Utility inside the jar. Usage java -cp humio.jar com.humio.main.DecryptAESBucketStorageFile <secret string> <encrypted file> <decrypted file>. Allows decrypting a file that was uploaded using bucket storage outside the system.
  • Bug fix: The Zookeeper status page now shows a warning when the commands it needs for the status page to work are not whitelisted on the ZK server.
  • Bug fix: Restart of queries using lookup/match/cidr when the uploaded file changes only worked for top-level functions, not when nested inside another function.
  • Bug fix: Query of segments only present in a bucket now works even if disabling further uploads to bucket storage.
  • Bug fix: Bucket storage, GCP variant: Remove temporary files after download from GCP. Previous versions left a copy in the tmp dir.
  • Bug fix: Top(x, sum=y) now also support non-integer values of y (even though the internal state is still an integer value)
  • Bug fix: #repo=* never matched but should always match.
  • Bug fix: Retention could in fail to delete obsolete files in certain cases.
  • Bucket storage: Support download after switching provider from S3 to GCP or vice versa.
  • Bucket storage: Continue cleaning the old buckets after switching provider from S3 to GCP or vice versa.
  • Bucket storage: Also keep copies of the “metadata files” that you use for lookup and match functions in the bucket and restore from there when needed.

1.7.1

Bug fixes

Release Date: 2020-01-06
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: true

Bug fixes and removal of limitations.

Changelog

  • Bug fix: Handle large global snapshot files (larger than 2 G).
  • Allow explicit auto as argument to the span parameter in bucket and timechart. This makes it easier to set span from a macro argument.
  • Remove 64 K restriction on individual fields to be parsed by parsers.
  • Bug fix: Reuse of live dashboard queries on the humio-search-all repository did not work correctly. As an effect the number of live queries could keep increasing.
  • Bug fix: Saved Queries/macros was not expanded when checking if a live dashboard query could reuse an existing query.
  • Bug fix: The Postmark integration was always assuming a humio.com from address. This has been fixed by introducing a new POSTMARK_FROM configuration parameter.

1.7.0

Join, Bucket Storage Backend, Query Quotas, UI Improvements

Release Date: 2019-12-17
Minimum previous Humio version: 1.6.10
Requires data migrations: false
Has changes to configuration: true

Joins

Humio now supports joins in the query language; the functionality is largely similar to what could previously be done by running a query, exporting it as a .csv, uploading said .csv file, and then using the match() function to filter/amend a query result. See docs.

Bucket Storage

Humio now supports storing segment files on Amazon S3 (and Google cloud storage) and compatible services to allow keeping more segment files than the local disks have room for and managing the local disk as a cache of these files. See docs.

New stable/preview release versioning

Stable release will have an even Minor version. If Minor is an odd number (like in this release), it is a preview release. Critical fixes will be back ported to the most recent stable release. More details can be found here.

Dashboard Improvements

To make it easier to integrate with external systems Humio dashboards, can now be passed URL parameters to set the dashboard’s global time interval. By passing query parameters ?time=<unix ms timestamp>&window=5m the dashboard will be opened with a 10m time window (5m before and after the the origin specified by time). The feature is not available for shared dashboards - since they do not support changing time intervals.

You can now also disable shared dashboards completely using the SHARED_DASHBOARDS_ENABLED=false configuration setting.

See the changelog for a more complete list of changes.

Changelog

  • Upgrading: After installling this version, it is not possible to roll back to a version lower than 1.6.10. Be on version 1.6.10 before upgrading to this version.
  • Top Feature: Bucket Storage with support for S3 and Google cloud storage, see description.
  • Top Feature: Joins allowing subqueries and joining data from multiple repositories, see description.
  • Top Feature: Query errors will now be highlighted as-you-type in on the search page.
  • Top Feature: The “Queries” page has been replaced with a dropdown on the Search page, that allows searching saved and recent queries.
  • Top Feature: Query quotas allowing limiting how many resources users can use when searching, see description
  • UI: Improved Query Monitor in the administration section, making it much easier to find expensive queries. See description
  • UI: Queries page removed, and delete and edit saved query functionality moved into “Queries” dropdown on search page.
  • UI: Add support for loading a specific time window when launching a dashboard, by setting time= and window= in the URL.
  • UI: Improve word-wrap and allow columns in the event list to be marked as ‘autosize’. Autosizing columns will adapt to the screen size when word-wrap is enabled.
  • UI: Word-wrap and event list orientation is now sticky in a session, meaning revisiting the search page will keep the previous selected options.
  • UI: Allow disabling automatically searching when entering a repository search page, on a per-repo basis.
  • UI: The time selector on dashboards now allow panning and zooming - like the one on the search page.
  • UI: Bugfix - Don’t show “unexpected error” screen when Auth Token expires.
  • UI: Bugfix - Ensure counts of fields and value occurrences on the event list are reliable.
  • Function: New function json:prettyPrint()
  • Function: New function xml:prettyPrint()
  • Function: New function callFunction, allows you to call a humio function by name. This is useful if you for instance want a dashboard where you can control what statistics your widgets show based on a parameter, e.g. timechart(function=callFunction(?statistic, field=response_time))
  • Function: The function top has a new max=field argument, that can be used to make it work as a more efficient alias a groupby/sort combination, like top(field, max=value, limit=5) is equivalent (and much faster than) groupby(field, function=max(value)) | sort(limit=5).
  • Function: The implementation of the percentile function has been updated to be more precise (and faster).
  • Config: COMPRESSION_TYPE=high is now the default compression type. Clusters running with default configuration, wil change to high compression unless the configuration COMPRESSION_TYPE=fast is set.
  • Config: Add SHARED_DASHBOARDS_ENABLED configuration setting which allows disabling access to the shared dashboards feature - if e.g. your organization has strict security policies.
  • Config: Autosharding can now bet set “sticky” which means fixed as set by user on a specific (input) datasource. The API also allows listing all autosharding rules, both system-manages and sticky.
  • New stable/preview release versioning scheme. See description.
  • Use case-insensitive comparison of usernames (historically an email address) when logging into Humio.
  • Java 13 is the recommended Java version. Docker images are now running Java 13.