Preview Releases

• Allow users group to be represented as a json string and not only array when logging in with oauth.
• New config: S3_ARCHIVING_IBM_COMPAT for compatility with S3 archiving to IBM Cloud Object Storage.

• Added IP Filter for readonly dashboard links, and started to audit log readonly dashboard access. In this initial version. The readonly ip filter can be configured with the graphql mutation: mutation { updateReadonlyDashboardIPFilter(ipFilter: “FILTER”) } The FILTER is expected in this format: ip-filter From Humio 1.25 this can be configured in the configuration UI.
• Added GraphQL queries and mutations for alerts and actions, which correspond to the deprecated REST endpoints for those entities.
• Added GraphQL mutations addAlertLabelV2, removeAlertLabelV2, addStarToAlertV2 and removeStarFromAlertV2.
• Added an option to make it easier to diagnose problems by detecting inconsistencies between globals in different Humio instances. Each Humio instance has its own copy of the global state which must all be identical. It has happened that they were not, so now we check and if there is a difference we report an error and dump the global state into a file.
• Added a new introduction message to empty repositories.
• Added mode parameter to match(), allowing different ways to match the key.
• Added support for CIDR matching on match() using mode=cidr.
• Improved performance when using match() with mode=cidr compared to using cidr() with file().
• Negated, non-strict match() or lookup is no longer allowed.
• Deprecated glob parameter on match(), use mode=glob instead.
• Deprecated file and column parameter on cidr(). Use match() with mode=cidr instead.
• Deprecated the ReadEvents enum variant from the ViewAction enum in GraphQL. Use the ReadContents variant instead, which has the same semantics, but a more accurate name. ReadEvents will be removed in a future release.
• Deprecated the REST endpoints for alerts and actions.
• Deprecated GraphQL mutations addAlertLabel, removeAlertLabel, addStarToAlert and removeStarFromAlert as they did not follow the standard for mutation input.
• The GraphiQL API Explorer is now available from inside Humio. You can access it using the Help->API Explorer menu.
• The GraphiQL API Explorer has been upgraded to a newer version. The new version includes a history of the queries that have been run.
• The SearchDomain.queries GraphQL field has been deprecated, and will be removed in a future release. Use SearchDomain.savedQueries instead.
• The SearchDomain.viewerCanChangeConnections GraphQL field has been deprecated, and will be removed in a future release. Use SearchDomain.isActionAllowed instead.
• Allow turning encryption of files stored in bucket storage off by explicitly setting S3_STORAGE_ENCRYPTION_KEY=off (similar for GCP_ )
• Removed the deprecated Repository.isFreemium GraphQL field.
• Removed support for shorthand IPv4 CIDR notation in cidr(). See section “Removed support for CIDR shorthand”.
• Changed the URL of the Kafka cluster page in the settings.
• Implemented toggle button for dark mode.
• UI enhancements for the new repository Access Permissions page.
• Mark required fields on the Accept Terms and Conditions page.
• Enforce accepting terms and conditions.
• Enforce permissions to enter Organization Settings page.
• Enforce permissions to enter creating new repository page.
• Refactor All Organizations page.
• Refactor Organization Overview page.
• Refactor client side action cache of allowed permissions.
• Refactor how the width of the repository name in the main navigation bar is calculated.
• It is again possible to sort the events on the test parser page.
• Made the S3 archiving save button work again.
• Restyled the alert dialogue.
• Improved memory use for certain numerical aggregrating functions
• Improved performance of free-text search using regular expressions
• Fixed the requirement condition for the time retention on a repository.
• Fixed an issue which caused ingest-to-multiple-repos() to break, when the parser used copyEvent to duplicate the input events into multiple repositories
• Fixed an issue causing undersized segment merging to repeatedly fetch the same segments, in cases where the merger job took too long to finish.
• Fixed an issue where regular expressions too large to handle would sometimes cause the query to hang. Now we report an error.
• Fixed an issue with the Missing Segments API that caused missing segments to not appear in the missing segments list if they had a replacement segment.
• Fixed an issue where changes to files would not propagate to parsers or event forwarders.
• Fixed an issue where Prometheus metrics always reported 0.0 for humio_primary_disk_usage
• Fixed a bug in event forwarding that made start(), end() and now() return the time at which the event forwarding rule was cached. Instead, now() will return the time at which the event forwarding rule was run. start() and end() were never meant to be used in an event forwarding rule and will return 0, which means Unix Epoch.
• Fixes a bug where events deleted with the delete-event API would appear deleted at first, but then resurface again after 24h. If user applying delete did not have permission to search the events being deleted.
• Fixed a bug which caused in() with values=[] to give incorrect results
• Fixed a bug which caused match() to give incorrect results in certain cases due to incorrect caching
• Fixed a bug which caused glob-patterns in match() to not match newline characters
• Fixed a bug which caused tag-filters in anonymous functions to not work in certain cases (causing to many events to be let through)
• Fixed a bug where the same regex pattern occurring multiple times in a query could cause incorrect results

• Improve performance of “decrypt step” in downloads from bucket storage
• New “prefetch from bucket” job. When a node starts with an empty disk it will download a relevant subset of segment files from the bucket in order to have them present locally for queries.
• Server: header in responses from from Humio HTTP server now includes (Vhost, NodeRole) after the version string.

• The default parser kv has been changed from using the parseTimestamp() function to use the findTimestamp() function. This will make it able to parse more timestamp formats. It will still only parse timestamps with a timezone. It also no longer adds a timezone field with the extracted timestamp string. This was only done for parsing the timestamp and not meant for storing on the event. To keep the old functionality, clone the kv parser in the relevant repositories and store the cloned parser with the name kv. This can be done before upgrading to this release. See docs.
• kvParse() now unescapes backslashes when they’re inside (' or ") quotes.
• kvParse() now only unescapes quotes and backslashes that are inside a quoted string.
• kvParse() now also unescapes single quotes. (')
• The findTimestamp() function has been changed, so that it no longer has a default value for the timezone parameter. Previously, the default was UTC. If no timezone argument is supplied to the function, it will not parse timestamps that do not contain a timezone. To get the old functionality, simply add timezone=UTC to the function. This can be done before upgrading to this release.
• The deprecated built-in parser json-for-notifier has been deleted. It has been replaced by the parser json-for-action.
• The deprecated built-in parser bro-json has been deleted. It has been replaced by the parser zeek-json.
• Split() no longer adds a @display field to the event it outputs.
• Make the thread dump job run on a dedicated thread, rather than running on the thread pool shared with other jobs.
• Added support for disaster recovery of a cluster where all nodes including Kafka has been lost, restoring the state present in bucket storage as a fresh cluster using the old bucket as read-only, and forming a fresh cluster from that. New Configs: S3_RECOVER_FROM_REPLACE_REGION and S3_RECOVER_FROM_REPLACE_BUCKET to allow modifying names of region/bucket while recovering to allow running on a replica, specifying read-only source using S3_RECOVER_FROM_* for all the bucket storage target parameters otherwise named S3_STORAGE_*
• Improve hit rate of query state cache by allowing similar but not identical queries to share cache when the entry in the cache can form the basis for both. The cache format is incompatible with previous versions, this is handled internally by handling incompatible cache entries as cache misses.
• Improve performance of writeJson() a bit.
• Improve number formatting in certain places by being better at removing trailing zeros.
• Change handling of groupBy() in live-queries which should in many cases reduce memory cost.
• The experimental function moment has been removed.
• subnet() now reports an error if its argument bits is outside the range 0 to 32.
• The transpose function now reports an error if the arguments header or column is provided together with the argument pivot.
• The replace() function now reports an error if the arguments replacement and with are provided at the same time.
• The replace() function now reports an error if an unsupported flag is provided in the flags argument.
• The functions worldMap() and geohash() now errors if requested precision is greater than 12.
• When on ephemeral disks, nodes being replaced with new ones on empty disks no longer download most of the segments they had before being replaced, but instead schedule downloads based on is being searhed.
• The Auth0 login page will no longer load a local version of the Auth-Lock library, but instead load a login script hosted on Auth0’s CDN. This may require opening access to https://cdn.auth0.com/ if hosting Humio behind a firewall.
• Lowered the severity level for some loggings for running alerts.
• Made loggings for running alerts more consistent and more structured. All loggings regarding a specific alert will contain the keys alertId, alertName and viewId. Loggings regarding the alert query will always contain the key externalQueryId and sometimes also the keys queryId with the internal id and query with the actual query string. If there are problems with the run-as-user, the id of that user is logged with the key user.
• Made loggings for running scheduled searches more consistent and more structured. All loggings regarding a specific alert will contain the keys scheduledSearchId, scheduledSearchName and viewId. Loggings regarding the alert query will always contain the key externalQueryId and sometimes also the keys queryId with the internal id and query with the actual query string. If there are problems with the run-as-user, the id of that user is logged with the key user.
• Prevent Humio from booting when Zookeeper has been reset but Kafka has not.
• Create, update and delete of an alert, scheduled search or action is now recorded in the audit log.
• Running test of a parser is no longer recorded in the audit log, and irrelevant fields are no longer recorded upon parser deletion.
• When using filters on dashboards, you can now easily reset the filter, either removing it completely, or using the default filter if one is present.
• Made sure the humio view humio default parser is only installed when missing, instead of overwriting it every time humio starts.
• When exporting a package, you now get a preview of the icon you’ve added for the package.
• Packages can now be updated with the same version but new content. This makes iterating over a package before finalizing it easier.
• Humio insights package installed if missing on the humio view when humio is started.
• Raised the parser test character length limit to 20000.
• Raised the note widget text length limit to 20000.
• Fixed a performance and a robustness problem with the function unit:convert(). The formatting of the numbers in its output may in some cases be different now.
• Fixed a number of potential concurrency issues.
• Fixed a memory leak in rdns() in cases where many different name servers are used.
• Fixed a bug in parseJson which resulted in failed JSON parsing if an object contained an empty key ("").
• Fixed a bug which caused eventInternals() to crash if used late in the pipeline.
• Fixed a bug which caused validation to miss rejecting window() inside window() and session().
• Fixed a bug which could cause saving of query state cache to take a rather long time.
• Fixed a bug which could potentially cause a query state cache file to be read in an incomplete state.
• Fixed a bug in upper() and lower() which could cause its output to be corrupted (in cases where no characters had been changed).
• Fixed a bug where analysis of a regex could consume extreme amounts of memory.
• Fixed a bug in lowercase() which caused the case lowercase(field=”*", include="values”) to not process all fields but only the field named “*".
• Fixed a bug where referenced saved queries were not referenced correctly after exporting them as part of a package.
• Fixed bugs in format() which caused output from ‘%e’/'%g’ to be incorrect in certain cases.
• Fixed an issue causing Humio to crash when attempting to delete an idle empty datasource right as the datasource receives new data.
• Fixed an issue with the validation of the query prefix set on a view for each repository within the view: Invoking macros is not allowed and was correctly rejected when creating a view, but was not rejected when editing an existing connection.
• Fixed an issue where merge of segments were reported as failed due to input files being deleted while merging. This is not an error, and is no longer reported as such.
• Fixed an issue where the segment mover might schedule too many segments for transfer at a time.
• Fixed an issue with lack of escaping in filename when downloading.
• Fixed an issue causing segment tombstones to potentially be deleted too early if bucket storage is enabled, causing an error log.
• Fixed an issue causing event redirection to break when using copyEvent to get the same events ingested into multiple repositories.
• Fixed an issue where repeating queries would not validate in alerts
• Fixed an issue where cancelled queries could be cached
• Bumped minimum supported versions of Chrome and Chromium from 60 to 69 due to updated dependencies - note added late

• Fixed an issue for on-prem users not on multitenant setup by reverted a metric change introduced in 1.18.0, jmx and Slf4j included an OrgId in all metrics for repositories.
• Fixed automatic installation of Humio Insights package to the humio repository

• Fixed bug where the format() function produced wrong output for some floating-point numbers
• Update dependencies with known vulnerabilities
• Do not retry a query when getting a HTTP 400 error
• Do not cache cancelled queries.
• Fixed bug in a saved query in the Humio Insights package
• Fixed an issue - Do not delete datasource before the segments have been deleted also in bucket storage if present there.

• Introduced humio insights package that is installed per default on startup on the humio repository
• Raised the limit for note widget text length to 20000
• Raised the parser test character length to 20000.
• No longer overwrite the humio parser in the humio repository on startup.
• Improved app loading logic.
• Kafka client inside Humio has been bumped from 2.4.1 to 2.6.0.
• Improve handling of broken local cache files
• New config QUERY_QUOTA_EXCEEDED_PENALTY with value 50 by default. When set >= 1.0 then this throttles queries from users that are over their quota by this factor rather than stopping their queries. Set to 0 to disable and revert to stopping queries.
• For ingest using a URL with a repository name in it, Humio now fails ingest if the repository in the URL does not match the repository of the ingest token. Previously, it would just use the repository of the ingest token.
• “Notifiers” have been renamed to “actions” throughout the UI and in log statements. The REST APIs have not been changed and all message templates can still be used.
• In the GraphQL API, on the Alert type, the notifiers field has been deprecated and will be removed in a later release. It has been replaced by the actions field.
• In the GraphQL API, the value ChangeAlertsAndNotifiers on the Permission enum has been deprecated and will be removed in a later release. It has been replaced by the ChangeTriggersAndActions value. The same is true for the ViewAction enum. On the ViewPermissionsType type, the administerAlertsfield has been deprecated and will be removed in a later release. It has been replaced by the administerTriggersAndActions field.
• The built-in json-for-notifier parser used by the Humio Repository action (formerly notifier) is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name json-for-action, see docs.
• The built-in bro-json parser is deprecated and will be removed in a later release. It has been replaced by an identical parser with the name zeek-json, see docs.
• The configuration option IP_FILTER_NOTIFIERS has been renamed to IP_FILTER_ACTIONS. The old name will continue to work.
• The configuration option HTTP_PROXY_ALLOW_NOTIFIERS_NOT_USE has been renamed to HTTP_PROXY_ALLOW_ACTIONS_NOT_USE. The old name will continue to work.
• New feature “Scheduled Searches” making it possible to run queries on a schedule and trigger actions (formerly notifiers) upon query results. See docs.
• New feature “Event forwarding” making it possible to forward events during ingest out of Humio to a Kafka server. See docs. Currently only available for on-prem customers.
• The Humio Repository action (formerly notifier) now replaces a prefix ‘#’ character in field names with @tag. so that #source becomes @tag.source. This is done to make them searchable in Humio. You can change the name by creating a custom parser. See docs for more details.
• The names of the metadata fields added by the Humio Repository action (formerly notifier) has been changed to accomodate that it can now also be used from scheduled searches. See docs for more details.
• New filter function test( <boolean expression> ) makes it convenient to test complex expressions.
• New ingest endpoint /api/v1/ingest/raw for ingesting singular webcalls as events. See docs.
• API Changes (Non-Documented API): getFileContent has been moved to a field on the SearchDomain type.
• API Changes (Non-Documented API): Queries and Mutations for Parser now expects an id field in place of a name field, when fetching and updating parsers.
• API Changes (Non-Documented API): Getting Alert by ID has been moved to a field on the SearchDomain type.
• New feature: “stateless Ingest-only nodes”: A node that the rest of the cluster does not know exists, but is capable of ingesting events into the ingest queue. Enable using NODE_ROLES=ingestonly.
• unit on timechart (and bucket) now works also when the function within uses nesting and anonymous pipielines.
• New feature “Custom ingest tokens” making it possible for root users to create ingest tokens with a custom string.
• The function parseCEF() now deals with extension fields with labels, i.e. cs1=Value cs1Label=Key becomes cef.label.Key=Value.
• Added asn function for retrieving the ASN number for a given IP address, see docs.
• New config AUTO_UPDATE_MAXMIND for enabling/disabling updating of all maxmind databases. Deprecates AUTO_UPDATE_IP_LOCATION_DB, but old config will continue to work.
• New config MAXMIND_IP_LOCATION_EDITION_ID for selecting the maxmind edition of the IP location database. Deprecates MAXMIND_EDITION_ID, but old config will continue to work.
• New function hash for computing hashes of fields. See docs.
• Renamed LOG4J_CONFIGURATION environment variable to HUMIO_LOG4J_CONFIGURATION. See Docs. The old variable will no longer work.
• New validation when creating an ingest token using the API that the parser, if specified, actually exists in the repository.
• Humio no longer deletes an existing humio-search-all view if the CREATE_HUMIO_SEARCH_ALL environment variable is false. The view instead becomes deleteable via the admin page.
• When a host dies and Humio reassigns digest, it will warn if a fallback host is picked that is in the same zone as existing replicas. Eliminate warning if falling back to a host in the null zone.
• Reduced the number of writes to global on restart, due to merge targets not being properly reused.
• Cluster management stats now shows segments as underreplicated if they are replicated to enough hosts, but are not present on all configured hosts.
• Add an error message to the event if the user is trying to redirect it to another repo using #repo, and the target repo is invalid.
• Make the query functions window and series be enabled by default. They can be disabled by seting the configuration options WINDOW_ENABLED and SERIES_ENABLED to false, respectively.
• The transfer job will delete primary copies shortly after transferring the segments to secondary storage. The copies would previously only be deleted once a full bulk had been moved.
• Added config option for Auth0 based sign on method: AUTH_ALLOW_SIGNUP defaults to true. The config is forwarded to the auth0 configuration for the lock widget setting: allowSignUp
• Reduce contention on the query scheduler input queue. It was previously possible for large queries to prevent each other from starting, leading to timeouts.
• Made cluster nodes log their own version as well as the versions of all other nodes. This makes it easier to tell which versions are running in the cluster.
• Introduction of the new log file humio-requests.log. Also the log format for the files humio-metrics.log and humio-nonsensitive.log has changed as described above. The guide for sending Humio logs to another Humio cluster has been updated.
• Custom made saved queries, alerts and dashboards in the humio repository searching for events of the kinds metrics, requests or nonsensitive may need to be modified. This is described in more detail here.
• Upgraded Log4j2 from 2.13.3 to 2.14.0.
• Removed config IDLE_POLL_TIME_BEFORE_DASHBOARD_QUERY_IS_CANCELLED_MINUTES. Queries on dashboards now have the same life cycle as other queries.
• Updated the permission checks when polling queries. This will results in dashboard links “created by users who are either deleted or lost permissions to the view” to get unauthorized. To list all dashboard links, run this graphql query as root: query { searchDomains {dashboards { readOnlyTokens { createdBy name token } } } }
• Added mutation to update the runAsUser for a read only dashboard token.
• Added timeout for TCP ingest listeners. By default the connection is closed if no data is received after 5 minutes. This can be changed by setting TCP_INGEST_MAX_TIMEOUT_SECONDS. See docs.
• Humio will only allow using Zookeeper for node id assignment (ZOOKEEPER_URL_FOR_NODE_UUID) when configured for ephemeral disks (USING_EPHEMERAL_DISKS). When using persistent disks, there is no need for the extra complexity added by Zookeeper.
• Fixed a rare issue where a node that was previously assigned digest could write a segment to global, even though it was no longer assigned the associated partition.
• Fixed an rare issue where the digest coordinator would assign digest fewer hosts than configured.
• Fixed an issue which caused free-text-search to not work correctly for large (>64KB) events.
• Fixed crash in CleanupDatasourceFilesJob when examining a file size fails due to that file being deleted concurrently.
• Fixed an issue where the segment rewrite job handling event deletion might rewrite segments sooner than configured.
• Fixed an issue where cancelling queries could produce a spurious error log.
• Fixed an issue with the cidr function that would make some IPv4 subnets accept IPv6 addresses and some strings that were not valid IP addresses.
• Fixed an issue that could cause node id assignment to fail when running on ephemeral disks and using Zookeeper for node id assignment. Nodes in this configuration will now try to pick a new id if their old id has been acquired by another node.
• Fixed an issue causing Humio to retain deleted minisegments in global for longer than expected.
• Fixed an issue where unit-convertion (by timechart) did not take effect through groupBy() and window().
• Fixed an issue causing queries using kvParse() to filter out too much in specific circumstances when filtering on a field assigned before kvParse().
• Fixed an issue causing queries using kvParse() to be executed incorrectly in certain circumstances when kvParse() assigned fields starting with a non-alphanumeric character.
• Fixed an issue causing the secondary storage transfer job to select and queue too many segments for transfer at once. The job will now stop and recalculate the bulk to transfer periodically.
• Fixed an issue with updating user profile, in some situations save failed.
• Fixed an issue where the filter and groupBy buttons on the search page would not restart the search automatically
• Fixed logic for when the organization owner panel should be shown in the User’s Danger zone.
• Fixed an issue where segment merge occasionally reported BrokenSegmentException when merging, while the segments where not broken.
• Fixed timeout issue in S3 Archving
• Fixed bug where repeating queries would not validate in alerts.
• Fixed a bug where fullscreen mode could end up blank

• New config option IP_FILTER_NOTIFIERS, to set up IP filters for Alert Notifications, see docs.
• New config option ENABLE_ALERTS makes it possible to disable alerts from running (enabled by default), see docs.
• New config option ALERT_DESPITE_WARNINGS makes it possible to trigger alerts even when warnings occur, see docs.
• New config option DEFAULT_MAX_NUMBER_OF_GLOBALDATA_DUMPS_TO_KEEP, see docs.
• New query function parameter to parseJson, removePrefixes, see docs.
• New query function concatArray, see docs.
• New query function parseCEF used to parse events which are formatted according to the Common Event Format(CEF), see docs.
• New experimental query function beta:repeating(), see docs.
• New experimental query function series(), enabled by config option SERIES_ENABLED=true, see docs.
• New experimental query function window(), enabled by config option WINDOW_ENABLED=true, see docs.
• The {events_html} notifier template will now respect the field order from the query, see docs.
• It is again possible to override a built-in parser in a repository by creating a parser with the same name.
• Periodically release object pools used by mapper pipeline, to avoid a possible source of memory leaks.
• Fix negating join expressions.
• Fixes a bug where join function in some circumstances would fetch subquery results from other cluster nodes more than once.
• Setting the default query for a view in the UI has been moved from the “Save as Query” to the View’s “Settings” tab.
• The notifier list is sorted when selecting notifiers for an alert.
• Improved wording of diagnostics regarding function arguments.
• Tweaked location of diagnostics regarding missing function arguments.
• API Changes (Non-Documented API): Saved Query REST API has been replaced by GraphQL.
• API Changes (Non-Documented API): View Settings REST API has been replaced by GraphQL.
• Allow running Humio on JDK-14 and JDK-15 to allow testing these new builds.
• Free-text search has been fixed to behave more in line with the specification.
• Refuse to boot if the global topic in Kafka does not contain the expected starting offset.
• Crash the node if an exception occurs while reading from the global Kafka topic, rather than trying to recover.
• Reduce the max fetch size for Kafka requests, as the previous size would sometimes lead to request timeouts.
• Improve logic attempting to ensure other live nodes can act as substitutes in case the preferred digest nodes are not available when writing new segments.
• Fixes the case where datasources receiving data might not be marked idle, causing Humio to retain too much ingest data in Kafka.
• Fixes the case where Humio would consider local node state when deciding which ingest data was safe to delete from Kafka.
• Fix several cases where Humio might attempt to write a message to Kafka larger than what Kafka will allow.
• Fixes the issue where Humio could behave incompatibly with Kafka versions prior to 2.3.0 if KAFKA_MANAGED_BY_HUMIO was true.
• Refuse to boot if the booting node would cause violations of the “Minimum previous Humio version” as listed in the release notes.
• Fixes an issue which caused free-text-search to not work correctly for large (>64KB) events.
• Fixes a bug where unit:convert couldn’t handle numbers in scientific notation.
• Rename a few scheduler threads so they reflect whether they’re associated with streaming queries (“streaming-scheduler”) or not (“normal-scheduler”)
• Fixes an issue where Humio might try to get admin access to Kafka when KAFKA_MANAGED_BY_HUMIO was false.
• If KAFKA_MANAGED_BY_HUMIO is true, Humio will ensure unclean leader election is disabled on the global-events topic.
• The Humio-search-all view will no longer be removed if CREATE_HUMIO_SEARCH_ALL is set to false. The view will instead become possible to delete manually via the admin UI.
• Reduce the number of merge target updates Humio will write to global on digest leader reassignment or reboot.
• Fixes a bug causing join() to not work after an aggregating function.
• Fixes a bug causing sort()/head()/tail() to work incorrectly after other aggregating functions.
• Fixes a bug causing the sub-queries of join() etc. to not see events with an @ingesttimestamp occurring later than the search time interval.
• Fixes an issue causing Humio to fail to upload files to bucket storage in rare cases.
• Switch from JDK to BouncyCastle provider for AES decrypt to reduce memory usage.
• Changed default TLS ciphers and protocols accepted by Humio, see docs.

• Generate ingest tokens in UUID format, replacing the current format for any new tokens being created.
• In the dialog for saving a search as an alert, the save button is no longer always grey and boring, but can actually save alerts again.
• Fixed a problem where the login link did not work in Safari and Firefox.
• Fixed a problem with scrolling on the login page on screens with low resolution.
• Made the login and sign up pages responsive to the device.
• Fixed a bug in the partition table optimizer that lead to unbalanced layouts.
• Fixed a bug causing an authentication error when trying to download a file when authenticating by proxy.
• Fixed an issue showing duplicate entries when searching for users.
• Fixed a memory leak when authenticating in AWS setups.
• Avoid overloading kafka with updates for the global database by collecting operations in bulk.
• Only consider fully replicated data when calculating which offsets can be pruned from Kafka.
• Improved naming of threads to get more usable thread dumps.
• Changed the query scheduling to account for the work of the overall query, rather than per job started. This allows fairer scheduling of queries hitting many dataspaces e.g. when using search-all.
• Changed priorities when fetching segments to a node which have been offline for a longer period. This avoids waiting too long before the cluster becomes fully synced.
• Improved handling of sub-queries polling state from the main query when using join().
• Fixed an issue where a slow data stream could cause Humio to retain more data in Kafka than necessary, as well as cause a restarted Humio node to reprocess too much data.
• Added logging to detect issues when truncating finished files.

• The job for updating the IP location database now uses the configured HTTP proxy, if present. See docs.
• Fixed a problem with AWS, where STS tokens would fail to authenticate.
• Fixed a problem in the UI, where the wrong timestamp was displayed as @ingesttimestamp.
• Revert login UI to same behavior as before 1.15.0.

• Bugfix: CSV files can no longer contain unnamed columns and also trailing commas are disallowed. Queries based on such files will now fail with an error.
• Improved error handling when a parser cannot be loaded. Before, this resulted in Humio returning an error to the data shipper. Now, data is ingested without being parsed, but marked with an error as described in Parser Errors.
• If automatically creating users upon login and syncing their groups from the authentication mechanims, the configuration ONLY_CREATE_USER_IF_SYNCED_GROUPS_HAVE_ACCESS now controls whether users should only be created if the synced groups have access to a repository or view. The default is false.
• Humio will set the field @ingesttimestamp on all events.
• S3 communication can be configured to not use an HTTP proxy. See docs.
• Upgrade to AWS SDK v2. When using Java system properties for configuring Humio bucket storage, use aws.secretAccessKey instead of aws.secretKey.
• New explicit signup and login pages for social login.
• Newly added users will by default get an email. See docs.
• Alert notifiers can be configured to not use an HTTP proxy. See docs.
• Field based throttling on alerts. See docs.
• New alert notifier type logging to a Humio repository. See docs.
• New alert notifier template {events_html} formatting events as an HTML table. See docs.
• Auto-balanced partition table suggestions. See ZONE, DIGEST_REPLICATION_FACTOR, STORAGE_REPLICATION_FACTOR in configuration. See docs.

• This release fixes a security issue. More information will follow when Humio customers have had time to upgrade. See: Humio Security Disclosures.
• Bugfix: export to file now allows for fieldnames with special characters.
• Bugfix: export to file can now include query parameters
• Bugfix: missing migration of non-default groups would result in alerts failing until the user backing the alert logs in again.

• This release fixes a security issue. For more information see: Humio Security Disclosures.
• Fix issue where a query could fail to search all segments if digest reassignment was occurring at the same time as the query.
• Fix issue where a node with no digest assignment could fail to delete local segment copies in some cases.

• This release fixes a security issue. For more information see: Humio Security Disclosures.
• Bugfix: avoid forbidden access error on shared dashboard links by ensuring correct use of time stamps

• Bugfix: all ingest methods now support the ALLOW_CHANGE_REPO_ON_EVENTS configuration parameter
• Bugfix: avoid saving invalid bucket storage configurations
• Bugfix: export to file no longer fails/timeouts on heavy sub queries
• Bugfix: joins will now propagate limit warnings from sub queries to the main query
• Bugfix: make sure join-subqueries gets canceled when the main query is canceled
• Default groups added

• Improved query scheduling on machines with many cores. This can improve search speeds significantly.
• Support for a new storage format for segment files that will be introduced in a later release (to support rollback)
• Bugfix: S3Archiving could write events twice in a special case (When a merge happens where all inputs have been archived, write in global that the merge-result was archived too).
• Bugfix: Bucket storage in GCP could did not clean up all tmp files

• Free text search now searches all fields rather than only @rawstring.
• Humio can now balance and reuse existing queries internally in the cluster. See Humio Configuration
• Internal communication in a Humio installation can now be encrypted using TLS. See TLS Configuration
• Added support for WebIdentityTokenCredentialsProvider on AWS.
• The data source for the ipLocation() query function is no longer shipped with humio but installed/updated separately.
• Introduced a new ChangeViewOrRepositoryDescription permission for editing the description of a view or repository. This was previously tied to ConnectView and any user with that permission will now have the new permission as well.

Elastic Bulk API change - Fluent Bit users might need to change the Fluent Bit configuration To ensure compatability with the newest Beats clients, the Elastic Bulk API has been changed to always return the full set of status information for all operations, as it is done in the official Elastic API

This can however cause problems when using Fluent Bit to ingest data into Humio.

Fluent Bit in default configuration uses a small buffer (4KB) for responses from the Elastic Bulk API, which causes problems when enough operations are bulked together. The response will then be larger than the response buffer as it contains the status for each individual operation. Make sure the response buffer is large enough, otherwise Fluent Bit will stop shipping data. See: https://github.com/fluent/fluent-bit/issues/2156 and https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch

Other changes

• Several improvements to memory handling
• Several improvements to query error handling
• Dashboard widgets now display an error if data is not compatible with the widget

• Allow for emergency logins if the primary login provider is having problems. Emergency user subsystem
• Query function selfJoin
• Query function selfJoinFilter
• Query function findTimestamp
• Query function unit:convert
• Query function formatDuration
• New configuration MAX_CHARS_TO_FIND_TIMESTAMP. Default value should work for most deployments. See docs
• New built-in parser zeek-json.
• Gauge widget now works for arbitrary numbers not only for aggregated numbers.

• Security: [critical] Fixed more security vulnerabilities discovered through proactive penetration testing (more information will be forthcoming).
• New query function fieldstats.
• More efficient “collect” function implementation.
• Allow more concurrent processing to take place in “export” query processing.
• Styling improvements in the “Style” panel for widgets.
• New Time Chart interpolation options.
• New options for dealing with missing data in Time Charts.
• Improves responsiveness of the recent queries dropdown, and limits the number of stored recent queries to 100 per user per repository.
• Allow dots in tagged field names.
• If at startup the global-snapshot.json file is missing, then try loading the “.1” backup copy.
• Bug fix: The segment queue length metric was not correct when segments got fetched from bucket storage by a query.
• Bug fix: the query metric only measured time for streaming queries, now it includes non-streaming as well.
• Bug fix: api-explorer not working due to CSP inline script.
• Improve disk space monitoring when using bucket storage.

• Added API to the list and deleted missing segments from global. See reference here.
• Security: [critical] Fixed a security vulnerability discovered through proactive penetration testing (more information will be forthcoming).

• This is a critical update. Self-hosted systems with access for non-trusted users should upgrade immediately. We will follow up with more details when this update has been rolled out.
• Bug fix: the health-check failed-http-status-check would get stuff in warn state, this has now been fixed

Role Based Access Control

Role Based Access Control (RBAC) through the UI is now the only permission model in Humio. Please see the Authorization documentation for more information.

Kafka version update

• Updated Humio to use Kafka 2.4. Humio can still use versions of Kafka down through 1.1.
• Be aware that updating Kafka also requires you to update Zookeeper to 3.5.6. There is a migration involved in updating Zookeeper. See Zookeeper’s migration FAQ here. Use the migration approach using an empty snapshot. The other proposed solution can loose data.
• Updated Kafka and Zookeeper Docker images to use Kafka 2.4. Updating to Kafka 2.4 should be straightforward using Humio’s Kafka/Zookeeper Docker images. Zookeper image will handle migration. Stop all Kafka nodes. Stop all Zookeeper nodes. Start all Zookeeper nodes on the new version. Start all Kafka nodes on the new version. Before updating Kafka/Zookeeper, we recommend backing up the Zookeeper data directory. Then, add the Zookeeper properties described below. If you are deploying Kafka/Zookeeper using other tools, for example Ansible scripts, be aware there is a migration involved in updating Zookeeper.
• When updating Kafka/Zookeeper we recommend setting these Zookeeper properties

  # Do not start the new admin server. Default port 8080 conflicts with Humio port.admin.enableServer=false
  # purge old snapshot files autopurge.purgeInterval=1
  # Allow 4 letter commands. Used by Humio to get info about the Zookeeper cluster 4lw.commands.whitelist=*
  

Query Function updates

• New caseSensitive option added to the parseTimestamp query function.
• Queries involving join can now be with ‘used export to file’ and the /query HTTP endpoint.
• New selectLast function, which is like select but aggregate.
• Improved (reduced) memory consumption for live groupby, and for groupby’s involving many distinct keys.

Health Check APIs

The overall health of a Humio system is determined by a set of individual health checks. For more information about individual checks see the Health Check page and the Health Check API page.

IPFIX

Humio’s NetFlow support has been extended to also support IPFIX. See Humio’s docs for IPFIX.

Vega + chart series colors

This version replaces our chart library with Vega. The goal is to create a better, customizable, and more interactive charting experience in Humio. This first iteration is largely just a feature replacement for the existing functionality, with a few exceptions

Support for controlling color and title in widgets

Each chart type now supports assigning colors to specific series. This will allow you to, for instance, assign red to errors and green to non-errors.

You can find the series configuration controls in the Style tab of the Search page.

Control widget styling directly from dashboards

Now, you can click Edit Styling in the widget’s menu and modify styling directly from the dashboard view.

Time Chart series roll-up

To prevent the charts from getting cluttered, you can adjust the maximum number of series that should be shown in the chart. Any series that are not part of the top-most series will be summed together and added to a new series called Other.

Interpolation types

Linear interpolation is now the default, and we have added a new type of interpolation: Basis.

Bar Chart styling support

You can now style your bar charts to control things like label position and colors.

Pie Chart styling support

You can now style your pie charts, and they will default to having a center radius (actually making them donuts!).

Disabling Vega

Since charts are such a central feature, we allow disabling the new implementation of widgets if you are experiencing issues with them. You can disable Vega charts globally using the ENABLE_VEGA_CHARTS=false flag.

• Add Chromium to the list of compatible browsers
• Allow webhook notifiers to optionally not validate certificates.
• Bug fix: join now accepts absolute timestamps in millis in start and end parameters.
• Bug fix: Stabilized sync of uploaded files within a cluster in combination with bucket storage.
• Bug fix: Allows “Force remove” of a node from a cluster.

• New log output option for the LOG4J_CONFIGURATION configuration now allows the built-in: log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
• ERROR logs get output to stderr instead of stdout to avoid breaking the potential stdout format.
• top() function allows limit up to 20,000 by default now. Used to be 1,000.

• Change: When the system starts with no users at all, the first user to log get root priviledges inside the system.
• The “query monitor” and “query quota” new share the definition of “cost points”. The definition has changed in such a way that quotas saved by version up to 1.7.1 and earlier are disregarded by this (and later) versions.
• New config: LIVEQUERY_STALE_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_STALE_CANCEL_COST_PERCENTAGE controls discard of live queries that have not been polled by a client for a while when the system experiences digest latency of more than the delay.
• New config: LIVEQUERY_CANCEL_TRIGGER_DELAY_MS and LIVEQUERY_CANCEL_COST_PERCENTAGE controls cancelling of live queries that have been consuming the most cost for the previous 30s when the system experiences digest latency of more than the delay. New metrics: livequeries-canceled-due-to-digest-delay, livequeries-rate-canceled-due-to-digest-delay and livequeries-rate
• New config: USING_EPHEMERAL_DISKS allows running a cluster on disks that may be lost when the system restarts by assuming that only copies in Bucket Storage and the events in Kafka are preserved across restarts. If the filesystem remains during restart this is also okay in this mode and more efficient then fetching the files from the bucket.
• New config: LOG4J_CONFIGURATION allows a custom log4j file. Or set to one of the built-in: log4j2-stdout.xml to get the log in plain text dumped on stdout, or log4j2-stdout-json.xml to get the log in NDJSON format, one line for each event on stdout.
• New Utility inside the jar. Usage java -cp humio.jar com.humio.main.DecryptAESBucketStorageFile <secret string> <encrypted file> <decrypted file>. Allows decrypting a file that was uploaded using bucket storage outside the system.
• Bug fix: The Zookeeper status page now shows a warning when the commands it needs for the status page to work are not whitelisted on the ZK server.
• Bug fix: Restart of queries using lookup/match/cidr when the uploaded file changes only worked for top-level functions, not when nested inside another function.
• Bug fix: Query of segments only present in a bucket now works even if disabling further uploads to bucket storage.
• Bug fix: Bucket storage, GCP variant: Remove temporary files after download from GCP. Previous versions left a copy in the tmp dir.
• Bug fix: Top(x, sum=y) now also support non-integer values of y (even though the internal state is still an integer value)
• Bug fix: #repo=* never matched but should always match.
• Bug fix: Retention could in fail to delete obsolete files in certain cases.
• Bucket storage: Support download after switching provider from S3 to GCP or vice versa.
• Bucket storage: Continue cleaning the old buckets after switching provider from S3 to GCP or vice versa.
• Bucket storage: Also keep copies of the “metadata files” that you use for lookup and match functions in the bucket and restore from there when needed.

• Bug fix: Handle large global snapshot files (larger than 2 G).
• Allow explicit auto as argument to the span parameter in bucket and timechart. This makes it easier to set span from a macro argument.
• Remove 64 K restriction on individual fields to be parsed by parsers.
• Bug fix: Reuse of live dashboard queries on the humio-search-all repository did not work correctly. As an effect the number of live queries could keep increasing.
• Bug fix: Saved Queries/macros was not expanded when checking if a live dashboard query could reuse an existing query.
• Bug fix: The Postmark integration was always assuming a humio.com from address. This has been fixed by introducing a new POSTMARK_FROM configuration parameter.

• Upgrading: After installling this version, it is not possible to roll back to a version lower than 1.6.10. Be on version 1.6.10 before upgrading to this version.
• Top Feature: Bucket Storage with support for S3 and Google cloud storage, see description.
• Top Feature: Joins allowing subqueries and joining data from multiple repositories, see description.
• Top Feature: Query errors will now be highlighted as-you-type in on the search page.
• Top Feature: The “Queries” page has been replaced with a dropdown on the Search page, that allows searching saved and recent queries.
• Top Feature: Query quotas allowing limiting how many resources users can use when searching, see description
• UI: Improved Query Monitor in the administration section, making it much easier to find expensive queries. See description
• UI: Queries page removed, and delete and edit saved query functionality moved into “Queries” dropdown on search page.
• UI: Add support for loading a specific time window when launching a dashboard, by setting time= and window= in the URL.
• UI: Improve word-wrap and allow columns in the event list to be marked as ‘autosize’. Autosizing columns will adapt to the screen size when word-wrap is enabled.
• UI: Word-wrap and event list orientation is now sticky in a session, meaning revisiting the search page will keep the previous selected options.
• UI: Allow disabling automatically searching when entering a repository search page, on a per-repo basis.
• UI: The time selector on dashboards now allow panning and zooming - like the one on the search page.
• UI: Bugfix - Don’t show “unexpected error” screen when Auth Token expires.
• UI: Bugfix - Ensure counts of fields and value occurrences on the event list are reliable.
• Function: New function json:prettyPrint()
• Function: New function xml:prettyPrint()
• Function: New function callFunction, allows you to call a humio function by name. This is useful if you for instance want a dashboard where you can control what statistics your widgets show based on a parameter, e.g. timechart(function=callFunction(?statistic, field=response_time))
• Function: The function top has a new max=field argument, that can be used to make it work as a more efficient alias a groupby/sort combination, like top(field, max=value, limit=5) is equivalent (and much faster than) groupby(field, function=max(value)) | sort(limit=5).
• Function: The implementation of the percentile function has been updated to be more precise (and faster).
• Config: COMPRESSION_TYPE=high is now the default compression type. Clusters running with default configuration, wil change to high compression unless the configuration COMPRESSION_TYPE=fast is set.
• Config: Add SHARED_DASHBOARDS_ENABLED configuration setting which allows disabling access to the shared dashboards feature - if e.g. your organization has strict security policies.
• Config: Autosharding can now bet set “sticky” which means fixed as set by user on a specific (input) datasource. The API also allows listing all autosharding rules, both system-manages and sticky.
• New stable/preview release versioning scheme. See description.
• Use case-insensitive comparison of usernames (historically an email address) when logging into Humio.
• Java 13 is the recommended Java version. Docker images are now running Java 13.