Timestamp formatting
Transform column definitions that use either the datetime
or epoch
datatypes require the additional definition of a format
property. This property lets Hydrolix know how to parse and interpret the column’s incoming time data.
You can optionally specify a resolution
property as well, in order to tune the granularity of the stored time data.
Formatting datetime
values
The datetime
datatype allows most any format of time representation in incoming data, provided that you can describe that format as a Go-style time format string. In that column definition’s format
property, you must indicate how the data source would represent Go’s “reference time” of 03:04:05 PM on January 2, 2006, in the Mountain Standard Time zone (UTC-07:00).
A handful of examples:
Possible data value | The literal format to specify |
---|---|
2020-12-01 | 2006-01-02 |
12-01-20 13:05:00 | 01-02-06 15:04:05 |
2020-12-01T08:05:00-0500 | 2006-01-02T15:04:05-0700 |
Formatting epoch
values
Use the epoch
datatype for Unix-style “epoch time”, or other representations of a timestamp a number of time-units that have passed since 1970-01-01 00:00:00 UTC, a.k.a. the epoch.
With epoch
, acceptable values of format
include the following:
Format | Meaning |
---|---|
ns |
nanoseconds since epoch |
us |
microseconds since epoch |
ms |
milliseconds since epoch |
cs |
centiseconds since epoch |
s |
seconds since epoch (a.k.a. Unix time) |
In all cases, the Hydrolix ingester accepts either integers or numerical strings as legal epoch
data.
Hydrolix treats all epoch
-based timestamps as UTC time.
Specifying resolution
Hydrolix supports storing time values with either second or millisecond resolution. You can set this granularity level for a given field by specifying a resolution
property within the appropriate output_columns
definition, as discussed in Column definitions.
Hydrolix uses seconds as its default stored-time resolution.
Resolution and truncation
If an incoming time value contains a finer resolution than you’ve set its target column definition to receive, Hydrolix will truncate the fractional-second part as necessary prior to storage.
For example, if you’ve set a given column to have millisecond resolution, and incoming data for that field includes a time down to the microsecond, Hydrolix will store the value after rounding down to the nearest millisecond.