Auto Values
Hydrolix has automatic values that can be included within a transform, listed below.
Additionally, there are WURFL-related auto values available for device identification when WURFL is enabled.
Auto Value Name | Description | Stream | Kafka | Batch |
---|---|---|---|---|
current_time | Current time | Yes | Yes | Yes |
index_pool | Pool name used by the indexer (stream peer, kafka peer, batch peer) | Yes | Yes | Yes |
input_filename | For batch only, specify the filename indexed. The filename is the full s3 path: s3://bucket/folder/subfolder/filename.csv | No | No | Yes |
intake_method | Specify the intake method (Batch, Stream / Kafka) | Yes | Yes | Yes |
intake_pool | Pool name used by the intake service. Blank if unassigned. | Yes | Yes | Yes |
kafka_topic | Specify the Kafka topic used to index the data | Yes | No | No |
project_name | Name of the project where the data will be indexed | Yes | Yes | Yes |
project_revision | Revision number of the project where the data will be indexed | Yes | Yes | Yes |
project_uuid | UUID of the project where the data will be indexed | Yes | Yes | Yes |
receive_time | Time when the data is received | Yes | Yes | Yes |
table_name | Name of the table where the data will be indexed | Yes | Yes | Yes |
table_revision | Revision number of the table where the data will be indexed. | Yes | Yes | Yes |
table_uuid | UUID of the table where the data will be indexed | Yes | Yes | Yes |
transform_name | Name of the transform used to index the data | Yes | Yes | Yes |
transform_revision | Revision number of the transform used to index the data | Yes | Yes | Yes |
transform_type | Transform type (csv or json). | Yes | Yes | Yes |
transform_uuid | UUID of the transform used to index the data. | Yes | Yes | Yes |
input_filename_segment_[0-12] | Extract part of the filename with batch/autoingest where the separator is / Example: s3://bucket/folder/subfolder/filename.csv s3 = input_filename_segment_0 bucket = input_filename_segment_2 folder = input_filename_segment_3 subfolder = input_filename_segment_4 filename.csv = input_filename_segment_5 | No | No | Yes |
To use these automatic values, add definitions like the following in your transform:
{
"name": "filename",
"datatype": {
"type": "string",
"index": true,
"source": {
"from_automatic_value": "input_filename"
}
},
},
The source should be from_automatic_value
, and the value name should be from the table above.
Updated about 2 months ago