CSV Features

Format details

To create a transform schema that handles CSV-formatted incoming data, Set its type property to "csv", and its format_details object to include the following configuration options:

Element Type Default Description
delimiter number, string , The delimiter substring.
escape number, string " The escape character.
skip_head number 0 The number of rows to skip before ingestion starts.
quote number, string " The quote character.
comment number, string # The comment character. Only single characters are supported.
skip_comments boolean false If true, then the ingester will not process lines beginning with the comment character.
windows_ending boolean false If true, then Hydrolix will expect incoming data to use Windows-style (CR-LF) line endings.

(Note that Hydrolix recognizes "\t" as a tab character for the purposes of CSV configuration.)

example

{
    "name": "my_special_transform",
    "type": "csv",
    "settings": {
        "format_details": {
            "skip_head": 2,
            "delimiter": ","
        },
        ...
    }
}