Compression

The compression property in a transform describes one or more compression algorithms that Hydrolix should expect to find already applied to the data package as a whole, and which it will need to uncompress prior to working with the data.

For example, setting the transform's compression property to "gzip" means that you expect the source data, in its entirety, to have had the GZIP compression method applied to it prior to its receipt by Hydrolix.

Recognized compression algorithms

Valid values for the compression property include the following:

ValueMeaning
gzipContent is compressed via gzip (LZ77 with 32-bit CRC).
zipContent is ZIP-encoded via zlib (RFC 1950)
deflateContent is encoded in zlib structure and the deflate compression algorithm.
bzip2Content is compressed with the bzip2 algorithm.
noneContent is not compressed. (Equivalent to not specifying compression at all.)

Note that, in streaming ingestion, the request document may have compression represented via the content-encoding header, but the data has its own compression potentially.

Handling multiple compression layers

To define multiple layers of compression, specify them in a comma-and-space-separated list:

"compression": "gzip, bzip2, zip"

The order matters: Hydrolix will attempt to apply decompression algorithms in the order specified, right-to-left.

In the above example, Hydrolix would apply zlib decompression to all received data, then further apply bzip2 decompression, and end with applying gzip decompression.


Did this page help you?