Exporting Data

Hydrolix data is normally stored in our proprietary, patented, high-density compression & indexing data format, HDX. This is what enables our breakthrough performance and cost savings. Unfortunately, it also means that Hydrolix-stored data is not readily accessible via other technologies.

In order to insure that customers can alway export their data out of HDX and back into other common formats, we've released a stand-alone product in the AWS Marketplace (HDXReader) which can be used outside of a Hydrolix cluster to extract data from an HDX archive.

HDXReader is a specialized tool designed to search and export HDX-format data archives stored in S3. Before you begin, be sure to identify the location (in S3) of the HDX data that you wish to process.

Using the HDXReader

HDXReader is a specialized tool designed to search HDX-format data archives stored in S3. Before you begin, be sure to identify the location (in S3) of the HDX format data that you wish to search.

Installing HDXReader

The HDXReader utility is available exclusively via the AWS Marketplace. Once the instance is running, access your instance via SSH using the username 'ubuntu' and your Amazon private key.

Configuring HDXReader

Initialize the HDXReader app by running the script enable-hdxreader. The enable-hdxreader script requires 3 arguments: The name of the S3 region and the name of the S3 bucket you wish to search, along with the path within that bucket where the HDX data resides. For example:

$ sudo enable-hdxreader {region} {bucket} {path}

If you do not yet have any private data stored in HDX format, you can test out HDXReader by pointing at the hdx-public bucket in us-east-2:

$ sudo enable-hdxreader us-east-2 hdx-public /hdxformat/sample_project/

If you wish to test HDXReader locally, you are invited to copy the contents of this bucket to your local S3 account.

Using HDXReader

You can issue SQL queries against your data by running the script hdxsql:

$ hdxsql "select * from sample_project.sample_table limit 1" | less