IO River

IO River is a multi-CDN platform provider that orchestrates and optimizes traffic across multiple CDNs.

IO River supports pushing unified logs from all underlying CDNs to an a Hydrolix HTTP endpoint. These logs contain structured fields such as request timestamps, client IPs, cache status, and response metrics, enabling visibility into performance, reliability, and content delivery for each underlying CDN.

Follow the instructions on this page to stream IO River logs into Hydrolix.

Setup steps

There are some prerequisites and three major steps covered in this document. Following these steps will result in an integrated setup between IO River and a running Hydrolix cluster:

1. Before you Begin: Complete these prerequisites first
2. Configure a Hydrolix Transform: Prepare Hydrolix to receive and transform IO River log data
3. Configure IO River log shipping: Add Hydrolix as a log target in IO River to send HTTP logs to Hydrolix
4. Verification: Follow these steps to verify the integration is working correctly

Before you begin

You will need a IO River account and a running Hydrolix deployment. Follow the instructions for your preferred deployment method if you have yet to deploy Hydrolix. From your running Hydrolix cluster, you will need the following information:

Gather the following information:

Item	Description	Example value	How to obtain this information
Cluster ingest URL	This is the hostname and ingest endpoint of your Hydrolix cluster.	https://hostname.hydrolix.live/ingest/event	The value of hydrolix_url in your hydrolixcluster.yaml including the preceding https://.
Table name	The destination table in the Hydrolix cluster for IO River CDN logs. Specified in the format: project_name.table_name.	ioriver_project.ioriver_cdn_logs	See create a table if you need to create a Hydrolix table in which to store your IO River CDN logs.
Basic authentication credentials	The username and password to authenticate with the Hydrolix endpoint and grant streaming ingest access to the destination table.	Username: hydrolix Password: $TRAEFIK_PASSWORD	For a more thorough explanation of basic authentication as it pertains to a Hydrolix cluster, see Enable Basic Authentication.
Table access token	For tables with token-based authentication enabled, this token is required.	Example value: 7bebd8ebsfalkj2j3jk	Enable token-based authentication for your table and note the generated token.

Keep these values on hand for later steps within this guide.

Get started

There are two steps covered within this document. Following these steps will result in an integrated setup between IO River and a running Hydrolix cluster:

1. Create a Hydrolix transform
2. Configure IO River log streaming

Configure a Hydrolix transform

Register a Hydrolix transform. The transform defines a schema to map fields from the IO River logs onto the Hydrolix table.

Using the Hydrolix transform for IO River, see Publish Transform for instructions.

You have two options for the publishing the transform:

1. UI: Registering a transform through the UI requires the following information:

   • Project name
   • Table name
   • The contents of the output_columns property

2. API: Registering a transform through the API requires the following information:

   • Project ID
   • Table ID
   • The entirety of the IO River transform json

If the transform you have created is the only one for your table, it will become the default transform for the table. If you have multiple transforms configured for the destination table, and the transform registered for IO River isn't the default transform, you should specify the transform for this data as a query parameter in the endpoint URL.

Verify that the Hydrolix transform exists. You can do so using either method:

• UI: In the UI by navigating to Data in the left-hand nav, selecting the table from under the Tables tab, and locating the transform name under Table transforms.
• API: Verify the transform you created is returned as part of the response from the Get transforms endpoint.

Configure IO River log streaming

Follow these steps to configure IO River log streaming to send data to your Hydrolix cluster:

1. From the IO River portal, select Logs then select Log Destinations from the sidebar.
2. Select the Add Destination button.

3. Fill out the form with these values:

   

4. Destination Name: Hydrolix (or choose your own)

5. Destination Type: Hydrolix
6. User: your username, to authenticate with the Hydrolix endpoint
7. Password: your password, to authenticate with the Hydrolix endpoint
8. Authentication Token: the table access token

9. Endpoint: Enter the following URL, replacing the {} fields:

   https://{hostname}.hydrolix.live/ingest/event
   

10. Project: Your project name

11. Table: Your table name
12. Log File Format: JSON List

Verification

To verify that the integration is working correctly:

1. Wait a few minutes for data to start flowing from IO River to Hydrolix.
2. In the Hydrolix UI, navigate to the query interface. (https://hostname.hydrolix.live/queries/).

3. Run a simple query against your able. For example:

   SELECT
     timestamp,
     request_host,
     request_method,
     request_full_path,
     response_status_code,
     response_time_to_first_byte_ms
   FROM integrations.ioriver
   WHERE timestamp > NOW() - INTERVAL '1 hour'
   LIMIT 10
   

4. Verify that you see results similar to your data in IO River.

Going further

• For more information on querying your Hydrolix cluster, see ClickHouse SQL.
• Explore your IO River logs in Grafana dashboards.

Troubleshoot

If you encounter issues with the integration:

Review the ingest logs in Hydrolix for any errors

• You can view and explore cluster logs using the cluster's System Health page.

Check ingest latency in table health

• Navigate to https://hostname.hydrolix.live/data/tables/. Select your project and table. This will take you to a page which displays some table health information.
• Check that the value for Ingest Latency indicates a time since you last sent data from IO River. Note that if the table is the destination for any other data streams, this number may be low due to data from another stream. If the latency indicates a time since the epoch (January 1st, 1970 at 00:00:00 UTC), then the table has received no data.

Ensure your transform correctly maps the fields you expect to see

• The Hydrolix transform provided acts a schema for storing HTTP requests from IO River log fields in Hydrolix. If you forward another data set to Hydrolix, see the Hydrolix documentation on transform output columns to modify the transform to correctly map the data you are sending.