Skip to content

Tencent EdgeOne

Tencent EdgeOne is an edge platform that provides traffic acceleration and security features, such as DDoS mitigation, a web application firewall (WAF), and bot management. EdgeOne Real-Time Log Push can send its logs to a Hydrolix cluster.

Setup steps⚓︎

Connect Tencent EdgeOne to a running Hydrolix cluster in these steps:

  1. Prerequisites: Gather the values the later steps need
  2. Configure a Hydrolix transform: Prepare Hydrolix to receive and transform EdgeOne log data
  3. Configure EdgeOne Real-Time Log Push: Set up Real-time Log Push in EdgeOne to send logs over HTTP to Hydrolix
  4. Verification: Confirm that data arrives

Prerequisites⚓︎

The integration uses EdgeOne Real-Time Log Push to send Tencent EdgeOne real-time logs directly to a Hydrolix cluster's HTTP Streaming API endpoint. The output columns specified in the Hydrolix transform correspond to EdgeOne log fields:

Tencent Cloud Services → EdgeOne Real-Time Log Push → Hydrolix HTTP Streaming API

This requires a Tencent Cloud account and a running Hydrolix cluster. To deploy one, see Deploy on Google GKE. Gather these values from the cluster:

Item Description Example value How to get it
Org ID The Hydrolix organization that owns the table. bc3f041b-ea52-45e1-b9d5-41819c465854 List orgs with the Config API.
Project name and ID The project that contains the table. Name: project_name
ID: c2445da3-ec63-42be-9f12-f104f9656f4c
See Create a project with the API.
Table name and ID The table that stores EdgeOne log data. Name: table_name
ID: 798dfcf4-7560-4b24-1337-29017619baa6
See Create a table using the API.
Authorization token An OAuth bearer token for the HTTP Streaming API. eyXrxkzoN2fRiiKpnV... See Acquire an authorization token.

Keep these values for the later steps.

Get started⚓︎

The integration has two parts:

Component Description
EdgeOne Real-Time Log Push Ships EdgeOne access logs in real time to destinations such as a Hydrolix cluster
Hydrolix transform Maps EdgeOne log fields to Hydrolix table columns

Configure a Hydrolix transform⚓︎

Publish the Hydrolix transform for Tencent EdgeOne to the table. See Publish a Transform.

Registering the transform in the UI requires:

  • Project name
  • Table name
  • The output_columns property from the transform

Registering it with the API requires:

  • Project ID
  • Table ID
  • The whole transform JSON

Either way, enable gzip compression in the transform. EdgeOne Log Push compresses logs with gzip before sending them.

Configure EdgeOne Real-Time Log Push⚓︎

Follow the EdgeOne Push to HTTP Server instructions, with these values on the Destination Information Page:

Parameter name Value
Address https://hostname.hydrolix.live/ingest/event
File compression Check Use Compress log files with gzip
Origin authentication None

Under Custom HTTP header, add at least the required headers for the HTTP Streaming API:

Key Value
Authorization Bearer and the token from Prerequisites
x-hdx-table project_name.table_name
x-hdx-transform The transform name from Configure a Hydrolix transform

content-type isn't in the list because EdgeOne Log Push sends it by default. With compression on, EdgeOne also sends Content-Encoding: gzip.

The table and transform can be query strings in the address instead. For example: https://hostname.hydrolix.live/ingest/event?table=project_name.table_name&transform=transform_name

Verification⚓︎

  1. Wait a few minutes for data to arrive. Log Push sends EdgeOne real-time logs, so no data arrives until EdgeOne generates some.
  2. In the Hydrolix UI, open the query interface at https://hostname.hydrolix.live/queries/.
  3. Query the EdgeOne table. For example:

    Query Recent EdgeOne Requests
    SELECT
      ClientState,
      EdgeSeverRegion,
      RequestID,
      RequestMethod,
      RequestUrl,
      EdgeResponseTime
    FROM project_name.table_name
    WHERE RequestTime > NOW() - INTERVAL '1 hour'
    LIMIT 100
    
  4. Check that the results match the requests in EdgeOne.

Troubleshoot⚓︎

Check these when data doesn't arrive in Hydrolix.

Verify your Hydrolix bearer token is valid and hasn't expired⚓︎

  • If you receive a 401 Unauthorized response, the authorization token may have expired. Retrieve a new one using the Hydrolix login API endpoint.

Review the ingest logs in Hydrolix for errors⚓︎

  • View and explore cluster logs on the cluster's System Health page.
  • You can also query cluster logs directly with Hydrologs.

Check ingest latency in table health⚓︎

Table health reports Ingest Latency for each table. A latency measured from the epoch (January 1st, 1970 at 00:00:00 UTC) means the table has never received data.

A recent latency doesn't prove your source is working. If the table also receives other streams, their data keeps the figure low on its own.

Check that the transform matches the log fields⚓︎

The transform on this page maps the standard EdgeOne log fields. To send other data, change the transform's output columns to match it.

Additional considerations⚓︎

A custom log field in EdgeOne Log Push needs a matching output column in the Hydrolix transform to be stored. See Transform Data: Output columns.