Tencent EdgeOne
Tencent EdgeOne is an edge platform that provides traffic acceleration and security features, such as DDoS mitigation, a web application firewall (WAF), and bot management. EdgeOne Real-Time Log Push can send its logs to a Hydrolix cluster.
Setup steps⚓︎
Connect Tencent EdgeOne to a running Hydrolix cluster in these steps:
- Prerequisites: Gather the values the later steps need
- Configure a Hydrolix transform: Prepare Hydrolix to receive and transform EdgeOne log data
- Configure EdgeOne Real-Time Log Push: Set up Real-time Log Push in EdgeOne to send logs over HTTP to Hydrolix
- Verification: Confirm that data arrives
Prerequisites⚓︎
The integration uses EdgeOne Real-Time Log Push to send Tencent EdgeOne real-time logs directly to a Hydrolix cluster's HTTP Streaming API endpoint. The output columns specified in the Hydrolix transform correspond to EdgeOne log fields:
This requires a Tencent Cloud account and a running Hydrolix cluster. To deploy one, see Deploy on Google GKE. Gather these values from the cluster:
| Item | Description | Example value | How to get it |
|---|---|---|---|
| Org ID | The Hydrolix organization that owns the table. | bc3f041b-ea52-45e1-b9d5-41819c465854 |
List orgs with the Config API. |
| Project name and ID | The project that contains the table. | Name: project_name ID: c2445da3-ec63-42be-9f12-f104f9656f4c |
See Create a project with the API. |
| Table name and ID | The table that stores EdgeOne log data. | Name: table_name ID: 798dfcf4-7560-4b24-1337-29017619baa6 |
See Create a table using the API. |
| Authorization token | An OAuth bearer token for the HTTP Streaming API. | eyXrxkzoN2fRiiKpnV... |
See Acquire an authorization token. |
Keep these values for the later steps.
Get started⚓︎
The integration has two parts:
| Component | Description |
|---|---|
| EdgeOne Real-Time Log Push | Ships EdgeOne access logs in real time to destinations such as a Hydrolix cluster |
| Hydrolix transform | Maps EdgeOne log fields to Hydrolix table columns |
Configure a Hydrolix transform⚓︎
Publish the Hydrolix transform for Tencent EdgeOne to the table. See Publish a Transform.
Registering the transform in the UI requires:
- Project name
- Table name
- The
output_columnsproperty from the transform
Registering it with the API requires:
- Project ID
- Table ID
- The whole transform JSON
Either way, enable gzip compression in the transform. EdgeOne Log Push compresses logs with gzip before sending them.
Configure EdgeOne Real-Time Log Push⚓︎
Follow the EdgeOne Push to HTTP Server instructions, with these values on the Destination Information Page:
| Parameter name | Value |
|---|---|
| Address | https://hostname.hydrolix.live/ingest/event |
| File compression | Check Use Compress log files with gzip |
| Origin authentication | None |
Under Custom HTTP header, add at least the required headers for the HTTP Streaming API:
| Key | Value |
|---|---|
Authorization |
Bearer and the token from Prerequisites |
x-hdx-table |
project_name.table_name |
x-hdx-transform |
The transform name from Configure a Hydrolix transform |
content-type isn't in the list because EdgeOne Log Push sends it by default. With compression on, EdgeOne also sends Content-Encoding: gzip.
The table and transform can be query strings in the address instead. For example: https://hostname.hydrolix.live/ingest/event?table=project_name.table_name&transform=transform_name
Verification⚓︎
- Wait a few minutes for data to arrive. Log Push sends EdgeOne real-time logs, so no data arrives until EdgeOne generates some.
- In the Hydrolix UI, open the query interface at
https://hostname.hydrolix.live/queries/. -
Query the EdgeOne table. For example:
-
Check that the results match the requests in EdgeOne.
Troubleshoot⚓︎
Check these when data doesn't arrive in Hydrolix.
Verify your Hydrolix bearer token is valid and hasn't expired⚓︎
- If you receive a
401 Unauthorizedresponse, the authorization token may have expired. Retrieve a new one using the Hydrolix login API endpoint.
Review the ingest logs in Hydrolix for errors⚓︎
- View and explore cluster logs on the cluster's System Health page.
- You can also query cluster logs directly with Hydrologs.
Check ingest latency in table health⚓︎
Table health reports Ingest Latency for each table. A latency measured from the epoch (January 1st, 1970 at 00:00:00 UTC) means the table has never received data.
A recent latency doesn't prove your source is working. If the table also receives other streams, their data keeps the figure low on its own.
Check that the transform matches the log fields⚓︎
The transform on this page maps the standard EdgeOne log fields. To send other data, change the transform's output columns to match it.
Additional considerations⚓︎
A custom log field in EdgeOne Log Push needs a matching output column in the Hydrolix transform to be stored. See Transform Data: Output columns.