| Batch (Intake) |
Batch ingest from a remote storage bucket. Includes Auto ingest where automated bucket notifications are used to notify new files being created. |
| Cloud Storage Bucket |
Hydrolix Storage bucket containing the Database files (partitions), service metadata, logs and system data. |
| Decay |
Ages out data within tables based on table settings. |
| Intake-api (Intake) |
API ingest end-point for batch, receives work from the turbine-api or from a notification queue. |
| Intake Head |
Streaming Ingest components for HTTP ingest of data. Includes Streaming Summary components for the aggregation of incoming data. |
| Kafka (Intake) |
Kafka intake components for ingestion of data from a customer’s Kafka infrastructure. |
| Keycloak |
Access and Authorization of components and users. |
| Kinesis (Intake) |
Kinesis intake components for ingestion of data from a customer’s Kinesis infrastructure. |
| Merge |
Compaction service that optimizes storage. |
| Operator |
Kubernetes Operator for the command and control of the Kubernetes infrastructure. |
| Postgres |
Database used for storage of “State” within the Hydrolix platform, includes metadata for tables and partitions, jobs and tasks state and auditing. |
| Prometheus |
Prometheus service for reporting metrics. |
| Query |
Query components for accessing projects and tables storing data. |
| RabbitMQ |
Queueing service used by the Batch Ingest service. |
| Reaper |
Table partition service that removers access to “old” partitions. |
| Redpanda |
Queueing mechanism for Streaming and merge. |
| Traefik |
Application layer (7) Load balancer. |
| Turbine-API |
Restful API for control of data and infrastructure |
| UI |
User interface. |
| Validator |
A debugging interface that allows the development and debugging of transforms. |
| Vector |
Logging component that retrieves logs from nodes and places them on a RedPanda Queue and within the Hydrolix storage bucket. |
| Version |
Simple service that returns the current version of Hydrolix. |
| Vacuum |
Cleans up stored files within storage that are no longer needed. |
| Zookeeper |
Manages available query-peer nodes. |
