Pod Descriptions

Hydrolix operator automatically deploys several pod to ensure that the service is running properly. Here is the complete list of pods deployed by Hydrolix and their usage:

Pod Name	Description
alter-head	Micro-service used to queue up work for the Alter Service .
alter-indexer	A sidecar process deployed with the alter-peer to provide HDX data extraction, data compression and indexing.
alter-peer	The Alter Service supports async modification of Hydrolix data.
autoingest	Micro-service used to poll for cloud object storage update notifications, and queue up work for batch-peers.
batch-head	Micro-service used to scan cloud object storage, and queue up work for batch-peers.
batch-indexer	A sidecar process deployed with the batch-peer to provide HDX data compression and indexing.
batch-peer	The Batch Service supports one-off import of legacy files from cloud object storage. Also used for automatic ingest of newly-added cloud object storage files (Autoingest).
decay	Micro-service used to Age out partitions which exceed a (configurable) time to live.
indexer	A sidecar process deployed alongside all of the ingest services (eg, batch-peer, kafka-peer, kinesis-peer, stream-peer, summary-peer) to provide HDX data compression and indexing.
init-cluster	Micro-service used only at cluster creation time.
init-turbine-api	Micro-service used only at cluster creation time.
intake-api	Micro-service used to queue up work for batch-heads.
intake-head	Process that receives and decompresses incoming messages, verifies data formats, applies transformation policies, and pushes indexed partitions to cloud object storage. Replaces stream-head, stream-peer, and summary-peer: refer to the migration guide to switch to intake-head.
kafka-indexer	A sidecar process deployed with the kafka-peer to provide HDX data compression and indexing.
kafka-peer	The Kafka Service supports real-time ingestion from a customer's kafka data stream.
keycloak	Open source component used for user authentication and authorization.
kinesis-coordinator	A sidecar process that enables dynamic coordination of Kinesis peer-to-shard mapping.
kinesis-indexer	A sidecar process deployed with the kinesis-peer to provide HDX data compression and indexing.
kinesis-peer	The Kinesis Service supports real-time ingestion from a customer's AWS Kinesis data stream.
load-sample-project	Micro-service used only at cluster creation time.
merge-head	Micro-service used to queue up work for the Merge Service.
merge-indexer	Sidecar process deployed with the merge-peer to provide HDX data extraction, data compression and indexing.
merge-peer	The Merge Service combines small, related partitions into larger ones to improve storage and query efficiency.
monitor-ingest	monitor-ingest produces one event per second as a synthetic heartbeat test.
operator	Micro-service which manages the scale and deployment of various pods.
otel	Open source component used for collecting distributed query traces.
postgres	Stateful database used to store customer configuration data and provide a shared catalog of active Hydrolix partitions.
prometheus	Open source component used for collecting system and application health metrics.
promwaltz	Micro-service which helps coordinate Prometheus service discovery. Restarts automatically on configuration changes.
query-head	The front-end of the Hydrolix Query Service, query heads parse and compile incoming SQL queries, identify candidate partitions, and distribute work across pools of query-peers.
query-peer	The back-end of the Hydrolix Query Service, query peers download a filtered set of column indexes and data blocks from cloud object storage and returns partially-aggregated results to the appropriate query-head.
rabbitmq	Open source component used for managing service task queues (e.g. merge, batch, and reaper).
reaper	Micro-service used to permanently delete partitions which have been marked as inactive by other services (e.g. decay or merge).
redpanda	Open source component which provides reliable buffering of incoming messages for stream peers and summary peers.
stream-head	The front-end of the Hydrolix Stream Service, stream heads receive and decompress incoming messages, verify data formats, and push events to Redpanda. Deprecated: refer to our migration guide to switch to the more efficient intake-head pods.
stream-indexer	Sidecar process deployed with the stream-peer to provide HDX data compression and indexing.
stream-peer	The back-end of the Hydrolix Stream Service, stream peers read messages from Redpanda, apply transformation policies, and push indexed partitions to cloud objects storage. Deprecated: refer to our migration guide to switch to the more efficient intake-head pods.
summary-indexer	A sidecar process deployed with the summary-peer to provide HDX data compression and indexing.
summary-peer	The Summary Service enables continuous update of materialized views by applying alternative transformation policies to incoming events to generate compact (lossy) partitions. Deprecated: refer to our migration guide to switch to the more efficient intake-head pods.
traefik	Open source component which provides authentication and load balancing across various endpoints (eg, config API, stream ingest, and query).
traefik-cfg	Micro-service which manages traefik configuration updates. Restarts automatically on configuration changes.
turbine-api	The Config Service is responsible for managing project, table and user permissions.
ui	Micro-service which supports the (optional) Hydrolix user interface.
vacuum	Micro-service used to identify orphaned or invalid partitions which may exist in object storage but are not referenced in the Hydrolix catalog.
validator	Micro-service which enables pre-publication testing of new transformation policies.
validator-indexer	A sidecar process deployed with the validator to replicate the behavior of a Hydrolix stream peer.
vector	Open source component used for pushing log files to cloud object storage.
version	Micro-service used to expose the current version of a Hydrolix cluster.
zookeeper	Open source component used to track membership of individual query peer pools.