K8 Platform

Hydrolix operator automatically deploys several pod to ensure that the service is running properly.
Here's the list of pod deployed by Hydrolix and their usage:

Pod Name	Description
batch-head	Call the intake-api service to list batch jobs created by users. Then list the files included from the cloud storage, and create a task per files. The task is created in postgres database.
batch-peer	Calls postgres database to get the different files to index. Download the file from cloud storage, generate a Hydrolix partition and send that partition to cloud storage.
intake-api	API used to generate batch jobs
keycloak	User and right management for Hydrolix platform
merge-head	Service which list the partitions that can be merged together. It create a job which is queued in rabbitMQ.
merge-peer	Service which merge partition together by reading tasks from rabbitMQ.
operator	Hydrolix operator which manages the scale and the deployment of the different pods
postgres	Database which stores metadata information for the partition (catalog service).
prometheus	Prometheus server fetching the different Hydrolix metrics
query-head	Service receiving query from end users, responsible of listing the partition required. Assign partition to the different query peer and get the response from each one.
query-peer	Download partition from cloud storage, execute the SQL Statement and provide response to query head.
rabbitmq	Distributed queueing system used by merge service, and reaper
redpanda	Kafka service to store streaming message retrieve by the stream-head.
stream-head	Service receiving HTTP POST request containing data to be indexed. Verify the format and send the data into redpanda
stream-peer	Service reading redpanda queue and generating Hydrolix partition from the data.
traefik	traefik load balancing between the different service (query, streaming data, API)
turbine-api	API service to manage project, table and user /rights management.
UI	User Interface service.
version	A small service to provide the version of the Hydrolix infrastructure
zookeeper	Zookeeper to manage and orchestrate which query peer is online and what capacity is available to distribute the partition.