Data Lifecycle

Hydrolix provides a Lifecycle mechanism for ageing out data from the system. The system has two settings one for how long data is made viewable and then how long partitions are kept.

👍

Age: Configure it!

To configure the age services for your platform information can be found here - Age service

537
ComponentDescriptionScale to 0
DecayThe Decay service evaluates partitions for their age and where the partition is older than a set period of time, the partition is made inactive. Once inactive the partition is then evaluated on if it should be deleted (based on another time based setting). Any partitions to be deleted are sent to a queueYes
QueuesTwo queues are operated, a listing queue for a list of partitions to be deleted and a dead-letter queue, where jobs are expired to after a period of time if they are not completed.Yes
ReaperThe Reaper service retrieves messages from the queue and will remove the partition from the storage bucket and catalog.Yes
HDX DB Storage BucketContains the database (including partitions), configuration and other state information concerning the platform. Forms part of the core infrastructure.No
CatalogContains metadata on the database, partitions and job tasks. Forms part of the core infrastructure.No