AWS Account Preparation
To ensure a successful deployment of a Hydrolix Data Platform we suggest preparing your AWS account for the install.
Hydrolix is a Data Platform that contains a number of components that are created and configured automatically at time of deployment. In order to be successful, it is important to set-up the AWS account to start.
We've broken this down to:
- AWS Account Preparation
- Users and Roles
- Marketplace Subscription
- Component Scaling
Account Preparation
We would suggest installing Hydrolix into a Sub account.
The installer creates everything, including the VPC, networking components, service components, roles and policies needed to run the service, see what is deployed take a look here.
This means the installer requires Administrative access or a policylike the below to be used.
Make sure to grab the Account ID, you'll need it for your licence.
Users and Roles
In order to successfully create your cluster you will need a user or policy that has rights to create and manage AWS components, roles and policies.
This can either be achieved through the use of a Administrative rights or by using the Policy below.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "cloudformation:CreateStack", "cloudformation:DeleteStack", "cloudformation:DescribeStackEvents", "cloudformation:DescribeStacks", "cloudformation:GetStackPolicy", "cloudformation:ListStacks", "iam:AttachRolePolicy", "iam:CreateRole", "iam:DeleteRole", "iam:DeleteRolePolicy", "iam:DetachRolePolicy", "iam:GetRole", "iam:GetRolePolicy", "iam:PassRole", "iam:PutRolePolicy", "lambda:CreateFunction", "lambda:DeleteFunction", "lambda:GetFunction", "lambda:GetFunctionCodeSigningConfig", "lambda:InvokeFunction", "lambda:UpdateFunctionCode", "s3:CreateBucket", "s3:DeleteBucket", "s3:DeleteObject", "s3:GetObject", "s3:ListAllMyBuckets", "s3:ListBucket", "s3:PutBucketPolicy", "s3:PutEncryptionConfiguration", "s3:PutObject", "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], "Resource": "*" } ] }
More information on AWS Policies can be found here - AWS Permissions and Polices
Marketplace Subscription
Subscribe to our AMI HDXReader from the Amazon Marketplace.
Just subscribe to the AMI, no need to deploy.
For more information on permission and market place usage you can read the AWS guide
Account Scaling
Some users may have scale limits on their account. If so you should ensure you have the ability to scale the following instance sizes. Note this scale is the initial basic scale that is deployed when the architecture is created.
More information on what is deployed can be found here - AWS Hydrolix Architecture
Service/Pool | Count | Server Type | Disk |
---|---|---|---|
Bastion | 1 | t3.micro | 10 |
Batch-peer | 1 | r5.2xlarge | 30 |
Grafana | 1 | t3.small | 10 |
Head (Query) | 1 | c5n.xlarge | 10 |
Query-peer | 1 | c5n.4xlarge | 10 |
Intake-misc | 1 | t3.small | 10 |
Merge-peer | 1 | r5.xlarge | 30 |
Prometheus | 1 | t2.medium | 50 |
Stream-head | 1 | r5.xlarge | 10 |
Stream-peer | 1 | r5.xlarge | 10 |
Superset | 1 | t3.medium | 10 |
Web | 1 | t3.small | 10 |
Zookeeper | 3 | t2.small | 10 |
Catalog | 2 | RDS (db.r5.large) | - |
S3 | - | S3 Bucket Created with your client_id | - |
Updated 3 months ago