RDS/S3/ECR/Redis/Secrets Manager/Service Account Setup
Table of Contents
- Prerequisites
- Create RDS
- Create S3 Bucket
- Create ECR Repository
- Create Redis
- Create Secrets Manager
- Create Kubernetes Service Account
Detailed Steps
For detailed explanations of {variables}, refer to the Terminology page
1. Prerequisites
Ensure the setup for the installation environment is complete. (Refer to 1. Setup Deploy Environment)
2. Create RDS
Please input the {variables} directly
- Create RDS Subnet Group
- Go to the AWS RDS Console, and click Subnet groups on the left-hand side
- Step 1: Create DB subnet group
- Step 2: Subnet group details
- Name: sng-rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Description: Required
- VPC: {AWS_VPC_NAME} select
- Step 3: Add subnets
- Availability Zones: {AWS_DEFAULT_REGION}a, {AWS_DEFAULT_REGION}c select
- Subnets: Check only Public subnet
- Step 4: Create
- Create RDS Instance
- Go to the AWS RDS Console
- Click the Create database button to start creating an RDS instance
- Step 1: Choose a database creation method
- Standard create
- Step 2: Engine options
- Engine type: MySQL
- Edition: MySQL Community
- Engine Version: MySQL 8.0.33
- Step 3: Templates
- Production
- Step 4: Availability and durability
- Multi-AZ DB instance → Select Multi AZ if redundancy is needed
- Step 5: Settings
- DB instance identifier: rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Master username: admin
- Master password: Select from 2 options
- Self managed: Create a password (recommended)
- Auto generate password: Automatically generated by RDS, can be checked once after creation
- Master password: User-created password
- Managed in AWS Secrets Manager: Automatically generated by AWS
- Go to AWS Secret Manager Console
- Click rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Secret value -> Retrieve secret value
- Self managed: Create a password (recommended)
- Step 6: Instance configuration
- DB instance class: Standard classes
- Instance type: db.m5.large
- Step 7: Storage
- Storage type: General Purpose SSD (gp2)
- Allocated storage: 50GiB
- Storage autoscaling: Uncheck Enable storage autoscaling
- Step 8: Connectivity
- Compute resource: Don't connect to an EC2 compute resource
- Network type: IPv4
- VPC: {AWS_VPC_NAME} select
- DB subnet group: sng-rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Public access: Yes
- VPC security group: Create new
- New VPC security group name: scg-rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Additional configuration -> Database port: 3310
- Step 9: Database authentication
- Database authentication options: Password authentication
- Step 10: Monitoring
- Performance Insights: Check Turn on Performance Insights
- Retention period: 7 days
- AWS KMS key: aws/rds
- Step 11: Create database
- Step 12: Additional security group settings (after RDS creation)
- Go to AWS EC2 Console and click Security Groups on the left-hand side
- Select scg-rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-mysql
- Click Edit Inbound rules
- Click Add rules
- Port range: 3310
- Source: Select the sg of eks (e.g., eks-cluster-sg-{AWS_CLUSTER_NAME})
- Save rules
-
Create RDS User and Grant Permissions
-
Check the address and admin account information of the created RDS, and store them in variables. (Update the variable list on the first page)
export DB_HOST=
export DB_PORT=3310
export DB_ADMIN_USERNAME=admin
export DB_ADMIN_PASSWORD=
export DB_APP_USERNAME=`echo ${INFRA_NAME}_APP | tr [:lower:] [:upper:]`
export DB_APP_PASSWORD= -
Access DB
mysql -h ${DB_HOST} -P ${DB_PORT} -u ${DB_ADMIN_USERNAME} -p
# Insert : ${DB_ADMIN_PASSWORD} -
Create DB USER and Grant Permissions
create user '${DB_APP_USERNAME}'@'%' identified by '${DB_APP_PASSWORD}';
GRANT SELECT, PROCESS, SHOW DATABASES, SHOW VIEW ON *.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `cachedb`.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `kubeflow`.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `metadb`.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `mlpipeline`.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `ai_conductor`.* TO `${DB_APP_USERNAME}`@`%`;
GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE, SHOW VIEW ON `edge_conductor`.* TO `${DB_APP_USERNAME}`@`%`;
-
- Additional Security Group settings for RDS access (Optional)
- To access RDS from outside, add an Inbound Rule.
- Go to AWS RDS Console and click Security Groups on the left-hand side
- Search and click scg-rds-{AWS_DEFAULT_REGION_ALIAS}-{INFRA-NAME}-{DEPLOY_ENV}-mysql
- Click Edit inbound rules in the Inbound rules tab to add an Inbound Rule.
3. Create S3 Bucket
Please input the {variables} directly
-
Three S3 Buckets are needed, each serving the following purposes:
- s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-aia: mellerikat operations bucket
- s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-kubeflow: Kubeflow operations bucket
- NOTE: Skip if installing edge app on-premise.
- s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-edgeapp: Edge App operations bucket
-
Create S3 Bucket
- Go to the AWS S3 Console
- Click the Create bucket button to create an S3 Bucket
- Step 1: General configuration Bucket name: s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-aia
- Step 2: Create bucket
- Create the following Buckets in the same way:
- s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-kubeflow
- NOTE: Skip if installing edge app on-premise.
- s3-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-edgeapp
4. Create ECR Repository
- Create image repositories for installing Kubeflow.
aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/ml-pipeline/api-server
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/oidc-authservice
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/kubeflowkatib/katib-db-manager
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/ml-pipeline/cache-server - Create image repositories for installing AI Conductor.
aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/ai-advisor/ai-conductor/aic-webserver
aws ecr create-repository --repository-name ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/ai-advisor/ai-conductor/aic-frontend
5. Create Redis
Please input the {variables} directly
-
mellerikat uses Redis to store login session information.
-
Create Security Groups for Redis
- Go to the AWS EC2 Console
- Click Security Groups in the left-hand menu
- Click the Create security group button to create a security group for Redis
- Step 1: Basic details
- Security group name: scg-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis
- Description: Required
- VPC: {AWS_VPC_NAME} select
- Step 2: Inbound rules
- Click Add rule
- Port range: 6379
- Source: Select eks sg (e.g., eks-cluster-sg-eks-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-1-28-eks-master)
- Step 3: Create security group
-
Create Redis
- Go to the AWS ElastiCache Console
- Click Redis caches in the left-hand menu
- Click the Create Redis Cache button to create Redis
- Step 1: Configuration
- Deployment option: Design your own cache
- Creation method: Cluster cache
- Cluster mode: Disabled
- Step 2: Cluster info
- Name: elasticache-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis
- Step 3: Location
- Location: AWS Cloud
- Multi-AZ: Enabled
- Step 4: Cluster settings
- Engine Version: 7.1
- Port: 6379
- Parameter group: default.redis7
- Node type:
- Recommended: cache.m6g.large
- Select Node type by referring to the AWS ElastiCache Node type
- Number of replicas: 1
- Step 5: Connectivity
- Network type: IPv4
- Subnet groups: Create a new subnet group
- Name: sng-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis-private
- VPC ID: {AWS_VPC_NAME} select
- Selected subnets: Manage
- Check only SubnetPrivate
- Next
- Step 6: Security
- Encryption at rest: Disabled
- Encryption in transit: Disabled
- Step 7: Selected security group
- Manage: scg-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis
- Step 8: Backup
- Enable automatic backups: Check
- Backup retention period: 1
- Backup window: Specify backup window
- Backup start time: 02:30
- Backup duration: 1
- Step 9: Maintenance
- Maintenance window: Specify maintenance window
- Maintenance start day: Tuesday
- Maintenance start time: 01:00
- Maintenance duration: 1
- Auto upgrade minor versions: Enabled
- Next
- Create
-
Additional settings after Redis creation
- Go to the AWS ElastiCache Console
- Click Redis caches in the left-hand menu
- In Redis caches, click elasticache-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis
- Click Modify
- In the Security section, check Enable for Encryption in transit
- Click Preview changes
- Click Modify
- Set Redis Variables
-
Go to the AWS ElastiCache Console
-
Click Redis caches in the left-hand menu
-
In Redis caches, click elasticache-{AWS_DEFAULT_REGION_ALIAS}-{INFRA_NAME}-{DEPLOY_ENV}-redis
-
In Cluster details, the Primary endpoint is {REDIS_HOST}:{REDIS_PORT}.
export REDIS_HOST=
export REDIS_PORT=6379
-
6. Create Secrets Manager
- mellerikat uses Secrets Manager to store RDS encryption information and access details for S3. Configure Secrets Manager as follows to save the required information.
# SECRETS MANAGER Variables (required for installing Kubeflow)
export AWS_SECRETS_MANAGER_S3=/parameter/${AWS_DEFAULT_REGION}/${INFRA_NAME}/${DEPLOY_ENV}/s3
export AWS_SECRETS_MANAGER_RDS=/parameter/${AWS_DEFAULT_REGION}/${INFRA_NAME}/${DEPLOY_ENV}/rds
# S3
aws secretsmanager create-secret --name ${AWS_SECRETS_MANAGER_S3} \
--secret-string '{"accesskey":"'${KUBEFLOW_USER_AWS_ACCESS_KEY_ID}'","secretkey":"'${KUBEFLOW_USER_AWS_SECRET_ACCESS_KEY}'"}' --region ${AWS_DEFAULT_REGION}
# RDS
aws secretsmanager create-secret --name ${AWS_SECRETS_MANAGER_RDS} \
--secret-string '{"username":"'${DB_APP_USERNAME}'","password":"'${DB_APP_PASSWORD}'","database":"kubeflow","host":"'${DB_HOST}'","port":"'${DB_PORT}'"}' --region ${AWS_DEFAULT_REGION}
```
7. Create Kubernetes Service Account
-
In this step, create policies for accessing resources such as S3 and ECR, and create a Service Account with those policies.
- If there is no Namespace for the Service Account, it will be created automatically during the Service Account creation.
- kubeflow
- ai-conductor
- edge-conductor
- edge-app
- If there is no Namespace for the Service Account, it will be created automatically during the Service Account creation.
-
Create IAM policy
- Create the Secret Manager Policy for mellerikat defined in policy-secret-manager-aia-r.yaml.
[Expand policy-secret-manager-aia-r.yaml]
cat <<EOT > policy-secret-manager-aia-r.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowGetRdsSecretViaSsm",
"Effect": "Allow",
"Action": [
"ssm:GetParameters",
"ssm:GetParameter",
"secretsmanager:GetSecretValue"
],
"Resource": [
"arn:aws:ssm:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:parameter/aws/reference/secretsmanager/*",
"arn:aws:secretsmanager:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:secret:/parameter/${AWS_DEFAULT_REGION}/${INFRA_NAME}/${DEPLOY_ENV}/*"
]
}
]
}
EOTaws iam create-policy --policy-name policy-secret-manager-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-secret-manager-aia-r.yaml
- Create the S3 Policy for mellerikat defined in policy-s3-aia-rwd.yaml.
[Expand policy-s3-aia-rwd.yaml]
cat <<EOT > policy-s3-aia-rwd.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3BucketDeliverableRWD",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-aia",
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-aia/*"
]
}
]
}
EOTaws iam create-policy --policy-name policy-s3-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-s3-aia-rwd.yaml
- Create the S3 Policy for Kubeflow defined in policy-s3-kubeflow-rwd.yaml.
[Expand policy-s3-kubeflow-rwd.yaml]
cat <<EOT > policy-s3-kubeflow-rwd.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3BucketKubeflowRWD",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-kubeflow",
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-kubeflow/*"
]
}
]
}
EOTaws iam create-policy --policy-name policy-s3-kubeflow-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-s3-kubeflow-rwd.yaml
- Create the ECR Policy for mellerikat defined in policy-ecr-aia-rwd.yaml.
[Expand policy-ecr-aia-rwd.yaml]
cat <<EOT > policy-ecr-aia-rwd.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ECRFullRWD",
"Effect": "Allow",
"Action": "ecr:*",
"Resource": [
"arn:aws:ecr:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:repository/ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}",
"arn:aws:ecr:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:repository/ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}/*"
]
}
]
}
EOTaws iam create-policy --policy-name policy-ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-ecr-aia-rwd.yaml
- Create the S3 Policy for Edge App defined in policy-s3-edge-app-rwd.yaml.
- NOTE: Skip if installing edge app on-premise.
[Expand policy-s3-edge-app-rwd.yaml]
cat <<EOT > policy-s3-edge-app-rwd.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "S3BucketEdgeAppRWD",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:GetObjectTagging",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-edgeapp",
"arn:aws:s3:::s3-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-edgeapp/*"
]
}
]
}
EOTaws iam create-policy --policy-name policy-s3-edgeapp-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-s3-edge-app-rwd.yaml
- Create the SQS Policy for Edge App defined in policy-sqs-edge-app-rwd.yaml.
- NOTE: Skip if installing edge app on-premise.
[Expand policy-sqs-edge-app-rwd.yaml]
cat <<EOT > policy-sqs-edge-app-rwd.yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SqsEdgeAppAllow",
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:SendMessage",
"sqs:ListQueues"
],
"Resource": [
"arn:aws:sqs:${AWS_DEFAULT_REGION}:${AWS_ACCOUNT_ID}:sqs-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV}-edgeapp"
]
}
]
}
EOTaws iam create-policy --policy-name policy-sqs-edgeapp-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} --policy-document file://policy-sqs-edge-app-rwd.yaml
-
Create Service Accounts
-
ai-conductor-sa
eksctl create iamserviceaccount \
--name ai-conductor-sa \
--role-name role-${INFRA_NAME}-${DEPLOY_ENV}-ai-conductor-sa \
--namespace ai-conductor \
--cluster ${AWS_CLUSTER_NAME} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-secret-manager-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-s3-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--approve \
--region ${AWS_DEFAULT_REGION} -
edge-conductor-sa
- NOTE: Skip if installing edge conductor on-premise.
eksctl create iamserviceaccount \
--name edge-conductor-sa \
--role-name role-${INFRA_NAME}-${DEPLOY_ENV}-edge-conductor-sa \
--namespace edge-conductor \
--cluster ${AWS_CLUSTER_NAME} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-secret-manager-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-s3-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--approve \
--region ${AWS_DEFAULT_REGION} -
edge-app-sa
- NOTE: Skip if installing edge app on-premise.
eksctl create iamserviceaccount \
--name edge-app-sa \
--role-name role-${INFRA_NAME}-${DEPLOY_ENV}-edge-app-sa \
--namespace edge-app \
--cluster ${AWS_CLUSTER_NAME} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-s3-aia-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-s3-edgeapp-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-sqs-edgeapp-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--attach-policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/policy-ecr-repo-${AWS_DEFAULT_REGION_ALIAS}-${INFRA_NAME}-${DEPLOY_ENV} \
--approve \
--region ${AWS_DEFAULT_REGION}
-