During provisioning, both the EKS Control Plane and the Autoscaling Group for the EKS Worker Nodes are automatically provisioned by the Rafay Controller based on input/configuration specified by the user.
Rafay needs to be configured with credentials in order to programmatically create and configure required EKS infrastructure on AWS in your account. These credentials securely managed as part of a cloud credential in the Rafay Controller.
The creation of a cloud credential is a "One Time" task. It can then be used to create clusters in the future when required. Please review Amazon EKS Credentials for additional instructions on how to configure this.
To guarantee complete isolation across Projects (e.g. BUs, teams, environments etc.), cloud credentials are associated with the a specific Rafay project.
Self Service Wizard¶
This approach is ideal for users that need to quickly provision and manage EKS clusters without having to become experts in Amazon EKS tooling, best practices and writing bespoke Infrastructure as Code (IaC).
The wizard prompts the user to provide critical cluster configuration details organized into logical sections:
- General (mandatory)
- EKS Control Plane
- Node Group Configuration
- IAM Roles
Only the "general" section is mandatory. Out of box details are provided for the remaining sections.
- Click on "Clusters" on the left panel
- Click on the "New Cluster" button
- Select "Create a New Cluster" and click Continue
- Select "Environment" as "Public Cloud"
- Select "Cloud Provider" as "AWS"
- Select "Kubernetes Distribution" as "Amazon EKS"
- Give it a name and click Continue
The Controller provides a quick start option where it is possible to quickly get started with an EKS Cluster. The user has to provide only two inputs for the quick start.
Required - Provide a name for your EKS cluster - Select the Cloud Credential
Optional - Select the AWS Region where you wish to provision your EKS Cluster. - Select the Kubernetes version - Select Cluster Blueprint
If nothing else is specified, the following defaults are used.
- Two (2) worker nodes
- AWS Region (US West- Oregon)
- Instance Type (m5.xlarge)
- Default Kubernetes version (v1.17)
- Default Cluster Blueprint
- VPC and Subnets automatically created
- Secure, private access to EKS Control Plane
- IAM Roles: Autoscaling Group, ECR
- Click on Create
Manual intervention is NOT REQUIRED unless there is an error or an issue to deal with. The entire process for provisioning with the cluster blueprint can take ~20 minutes.
The process comprises multiple distinct phases. In addition to creating the worker nodes, the Controller also fully prepares the Cluster Worker Nodes to become ready to accept workloads.
EKS Control Plane
The Controller provisions the EKS Control Plane and creates a node group that will contain the requested worker nodes.
The provisioning process will temporarily create a bootstrap node in your VPC for secure, automated provisioning of the software components of the Kubernetes Operator and blueprint. This bootstrap node will be automatically removed once the bootstrap process is complete.
Cloud Formation Stacks
Both during and after provisioning, users can view the AWS Cloudformation stacks for both the EKS Control Plane and Worker Nodegroups in the AWS Console. A separate stack is created for the EKS Cluster control plane and the worker node nodegroup. An illustrative example is shown below.
The AWS EC2 nodes backing the Worker Nodes can be viewed on the AWS Console (EC2 Service). Note that the names of the nodes start with the EKS cluster's name in the Console. By default, the worker nodes are spread across multiple availability zones to ensure high availability.
An illustrative example is shown below.
Once the EKS Cluster's worker nodes are operational, the Kubernetes Operator is automatically deployed on the newly provisioned worker nodes to bring it under centralized management in the Controller.
Once the Kubernetes Operator is deployed on the cluster, the specified cluster blueprint is applied. This step will involve the download and configuration of the software addons configured in the blueprint.
The time required for this step is dependent on the size and complexity of the add-ons specified in the cluster blueprint.
Once all the steps are complete, the cluster is sucessfully provisioned as per the specified configuration. Users can now view and manage the Amazon EKS Cluster in the specified Project in the Controller just like any other cluster type.
Once sucessfully provisioned, the user can view the dashboards. See example below.
Cluster provisioning can fail if the user had misconfigured the cluster configuration (e.g. wrong cloud credentials) or encountered soft limits in their AWS account for resources.
When this occurs, the user is presented with an intuitive error message. Users are allowed to edit the configuration and retry provisioning.
Users can also fully automate the entire cluster provisioning process without any form of inbound access to their VPC. They can create and manage version controlled EKS Cluster Specs in their Git repos and use it to declaratively provision clusters using their existing pipelines in platforms such as Jenkins.
Jenkins pipeline examples and EKS Cluster specs are available in this public Git repo. An illustrative worfklow is shown below.