CLI
For purposes of automation, it is strongly recommended that users create and manage version controlled "cluster templates" to provision and manage the lifecycle of clusters. This is well suited for scenarios where the multiple cluster creation is required with the same configuration
Step 1: Create Cluster Template¶
Command to create Cluster Template:
./rctl apply -f <clustertemplate.yaml>
Below is an example of yaml file to create a cluster template named demotest with the configuration details provided in Step 1
kind: ClusterTemplate
metadata:
name: demotest
description: 'My GKE Cluster template'
clusterType: Gke
project: defaultproject
To create this cluster template, use the command
./rctl apply -f demo-template.yaml
Output
{
"id": "q6kn4ml",
"name": "demotest",
"description": "My GKE Cluster template",
"cluster_type": "Gke",
"created_at": "2023-04-26T10:17:48.940538Z",
"modified_at": "2023-04-26T10:17:48.940538Z",
"organization_id": "7w2lnkp",
"partner_id": "rx28oml",
"project_id": "rx28oml",
"request_type": "TEMPLATE_FILE",
"input_template": "kind: ClusterTemplate\nmetadata:\n clusterType: Gke\n description: My GKE Cluster template\n name: mytemplate\n project: defaultproject\n",
"share_mode": "CUSTOM"
}
Step 2: Create Cluster Template Snapshot¶
Command to create cluster Template Snapshot:
./rctl apply -f <clustertemplatesnapshot.yaml>
Below is an example of a cluster template snapshot yaml file named "demo_template.yaml". This configuration file is to set the required parameters on the cluster template. Users can modify these parameters when required and create multiple versions
kind: ClusterTemplateSnapshot
metadata:
name: mysnapshot
description: 'My GKE Snapshot'
templateName: mytemplate
project: defaultproject
parameters:
spec_config_controlPlaneVersion:
default: "1.23"
override:
allow: true
values: ["1.23", "1.24"]
restricted: true
spec:
blueprint:
name: minimal
version: latest
cloudCredentials: gke-cred
config:
controlPlaneVersion: '{{ spec_config_controlPlaneVersion }}'
description: ''
location:
defaultNodeLocations:
- us-west1-c
type: zonal
zone: us-west1-c
network:
enableVPCNativeTraffic: true
maxPodsPerNode: 110
name: default
networkAccess:
privacy: public
nodeSubnetName: default
nodePools:
- machineConfig:
bootDiskSize: 100
bootDiskType: pd-standard
imageType: COS_CONTAINERD
machineType: e2-standard-4
name: default-nodepool
nodeVersion: '1.23'
size: 3
preBootstrapCommands: []
project: dev-382813
type: Gke
Important
Users can parameterize the fields in the template with overrides and restricted values. In the above config file, the parameterized fields are maxSize, instanceType, region within the double curly braces
To create this cluster template snapshot, use the command
./rctl apply -f demo-template.yaml
Output
{
"id": "3mx6vmr",
"template_id": "q6kn4ml",
"template_name": "mytemplate",
"name": "mysnapshot",
"cluster_type": "Gke",
"created_at": "2023-04-26T10:29:54.142419Z",
"modified_at": "2023-04-26T10:29:54.142419Z",
"project_id": "rx28oml",
"partner_id": "rx28oml",
"organization_id": "7w2lnkp",
"request_type": "TEMPLATE_FILE",
"cloud_credential": "gke-cred",
"blueprint_name": "minimal",
"blueprint_version": "latest",
"gen_template": "kind: ClusterTemplateSnapshot\nmetadata:\n description: My GKE Snapshot\n name: mysnapshot\n project: defaultproject\n templateName: mytemplate\nparameters:\n spec_config_controlPlaneVersion:\n default: \"1.23\"\n override:\n allow: true\n values:\n - \"1.23\"\n - \"1.24\"\n restricted: true\nspec:\n blueprint:\n name: minimal\n version: latest\n cloudCredentials: gke-cred\n config:\n controlPlaneVersion: '{{ spec_config_controlPlaneVersion }}'\n description: \"\"\n location:\n defaultNodeLocations:\n - us-west1-c\n type: zonal\n zone: us-west1-c\n network:\n enableVPCNativeTraffic: true\n maxPodsPerNode: 110\n name: default\n networkAccess:\n privacy: public\n nodeSubnetName: default\n nodePools:\n - machineConfig:\n bootDiskSize: 100\n bootDiskType: pd-standard\n imageType: COS_CONTAINERD\n machineType: e2-standard-4\n name: default-nodepool\n nodeVersion: \"1.23\"\n size: 3\n preBootstrapCommands: []\n project: dev-382813\n type: Gke\n",
"input_template": "kind: ClusterTemplateSnapshot\nmetadata:\n description: My GKE Snapshot\n name: mysnapshot\n project: defaultproject\n templateName: mytemplate\nparameters:\n spec_config_controlPlaneVersion:\n default: \"1.23\"\n override:\n allow: true\n values:\n - \"1.23\"\n - \"1.24\"\n restricted: true\nspec:\n blueprint:\n name: minimal\n version: latest\n cloudCredentials: gke-cred\n config:\n controlPlaneVersion: '{{ spec_config_controlPlaneVersion }}'\n description: \"\"\n location:\n defaultNodeLocations:\n - us-west1-c\n type: zonal\n zone: us-west1-c\n network:\n enableVPCNativeTraffic: true\n maxPodsPerNode: 110\n name: default\n networkAccess:\n privacy: public\n nodeSubnetName: default\n nodePools:\n - machineConfig:\n bootDiskSize: 100\n bootDiskType: pd-standard\n imageType: COS_CONTAINERD\n machineType: e2-standard-4\n name: default-nodepool\n nodeVersion: \"1.23\"\n size: 3\n preBootstrapCommands: []\n project: dev-382813\n type: Gke\n",
"publish": true
}
Step 3: Create Cluster out of template¶
Command to create cluster out of template:
./rctl apply -f <cluster.yaml>
An example of a yaml file named demo-cluster.yaml to create a cluster using the template created in Step 2:
apiVersion: v2
kind: Cluster
metadata:
description: 'Cluster using mytempalte'
name: demo-cluster
project: defaultproject
spec:
type: Gke
clusterTemplateRef:
name: mytemplate
version: mysnapshot
values:
spec_config_controlPlaneVersion:
value: '1.24'
where name: demo-template is the selected template for the new cluster
To create a cluster out of template, use the command
./rctl apply -f cluster.yaml
Output
{
"apiVersion": "infra.k8smgmt.io/v3",
"kind": "Cluster",
"metadata": {
"name": "mycluster",
"displayName": "mycluster",
"createdAt": "2023-04-26T10:32:36.025652Z",
"modifiedAt": "2023-04-26T10:32:36.865594147Z",
"labels": {
"rafay.dev/clusterName": "mycluster",
"rafay.dev/clusterType": "gke"
},
"organizationID": 3,
"partnerID": 1,
"projectID": 1,
"namespaces": null,
"id": "3mx6vmr"
},
"spec": {
"clusterType": "Gke",
"config": "YXBpVmVyc2lvbjogaW5mcmEuazhzbWdtdC5pby92MgpraW5kOiBDbHVzdGVyCm1ldGFkYXRhOgogIGNsdXN0ZXJUZW1wbGF0ZVJlZjoKICAgIG5hbWU6IG15dGVtcGxhdGUKICAgIHByb2plY3Q6IGRlZmF1bHRwcm9qZWN0CiAgICB2ZXJzaW9uOiBteXNuYXBzaG90CiAgZGVzY3JpcHRpb246IENsdXN0ZXIgdXNpbmcgbXl0ZW1wYWx0ZQogIG5hbWU6IG15Y2x1c3RlcgogIHByb2plY3Q6IGRlZmF1bHRwcm9qZWN0CnNwZWM6CiAgYmx1ZXByaW50OgogICAgbmFtZTogbWluaW1hbAogICAgdmVyc2lvbjogbGF0ZXN0CiAgY2xvdWRDcmVkZW50aWFsczogZ2tlLWNyZWQKICBjb25maWc6CiAgICBjb250cm9sUGxhbmVWZXJzaW9uOiAiMS4yNCIKICAgIGRlc2NyaXB0aW9uOiAiIgogICAgbG9jYXRpb246CiAgICAgIGRlZmF1bHROb2RlTG9jYXRpb25zOgogICAgICAtIHVzLXdlc3QxLWMKICAgICAgdHlwZTogem9uYWwKICAgICAgem9uZTogdXMtd2VzdDEtYwogICAgbmFtZTogbXljbHVzdGVyCiAgICBuZXR3b3JrOgogICAgICBlbmFibGVWUENOYXRpdmVUcmFmZmljOiB0cnVlCiAgICAgIG1heFBvZHNQZXJOb2RlOiAxMTAKICAgICAgbmFtZTogZGVmYXVsdAogICAgICBuZXR3b3JrQWNjZXNzOgogICAgICAgIHByaXZhY3k6IHB1YmxpYwogICAgICBub2RlU3VibmV0TmFtZTogZGVmYXVsdAogICAgbm9kZVBvb2xzOgogICAgLSBtYWNoaW5lQ29uZmlnOgogICAgICAgIGJvb3REaXNrU2l6ZTogMTAwCiAgICAgICAgYm9vdERpc2tUeXBlOiBwZC1zdGFuZGFyZAogICAgICAgIGltYWdlVHlwZTogQ09TX0NPTlRBSU5FUkQKICAgICAgICBtYWNoaW5lVHlwZTogZTItc3RhbmRhcmQtNAogICAgICBuYW1lOiBkZWZhdWx0LW5vZGVwb29sCiAgICAgIG5vZGVWZXJzaW9uOiAiMS4yMyIKICAgICAgc2l6ZTogMwogICAgcHJlQm9vdHN0cmFwQ29tbWFuZHM6IFtdCiAgICBwcm9qZWN0OiBkZXYtMzgyODEzCiAgdHlwZTogR2tlCg==",
"blueprint": "minimal",
"blueprintVersion": "latest",
"providerID": "wg29ek0",
"agentName": "agent-cluster-mycluster",
"configJson": {
"name": "mycluster",
"network": {
"name": "default",
"networkAccess": {
"privacy": "public"
},
"maxPodsPerNode": 110,
"nodeSubnetName": "default",
"enableVPCNativeTraffic": true
},
"project": "dev-382813",
"location": {
"type": "zonal",
"zone": "us-west1-c",
"defaultNodeLocations": ["us-west1-c"]
},
"nodePools": [
{
"name": "default-nodepool",
"size": 3,
"nodeVersion": "1.23",
"machineConfig": {
"imageType": "COS_CONTAINERD",
"machineType": "e2-standard-4",
"bootDiskSize": 100,
"bootDiskType": "pd-standard"
}
}
],
"controlPlaneVersion": "1.24"
},
"clusterTemplateRef": {
"name": "mytemplate",
"version": "mysnapshot",
"nameHashId": "q6kn4ml",
"versionHashId": "3mx6vmr"
},
"inputType": 2
},
"status": {
"conditions": [
{
"type": "ClusterInitialized",
"status": "Pending",
"lastUpdated": "2023-04-26T10:32:36.094613123Z",
"reason": "creating",
"duration": ""
},
{
"type": "ClusterBootstrapNodeInitialized",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025597908Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterProviderInfraInitialized",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025598349Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterSpecApplied",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.02559884Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterControlPlaneReady",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025599Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterCNISpecApplied",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025599612Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterOperatorSpecApplied",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025599762Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterHealthy",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025599902Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterPivoted",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025600052Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterBootstrapNodeDeleted",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025601645Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterDeleted",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025601796Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterActive",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025601926Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterUpgraded",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025602066Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterNodesScaled",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025602367Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterProvisioned",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025602507Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterFailed",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025602617Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterNodePoolAdded",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025602778Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterNodePoolDeleted",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.02560409Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterNodePoolUpgraded",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025604421Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterNodePoolScaled",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025604581Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterConfigCreated",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025604731Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterHardwareCSVCreated",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025604881Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterEKSCTLInstalled",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025605072Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterWorkerNodeGroupsReady",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025605212Z",
"reason": "pending",
"duration": ""
},
{
"type": "ClusterMigrationNeeded",
"status": "NotSet",
"lastUpdated": "2023-04-26T10:32:36.025605352Z",
"reason": "pending",
"duration": ""
}
],
"projects": [
{
"projectID": "rx28oml"
}
],
"health": {
"clusterID": "3mx6vmr",
"health_status_modified_at": "2023-04-26T10:32:36.040706Z"
},
"cluster_fqdn": "3mx6vmr.v2private.7w2lnkp.edge.dev.rafay-edge.net"
}
}
To know the status of the cluster template apply operation, enter the below command with the generated task id 3mx6vmr
./rctl status apply 3mx6vmr
Share Cluster Template¶
- To share a cluster template to one or more project(s), create a cluster template file with sharing parameter set to true and provide the project name(s) as shown in the below config file
kind: ClusterTemplate
metadata:
name: newtemplate
description: 'My GKE cluster template'
clusterType: Gke
project: defaultproject
sharing:
enabled: true
projects:
- name: "newproject"
Run the below command to share the templates
./rctl apply -f <shared-template.yaml>
Output
{
"id": "pd270k4",
"name": "newtemplate",
"description": "My GKE cluster template",
"cluster_type": "Gke",
"created_at": "2023-04-26T10:13:13.881551Z",
"modified_at": "2023-04-26T10:13:13.881551Z",
"organization_id": "7w2lnkp",
"partner_id": "rx28oml",
"project_id": "rx28oml",
"request_type": "TEMPLATE_FILE",
"input_template": "kind: ClusterTemplate\nmetadata:\n clusterType: Gke\n description: My GKE cluster template\n name: newtemplate\n project: defaultproject\n sharing:\n enabled: true\n projects:\n - name: newproject\n",
"share_mode": "CUSTOM"
}
Get Cluster Templates¶
- Command to get the list of cluster template details:
./rctl get clustertemplate
Example
./rctl get clustertemplate
+------------+--------------+-----------------------------+
| NAME | CLUSTER TYPE | CREATED AT |
+------------+--------------+-----------------------------+
| mytemplate | Gke | 2023-04-26T10:05:52.798931Z |
+------------+--------------+-----------------------------+
| test1 | Gke | 2023-04-20T20:32:34.045138Z |
+------------+--------------+-----------------------------+
- Command to get a specific cluster template details:
./rctl get clustertemplate <template-name>
Example:
./rctl get clustertemplate mytemplate
+------------+--------------+-----------------------------+
| NAME | CLUSTER TYPE | CREATED AT |
+------------+--------------+-----------------------------+
| mytemplate | Gke | 2023-04-26T10:05:52.798931Z |
+------------+--------------+-----------------------------+
- Command to get all Cluster Template Snapshots:
./rctl get clustertemplatesnapshot <template-name>
Example
./rctl get clustertemplatesnapshot mytemplate
+------------+--------------+---------------+-----------------------------+---------+
| NAME | CLUSTER TYPE | TEMPLATE NAME | CREATED AT | PUBLISH |
+------------+--------------+---------------+-----------------------------+---------+
| mysnapshot | Gke | mytemplate | 2023-04-26T10:27:04.959949Z | true |
+------------+--------------+---------------+-----------------------------+---------+
- Command to get a Cluster Template Snapshot with a specific name
./rctl get clustertemplatesnapshot <template_name> <snapshot_name>
Example:
./rctl get clustertemplatesnapshot mytemplate mysnapshot
+------------+--------------+---------------+-----------------------------+---------+
| NAME | CLUSTER TYPE | TEMPLATE NAME | CREATED AT | PUBLISH |
+------------+--------------+---------------+-----------------------------+---------+
| mysnapshot | Gke | mytemplate | 2023-04-26T10:27:04.959949Z | true |
+------------+--------------+---------------+-----------------------------+---------+
Delete Cluster Template and Snapshot¶
- Command to delete a cluster template
./rctl delete clustertemplate <template-name>
Example:
./rctl delete clustertemplate demotest
Output
Deleted demotest
Important
Deleting a cluster template deletes all the cluster template snapshots under this specific template
- Command to delete a cluster template snapshot
./rctl delete clustertemplatesnapshot <template-name> <clustersnapshot-name>
Example:
./rctl delete clustertemplatesnapshot demotest v1
Output
Deleted v1
Note: Users cannot delete a cluster template which is associated with a cluster