Part 3: Blueprint

What Will You Do

In this part of the self-paced exercise, you will create a custom cluster blueprint with a Karpenter add-on, based on declarative specifications.

---

Step 1: Create Namespace

In this step, you will create a namespace for Karpenter. The "namespace.yaml" file located in /getstarted/karpenter/namespace" contains the declarative specification.

The following items may need to be updated/customized if you made changes to these or used alternate names.

• value: karpenter-cluster

kind: ManagedNamespace
apiVersion: config.rafay.dev/v2
metadata:
  name: karpenter
  description: namespace for karpenter
  labels:
  annotations:
spec:
  type: RafayWizard
  resourceQuota:
  placement:
    placementType: ClusterSpecific
    clusterLabels:
    - key: rafay.dev/clusterName
      value: karpenter-cluster

• Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you forked the Git repository
• Navigate to the folder "/getstarted/karpenter/namespace"
• Type the command below

rctl create namespace -f namespace.yaml

If you did not encounter any errors, you can optionally verify if everything was created correctly on the controller.

• Navigate to the "defaultproject" project in your Org
• Select Infrastructure -> Namespaces
• You should see an namesapce called "karpenter"

---

Step 2: Create Addons

In this step, you will create two custom addons, one for Karpenter and a second for the Karpenter Provisioner. The specification files for this section are located in /getstarted/karpenter/addon".

The following details are used to build the Karpenter addon declarative specification.

• "v1" because this is our first version
• The addon is part of the "defaultproject"
• Name of addon is "karpenter-addon"
• The addon will be deployed to a namespace called "karpenter"
• You will be using a "custom-values.yaml" as an override which is located in the folder "/getstarted/karpenter/addon"
• The "karpenter" chart will be used from the previously created repository named "karpenter-repo"

The following items may need to be updated/customized if you made changes to these or used alternate names.

• project: defaultproject
• namespace: karpenter

apiVersion: infra.k8smgmt.io/v3
kind: Addon
metadata:
  name: karpenter-addon
  project: defaultproject
spec:
  artifact:
    artifact:
      catalog: default-rafay
      chartName: karpenter
      chartVersion: v0.27.0
      valuesPaths:
      - name: file://custom-values.yaml
    options:
      maxHistory: 1
      timeout: 1m0s
    type: Helm
  namespace: karpenter
  sharing:
    enabled: false
  version: v0

• Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you forked the Git repository
• Navigate to the folder "/getstarted/karpenter/addon"
• Type the command below

rctl create addon version -f karpenter-addon.yaml --v3

If you did not encounter any errors, you can optionally verify if everything was created correctly on the controller.

• Navigate to the "defaultproject" project in your Org
• Select Infrastructure -> Addons
• You should see an addon called "karpenter-addon"

Next, we will create the second custom addon for the Karptenter Provisioner.

Note that the "cluster-name" is set to match the name of the cluster and the AWS tags that were specified in the cluster spec file.

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]
    - key: "node.kubernetes.io/instance-type" 
      operator: In
      values: ["t3.micro","t3.small","t3.medium","t3.large","t3.xlarge","t3a.micro","t3a.small","t3a.medium","t3a.large","t3a.xlarge"]
  limits:
    resources:
      cpu: 1000
  ttlSecondsAfterEmpty: 30
  providerRef:
    name: default
---
apiVersion: karpenter.k8s.aws/v1alpha1
kind: AWSNodeTemplate
metadata:
  name: default
spec:
  subnetSelector:
    cluster-name: {{{ .global.Rafay.ClusterName }}}
  securityGroupSelector:
    cluster-name: {{{ .global.Rafay.ClusterName }}}
  tags:
    email: [email protected]
    env: qa

The following details are used to build the provisioner addon declarative specification.

• "v1" because this is our first version
• The addon is part of the "defaultproject"
• Name of addon is "provisioner-addon"
• The addon will be deployed to a namespace called "karpenter"
• You will be using the "provisioner.yaml" which is located in the folder "/getstarted/karpenter/addon"

The following items may need to be updated/customized if you made changes to these or used alternate names.

• project: defaultproject
• namespace: karpenter

kind: AddonVersion
metadata:
  name: v1
  project: defaultproject
spec:
  addon: provisioner-addon
  namespace: karpenter
  template:
    type: NativeYaml
    yamlFile: provisioner.yaml

• Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you forked the Git repository
• Navigate to the folder "/getstarted/karpenter/addon"
• Type the command below

rctl create addon version -f provisioner-addon.yaml

If you did not encounter any errors, you can optionally verify if everything was created correctly on the controller.

• Navigate to the "defaultproject" project in your Org
• Select Infrastructure -> Addons
• You should see an addon called "provisioner-addon"

---

Step 3: Create Blueprint

In this step, you will create a custom cluster blueprint with the Karpenter addon and the provisioner addon. The "blueprint.yaml" file contains the declarative specification.

• Open Terminal (on macOS/Linux) or Command Prompt (Windows) and navigate to the folder where you forked the Git repository
• Navigate to the folder "/getstarted/karpenter/blueprint"

The following items may need to be updated/customized if you made changes to these or used alternate names.

• project: "defaultproject"

apiVersion: infra.k8smgmt.io/v3
kind: Blueprint
metadata:
  name: karpenter-blueprint
  project: defaultproject
spec:
  base:
    name: minimal
    version: 1.22.0
  type: custom
  customAddons:
  - name: karpenter-addon
    version: v1
  - name: provisioner-addon
    version: v1
  version: v1

• Type the command below

rctl apply -f blueprint.yaml

If you did not encounter any errors, you can optionally verify if everything was created correctly on the controller.

• Navigate to the "defaultproject" project in your Org
• Select Infrastructure -> Blueprint
• You should see an blueprint called "karpenter-blueprint

---

Step 4: Update Cluster Blueprint

In this step, you will update the cluster to use the previously created custom blueprint with the Karpenter addon and the Provisioner addon.

• Open Terminal (on macOS/Linux) or Command Prompt (Windows)
• Type the command below

rctl update cluster karpenter-cluster --blueprint karpenter-blueprint --blueprint-version v1

If you did not encounter any errors, you can optionally verify if everything was created correctly on the controller.

• Navigate to the "defaultproject" project in your Org
• Select Infrastructure -> Clusters
• You should see the cluster is now using the "karpenter-blueprint

• Navigate to Infrastructure -> Clusters
• Click on "KUBECTL" in the "karpenter-cluster" cluster card
• Type the command below

kubectl get pods --namespace karpenter

• You should see a result like the following.

NAME                              READY   STATUS    RESTARTS   AGE
karpenter-addon-b6cb889dd-vfqtj   1/1     Running   0          6m39s
karpenter-addon-b6cb889dd-x47n9   1/1     Running   0          6m39s

Recap

As of this step, you have created a "cluster blueprint" with Karpenter and the Karpenter Provisioner as addons, and applied this blueprint to the existing cluster. You are now ready to move onto the next step where you will deploy a test workload to scale the cluster with Karpenter.

---