System Sync
System Sync enables two Ways or bidirectional Sync to maintain the configuration in the system (system database) and Git repository sync. Any configuration changes performed in the Git repository get reflected in the system, and vice-versa is supported. To achieve this bidirectional sync, external and internal triggers are required. External triggers (Pipeline Triggers) notify whenever resources are modified in a Git repository, and internal triggers notify whenever resources are modified in an internal artifact store.
Cluster Resource System Sync flow¶
The following are sequence diagrams that illustrate actions in the System sync flow for cluster resource.
Cluster Provision Click Ops flow
sequenceDiagram
autonumber
participant Admin
participant Rafay
participant AWS/Azure
participant Customer Git Repo
rect rgb(191, 223, 255)
note right of Admin: Day-1:Cluster Provisioning
Admin->>Rafay: Configure Cluster via Web Console
Rafay->>AWS/Azure: Provision Cluster
Rafay->>Customer Git Repo: “Auto Generate” Cluster Spec
Rafay->>Admin: Cluster Provisioned
end
Update Cluster GitOps flow
sequenceDiagram
autonumber
participant Admin
participant Rafay
participant AWS/Azure
participant Customer Git Repo
rect rgb(191, 223, 255)
note right of Admin: Day2: Scale and upgrade via gitops
Admin->>Customer Git Repo: Git PR to update cluster spec
Customer Git Repo->> Rafay: Webhook notification
Rafay->>AWS/Azure: Bring cluster to desired state
Rafay->>Admin: Cluster Updated
end
Update Cluster Click Ops flow
sequenceDiagram
autonumber
participant Admin
participant Rafay
participant AWS/Azure
participant Customer Git Repo
rect rgb(191, 223, 255)
note right of Admin: Day2: Scale and upgrade via ClickOps
Admin->>Rafay:Update Cluster via WebConsole
Rafay->> AWS/Azure: Update Cluster
Rafay->>Customer Git Repo: Auto Update Cluster spec
Rafay->>Admin: Cluster Updated
end
Pre-requisites¶
System Sync Stage requires the below setup to complete the sync successfully
Github Repository
- User should create a valid Git repository
- Update the created repository in the Repositories page with an appropriate name
GitOps Agents
- Create GitOps Agents and associate with the K8s Cluster for secured connectivity to the repositories
- Git Repo used for System-Sync must be associated with the GitOps agent. This helps to access the repositories available in the private network
System Sync Operation¶
This stage is responsible for resource config sync between Git Repository and the system
- Provide a friendly name for the stage
- Select System Sync
- To sync the resource specification from Git Repository to System, select Git To System Sync checkbox
- Select the Repository from where the resource details are to be cloned
- Provide the Tag/Branch of the repository
- Provide the appropriate Folder path of the repository
- To sync the resource specification (updated through Controller) from system to Git Repository to System, select System To Git Sync checkbox
- Select the Repository to which the resource details must be synced
- Provide the Tag/Branch of the repository
- Provide the appropriate Folder path of the repository
- Select Use Source As Destination checkbox to use the same source destination (or) uncheck this option to provide a different (system or repo) destination
Include / Exclude Resources¶
Include Resources allows the users to add all the resources or add only the required resources anytime. Click Add Resource to include one or more resources. This operation sync only the selected resource(s) in Git Repository and System. Below are the supported resources
- Add-On
- Blueprint
- Cluster
- GitOps Agent
- Infrastructure Provisioner
- Namespace
- OPA Constraint
- OPA Constraint Template
- Overrides
- OPA Policy
- Overrides
- Pipeline
- Repository
- Secret Sealer
- Secret Store
- Workload
- Workload Template
Important
Add-On from Catalog and Workload from Catalog system sync can be performed using the respective YAML spec available in the Git Repo. The parameter catalog: default-bitnami
in the workload and add-on YAML spec represents the catalog functionality
By default, All Resources are selected from the drop-down
Exclude Resource allow the users to remove the resource(s) anytime. Click Add Resource to exclude one or more resources
During System to Git Sync, regardless of whether any resource configuration files are available to sync, a new project folder with a git-ignore file named .gitkeep will be created in the selected git repo when triggering a pipeline. Users can add or remove the resources at any time. A project folder cannot be empty in the git repo, thus deleting the .gitkeep file where the project folder doesn't have any other resource configuration files, results in deleting the project folder itself. When you delete that .gitkeep file, the same file will get created on the next pipeline run.
Pre-Conditions (Optional)
Stages can be configured to execute ONLY if the expression matches the specified pre-conditions
Stage Variables (Optional)
All the variables available for a given stage are fetched as a sorted list according to their scope (Organization -> Project -> Pipeline -> Trigger -> Stage). These variables are evaluated with the environment. The environment is then updated with the variable according to their scope
Click Save to add the system sync stage to the pipeline or Cancel to abort the process. Click Save and Go To Triggers to complete the sync process and trigger the changes
Refer Triggers for more information
Success System Sync¶
The System Sync job begins automatically in the pipeline. The initial status is In Progress, later changes to Success. Use the Run button to trigger the pipeline for the changes performed manually in the system
Users can view the recently triggered changes on the Triggers page. Also, this creates a pipeline in Git Repo with all the selected resources in the specified project. Below is an example of defaultproject and the list of included resources
Below is an example of the add-ons resource in Git Repository where the users can view or update the resource specs based on the selected direction during system sync stage creation
Git Directory Structure for resource(s) sync: Repo_name/folder_name (provided by the customer)/projects/project_name/resource_name (example: cluster, addons, blueprint)
Important
When a resource contains a secret value/key, the system sync fails for that specific resource, but continues with the other resources. Currently, Infrastructure Provisioners is the only resource that can contain a secure text. Refer Secret Sealer on how to configure secrets for Infra Provisioners.
Refer Infrastructure Provisioners for more details on Infrastructure Provisioners configurations
Cluster Resource¶
The resource Cluster system sync is applicable only for EKS and AKS clusters. Below is an example of the EKS cluster resource yaml for cluster system sync process
apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
name: demo-cluster
project: default
spec:
blueprintConfig:
name: demo-bp
version: v1
cloudCredentials: demo_aws
config:
managedNodeGroups:
- amiFamily: AmazonLinux2
desiredCapacity: 1
iam:
withAddonPolicies:
autoScaler: true
instanceType: t3.xlarge
maxSize: 2
minSize: 0
name: managed-ng-1
version: "1.22"
volumeSize: 80
volumeType: gp3
metadata:
name: demo-cluster
region: us-west-2
version: "1.22"
network:
cni:
name: aws-cni
params:
customCniCrdSpec:
us-west-2a:
- securityGroups:
- sg-09706d2348936a2b1
subnet: subnet-0f854d90d85509df9
us-west-2b:
- securityGroups:
- sg-09706d2348936a2b1
subnet: subnet-0301d84c8b9f82fd1
vpc:
clusterEndpoints:
privateAccess: false
publicAccess: true
nat:
gateway: Single
subnets:
private:
subnet-06e99eb57fcf4f117:
id: subnet-06e99eb57fcf4f117
subnet-0509b963a387f7fc7:
id: subnet-0509b963a387f7fc7
public:
subnet-056b49f76124e37ec:
id: subnet-056b49f76124e37ec
subnet-0e8e6d17f6cb05b29:
id: subnet-0e8e6d17f6cb05b29
proxyConfig: {}
type: aws-eks
System Sync Output¶
On a successful run, select the job and click Output to view any updates/modifications triggered recently within the system. Below is an example of the resource updates triggered internally in the controller.
Event Payload
Click Event Payload to view in which resource the changes are performed, user id, type of operation, etc.
- The changes performed internally through the controller, i.e., System To Git Sync, the Trigger type in the controller is Internally Triggered
- The changes performed externally through Git repository, i.e., Git to System Sync, the Trigger type is Webhook
- Manually running the job after a change using the Run icon, the Trigger type is Manually Triggered
System Sync Pipeline Sharing¶
Users can share a system sync pipeline with All Projects/Specific Projects/None. When sharing the system sync pipeline and any resources containing a secret value/key, system sync fails for that specific resource and proceeds with the other resources
Users can also create a new project in Git Repository and use the existing pipeline of any project
Important
To use GitOps for cluster operations, users must upgrade their existing GitOps Agent