This article was written by Sachin Kulkarni of Trilio.
Introduction
Organizations are moving towards Kubernetes as an operating environment, and protecting the data is paramount. It’s their top-most priority to protect the business-critical data and set up a business continuity plan in case of a disaster. A cloud-native backup and Disaster Recovery (DR) solution is the need of the hour, and the answer is TrilioVault for Kubernetes (TVK).
TVK is a cloud-native, application-centric data protection platform designed to support the scale, performance, and mobility requirements of Kubernetes container environments across any public or hybrid cloud environment. It offers backup and DR solutions for the entire application, including data, metadata, and Kubernetes objects. It is protected and can be restored at any point in time.
RHACM is a management solution designed to help organizations extend and scale Red Hat OpenShift, the leading enterprise Kubernetes platform. It enables management consistency across hybrid cloud deployments, including on-premises and public clouds.
Integration between TVK and RHACM provides an automated solution to protect containers, Kubernetes, and cloud infrastructure using the policy framework. This powerful framework allows users to govern multiple clusters in the data center using policies. The TVK policies enable the users to protect their OpenShift clusters and provide continued protection for existing and new clusters.
This blog provides a walk-through on how to deploy the TVK policies on RHACM. It’s an efficient way to protect your cloud workloads.
Prerequisites
Be sure to meet the following prerequisites:
- Install Red Hat OpenShift Kubernetes (version 4.8 and later) clusters.
- Meet TrilioVault prerequisites. See Prerequisites for using TrilioVault for Kubernetes
- Create an environment that supports CSI Driver with snapshot capability. See CSI Drivers and Snapshot capabilities for more information.
- Install
VolumeSnapshotCRDs. See Installing VolumeSnapShot CRDs.
Install RHACM
RHACM can be used to comply with enterprise and industry standards for aspects, such as security and regulatory compliance, resiliency, and software engineering. Deviation from defined values for such standards represents a configuration drift, which can be detected using the built-in configuration policy controller of RHACM.
Install RHACM version 2.6 as outlined in the product documentation.
Deploy TrilioVault for Kubernetes TVK policies
In the RHACM policy collection from the community folder, there are three TVK policies available. View the following table for the TVK policy descriptions:
| Policy | Description | Prerequisites |
| Policy to install TVK Operator | Use this policy to install TrilioVault for Kubernetes Operator and a trial license on OpenShift clusters with the label protected-by=triliovault. | Use this policy to create a namespace-based backup using TKO and Kyverno template on OpenShift clusters with the label protected-by=triliovault. It creates a backup of the namespaces that have the following label, protected-by=tvk-ns-backup. |
| Policy to create namespace-based backup using TrilioVault for Kubernetes | Use this policy to create namespace-based backups using TVK on OpenShift clusters with the label protected-by=triliovault. | Requires OpenShift 4.8 or later. Note: TVK must be installed to use this policy. On the hub cluster, create a secret named aws-s3-secret with S3 credentials and a ConfigMap named aws-s3-configmap with S3 bucket name, region name & threshold capacity in the namespace, where this policy is created (details given in the policy). For more information, refer to the Trilio documentation. |
| Policy to create namespace-based backup using TVK and Kyverno template | Use this policy to create a namespace-based backup using TKO and Kyverno templates on OpenShift clusters with the label protected-by=triliovault. It creates a backup of the namespaces that have the following label, protected-by=tvk-ns-backup. | Requires OpenShift 4.8 or later. Note: Kyverno controller must be installed to use the kyverno policy. See the Policy to install Kyverno. TVK must be installed to use this policy. On the hub cluster, create a secret named aws-s3-secret with S3 credentials, and a ConfigMap named aws-s3-configmap with S3 bucket name, region name & threshold capacity in the namespace, where this policy is created (details given in the policy). For more information, refer to the Trilio documentation. |
Let’s start with the policy to install TrilioVault for Kubernetes (TVK).
Install TVK Using a Policy
Let’s use the policy to install TVK. This policy installs the latest TVK operator on any Openshift cluster with a label protected-by=triliovault.
For demo purposes, there are two OpenShift Container Platform clusters; one of them runs RHACM and is the hub cluster. You can view the clusters from the RHACM console Clusters page. View the following screen capture:
Note: Notice the clusters are prelabeled with the label protected-by=triliovault. There are no TVK policies created on the RHACM cluster yet, so let’s start with our first example.
From the navigation menu, select Governance. This is also referred to as the policy engine. Then click the Create policy button as shown in the following screen capture:
The Create policy form is displayed. By default, the YAML editor is also displayed alongside the form.
Let’s go to the policy from the GitHub page, click the Raw button to view the contents of the policy YAML, copy the YAML code from GitHub, and paste it into the YAML editor of RHACM.
Note: Before pasting the policy contents into RHACM, be sure to clear the YAML editor. Typically, you can use the shortcut, <ctrl>-a <ctrl>-c from the GitHub window, and <ctrl>-a <ctrl>-v from the RHACM window. After you paste the policy into that YAML editor, your console might resemble the following image:
In the last line of the policy code of the PlacementRule section, notice that this policy uses labels vendor=OpenShift and protected-by=triliovault. This indicates that the policy is deployed on all OpenShift clusters with the user-defined label, protected-by=triliovault. Before you create the policy, you need to select a namespace for the policy to be run. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the policy form, select the default namespace or any other namespace available on the hub cluster. You can create some specific policy-engine namespaces in advance to be able to group them more efficiently. For this example, select Inform as the remediation action.
Before you create the policy, let’s verify that TVK is not installed. Recheck the list of installed operators on the cluster itself, from the OpenShift console:
It is verified that TVK is not installed, so let’s create the policy by clicking the Submit button in the Create policy dialog box. After a couple of moments, the policy is listed in the policy table. Notice that there is a cluster violation. Select the install-tvk policy to view the policy details and results. View the following image:
Select the Results Tab to view the listed violations:
The required operator elements are missing, which is why the policy failed. Let’s change the remediation action by navigating to the Governance Overview tab, selecting the Actions icon (three vertical dots), and clicking Enforce:
The Enforce policy dialog box is displayed. Click Enforce to confirm the action. View the following image:
It takes a few minutes to enforcement of the policy. Recheck the details of the policy from the Results tab. View the following image:
And you can confirm that the operator has been installed on your cluster with the protected-by=triliovault label.
This concludes how to install TrilioVault for Kubernetes (TVK) on the clusters managed by RHACM. This policy also protects new OpenShift Container Platform clusters with a label protected-by=triliovault.
Protect a Namespace in the Cluster Using the TVK Backup Creation Policy
Now that the TVK is installed and a trial license is created (as seen in the image, where TVK is listed as an installed operator), let’s start using TVK. To get started with TVK in your environment, be sure that all prerequisites are fulfilled as listed in the Prerequisites section. Once done, you must perform the following steps:
- Create a TVK Target – Location where backups are stored
- Create a backup plan – Details about the backup
- Create a namespace-based backup
Creating a policy that creates a namespace-based backup using TVK helps to achieve the earlier-mentioned steps for a single namespace. Let’s take a look at how to apply the policy. For this example, the namespace is mysql, with the MySQL application running. Run the following command to receive a list of mysql applications:
# oc get all -n mysql NAME READY STATUS RESTARTS AGE pod/k8s-demo-app-mysql-78dbddc54f-dtpd6 1/1 Running 0 1d NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/k8s-demo-app-mysql ClusterIP None <none> 3306/TCP 1d NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/k8s-demo-app-mysql 1/1 1 1 1d NAME DESIRED CURRENT READY AGE replicaset.apps/k8s-demo-app-mysql-78dbddc54f 1 1 1 1d #
Navigate to the create-tvk-backup policy in the policy-collection repository. As mentioned in the comments of the create-tvk-backup policy, you must create a secret and ConfigMap with AWS S3 details for TVK target creation. Your secret and ConfigMap might resemble the following content:
cat <<EOF | oc apply -f - -n default apiVersion: v1 kind: Secret metadata: name: aws-s3-secret namespace: default type: Opaque stringData: accessKey: "PROVIDE_ACCESS_KEY" secretKey: "PROVIDE_SECRET_KEY" EOF
cat <<EOF | oc apply -f - -n default apiVersion: v1 kind: ConfigMap metadata: name: aws-s3-configmap namespace: default data: bucketName: "PROVIDE_S3_BUCKET_NAME" region: "PROVIDE_REGION" thresholdCapacity: "100Gi" EOF
Replace the test namespace used in the policy mysql to create the backup. You can make this change from the YAML editor in RHACM.
In the PlacementRule section, notice that this policy is defined to be applied on all clusters with labels vendor=OpenShift and protected-by=triliovault. This policy is expected to be deployed on all OpenShift clusters with user-defined label protected-by=triliovault. Before you select the Create button, you still need to select a namespace in which this policy needs to be applied. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the Create policy form, you can select the default namespace on the hub cluster. View the following image:
Click Submit to create the policy. Afterwards a Policy created banner appears. View the following image:
Notice, that RHACM detected a cluster violation. This is because the remediation action must be set to enforce to create a backup for the TVK operator. Return to the Policies tab and select the Actions icon for the create-tvk-backup policy. Select the Enforce option.
From the Enforce policy box, click Enforce to confirm the remediation action change:
It takes a few minutes for the policy to be enforced. Select the policy name to see the details and results. Select the Results tab to view the cluster violations. Based on the following image, there are no longer any violations:
As expected and displayed in the previous screenshot, the policy created a secret that uses AWS S3 credentials, a target to store backup data in the specified S3 bucket, BackupPlan with a daily backup schedule, and a first full backup of the namespace mysql. With this, the namespace mysql is protected, and a new backup is used daily.
Let’s also log in to the TVK console and sign in through your OpenShift cluster. Refer to OpenShift Environments in the Trilio documentation for more details. View the following image:
View the backups created for the mysql namespace from the TrilioVault management console, as shown in the following image:
This concludes how to create a namespace-based backup using the TVK policy on OpenShift clusters with the label, protected-by=triliovault, which is managed by RHACM.
However, you must update the namespace name to create the backup manually. Note, it is expected that the same namespace is present on all the OpenShift clusters with the label protected-by=triliovault.
Continue reading to learn how to protect multiple namespaces on any OpenShift clusters, matching the placement rule managed by RHACM.
Protect Multiple Namespaces in Any Cluster Using the TVK Backup Creation Policy
Before starting, be sure that all prerequisites are fulfilled as listed in the previous section, Protect a namespace in the cluster using the TVK backup creation policy.
Creating a policy that creates a namespace-based backup using TVK and a Kyverno template, creates a backup of the namespaces that have the protected-by=tvk-ns-backup label on all the OpenShift clusters with a protected-by=triliovault label. In addition to installing TVK, the Kyverno controller must be installed to use the kyverno policy. See the Policy to install Kyverno from the policy-collection.
Note – Grant additional privileges to the Kyverno service account for the namespaces.
Once the prerequisites are met, this policy performs the following steps for each existing or newly created namespace, with the protected-by=tvk-ns-backup label:
- Create a TVK Target – Location where backups are stored.
- Create a backup plan – Details about the backup
- Create a namespace-based backup
As a reminder, TVK is already installed with the install-tvk policy. Now, let’s create the policy-kyverno-tvk-create-ns-backup policy. Click the Create Policy button. Return to the policy example in the policy-collection repository to copy the contents, and paste them in the RHACM YAML editor. View the following image:
In the PlacementRule section, notice that this policy is defined to be applied on all clusters with the labels vendor=OpenShift and protected-by=triliovault. This policy is expected to be deployed on all OpenShift clusters with the user-defined label, protected-by=triliovault. Before you select the Create button, you still need to select a namespace in which this policy needs to be applied. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the Create policy form, you can select the default namespace on the hub cluster. View the following image:
Click Submit to create the policy. After the policy is created, the policy details are displayed similarly in the following image:
Similar to the previous example in the earlier section, RHACM detected a cluster violation. There is a cluster violation because the remediation action must be set to enforce. From the Details tab, select the Actions drop-down menu and click Enforce. View the following image:
Click Enforce to confirm that action from the Enforce policy box.
It takes a few minutes for the policy to be enforced. After you select the policy name, select the Results tab:
The previous screenshot shows that the policy created a Kyverno policy on the target cluster. This Kyverno policy monitors all the namespaces and creates a target to store backup data in the specified S3 bucket, BackupPlan with a daily backup schedule, and a first, full backup for all the namespaces that have the protected-by=tvk-ns-backup label. This is also applied to any new namespace created with the same label.
Let’s log in to the TVK console through OpenShift to check if the backups are created:
As seen in the previous screenshot, there are no backups from the Monitoring page in the Cluster Dashboard section. This is because there are no namespaces in the target cluster with the label protected-by=tvk-ns-backup.
For this demo, let’s create a namespace named postgres with the PostgreSQL application running. Run the following command:
# oc get all -n postgres NAME READY STATUS RESTARTS AGE pod/postgres-demo-0 1/1 Running 0 5m28s pod/postgres-demo-1 1/1 Running 0 5m3s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/postgres NodePort 172.30.17.37 <none> 5432:30992/TCP 5m29s NAME READY AGE statefulset.apps/postgres-demo 2/2 5m29s #
Apply the protected-by=tvk-ns-backup label to the postgres namespace with the following command:
# oc label ns postgres protected-by=tvk-ns-backup namespace/postgres labeled #
It takes a few minutes for the Kyverno policy to come into effect. When you check the details, notice that the resource details within the namespace are displayed using the following command:
# oc get target,backupplan,backup -n postgres NAME TYPE THRESHOLD CAPACITY VENDOR STATUS BROWSING ENABLED target.triliovault.triliostg.wpengine.com/tvk-postgres-s3-target ObjectStore 100Gi AWS Available NAME TARGET RETENTION POLICY INCREMENTAL SCHEDULE FULL BACKUP SCHEDULE STATUS backupplan.triliovault.triliostg.wpengine.com/tvk-postgres-backupplan tvk-postgres-s3-target trilio-daily-schedule-policy Available NAME BACKUPPLAN BACKUP TYPE STATUS DATA SIZE CREATION TIME START TIME END TIME PERCENTAGE COMPLETED BACKUP SCOPE DURATION backup.triliovault.triliostg.wpengine.com/tvk-postgres-backup tvk-postgres-backupplan Full Available 112328704 2022-09-29T10:15:03Z 2022-09-29T10:15:03Z 2022-09-29T10:23:22Z 100 Namespace 8m19.479134026s #
Let’s check the Monitoring page for the Cluster Dashboard in the TVK console. In the following screen capture, notice a backup is created for the postgres namespace:
This concludes how to create a namespace-based backup using Kyverno-based, TVK policy for all the namespaces that have the protected-by=tvk-ns-backup label on the OpenShift clusters with the protected-by=triliovault label, which is managed by RHACM. Apply the protected-by=tvk-ns-backup label for any namespace to be protected.
Conclusion
TrilioVault for Kubernetes (TVK) is a cloud-native, application-centric, data protection platform designed to support the scale, performance, and mobility requirements of Kubernetes container environments across any public or hybrid cloud environment. It offers backup and recovery of the entire application, including data, metadata, and Kubernetes objects. It is protected and can be restored at any point in time.
TVK provides the following wide range of helpful features:
- Native Kubernetes application
- Storage of metadata and all application resources to a specified target
- Support for Helm, Label, and Operators application deployment types and support for S3 or NFS-based backup targets
- Provisioning of application hooks to provide data-consistent backups
- Support for RHACM policies to install TVK and create backups for namespaces
TVK provides a strong platform for enterprise database backups. For more information on TVK, request a demo or download your free trial today.
References
- RHACM and Policies – An Introduction blog
- Implement Policy-based Governance Using Configuration Management of Red Hat Advanced Cluster Management for Kubernetes blog
- RHACM Installation documentation
- Welcome to TrilioVault For Kubernetes (TVK) documentation
