Deploying Trilio For Kubernetes with OpenShift & Red Hat Advanced Cluster Management Policies

This article was written by Sachin Kulkarni of Trilio.

Introduction

Organizations are moving towards Kubernetes as an operating environment, and protecting the data is paramount. It’s their top-most priority to protect the business-critical data, and set up a business continuity plan in case of a disaster. A cloud-native backup and Disaster Recovery (DR) solution is the need of the hour, and the answer is TrilioVault for Kubernetes (TVK).

TVK is a cloud-native, application-centric data protection platform designed to support the scale, performance, and mobility requirements of Kubernetes container environments across any public or hybrid cloud environment. It offers backup and DR solutions for the entire application, including data, metadata, and Kubernetes objects. It is protected and can be restored from any point-in-time.

RHACM is a management solution designed to help organizations extend and scale Red Hat OpenShift, the leading enterprise Kubernetes platform. It enables management consistency across hybrid cloud deployments, including on-premises and public clouds.

Integration between TVK and RHACM provides an automated solution to protect containers, Kubernetes, and cloud infrastructure using the policy framework. This powerful framework allows users to govern multiple clusters in the data center using policies. The TVK policies enable the users to protect their OpenShift clusters and provide continued protection for existing and new clusters.

This blog provides a walk-through on how to deploy the TVK policies on RHACM. It’s an efficient way to protect your cloud workloads.

Prerequisites

Be sure to meet the following prerequisites:

• Install Red Hat OpenShift Kubernetes (version 4.8 and later) clusters.
• Meet TrilioVault prerequisites. See Prerequisites for using TrilioVault for Kubernetes
• Create an environment that supports CSI Driver with snapshot capability. See CSI Drivers and Snapshot capabilities for more information.
• Install VolumeSnapshot CRDs. See Installing VolumeSnapShot CRDs.

Install RHACM

RHACM can be used to comply with enterprise and industry standards for aspects, such as security and regulatory compliance, resiliency, and software engineering. Deviation from defined values for such standards represents a configuration drift, which can be detected using the built-in configuration policy controller of RHACM.

Install RHACM version 2.6 as outlined in the product documentation.

Deploy TrilioVault for Kubernetes TVK policies

In the RHACM policy collection from the community folder, there are three TVK policies available. View the following table for the TVK policy descriptions:

Let’s start with the policy to install TrilioVault for Kubernetes (TVK).

Install TVK Using a Policy

Let’s use the policy to install TVK. This policy installs the latest TVK operator on any Openshift cluster with a label protected-by=triliovault.

For demo purposes, there are two OpenShift Container Platform clusters; one of them runs RHACM and is the hub cluster. You can view the clusters from the RHACM console Clusters page. View the following screen capture:

Note: Notice the clusters are prelabeled with the label protected-by=triliovault. There are no TVK policies created on the RHACM cluster yet, so let’s start with our first example.

From the navigation menu, select Governance. This is also referred to as the policy engine. Then click the Create policy button as shown in the following screen capture:

The Create policy form is displayed. By default, the YAML editor is also displayed alongside the form.

Let’s go to the policy from GitHub page, click the Raw button to view the contents of the policy YAML, copy the YAML code from GitHub and paste into the YAML editor of RHACM.

Note: Before pasting the policy contents into RHACM, be sure to clear the YAML editor. Typically, you can use the shortcut, <ctrl>-a <ctrl>-c from the GitHub window, and <ctrl>-a <ctrl>-v from the RHACM window. After you paste the policy into that YAML editor, your console might resemble the following image:

In the last line of the policy code of the PlacementRule section, notice that this policy uses the labels vendor=OpenShift and protected-by=triliovault. This indicates that the policy is deployed on all OpenShift clusters with the user-defined label, protected-by=triliovault. Before you create the policy, you need to select a namespace for the policy to be run in. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the policy form, select the default namespace or any other namespace available on the hub cluster. You can create some specific policy-engine namespaces in advance to be able to group them more efficiently. For this example, select Inform as the remediation action.

Before you create the policy, let’s verify that TVK is not installed. Recheck the list of installed operators on the cluster itself, from the OpenShift console:

It is verified that TVK is not installed, so let’s create the policy by clicking the Submit button in the Create policy dialog box. After a couple of moments, the policy is listed in the policy table. Notice that there is a cluster violation. Select the install-tvk policy to view the policy details and results. View the following image:

Select the Results Tab to view the listed violations:

The required operator elements are missing, which is why the policy failed. Let’s change the remediation action by navigating to the Governance Overview tab, select the Actions icon (three vertical dots), and click Enforce:

The Enforce policy dialog box is displayed. Click Enforce to confirm the action. View the following image:

It takes a few minutes for enforcement of the policy. Recheck the details of the policy from the Results tab. View the following image:

And you can confirm that the operator has been installed on your cluster with the protected-by=triliovault label.

This concludes how to install TrilioVault for Kubernetes (TVK) on the clusters managed by RHACM. This policy also protects new OpenShift Container Platform clusters with a label protected-by=triliovault.

Protect a Namespace in the Cluster Using the TVK Backup Creation Policy

Now that the TVK is installed and a trial license is created (as seen in the image, where TVK is listed as an installed operator), let’s start using TVK. To get started with TVK in your environment, be surethat all prerequisites are fulfilled as listed in the Prerequisites section. Once done, you must perform the following steps:

• Create a TVK Target – Location where backups are stored
• Create a backup plan – Details about the backup
• Create a namespace-based backup

Creating a policy that creates a namespace-based backup using TVK helps to achieve the earlier mentioned steps for a single namespace. Let’s take a look at how to apply the policy. For this example, the namespace is mysql, with the MySQL application running. Run the following command to receive a list of mysql applications:

# oc get all -n mysql
NAME                                                                  READY   STATUS      RESTARTS   AGE
pod/k8s-demo-app-mysql-78dbddc54f-dtpd6                               1/1     Running     0          1d

NAME                         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
service/k8s-demo-app-mysql   ClusterIP   None         <none>        3306/TCP   1d

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/k8s-demo-app-mysql   1/1     1            1           1d

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/k8s-demo-app-mysql-78dbddc54f   1         1         1       1d
#

Navigate to the create-tvk-backup policy in the policy-collection repository. As mentioned in the comments of the create-tvk-backup policy, you must create a secret and ConfigMap with AWS S3 details for TVK target creation. Your secret and ConfigMap might resemble the following content:

cat <<EOF | oc apply -f - -n default
apiVersion: v1
kind: Secret
metadata:
  name: aws-s3-secret
  namespace: default
type: Opaque
stringData:
  accessKey: "PROVIDE_ACCESS_KEY"
  secretKey: "PROVIDE_SECRET_KEY"
EOF

cat <<EOF | oc apply -f - -n default
apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-s3-configmap
  namespace: default
data:
  bucketName: "PROVIDE_S3_BUCKET_NAME"
  region: "PROVIDE_REGION"
  thresholdCapacity: "100Gi"
EOF

Replace the test namespace used in the policy with mysql to create the backup. You can make this change from the YAML editor in RHACM.

In the PlacementRule section, notice that this policy is defined to be applied on all clusters with labels vendor=OpenShift and protected-by=triliovault. This policy is expected to be deployed on all OpenShift clusters with user-defined label protected-by=triliovault. Before you select the Create button, you still need to select a namespace in which this policy needs to be applied. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the Create policy form, you can select the default namespace on the hub cluster. View the following image:

Click Submit to create the policy. Afterwards a Policy created banner appears. View the following image:

Notice, that RHACM detected a cluster violation. This is because the remediation action must be set to enforce to create a backup for the TVK operator. Return to the Policies tab and select the Actions icon for the create-tvk-backup policy. Select the Enforce option.

From the Enforce policy box, click Enforce to confirm the remediation action change:

It takes a few minutes for the policy to be enforced. Select the policy name to see the details and results. Select the Results tab to view the cluster violations. Based on the following image, there are no longer any violations:

As expected and displayed in the previous screenshot, the policy created a secret that uses AWS S3 credentials, a target to store backup data in the specified S3 bucket, a BackupPlan with a daily backup schedule, and a first full backup of the namespace mysql. With this, the namespace mysql is protected, and a new backup is used daily.

Let’s also log in to the TVK console and sign in through your OpenShift cluster. Refer to OpenShift Environments in the Trilio documentation for more details. View the following image:

View the backups created for the mysql namespace from the TrilioVault management console, as shown in the following image:

This concludes how to create a namespace-based backup using the TVK policy on OpenShift clusters with the label, protected-by=triliovault, which is managed by RHACM.

However, you must update the namespace name to create the backup manually. Note, it is expected that the same namespace is present on all the OpenShift clusters with the label protected-by=triliovault.

Continue reading to learn how to protect multiple namespaces on any OpenShift clusters, matching the placement rule managed by RHACM.

Protect Multiple Namespaces in Any Cluster Using the TVK Backup Creation Policy

Before starting, be sure that all prerequisites are fulfilled as listed in the previous section, Protect a namespace in the cluster using the TVK backup creation policy.

Creating a policy that creates a namespace-based backup using TVK and a Kyverno template, creates a backup of the namespaces that have the protected-by=tvk-ns-backup label on all the OpenShift clusters with a protected-by=triliovault label. In addition to installing TVK, the Kyverno controller must be installed to use the kyverno policy. See the Policy to install Kyverno from the policy-collection.

Note – Grant additional privileges to the Kyverno service account for the namespaces.

Once the prerequisites are met, this policy performs the following steps for each existing or newly created namespace, with the protected-by=tvk-ns-backup label:

• Create a TVK Target – Location where backups are stored.
• Create a backup plan – Details about the backup
• Create a namespace-based backup

As a reminder, TVK is already installed with the install-tvk policy. Now, let’s create the policy-kyverno-tvk-create-ns-backup policy. Click the Create policy button. Return to the policy example in the policy-collection repository to copy the contents, and paste them in the RHACM YAML editor. View the following image:

In the PlacementRule section, notice that this policy is defined to be applied on all clusters with the labels vendor=OpenShift and protected-by=triliovault. This policy is expected to be deployed on all OpenShift clusters with the user-defined label, protected-by=triliovault. Before you select the Create button, you still need to select a namespace in which this policy needs to be applied. This is for internal organization reasons only; it does NOT affect the results of the policy engine itself. From the Create policy form, you can select the default namespace on the hub cluster. View the following image:

Click Submit to create the policy. After the policy is created, the policy details are displayed similar to the following image:

Similar to the previous example in the earlier section, RHACM detected a cluster violation. There is a cluster violation because the remediation action must be set to enforce. From the Details tab, select the Actions drop-down menu and click Enforce. View the following image:

Click Enforce to confirm that action from the Enforce policy box.

It takes a few minutes for the policy to be enforced. After you select the policy name, select the Results tab:

The previous screenshot shows that the policy created a Kyverno policy on the target cluster. This Kyverno policy monitors all the namespaces and creates a target to store backup data in the specified S3 bucket, a BackupPlan with a daily backup schedule, and a first, full backup for all the namespaces that have the protected-by=tvk-ns-backup label. This is also applied to any new namespace created with the same label.

Let’s log in to the TVK console through OpenShift to check if the backups are created:

As seen in the previous screenshot, there are no backups from the Monitoring page in the Cluster Dashboard section. This is because there are no namespaces in the target cluster with the label protected-by=tvk-ns-backup.

For this demo, let’s create a namespace named postgres with the PostgreSQL application running. Run the following command:

# oc get all -n postgres
NAME                  READY   STATUS    RESTARTS   AGE
pod/postgres-demo-0   1/1     Running   0          5m28s
pod/postgres-demo-1   1/1     Running   0          5m3s

NAME               TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/postgres   NodePort   172.30.17.37   <none>        5432:30992/TCP   5m29s

NAME                             READY   AGE
statefulset.apps/postgres-demo   2/2     5m29s
#

Apply the protected-by=tvk-ns-backup label to the postgres namespace with the following command:

# oc label ns postgres protected-by=tvk-ns-backup
namespace/postgres labeled
#

It takes a few minutes for the Kyverno policy to come into effect. When you check the details, notice that the resource details within the namespace are displayed using the following command:

# oc get target,backupplan,backup -n postgres
NAME                                                  TYPE          THRESHOLD CAPACITY   VENDOR   STATUS      BROWSING ENABLED
target.triliovault.trilio.io/tvk-postgres-s3-target   ObjectStore   100Gi                AWS      Available

NAME                                                       TARGET                   RETENTION POLICY   INCREMENTAL SCHEDULE   FULL BACKUP SCHEDULE           STATUS
backupplan.triliovault.trilio.io/tvk-postgres-backupplan   tvk-postgres-s3-target                                             trilio-daily-schedule-policy   Available

NAME                                               BACKUPPLAN                BACKUP TYPE   STATUS      DATA SIZE   CREATION TIME          START TIME             END TIME               PERCENTAGE COMPLETED   BACKUP SCOPE   DURATION
backup.triliovault.trilio.io/tvk-postgres-backup   tvk-postgres-backupplan   Full          Available   112328704   2022-09-29T10:15:03Z   2022-09-29T10:15:03Z   2022-09-29T10:23:22Z   100                    Namespace      8m19.479134026s
#

Let’s check the Monitoring page for the Cluster Dashboard in the TVK console. In the following screen capture, notice a backup is created for the postgres namespace:

This concludes how to create a namespace-based backup using Kyverno-based, TVK policy for all the namespaces that have the protected-by=tvk-ns-backup label on the OpenShift clusters with the protected-by=triliovault label, which is managed by RHACM. Apply the protected-by=tvk-ns-backup label for any namespace to be protected.

Conclusion

TrilioVault for Kubernetes (TVK) is a cloud-native, application-centric, data protection platform designed to support the scale, performance, and mobility requirements of Kubernetes container environments across any public or hybrid cloud environment. It offers backup and recovery of the entire application, including data, metadata, and Kubernetes objects. It is protected and can be restored from any point-in-time.

TVK provides the following wide range of helpful features:

• Native Kubernetes application
• Storage of metadata and all application resources to a specified target
• Support for Helm, Label, and Operators application deployment types and support for S3 or NFS-based backup targets
• Provisioning of application hooks to provide data consistent backups
• Support for RHACM policies to install TVK and create backups for namespaces

TVK provides a strong platform for enterprise database backups. For more information on TVK, request a demo or download your free trial today.

References

• RHACM and Policies – An Introduction blog
• Implement Policy-based Governance Using Configuration Management of Red Hat Advanced Cluster Management for Kubernetes blog
• RHACM Installation documentation
• Welcome to TrilioVault For Kubernetes (TVK) documentation