Why OpenShift Architecture is Ideal for Enterprise Containerization

OpenShift infrastructure components

In a typical OpenShift deployment, there are two different node types: control plane nodes (also called master nodes) and worker nodes. In certain deployment architectures, a select set of worker nodes is designated only for running cluster infrastructure workloads, which are called infra nodes. 

Control plane nodes

These nodes run the OpenShift control plane components, which include:

• Core Kubernetes components such as kube-apiserver, kube-controller-manager, kube-scheduler, etc.
• OpenShift components/operators such as the Cluster Version Operator, MachineConfig Operator, etc. 

For a highly available configuration, you should run at least three control plane nodes. These nodes do not run user applications.

Worker nodes (compute nodes)

Worker nodes run user applications. They provide computing, networking, and storage resources for workloads. You can add worker nodes as needed, thus scaling the cluster.  

Worker nodes require kubelet and the CRI-O container runtime to run and manage pods.

Infra nodes

While modern OpenShift architectures do not have mandatory infra nodes in the design, some worker nodes can be designated as infrastructure nodes to host and offload cluster infrastructure services such as ingress controllers, the OpenShift Image Registry, monitoring tools like Prometheus and Grafana, etc. This can be achieved by tainting and labelling nodes.

OpenShift container operating system

Red Hat Enterprise Linux (RHEL) CoreOS (RHCOS) is the operating system for deploying OpenShift clusters. It is a purpose-built commercial product for enterprises and is the default host OS for both the control plane (master nodes) and the worker nodes. A key feature of RHCOS is its immutability, where the core file system is read-only, preventing configuration drift and simplifying management. 

It is based on the CoreOS Container Linux project, which is now discontinued. Fedora CoreOS is the successor to CoreOS and is the open source community upstream for RHCOS.

Operators

Operators leverage custom resource definitions, an extension capability of Kubernetes. Using operators, you can create custom object types for any use case. 

Red Hat OpenShift uses operators to manage platform components and applications. Operators also perform container engine installation and storage, networking, and image registry configuration. 

Operators are developed using the Operator Framework, a set of developer tools and Kubernetes components that aid operator development, management, and discovery.

MachineConfig Operator (MCO)

The MCO is used to manage and configure cluster nodes, such as the control plane, infra, and worker nodes. It ensures that all the nodes are in sync with the desired state of the OpenShift cluster. The MCO applies operating system configurations such as kernel arguments, systemd configurations, kubelet configuration, etc. It also upgrades RHCOS on the nodes automatically and reboot nodes if necessary.

Cluster operators

Cluster operators manage the lifecycle of core OpenShift components such as the Cluster Version Operator (CVO), Cluster Network Operator, Cluster Monitoring Operator, Cluster Ingress Operator, and Machine Config Operator (MCO). 

The following is a partial list of cluster operators you can see when running the oc get clusteroperators command.

$ oc get clusteroperators 
NAME                          VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE   
authentication                4.10.31   True        False         False      2d4h    
baremetal                     4.10.31   True        False         False      59d     
cloud-controller-manager      4.10.31   True        False         False      59d     
cloud-credential              4.10.31   True        False         False      59d     
cluster-autoscaler            4.10.31   True        False         False      59d     
config-operator               4.10.31   True        False         False      59d     
console                       4.10.31   True        False         False      2d4h    
csi-snapshot-controller       4.10.31   True        False         False      59d     
dns                           4.10.31   True        False         False      59d     
etcd                          4.10.31   True        False         False      59d     
image-registry                4.10.31   True        False         False      58d     
ingress                       4.10.31   True        False         False      59d     
insights                      4.10.31   True        False         False      59d     
kube-apiserver                4.10.31   True        False         False      59d     
kube-controller-manager       4.10.31   True        False         False      59d     
kube-scheduler                4.10.31   True        False         False      59d     
machine-api                   4.10.31   True        False         False      59d     
machine-approver              4.10.31   True        False         False      59d     
machine-config                4.10.31   True        False         False      59d     
marketplace                   4.10.31   True        False         False      59d     
monitoring                    4.10.31   True        False         False      59d     
network                       4.10.31   True        False         False      59d     
node-tuning                   4.10.31   True        False         False      58d     
openshift-apiserver           4.10.31   True        False         False      39d     
openshift-controller-manager  4.10.31   True        False         False      28d     
openshift-samples             4.10.31   True        False         False      58d     
operator-lifecycle-manager    4.10.31   True        False         False      59d     
service-ca                    4.10.31   True        False         False      59d     
storage                       4.10.31   True        False         False      59d

OpenShift cluster components

An OpenShift cluster contains the usual Kubernetes components such as the API Server, Scheduler, Controller Manager, and etcd. Here’s a quick introduction to these components:

• API Server: Exposes the Kubernetes API, validates all incoming requests, and acts as the frontend for interacting with the cluster. 
• Scheduler: Selects a suitable node for pods to run based on resource requirements and other constraints.
• Controller Manager: Runs different controllers that collectively work to maintain the cluster’s desired state. 
• etcd: A highly available key-value data store that acts as the single source of truth for cluster data, configurations, status, metadata, etc. 

OpenShift-specific components

The following is a list of OpenShift cluster components that enhance OpenShift’s capability, user experience, and security functionality: 

• OpenShift API Server builds on the Kubernetes API Server and provides additional APIs for OpenShift-specific resources like projects, builds, images, and DeploymentConfigs. You communicate through the OpenShift API Server for all interactions with the cluster. 
• OpenShift Cluster Version Operator (CVO) is an important component of OpenShift that is responsible for managing the lifecycle of the entire cluster and its core components. Specifically, the CVO is used to enable and perform OpenShift cluster upgrades. 

Container engine

CRI-O is the default container engine in OpenShift and runs on all nodes in the cluster. It was purpose-built for Kubernetes and OpenShift as a lightweight, faster, and secure alternative to the Docker container runtime. It is compliant with the Kubernetes Container Runtime Interface (CRI) standard, making it compatible with any OCI-compatible runtime, such as runc. 

The minimal nature of CRI-O makes it more secure—it contains only the components that Kubernetes needs. This also means faster container (and thus pod) startup and lower resource usage. It also includes security tools like SELinux, seccomp, AppArmor, and cgroups. Hence, it is more stable, compatible, and reliable in production. 

The kubelet running on each node talks to the CRI-O through the CRI API, which pulls container images, uses runc to start containers, and manages their lifecycle.  

Before OpenShift 4.x, the container runtime was Docker. 

Networking

OpenShift builds on the Kubernetes networking model and uses the Container Network Interface (CNI) for pod networking, services, and network security policies. OVN-Kubernetes is the default CNI in OpenShift version 4 onwards. It also includes Multus CNI to support attaching multiple networks to a pod for use cases like telecom and Network Function Virtualization (NFV). 

The following OpenShift components provide various networking capabilities: 

• OpenShift DNS: This component deploys and manages CoreDNS as a DaemonSet, which provides name resolution service internally, such as for pods and services. The default cluster domain is cluster.local. 
• OpenShift ingress: The ingress component manages and exposes HTTPS routes, including support for TLS from the outside world to the services running inside the cluster. These are HA-proxy-based ingress.
• OpenShift network: The network component provides cluster-wide networking like pod-to-pod, pod-to-service, and service-to-outside connectivity. Capabilities such as network isolation using network policies, load balancing, and service discovery are also managed by this component. 

Storage

OpenShift uses the Container Storage Interface (CSI), like Kubernetes, providing a pluggable interface to support various persistent storage backends such as block, file, and object. Third-party storage vendors can then offer CSI drivers that integrate seamlessly with their storage solutions. Popular examples include NetApp, Dell, Ceph, AWS EBS, Azure Disk, Google PD, etc. Red Hat OpenShift Container Platform includes a default StorageClass that supports AWS and OpenStack out of the box. 

Red Hat offers Red Hat OpenShift Data Foundation (previously called OpenShift Container Storage) to implement persistent storage capabilities for applications.

Compute node components

• The Operator Lifecycle Manager, part of the Operator Framework, is responsible for managing the lifecycle of operators. Since OpenShift relies heavily on operators for cluster and user workload management, it is an important component. The OLM manages the installation, upgrade, and removal of operators, handles dependency management between operators, and supports multi-tenancy. The OperatorHub, where users browse and install operators, relies heavily on OLM, acting as a package manager for operators. 
• Machine management is responsible for managing the cluster nodes themselves. This allows the provisioning of new nodes, managing configuration of these nodes, scaling node pools, and, importantly, executing healing actions when a node is unhealthy. The Machine API Operator (MAO) is used for this purpose. The operator deploys various controllers that facilitate the actions above: the Machine, MachineSet, and MachineHealthCheck controllers. The ClusterAutoscaler and MachineAutoscaler custom resources then add or remove nodes based on demand. 

Observability

The Red Hat OpenShift Container Platform consists of a number of observability components included by default. These provide core monitoring and logging capabilities for OpenShift cluster components, with an option to enable monitoring for user-defined projects. Alerts are also included to notify cluster administrators of potential issues. Additionally, default dashboards provide a visual status of the cluster’s state.   

Backing up OpenShift clusters

Trilio for Red Hat OpenShift can back up and recover OpenShift environments, including OpenShift virtualization environments. It can also be used for migration scenarios to move between infrastructures. It is a scalable and agentless solution with self-service capabilities, native integration with role-based access control (RBAC), and support for immutable backups protecting from ransomware.  

Trilio integrates natively with OpenShift, is packaged as an OpenShift operator, and is installed from the OperatorHub. It uses custom resource definitions to create and manage backup operations such as backup plans, restores, and targets.