The Challenges of Upgrading OpenStack Clouds
For businesses that have successfully navigated the hurdle of executing their first OpenStack cloud implementation, the complexity of moving to newer OpenStack releases can come as a shock. Without the right level of operational expertise, upgrading OpenStack can easily become a daunting challenge.
Upgrading vs. Updating OpenStack
There is a distinction between upgrading and updating OpenStack. Updating involves the application of fixes for bugs and security vulnerabilities to OpenStack components and its underlying OS (operating system). An upgrade is concerned with migrating to a newer, more stable OpenStack release.
How to Upgrade an OpenStack Deployment
The task of upgrading an OpenStack deployment is challenging for a number of reasons. The OpenStack cloud consists of several distributed software components that work collaboratively to deliver the required services. All these disparate software components, including OS dependencies, must be concurrently upgraded.
Each OpenStack release comes with new features and may require a new hardware configuration. A company may have to allocate more memory and disk space, and add or upgrade CPUs.
Simplify your OpenStack update with native workload recovery.
Learn how you can upgrade OpenStack with TrilioVault
Upgrading Existing Dependencies
Although an OpenStack release will introduce new system dependencies, existing ones must be upgraded. Otherwise, OpenStack service will either terminate with runtime failure or fail to start.
Remember to upgrade existing dependencies alongside the OpenStack services. All OpenStack components should be installed from packages with tested and correctly defined dependencies.
Unclean Shutdown
Upgrading OpenStack entails shutting down each OpenStack service and leaving enough time for them to complete active requests. In addition, it’s advisable to notify the message queue of the services’ unavailability before commencing an upgrade.
Once a service completes all received requests, it should notify the message queue to quit sending new requests.
Upgrading Services in Order
You may break the cloud if you upgrade OpenStack services in the wrong order. Here is a reasonable way to perform upgrades on your controllers:
- MariaDB
- MongoDBIdentity (keystone)
- Image service (glance)
- Block Storage (cinder)
- Orchestration (heat)
- Telemetry (ceilometer)
- Compute (nova)
- Networking (neutron)
- Dashboard (horizon)
- Fix Apache Configuration
- Fix Dashboard Configuration
Removed or Deprecated Features, APIs, and Plugins
If there are other software or custom scripts using OpenStack API, the upgrade may fail. This is because each release introduces new API versions and deprecates or removes old APIs. The same goes for vendor-specific plugins and specific architectural features of your existing OpenStack deployment.
Thoroughly read through the release notes to identify any such changes and possible remedial action.
Steps for Success in an OpenStack Upgrade
Release of an OpenStack upgrade can be a challenging task. It requires the right approach, careful planning, and precise execution to minimize downtime of the cloud environment. Because of the complexity and headaches involved, cloud operators prefer to skip one or more releases before doing an upgrade.
There are several key steps to ensure the success of a planned OpenStack upgrade:
- Going through the OpenStack release notes to identify potential incompatibilities between your existing deployment and the upgraded release.
- Determine the maximum acceptable downtime and notify users about possible service interruption.
- Back up all data, especially databases and configuration files.
- Plan for rolling back a failed upgrade.
- Verify that the upgrade will work by testing in a test cloud with the same automation scripts used to deploy the production cloud.
The end of the “Big Tent” approach
To help resolve the challenges of upgrading to a new release, the OpenStack community previously took a “Big Tent” approach for new releases. With this model, cloud operators were able to select preferred components and incrementally upgrade modules with near zero downtime. However, the OpenStack community is ditching this approach in favor of modularization and stand-alone services, as evidenced in the OpenStack Keystone release.
How Trilio Can Help
TrilioVault’s agentless, software-only solution can help enterprises upgrade their OpenStack environment by backing up the entirety of the old one (including all that valuable metadata, such as configurations) and restore it to the new, updated one. Cloud admins can choose to re-use existing hardware or leverage new hardware (in-place vs. out-of-place upgrade).
Trilio is the only native OpenStack backup and recovery platform that gives administrators and tenants the ability to restore entire workloads in one click. Connect with a team member today to learn how Trilio can help resolve OpenStack upgrade challenges.