Today’s world is driven by digital transformations, AI, and IoT, creating exponential growth in quantities of unstructured data. This has led to a need for object storage systems that address the critical needs of modern data management. First of all, the storage systems must have cloud-native compatibility, integrating with containers, microservices, and big data frameworks. And second, they need to be scalable enough to handle petabytes of data while offering very high levels of durability and reliability.
Many cloud service providers, such as Amazon Simple Storage Service, Azure Blob Storage, and IBM Cloud Object Storage, have fulfilled these requirements. Some cloud platform distributions also offer on-premises object storage solutions.
In this article, we discuss the features and architecture of OpenStack Object Storage, commonly called Swift. This open-source platform offers many of the same features as commercial offerings at a fraction of the cost. With its flexibility, highly distributed architecture, and substantial community support, the platform is well-suited as an enterprise-grade object storage solution.
Summary of key features of OpenStack Swift
Feature | Description |
Scalability | Swift is designed to scale by adding nodes and is capable of handling petabytes of storage and billions of objects. |
High availability | Components are replicated across multiple nodes to ensure high availability. |
RESTful API | Swift provides a standard HTTP interface for integration with applications and tools. |
Cost efficiency | Swift is an open-source and community-driven platform offering enterprise-grade features without vendor lock-in. |
User management | Swift integrates with Keystone for authentication and provides built-in access control for objects. |
Data management | Swift has multiple processes to ensure cluster health and data availability. |
What is OpenStack Swift?
When working with cloud platforms, there are different types of storage options available. The most common ones are block storage, file storage, and object storage. Each of these helps to meet the needs of specific use cases:
- Block storage functions like raw disks attached to your virtual machine. You can create your own file system and partitions on the storage based on your requirements.
- File storage offers a shared file system accessible over a standard network protocol like NFS or SMB. The file storage can be accessed by multiple machines and is typically used for shared content repositories and shared home directories.
- Object storage provides an HTTP/API interface and is used to store large amounts of unstructured data. This includes backup files, images, videos, log files, etc. Object storage is most commonly used to provide reliable storage for backups, host static content for websites, and host media.
OpenStack began as a joint project of NASA and RackSpace Hosting in 2010. The first release of OpenStack only included two projects: Swift object storage, based on RackSpace CloudFiles, and Nova compute provisioning engine based on NASA Nebula cloud technology.
OpenStack Swift is an open-source object storage system built for high availability and scalability. It is designed for the storage and retrieval of large amounts of unstructured data. The data is stored as files (called objects) in a flat namespace rather than a hierarchical file system. The objects are stored across multiple nodes to ensure reliability and fault tolerance.
The object store can be used to store different types of files, such as backups, songs, pictures, and videos. Swift provides a RESTful API interface, which can be used for interaction using standard HTTP requests (GET, PUT, POST, HEAD, and DELETE). Swift can be used for backups and archiving of data, as media storage, or for delivering static content in web pages.
Automated OpenStack & OpenShift Data Protection & Intelligent Recovery
Enjoy native OpenStack integration, documented RESTful API, and native OpenStack CLI
Restore Virtual Machines with Trilio’s One-Click Restore
Select components of your build to recover via the OpenStack dashboard or CLI
Key features of Swift
Scalability
Swift is designed with a scale-out architecture that supports both vertical scaling (increasing the capacity of existing nodes) and horizontal scaling (adding more nodes to the cluster). With this multi-dimensional scalability, Swift can support petabytes of storage with billions of objects.
High availability
Swift has a highly available architecture, providing a high level of durability. Its distributed design means that there is no downtime or data loss even if parts of the system or storage fail. Swift has built-in replication with consistency checks. It also has data redundancy, with typically three copies of objects, which ensures that the data can survive disk crashes and server outages.
RESTful API
Swift provides a RESTful API interface compatible with AWS S3 storage. The Swift API can be integrated into existing applications and automation tools and provides most of the S3 feature set. The APIs can be used to get objects from the storage or the associated metadata. The API also provides endpoints to create, modify, and delete objects.
Cost efficiency
Swift is an open-source platform designed to be deployed on commodity hardware. With no vendor lock-in, Swift offers cost-optimal storage with enterprise-grade features and performance.
User management
In Swift, a user is represented as a logical entity called an account. The account is tied to an OpenStack project. Swift doesn’t handle user authentication by itself; rather, it integrates with Keystone to handle the authentication. Swift offers access control of objects, and the account owner can configure ACLs to manage read and write access.
Data management
Swift has built-in utilities for monitoring cluster health and nodes and for the management of data. Swift has multiple background processes called replicators, auditors, and updaters (discussed below) that ensure data availability and consistency across multiple nodes in a Swift cluster.
Learn how Trilio’s partnership with Canonical helps better protect your data
Swift architecture
OpenStack Swift consists of multiple components that work together to offer a reliable, high-performance, and cost-effective object storage platform. These components include Proxy servers, the ring, storage nodes, and different services, including account, container, and object.
OpenStack Swift Components Architecture (Source)
Proxy server
The proxy server connects all components of the Swift architecture and is the entry point for all client requests. It handles incoming HTTP requests (via Swift’s RESTful API) from users or applications to store, retrieve, or delete objects. The proxy server also handles backend failures by redirecting object requests from failed nodes to handoff servers via the ring. Swift supports deploying multiple proxy servers as a cluster to avoid the proxy itself becoming a bottleneck or a single point of failure. The proxy servers are configured behind HTTP load balancers, such as Nginx or HAProxy. The load-balancers can perform a health check of the proxy servers and distribute client connections accordingly.
The ring
The ring is the data structure that maps entities to their physical location on disks. The ring divides the storage cluster into partitions (virtual buckets). Each partition is assigned to multiple storage nodes for replication (typically three replicas by default). Each entity in the Swift cluster (accounts, containers, and objects) has its own ring. When any operations need to be performed by components of Swift, these entities need to interact with the appropriate ring to determine the location of the subject entity in the cluster.
Zones
Swift has configurable failure boundaries called zones. The concept is to divide the replicas across multiple zones so that different types of failures do not impact data availability. For very large deployments, each rack of servers can be defined as a zone. For smaller infrastructure, a single server or even individual drives can be configured as separate zones.
Storage policies
Swift supports different service levels via storage policies. These policies can be defined based on replication count (two times, three times, etc.) and high and low performance tiers using different backends (HDD or SSD). These policies are assigned per container when they are created and are applicable for the lifetime of the container.
Learn about the features that power Trilio’s intelligent backup and restore
Accounts, containers, and objects
The OpenStack Swift architecture is based on a three-tier hierarchy:
- An account is the top-level entity that represents an OpenStack project or user namespace. It manages the access control and metadata, including the number of objects and total storage. It also contains the list of containers in the account.
- Containers package applications with their dependencies for isolation and consistent execution, unlike simple folders. Access control is managed by container platforms (like Kubernetes) through mechanisms beyond traditional file ACLs. Container metadata is stored in registries and platform control planes, not within the containers themselves.
- Objects are the actual files (videos, images, documents) that are replicated across multiple nodes for redundancy. The objects include metadata like the type of file, its size, and custom tags.
Accounts and Containers SQLite Databases (Source)
Object server
The objects in a Swift cluster are stored on object Servers. These servers store the objects as binary files with the metadata of the objects stored in the files’ extended attributes (xattr).
Container server
The container server has a SQLite database that has a listing of objects in containers, the total number of objects, and the total storage used by the containers. The SQLite database is replicated across the cluster to ensure high availability.
Account server
The account server, like the container server, also has a SQLite database, which has a listing of containers.
Replication
Swift uses replication to ensure data availability and cluster fault tolerance by maintaining multiple copies of objects, containers, and account data across different nodes. Object replication is done via rsync to copy files to all the peer nodes. Container and account replication push updates via HTTP to peer nodes or perform rsync of complete databases if required.
Updaters
Swift runs background processes called updaters that ensure the consistency of objects, containers, and accounts across the cluster. For instances of high load when updates cannot be pushed to all peers, the entities are queued for later update. The request continues to be served from nodes that have received the updates while the updater process pushes the failed updates.
Auditors
Swift auditors are background processes that ensure data integrity and consistency by detecting and fixing corrupt or missing data in the cluster. Auditors run in the background and scan stored objects for corruption or checksum failures. Whenever a corrupt object is found, it is quarantined and the replication process will copy the object from another peer node.
Comparison of Swift with Ceph Object Gateway
OpenStack supports multiple object storage options. Swift is the OpenStack native storage component. Ceph is another popular option to Swift. Ceph is a multi-purpose storage platform that supports block, file, and object storage. Ceph provides S3-compatible APIs using RADOS Gateway, which also supports Swift-compatible APIs and can seamlessly integrate with OpenStack. The following is a brief comparison of Swift with Ceph Object Gateway.
OpenStack Swift | Ceph (RADOS Gateway) | |
Purpose | Native OpenStack object storage for scalable, durable, and highly available data | Distributed storage providing object, block, and file storage for OpenStack |
Storage Types | Object storage only | Object, block, and file storage |
Scalability | Highly scalable, designed for massive datasets with horizontal scaling | Highly scalable, with automatic data distribution and no single point of failure |
Performance | Optimized for large objects and high durability; may be slower for small objects | Generally faster for mixed workloads; better performance for small objects. |
Use Cases | Archival, backups, and large-scale object storage | Mixed workloads, including VM storage, backups, and object storage. |
Consistency Model | Eventual consistency model, background replication processes ensure that all replicas eventually converge to the same state | Strong consistency, write operations are completed only after all replicas are written and synchronized across the cluster |
OpenStack Integration | OpenStack native component, integrates with Keystone for authentication | RADOS Gateway provides Swift compatible APIs for seamless integration with OpenStack |
OpenStack Swift CLI operations
OpenStack Swift has a Python-based CLI client called swift whose commands provide a command-line interface for interacting with the object store APIs. The client can be used to upload/download objects, display information, list containers and objects, and add metadata to objects. The client can be installed on Linux or MacOS as follows.
- Linux installation with pip:
$ pip3 install python-swiftclient
- Ubuntu/Debian installation:
$ sudo apt install python3-swiftclient
- MacOS installation:
brew install python-swiftclient
Here are some of the typical uses of the CLI client. Note that as a prerequisite, you would need to authenticate with Keystone using the OpenStack RC file:
$ source openstackprojectrc.sh
- Display information about the account, like number of containers and objects, bytes used, and quota allocated:
$ swift stat
Account: v1
Containers: 2
Objects: 1
Bytes: 1180728
Quota Bytes: 10485760
Containers in policy "default-placement": 2
Objects in policy "default-placement": 1
Bytes in policy "default-placement": 1180728
Objects in policy "default-placement-bytes": 0
Bytes in policy "default-placement-bytes": 0
Meta Quota-Count: 5
Server: Apache/2.4.41 (Ubuntu)
X-Timestamp: 1743582270.85084
X-Account-Bytes-Used-Actual: 1183744
- List containers in the account:
$ swift list
documents
pictures
- Upload files to the pictures container:
$ swift upload pictures *.jpg
Blue-2.jpg
Green-1.jpg
- List objects in the pictures container:
$ swift list pictures
Blue-2.jpg
Green-1.jpg
- Download an object from the pictures container:
$ swift download pictures Green-1.jpg
Green-1.jpg [auth 0.099s, headers 0.251s, total 0.609s, 3.340 MB/s]
- Configure ACL with read permissions for everyone:
$ swift post --read-acl .r:* pictures
- Display information about the pictures container:
$ swift stat pictures
Account: v1
Container: pictures
Objects: 2
Bytes: 4039791
Read ACL: .r:*
Write ACL:
Sync To:
Sync Key:
Server: Apache/2.4.41 (Ubuntu)
X-Timestamp: 1740892452.51354
X-Container-Bytes-Used-Actual: 4042752
X-Storage-Policy: default-placement
X-Storage-Class: STANDARD
Last-Modified: Wed, 02 Apr 2025 08:36:32 GMT
- Delete an object from the pictures container:
$ swift delete pictures Blue-2.jpg
Blue-2.jpg
Swift Operations via OpenStack Dashboard (Horizon)
The Swift CLI client provides full control over object storage using commands. The client is ideal for automation and scripting but requires knowledge of the commands and syntax. The Swift object store operations can also be performed via the OpenStack Horizon dashboard, which offers a visual representation to manage containers and objects.
An end user can log into the OpenStack dashboard and perform Swift object store operations by accessing the menu under Project > Object Store. Here are some of the Swift operations accessible through the Dashboard.
- List containers by accessing menu Project > Object Store > Containers:
- Create a new container by accessing the menu Project > Object Store > Containers and select the “+ Containers” button. Enter the container name, select the storage policy and container access mode. Select Submit to complete the operation:
- Upload files to the documents container by accessing Project > Object Store > Containers. Select the documents container and press the Upload button. In the popped-up window, select Browse and then Upload File:
- Enable the public read-only access ACL by accessing the menu Project > Object Store > Containers. Select the documents container and enable the Public Access checkbox. A link will be provided in front of the checkbox box that can be accessed from anywhere:
Find out how Vericast solved K8s backup and recovery with Trilio
Last thoughts
When you need object storage, you have several options to consider. One choice is to use a public cloud service, such as AWS S3; another is to set up an on-premises storage solution. On-premises options allow you to comply with local data residency rules while providing excellent latency and network performance.
For on-prem storage, you can deploy a commercial solution, which is typically charged based on the capacity and the feature-set licensing. However if you have the required skilled staff, you have the option to go with OpenStack Swift. This option lets you decide on your architecture, choose an appropriate redundancy configuration, and use any commercial off-the-shelf hardware (COTS). You get an enterprise-grade platform without being locked to a vendor and being continuously charged for support and licensing for the lifespan of the product.
Like This Article?
Subscribe to our LinkedIn Newsletter to receive more educational content