How to Backup and Restore InFluxDB Databases on Kubernetes
How-To: Backup & Restore InfluxDB Databases with TrilioVault for Kubernetes
In this blog post, you will find out how to backup and restore InfluxDB time-series databases to S3 compatible or NFS targets.
InFluxDB
InfluxDB is an open source time-series database developed by InfluxData. It is optimized for fast, high-availability storage and retrieval of time-series data for use with operations monitoring, application metrics, Internet of Things sensor data, real-time analytics and more. InfluxDB is designed to handle high write and query loads.
Why Backup?
Data loss is a serious problem. Losing files means losing time and money while you restore or recover information essential to the business. In the case of a server failure or outage, valuable data could be lost from the database forever. Losing files and documents often has a lasting impact on the company’s financial health. Consequently, backup and recovery are essential to the overall viability of the business.
The dynamic architecture of Kubernetes puts a lot of demands on a backup and recovery solution, so finding the right one can be difficult. In Kubernetes, applications are microservices-based and dynamic in nature. Applications can be deployed using labels, Helm charts or Operators. There are also multiple namespaces and resources that are either tied to a specific namespace or common across all namespaces. As such, metadata is vital for application-centric backup.
Considering all of the above, you need to identify software that can provide an application-centric, cloud-native data protection solution. TrilioVault for Kubernetes (TVK) offers a platform that satisfies these requirements.
The image below provides a quick overview of how backup works in TVK:
Now, take a look at how TVK performs application consistent backup and restore of InfluxDB.
Pre-requisite: Install TrilioVault for Kubernetes (TVK) and configure using the following steps:
- Install Test CSI Driver — Leverage the test hostpath driver if your environment does not support the driver with Snapshot capabilities today.
- Software Access and Installation — Access software and install it based on the specific directions for your environment.
- License — Leverage the Free Trial or Free Basic Edition by following instructions from the licensing page. If you already have an enterprise license, you can skip this step.
Create a target where backups will be stored
The Backup Target CRD specifies the backup storage media. TrilioVault supports either AWS S3 compatible object storage or NFS. All the backups created will be saved on the backup target specified in the Application CR spec. An Amazon S3 target example is provided below:
apiVersion: triliovault.trilio.io/v1
kind: Target
metadata:
name: demo-s3-target
spec:
type: ObjectStore
vendor: AWS
objectStoreCredentials:
url: "https://s3.amazonaws.com"
accessKey: "AaBbCcDdEeFf"
secretKey: "BogusKeyEntry"
bucketName: "S3_Bucket_US_East"
region: "us-east-1"
thresholdCapacity: 1000Gi
$ kubectl create -f tv-backup-target.yaml
InfluxDB databases can be deployed on Kubernetes using Helm charts. In the example below, a new release called my-release is created from helm chart bitnami/influxdb using the helm install command.
$ helm repo add bitnami https://charts.bitnami.com/bitnami
$ helm repo update
$ helm install my release bitnami/influxdb
Add Test Data
Let’s add some test data to the InfluxDB database. Please follow the commands below to create a test database, table and records.
$ Root$ ocexec -it influxdb-deployment-54c5c78664-f7z7n -- influx
Connected to https://localhost:8086 version 1.7.4
InfluxDB shell version 1.7.4
Enter an InfluxQL query
>
>auth
username: xxxx
password: xxxx
>
>CREATE DATABASE pirates
>use pirates
Using database pirates
>INSERT treasures,captain_id=dread_pirate_roberts value=801
>INSERT treasures,captain_id=flint value=29
>INSERT treasures,captain_id=sparrow value=38
>INSERT treasures,captain_id=tetra value=47
>INSERT treasures,captain_id=crunch value=109
>
>SELECT * FROM treasures
name:treasures
time captain_id value
---- ---------- -----
1613634634180857624 dread_pirate-roberts 801
1613634654035942732 flint 29
1613634666125641798 sparrow 38
1613634676936723375 tetra 47
1613634694340402128 crunch 109
>
>exit
Create Hooks
Hooks enable the ability to inject commands into pods or containers before and after a backup via pre/post commands. As a result, customers can take application consistent backups of their application.
- InfluxDB has its own built-in backup and restore capabilities. Their scope can range from individual databases to shards, policies, etc. A backup creates a copy of the metastore and shard data at a point-in-time and stores the copy in the specified directory.
- A full backup creates a copy of the metastore and shard data.
- An incremental backup creates a copy of whatever metastore and shard data has changed since the last incremental backup.
- If there are no existing incremental backups, the system automatically performs a complete backup.
- Use the InfluxDB backup command inside the TVK backup hook (see below).
apiVersion: triliovault.trilio.io/v1
kind: Hook
metadata:
name: influx-hook
spec:
pre:
execAction:
command:
- "bash"
- "-c"
- "bkpfile=/tmp/snapdate +%Y%m%dT%H%M; influxdbackup $bkpfile"
ignoreFailure: false
maxRetryCount: 1
timeoutSeconds: 10
post:
execAction:
command:
- "bash"
- "-c"
- "echo 'post hook action completed'"
ignoreFailure: false
maxRetryCount: 1
timeoutSeconds: 10
$kubectl create -f tv-influx-hook.yaml
Create BackupPlan
The BackupPlan specifies the backup job. The specification includes the backup schedule, backup target and the resources to backup.
apiVersion: triliovault.trilio.io/v1
kind: BackupPlan
metadata:
name: influx-hook-label-backup-plan
spec:
backupNamespace: default
backupConfig:
target:
name: demo-s3-target
backupPlanComponents:
custom:
- matchLabels:
app:influxdb
hookConfig:
mode: Sequential
hooks:
- hook:
name: influx-hook
podSelector:
labels:
- matchLabels:
app: influxdb
regex: influxdb*
$kubectl create -f tv-influx-backupplan.yaml
Create Backups
The backup takes either a full or incremental backup of the resources specified in the BackupPlan spec. The first backup of the application will always be a full backup, even if the user specifies their backup type as incremental.
apiVersion: triliovault.trilio.io/v1
kind: Backup
metadata:
name: influx-hook-label-full-backup
spec:
type: Full
scheduleType: Periodic
backupPlan:
name: influx-hook-label-backup-plan
$kubectl create -f tvk-mysql-backup.yaml
The Restore
The restore specifies which backup resources should be restored. Resources can be restored to the same namespace or to a different one. Let’s take a look at restoring to a different namespace, e.g. “restore-ns.” This needs to be created if it does not already exist.
apiVersion: triliovault.trilio.io/v1
kind: Restore
metadata:
name: influx-label-restore
spec:
backupPlan:
name: influx-hook-label-backup-plan
source:
type: Backup
backup:
name: influx-hook-label-full-backup
target:
name: demo-s3-target
restoreNamespace: restore-ns
skipIfAlreadyExists: true
$kubectl create -f tv-influx-restore.yaml
Conclusion
When you use this procedure, backing up a time-series database like InfluxDB with TrilioVault for Kubernetes is as easy as it gets. The backups are application-consistent, and the “hooks” allow the user to perform any pre/post backup actions. Furthermore, TrilioVault for Kubernetes provides a wide range of helpful features, including:
- Native Kubernetes application
- Stores metadata and all application resources to a specified target
- Supports application deployment types of Helm/Label/Operators and S3 or NFS-based backup targets
- Provides application hooks to ensure data consistent backups
Altogether, TVK provides a strong platform for enterprise database backups.
For more information on TrilioVault for Kubernetes request a demo today.
Sachin Kulkarni, Solutions Engineer for Trilio