Kubernetes storage in 2026: revisit Rook-Ceph, Longhorn and OpenEBS before you put another database on a persistent volume

Running stateless services on Kubernetes is routine now. Running state is where the calm ends. The moment a database or a file store asks for a PersistentVolumeClaim, somebody must answer: where do those bytes live, how many copies exist, and what happens when a node dies at 2 AM?

This post is about the storage layer that answers that question on clusters you run yourself: on bare metal, on rented servers or on plain VMs. It is not about backups, which we covered in Kubernetes backup in 2026, and it is not about the database-level copy of your data, which is the job of tools in PostgreSQL high availability with Patroni and CloudNativePG and PostgreSQL backups in 2026. If you need S3-style object storage rather than block volumes, see MinIO and its alternatives. And if your cluster is small, built with k3s, k0s or Talos, the storage choice is tied closely to that choice; see lightweight Kubernetes in 2026.

Here we compare the three open source systems most self-hosting teams shortlist: Rook with Ceph, Longhorn and OpenEBS. We also bring in simple local volumes (local-path-provisioner, TopoLVM and OpenEBS Local PV), because for many databases they are the honest answer. We end with a small, safe drill and a plain list of what we did not run.

A scenario to keep us honest

Imagine a company in Nashik that builds booking, cold-storage and export documentation software for grape and onion exporters across north Maharashtra. The company is invented for this post. It has about 35 engineers and a three-person platform team.

The setup today:

• A production Kubernetes cluster of seven rented bare-metal servers, built with kubeadm on Kubernetes 1.33. Each server has two local NVMe drives.
• Longhorn was installed in 2023 from the Helm chart and is now on the 1.11 line, using the default V1 data engine and three replicas for every volume.
• PostgreSQL runs through CloudNativePG with three instances per cluster. Each instance has its own Longhorn volume, and each of those volumes has three Longhorn replicas. So every committed row exists nine times on disk.
• A few smaller stateful apps (a document store for export certificates, a search index, a shared upload folder) also use Longhorn volumes.
• Backups go to an S3-compatible bucket outside the cluster.

Three things push the team to revisit this. First, during the grape export peak in March, a node was rebooted for a kernel patch and Longhorn rebuilt replicas for a long time while database write latency climbed. Second, Longhorn 1.13.0 needs Kubernetes 1.34 or later, so the next storage upgrade is blocked on a cluster upgrade. Third, one engineer wants to move everything to Rook-Ceph “because that is what large companies use”, another has read that OpenEBS Mayastor is faster on NVMe, and the DBA asks a simple question: if CloudNativePG already keeps three copies, why does each copy need three more?

We will come back to Nashik in the worked plan.

The short version

• First decide where replication should happen: in the database, in the storage layer, or both. Doing it twice by accident is common and expensive.
• Rook (1.21.1) runs Ceph (Tentacle 20.2.4 is the latest stable) inside Kubernetes. It gives block, shared file and S3 object storage from one system. It is the most capable and the most demanding of the three. Rook is CNCF graduated.
• Longhorn (1.13.0) is the simpler replicated block storage from the Rancher and SUSE world. Each volume gets its own engine and replicas, with a friendly UI, snapshots and backups to S3 or NFS. Its newer SPDK-based V2 engine became generally available in 1.12.0. Longhorn is CNCF incubating.
• OpenEBS (4.6.2) is now an umbrella of a few engines: Mayastor for replicated NVMe-oF storage, and Local PV engines for LVM, ZFS, hostpath and rawfile. It was archived by the CNCF in February 2024 and re-admitted to the Sandbox in October 2024.
• For databases that replicate themselves, a local volume (OpenEBS LVM, TopoLVM, or local-path for small cases) is often faster, cheaper and easier to reason about. You trade automatic volume movement for simplicity.
• 2026 was a heavy security year for Ceph. The CephX authentication fix (CVE-2025-30156) needs both an upgrade and a key rotation, not only a new image.
• Whatever you choose, test a node failure and a restore before you trust it with a database.

Where things stand on 28 October 2026

Rook. The latest release is Rook v1.21.1, published on GitHub at 20:35 UTC on 26 October 2026, which is 2:05 AM IST on 27 October. Rook v1.21.0 came out on 7 October 2026 (12:08 AM IST). The older lines still receive patches: v1.20.9 and v1.19.12 were both published on 15 October 2026 IST. The v1.21 documentation lists Kubernetes v1.32 to v1.37 as supported, and Ceph Squid v19.2.0 or newer and Tentacle v20.2.1 or newer as supported Ceph versions. Rook is licensed under Apache 2.0.

Ceph. The Ceph releases page lists two active lines. Tentacle (20.2.x) started on 18 November 2025 and its latest release is 20.2.4 from 19 August 2026. Squid (19.2.x) started on 26 September 2024 and its latest release is 19.2.7 from 12 October 2026, with an estimated end of life of 31 October 2026, which is three days from today. Reef reached end of life on 20 March 2026. The next major, Umbrella (v21), has release candidate tags in the Ceph repository, and Rook 1.21 says it will support it once v21.2.0 is released. Ceph itself is mostly LGPL 2.1 or LGPL 3.0.

Longhorn. The latest stable release is Longhorn v1.13.0, published on 29 September 2026 at 2:43 PM IST. The 1.12 line is at v1.12.1 (14 August 2026), with v1.12.2-rc1 published on 15 October 2026 as a pre-release, and the 1.11 line is at v1.11.3 (1 July 2026). v1.12.0 (2 June 2026) made the V2 Data Engine generally available. Longhorn is licensed under Apache 2.0.

OpenEBS. The umbrella release is OpenEBS v4.6.2, published on 7 October 2026 at 5:11 PM IST. It bundles Replicated PV Mayastor v2.12.2, Local PV ZFS v2.11.2, Local PV LVM v1.10.1, Local PV Hostpath v4.6.0 and Local PV Rawfile v0.15.3. OpenEBS 4.6.0 came out on 26 August 2026. The project is licensed under Apache 2.0.

Local volume helpers. Rancher’s local-path-provisioner is at v0.0.37 (5 August 2026). TopoLVM is at v0.42.0 (8 October 2026), with Helm chart 17.3.0 on the same day; its README lists Kubernetes 1.34 to 1.36 as supported. The Kubernetes CSI external-snapshotter, which provides the VolumeSnapshot objects all of these rely on, is at v8.7.0 (13 October 2026).

CNCF status. The CNCF project pages say Rook was accepted on 29 January 2018, moved to Incubating on 25 September 2018 and to Graduated on 7 October 2020. Longhorn was accepted on 11 October 2019 and moved to Incubating on 4 November 2021. OpenEBS shows Sandbox, accepted on 10 October 2024. That date is the re-admission: OpenEBS first joined the Sandbox in 2019, was archived after a TOC vote in February 2024, and came back after consolidating its engines and fixing its governance documents.

Security in 2026

Ceph and Rook. On 19 August 2026, Ceph released Tentacle 20.2.4 and Squid 19.2.6 as urgent hotfixes for four CVEs:

• CVE-2025-30156, an authentication bypass in CephX. The project’s advisory explains that CephX used AES-128-CBC without authentication and with a fixed IV, which allowed an attacker with one low-privilege key and a network position to forge credentials for privileged daemons. The fix adds a new key type, aes256k.
• CVE-2026-39944, the same root cause applied to RGW STS session tokens.
• CVE-2026-50152, an authorisation flaw in the Monitor subscription handler.
• CVE-2026-54330, an RGW SigV4 verification flaw where a holder of a presigned PUT URL could add unsigned x-amz headers and gain more access than intended.

Two details matter for operators. First, upgrading the image is not enough for CVE-2025-30156; keys must be rotated to the new type. Rook’s advisory asks users to move to Rook v1.20.6 or v1.19.10 or later, then Ceph 20.2.4 or 19.2.6 or later, and then trigger rotation through the CephCluster spec. Rook does not rotate CSI keys, and using aes256k keys with kernel RBD or CephFS clients needs Linux kernel 7.0 or later (or a distribution backport). So many clusters will run a mixed state for a while, and you should know which keys are still old. Second, the stricter SigV4 checks in 19.2.6 and 20.2.4 also rejected some valid requests, such as presigned PUTs with an unsigned Content-Type. The Ceph advisory says these regressions affect boto3, aws-sdk-php and minio-go based tools such as Thanos, Loki and Mimir, and that they are fixed in Squid 19.2.7 and the next Tentacle point release. If you use RGW, test your clients before upgrading the gateways.

Rook itself also tightened defaults in 1.21: Ceph msgr2 is required by default (it needs kernel 5.11 or later on clients), v1.21.1 enforces secure msgr2 mode for monitor connections, and it disables insecure global ID reclaim instead of only muting the warning.

Longhorn. The repository’s own advisories page lists two high-severity issues from December 2021 (CVE-2021-36779 and CVE-2021-36780) about what privileged Longhorn pods exposed. In 2026, scanner feeds such as OSV list many CVEs against longhorn-manager builds; the ones we checked are in Go libraries (for example CVE-2026-84304 in grpc-go), fixed by rebuilds. Stay on a supported patch release.

OpenEBS. The repository lists two critical advisories from May 2025 about a GitHub token leaking through workflow artifacts in the Mayastor repositories: a build pipeline issue, not a data path bug, but a reason to pull images by digest. Separately, 4.6.2 fixed fresh installs failing with ImagePullBackOff because the bundled Loki pulled MinIO images from a registry that no longer allows anonymous pulls.

Lessons. All three run privileged pods with host access on every storage node. Patch on a schedule, restrict who can reach their APIs and UIs, and read upgrade notes for manual security steps.

The first question: where should the copies live?

Before comparing features, write down your answer to this: for each stateful workload, who keeps the extra copies?

• The application replicates. PostgreSQL with streaming replication, MySQL group replication, Kafka, Elasticsearch, Cassandra and similar systems keep their own copies on different nodes. Storage-level replication on top of this gives you copies of copies. It costs disk, network and write latency, and it gives little extra safety if the database replicas are already on different nodes.
• The application does not replicate. A single-instance app, a shared upload folder, a legacy service or a small tool’s SQLite file needs the storage layer to keep copies, or it needs a good backup and an accepted restore time.
• Many pods need the same files (RWX). This needs a shared file system: CephFS, Longhorn’s NFS-based RWX volumes, or an external NFS server.

Most clusters have all three kinds, so the answer is often two storage classes, not one product.

What each tool is

Rook and Ceph

Ceph is a distributed storage system that has been around for close to twenty years. It stores data as objects in RADOS, spread across many OSD daemons (usually one per disk), with monitors keeping the cluster map and quorum and a CRUSH algorithm deciding placement. On top of RADOS it offers RBD block devices, the CephFS shared file system and the RGW S3-compatible object gateway.

Rook is a Kubernetes operator that deploys and manages Ceph as pods. You describe the cluster in a CephCluster resource and pools, file systems and object stores in their own resources, and Rook reconciles them. Since v1.20, Rook no longer deploys the Ceph CSI drivers directly; they are managed through the ceph-csi-operator. Rook can also connect Kubernetes to an existing external Ceph cluster, which is a useful option when a storage team already runs Ceph.

Longhorn

Longhorn is distributed block storage built for Kubernetes. Each volume has an engine (the controller) and a set of replicas on different nodes. With the default V1 engine, replicas are sparse files on a host file system (ext4 or XFS) under /var/lib/longhorn, and the volume is exposed to the node through iSCSI. With the V2 engine, Longhorn uses SPDK on raw block disks and NVMe over TCP. Longhorn has its own web UI, recurring snapshot and backup jobs, backups to S3-compatible storage or NFS, disaster recovery volumes in a second cluster, and RWX volumes served by a per-volume NFS share manager.

OpenEBS

OpenEBS today is a family of engines under one Helm chart and one project:

• Replicated PV Mayastor, an SPDK-based engine written mostly in Rust. Each volume has a nexus that fans writes out to replicas on DiskPools across nodes, and it is exposed to the node over NVMe-oF TCP.
• Local PV LVM and Local PV ZFS, CSI drivers that carve volumes from an LVM volume group or a ZFS pool on each node, with snapshots, resizing and capacity-aware scheduling.
• Local PV Hostpath, a simple directory-based local provisioner, similar in spirit to local-path-provisioner.
• Local PV Rawfile, which stores each volume as a file on the node with its own file system.

The older engines (Jiva, cStor and others) were archived in the 2024 clean-up, so old guides that start with cStor describe a different era.

Simple local volumes

Rancher’s local-path-provisioner creates hostPath or local volumes in a directory on each node; its README says it does not enforce capacity limits. TopoLVM is a CSI driver that creates LVM logical volumes and reports each node’s free capacity to the scheduler. Neither copies data: if the node or disk is lost, recovery comes from the application’s replicas or backups.

Side by side

Licence and governance

• Rook: Apache 2.0, CNCF graduated since 7 October 2020. Ceph itself is LGPL 2.1 or 3.0 and governed by the Ceph Foundation, with large contributions from IBM, Red Hat, CLYSO and others.
• Longhorn: Apache 2.0, CNCF incubating since 4 November 2021. Started at Rancher Labs (GitHub repository created April 2017, 1.0.0 released on 30 May 2020); SUSE acquired Rancher in 2020 and sells commercial support.
• OpenEBS: Apache 2.0, CNCF Sandbox, re-admitted on 10 October 2024. The main sponsor is DataCore, which acquired MayaData, the company behind OpenEBS, in 2021.

The CNCF level is one signal; release pace and issue response are others. All three shipped several releases in 2026.

Architecture

• Rook-Ceph: a full storage cluster. You run monitors (usually three or five), managers, OSDs (one per disk), and optionally MDS for CephFS and RGW for object storage. Clients talk directly to many OSDs.
• Longhorn: one engine and N replicas per volume, controlled by longhorn-manager on every node. Simple to picture: each volume is its own small replicated system. v1.13.0 adds a longhorn-global-manager Deployment to reduce API server load in large clusters.
• OpenEBS Mayastor: io-engine pods on storage nodes own DiskPools; a control plane places replicas and nexus targets. Local PV engines have no data path of their own beyond the node’s LVM, ZFS or file system.

Data path and performance

• Rook-Ceph: block I/O goes from the kernel RBD client (or rbd-nbd) straight to OSDs, with writes acknowledged after all replicas in the placement group have them. Ceph performance depends heavily on network (separate, fast links help), on flash for OSDs, and on CPU and RAM per OSD. Rook 1.21 adds RBD quality of service through VolumeAttributesClass with cgroup v2 io.max.
• Longhorn V1: data goes through the engine process and iSCSI, then over the network to replicas. It is easy to run but adds hops. Data locality settings (best-effort or strict-local) can keep one replica on the pod’s node to cut network reads.
• Longhorn V2 and Mayastor: both use SPDK with polling and huge pages, and NVMe-oF TCP to the node. Longhorn’s documentation describes V2 as giving lower latency and higher IOPS and throughput than V1; we did not measure it. Both reserve dedicated CPU cores for polling. Longhorn 1.13.0 adds a full interrupt mode for idle or light workloads, and Rook 1.21.1 enabled SPDK interrupt mode for its NVMe-oF gateway to reduce idle CPU.
• Local PVs: the pod writes to a local disk through the node’s file system. No network hop, no storage replication. For a database that replicates itself, this is usually the fastest path you can get.

We ran no benchmarks for this post and quote no vendor numbers. Measure on your own hardware with your own workload, for example with fio and a replay of your database load.

Replication and failure handling

• Rook-Ceph: replicated pools (three copies is the common default in Rook’s examples) or erasure-coded pools for object and file data. CRUSH rules decide failure domains: host, rack, zone. When an OSD fails, Ceph re-balances data across the rest of the cluster, so recovery is spread over many disks. Rook 1.21.0 adds automated OSD replacement that preserves the CRUSH position. Monitors need a majority to keep quorum.
• Longhorn: per-volume replica count (often three), with replica rebuilding when a node or replica is lost. Rebuild speed depends on volume size and network, and rebuilds compete with application I/O, which is what hurt Nashik in March. v1.13.0 adds a volumeTopology parameter to keep replicas within a zone or region, and a scheduler extender that can place a restarted pod on a node that already holds its replicas (it needs a kube-scheduler configuration change, which managed services do not allow).
• OpenEBS Mayastor: per-volume replica count set in the StorageClass (repl), with rebuilds when a replica is lost. Optional NVMe multipath (nvme_core.multipath=Y) is needed for its HA path switching.
• Local PVs: no storage-level replication. A volume is tied to its node. If the node dies, the pod cannot start elsewhere with the same data; the application’s other replicas take over, or you restore.

Snapshots and backup

• All three support Kubernetes CSI VolumeSnapshots, which need the external-snapshotter CRDs and controller in the cluster. OpenEBS 4.6.0 lets you turn off its bundled snapshot controller if the cluster already runs one.
• Rook-Ceph: RBD and CephFS snapshots and clones, RBD mirroring to a second Ceph cluster, and RGW multisite for object data. Off-cluster backup is usually done with Velero, Kanister or the database’s own tools.
• Longhorn: snapshots, recurring backup jobs to S3 or NFS, DR volumes in another cluster, and system backups. v1.13.0 adds volume group snapshots and age-based retention, but the release notes are clear that group members are snapshotted independently, not at one point in time.
• OpenEBS: Mayastor snapshots and restores (4.6.0 added a bestEffort restore policy), LVM and ZFS snapshots on the node, and ZFS backup through Velero. 4.6.2 fixed a bug where a failed ZFS send could be reported as a successful backup, so the first backups after upgrading may reveal old failures.
• For databases, a storage snapshot is not a database backup unless you have tested that the database restores cleanly from it. Use the database tool (pgBackRest, WAL-G, Barman, or the operator’s backup) as the primary backup.

Operations and upgrades

• Rook-Ceph: two upgrade tracks, Rook and Ceph. 1.21 has breaking changes: Helm OCI tags lost the v prefix, msgr2 is required by default, and new NFS installs must define the .nfs pool. Day two work means watching placement group health, replacing OSDs and keeping capacity well below full.
• Longhorn: one product, one UI, one upgrade. Each minor release has a minimum Kubernetes version (1.34 for 1.13.0, because of the CSI provisioner it ships), and V2 live upgrade to 1.13.0 is supported only from 1.12.2, which is still at release candidate stage today. The documentation also warns against open-iscsi 2.1.12 on hosts because of attach failures.
• OpenEBS: one Helm chart for all engines plus a kubectl plugin for upgrades. You can install only the engines you need. Local PV engines are small and quiet to operate; Mayastor needs the same care as any replicated storage system.

Hardware and kernel needs

• Rook-Ceph: raw disks, partitions or logical volumes with no file system for OSDs; lvm2 on hosts in some setups; udev running; the rbd kernel module (missing on some images such as GKE Container-Optimised OS); kernel 4.17 or later recommended for CephFS quotas; 5.11 or later for msgr2 by default; 7.0 or later for aes256k CSI keys. Plan RAM and CPU per OSD and a fast network.
• Longhorn V1: open-iscsi with iscsid running, an NFSv4 client for RWX and NFS backups, cryptsetup and dmsetup for encryption, ext4 or XFS for the data path, and root and privileged pods.
• Longhorn V2: AMD64 with SSE4.2 or ARM64, kernel 6.7 or later recommended, vfio_pci, uio_pci_generic and nvme-tcp modules, 1024 huge pages of 2 MiB per node, raw block disks (local NVMe recommended), and dedicated CPU cores. The 1.13.0 notes also warn about a UBLK kernel panic on kernel 6.17 and an ARM64 NVMe issue.
• OpenEBS Mayastor: x86-64 with SSE4.2, kernel 5.13 or later (5.15 tested), nvme-tcp loaded, two CPU cores, 1 GiB RAM and 2 GiB of 2 MiB huge pages reserved for each io-engine pod.
• Local PVs: an LVM volume group or ZFS pool (or just a directory) on each node. That is all.

A worked plan for Nashik

Step 1. Make an inventory. List every PVC with its size, access mode, storage class, owning app, and whether the app replicates itself. In Nashik, most of the disk space is PostgreSQL, which already has three instances per cluster on different nodes.

Step 2. Measure the double replication. For each CloudNativePG cluster, count copies: three instances times three Longhorn replicas. Note the disk space, the network traffic during rebuilds, and the write latency at peak.

Step 3. Decide two storage classes. One class for workloads that replicate themselves (local PVs), and one for workloads that do not (replicated storage).

Step 4. Pick the local PV tool. The nodes already have NVMe, so the team puts a dedicated LVM volume group on the second NVMe drive of each database node and uses OpenEBS Local PV LVM or TopoLVM. Both are CSI drivers with capacity-aware scheduling, snapshots and resizing. local-path-provisioner is ruled out for production databases because it does not enforce capacity limits. The class uses volumeBindingMode WaitForFirstConsumer and reclaimPolicy Retain.

Step 5. Move one database first. Create a new CloudNativePG cluster on the local class as a replica of the old one (or restore it from backup), let it catch up, switch over in a quiet window, and keep the old cluster for a few days. Then repeat for the others. Each PostgreSQL row now lives three times, not nine.

Step 6. Keep replicated storage for the rest. The document store, the search index’s small single node and the shared upload folder stay on replicated volumes. Now comes the real choice.

• Path A (chosen): stay on Longhorn. Upgrade Longhorn to the latest 1.11 and then 1.12 patch, upgrade Kubernetes to 1.34, then move to 1.13. Keep the V1 engine for now; trial V2 on one node later only if latency is a measured problem, since it needs huge pages and dedicated cores. Set replica counts per class (two for easily rebuilt data, three for the rest) and set a rebuild concurrency limit so rebuilds do not swamp a peak.
• Path B: move to Rook-Ceph. This makes sense only if the team also needs S3 storage and CephFS from the same system, can dedicate at least three nodes with spare disks, CPU and RAM to Ceph, has a fast storage network, and has someone willing to learn Ceph properly. If chosen, start on Tentacle 20.2.4 with aes256k keys from the first day.

Step 7. Test failure on purpose. In a maintenance window, drain and power off one storage node. Watch how long replica rebuilds take, what happens to database latency, and whether CloudNativePG fails over cleanly on the local PVs. Write down the numbers.

Step 8. Test restore. Restore one database from the database backup into a new namespace, and restore one replicated volume from its backup. Time both. This is the number to give the business.

When each tool is better

• Rook-Ceph is better when you need block, file and object storage from one system, have enough nodes and disks, want mature failure domains and erasure coding, or already have Ceph skills.
• Longhorn is better when a small platform team wants replicated block storage it can understand quickly, with a UI and built-in backups, on a handful to a few dozen nodes.
• OpenEBS is better when you want LVM or ZFS local PVs for self-replicating databases, or SPDK-based replicated volumes on NVMe with cores and huge pages to spare.
• local-path-provisioner or TopoLVM is better for test clusters, k3s edge sites and self-replicating databases where simplicity matters more than moving volumes between nodes.

When not to change

• If your current storage is on a supported release, your node failures and restores are tested, and the team can explain what happens when a disk dies, do not migrate only because another tool is popular.
• On a cloud provider with a good block storage CSI driver, these three usually add cost and complexity without much gain.
• If the real problem is double replication or bad replica placement, fix the storage classes first. That may solve most of the pain without changing products.
• If nobody on the team will own Ceph, do not adopt Rook-Ceph. An unloved Ceph cluster is worse than a well-understood Longhorn.

A short history

• Ceph started as Sage Weil’s research work at the University of California, Santa Cruz in the mid 2000s. Inktank was built around it and acquired by Red Hat in 2014; the Ceph Foundation was formed in 2018, and in 2023 the Red Hat Ceph team moved to IBM.
• Rook began on GitHub in July 2016, joined the CNCF in January 2018 and graduated in October 2020. Over time it focused on Ceph alone.
• Longhorn was started at Rancher Labs (the GitHub repository dates from April 2017), reached 1.0.0 on 30 May 2020, and came under SUSE when SUSE acquired Rancher in 2020. The V2 engine on SPDK became generally available in 1.12.0 in June 2026.
• OpenEBS began in 2016 (its GitHub repository was created on 1 August 2016) and was driven by MayaData, later acquired by DataCore. It joined the CNCF Sandbox in 2019, went through many engines, was archived in February 2024 and re-admitted in October 2024 after cutting down to four engines; Rawfile is a fifth in the 4.6 line.

Operational costs

• Disk. Every replica is a full copy (erasure coding in Ceph reduces this for suitable data). Double replication with a self-replicating database multiplies the bill.
• Network. Replicated writes cross the network on every write, and rebuilds can saturate links. Ceph strongly benefits from a separate storage network.
• CPU and RAM. Ceph OSDs and monitors need real memory and CPU. Longhorn V2 and Mayastor reserve whole cores for polling and 2 GiB of huge pages per node.
• People and upgrades. Ceph needs the most knowledge; Longhorn and OpenEBS Local PV the least. Longhorn ties releases to minimum Kubernetes versions, Rook has two upgrade tracks, and OpenEBS has several engines in one chart. Each needs a written runbook.

Decision guide

• Does the workload replicate itself across nodes? Use local PVs (OpenEBS LVM or ZFS, or TopoLVM) and let the application handle copies.
• Does it need block storage that survives node loss, on a small or medium cluster with a small team? Start with Longhorn V1.
• Do you need block, file and S3 together, at scale, with people to run it? Rook-Ceph.
• Do you have fast NVMe, spare cores and a real latency problem? Trial Longhorn V2 or OpenEBS Mayastor on a few nodes and measure.
• On a public cloud, or a dev or edge cluster? Use the provider’s CSI driver, or local-path-provisioner for dev and edge.

Checklist

• Each stateful workload is labelled with who keeps its copies.
• No accidental double replication for databases.
• Local PV classes use WaitForFirstConsumer and Retain for database data.
• Replica counts and failure domains (host, zone) are set per class, not left at defaults by habit.
• Storage software is on a supported patch release; for Ceph, Squid users have a plan before its estimated end of life on 31 October 2026.
• Ceph clusters have rotated CephX keys to aes256k after the August 2026 fix, and the team knows which CSI keys still wait for kernel support.
• RGW clients were tested against the stricter SigV4 checks before upgrading gateways.
• VolumeSnapshot CRDs and one snapshot controller, not two.
• Database backups use the database’s own tool and are restore-tested.
• A node failure drill has been run and timed.
• Storage UIs and APIs are not reachable from the internet or from every namespace.

Common mistakes

• Running a replicated database on replicated storage without thinking. Nine copies and slow rebuilds are the usual result.
• Using Immediate binding for local volumes. The volume gets created on one node and the pod is scheduled on another, or never scheduled.
• Treating a storage snapshot as a database backup. Test that the database actually starts and is consistent from it.
• Ignoring minimum Kubernetes versions. Longhorn 1.13.0 will not install below Kubernetes 1.34.
• Turning on SPDK engines on shared, busy nodes. Dedicated polling cores and huge pages will be taken from your other workloads.
• Letting Ceph get near full. Recovery needs free space; plan capacity alerts well below the limits.
• Writing StorageClass parameters as numbers. Parameters are strings; numberOfReplicas: 3 without quotes is rejected by the API server.

The drill we ran

All three storage systems need root, privileged pods, kernel modules and real disks on every node, and our test machine is shared. So we did not install or run Rook, Ceph, Longhorn or OpenEBS. We did a smaller, safe drill as an ordinary user, in a scratch folder, with no sudo, no system packages and no services.

1. Read-only pre-flight. Kernel 6.12, HugePages_Total 0 (so no Longhorn V2 or Mayastor here without 1024 pages of 2 MiB), none of rbd, nvme_tcp or iscsi_tcp loaded, and we were not root.
2. A Python virtual environment with PyYAML 6.0.3.
3. Five example files. StorageClasses for Rook-Ceph RBD, Longhorn V1, OpenEBS Mayastor and OpenEBS Local PV LVM, and a PostgreSQL PVC on the local class.
4. A small checker. It parsed each file and checked three simple rules: every StorageClass parameter must be a string, local provisioners must use WaitForFirstConsumer (with a warning on Delete for a database class), and PVC sizes written in decimal units such as 50G get a warning in case 50Gi was meant.
5. Three negative tests. A Longhorn class with numberOfReplicas: 3 unquoted parsed fine as YAML, and only our checker flagged it; the Kubernetes API expects strings here and would reject it. A local LVM class with Immediate binding and Delete was flagged. A PVC asking for 50G was flagged. All five good files passed.

Not run: no cluster, no kubectl apply or server-side dry run, no CSI driver, no storage install, no fio or other benchmark, no node failure test and no snapshot or restore test. Provisioner names and parameters come from the projects’ documentation and examples and were not validated against live drivers. Parsing YAML proves syntax and our own simple rules only. For the real test, build a throwaway cluster on VMs with spare disks and run Steps 7 and 8 of the plan.

Unlearn, re-learn

• Unlearn “every PVC needs replicated storage”. Re-learn to ask who keeps the copies, and use local volumes where the application already does.
• Unlearn “Ceph is what serious teams use”. Re-learn that Ceph is right when you need what it offers and have people to run it.
• Unlearn “Longhorn is only for labs”. Re-learn that its limits are rebuild time, network and scale, not seriousness.
• Unlearn “OpenEBS means cStor and Jiva”. Re-learn that today it is Mayastor plus Local PV engines.
• Unlearn “a new image fixes the CVE”. Re-learn that the 2026 CephX fix needed key rotation and kernel support.
• Unlearn “we have snapshots, so we have backups”. Re-learn to restore on purpose and time it.

Sources

• Rook: Rook releases, Rook v1.21 prerequisites, Rook upgrade guide (v1.21), Ceph upgrade notes in Rook, Rook CephX keys and rotation, Rook advisory for CVE-2025-30156, CNCF Rook project page
• Ceph: Ceph releases index, Squid 19.2.6 and Tentacle 20.2.4 CVE release, CVE-2025-30156 advisory GHSA-7q3q-3975-qw3q, CVE-2025-30156 documentation, CVE-2026-54330 documentation, Ceph licence file
• Longhorn: Longhorn releases, Longhorn v1.13.0 release notes, Longhorn 1.13.0 installation requirements, Longhorn security advisories, OSV entry for CVE-2026-84304 in longhorn-manager builds, CNCF Longhorn project page
• OpenEBS: OpenEBS releases, OpenEBS v4.6.0 release notes, OpenEBS v4.6.2 release notes, Mayastor releases, Local PV LVM releases, Local PV ZFS releases, OpenEBS prerequisites, OpenEBS security advisories, CNCF TOC archival issue, Sandbox re-application, CNCF OpenEBS project page
• Local volumes and CSI: local-path-provisioner, TopoLVM, CSI external-snapshotter releases, Kubernetes storage classes, Kubernetes volume snapshots
• Related posts on Revisit.Tech: Kubernetes backup in 2026, PostgreSQL high availability with Patroni and CloudNativePG, PostgreSQL backups in 2026, MinIO and its alternatives, lightweight Kubernetes in 2026