This October, Ana McTaggart, Michael Hackett and myself presented a fast-paced security overview of all things Ceph and Rook at the newfangled CNCF Security Conference.

This was a mixed virtual and in-person event. We elected to go virtual, but we saw from pictures the in-person session was well attended, despite travel restrictions. Yay vaccines!

Due to the virtual format, this talk was pre-recorded and I presented at a brisk pace. This inevitably happens when it is late in the evening and I had too much caffeine. Have some coffee yourself, then listen to the recording:

Our slides are available as a PDF and can be viewed inline below — we are very interested in your feedback and comments or any unanswered questions you may have. Find me on Twitter and share your thoughts!

Comments? Discuss on Hacker News.

Introducing the Pacific codebase of Red Hat’s flagship storage product

with Marcel Hergaarden (Red Hat)

Red Hat Ceph Storage 5 is now generally available. This release includes support for NFSv4, additional disaster recovery capabilities for CephFS and RBD, as well as new security features and performance improvements.

New features

Red Hat Ceph Storage 5 introduces significant new functionality in several key areas:

• The new, integrated Cephadm control plane and newly-stable management API lay a foundation for customized automation. With these improvements we avoid requiring mastery of a DevOps configuration system to deploy and manage Ceph clusters.
  
• CephFS now supports the NFS 4 protocol, enabling AI/ML workloads requiring a filesystem interface alongside object storage. CephFS is also adding geo-replication capabilities for disaster-recovery (DR) multi-cluster configurations...

The Pacific Ocean spans a quarter of the Earth’s surface. While our ambitions for Red Hat Ceph Storage 5 are slightly smaller, it is only just so. Releasing RHCS 5 today on Mary Shelley’s day is a fitting tribute to the amazing creative effort of the more than 195 people who worked on the product directly — and the many, many others who did so indirectly as part of the largest Open Source Community any enterprise storage product enjoys.

UX at the Shell: Cephadm

RHCS 5 maintains compatibility with existing Ceph installations while enhancing Ceph’s overall manageability. Further extending the management user interface introduced last year to provide operators with better oversight of their storage clusters, RHCS 5 does not require mastering the internal details of the RADOS distributed system, further lowering the learning curve to SDS adoption. Operators can manage storage volumes...

with Sage Weil (Red Hat)

Sage and I renewed the now annual tradition of delivering a recorded roadmap session for the Red Hat Summit virtual audience. This year, we have been selected for the launch of Red Hat TV, which will happen later this summer, so our session’s release may be delayed, becoming available at a time separate from the virtual Red Hat Summit event itself.

I covered the upcoming Red Hat Ceph Storage 5, based on upstream’s Pacific release, and Sage illustrated the Community plans for the Quincy release coming next year. We tried to compensate for the lack of interaction by packing a lot of material for the attendees of the session, so brew a cup of tea and block your calendar, because you will need a full hour to go through all that we are throwing at you!

Our slides are available as a PDF and can be viewed inline below — we are not including our backup slides...

The Red Hat Ceph Storage life cycle: upgrade scenarios and long-lived deployments

with Sean Murphy (Red Hat)

Different industries have varying requirements for the software systems on which their respective businesses rely. Some operators choose to quickly embrace the latest and greatest release when facing change and integration updates. Others defer upgrades for as long as possible, trying to continue on a tried-and-true combination of software components until end-of-support (or security patching) forces a change.

The distinction is somewhat artificial, as most operators really adopt a combination of the two strategies for different parts of their infrastructure. The Red Hat Ceph Storage life cycle aims to address both faster and slower movers. In this post, we’ll share how we’re helping customers stay current while also providing longer life cycle options where needed.

Software

Highlights of Alpha 4 release include the new integrated installer

with Daniel Pivonka (Red Hat) and Paul Cuzner (Red Hat)

We’re delighted to announce availability of the new Alpha 4 release of Red Hat Ceph Storage 5, built on the upstream project’s Pacific release cycle. This post is the first of a series that will walk you through the enhancements coming with the next major upgrade of Ceph Storage—
well ahead of their production release—and give the details needed to facilitate testing with early-access releases.

Today’s post centers on the new Cephadm interface to the orchestration API, which is intended to become the preferred bare-metal installation and management method for Ceph across the broader vendor community. You can find download details for early access releases at the end of this blog. Now, without further ado, on to what is new.

A short history

In the recent...

Deploying a containerized Red Hat Ceph Storage 4 cluster for Red Hat Open Stack Platform 16

with John Fulton (Red Hat) and Gregory Charot (Red Hat)

Ceph is the most popular storage backend for OpenStack by a wide margin, as has been reported by the OpenStack Foundation’s survey every year since its inception. In the latest survey, conducted during the Summer of 2019, Ceph outclassed other options by an even greater margin than it did in the past, with a 75% adoption rate.

Red Hat’s default method for installing OpenStack is with a tool called director. This tool can configure OpenStack to connect with an existing Ceph cluster, or it can deploy a new Ceph cluster as part of the process to create a new OpenStack private cloud. Because director integrates ceph-ansible, the same deployment options described in our previous post dedicated to it remain available from director. This post...

Sage, Uday, and I put our best efforts into making sure that the new virtual venue for the Red Hat Summit would not diminish customer access and visibility into our future plans for Ceph. We delivered an unprecedented 18-month roadmap for the downstream, enterprise-class supported product, showcasing the “secret deck” that is usually reserved for internal planning use within our team — I usually make a roadmap statement to customers only when we have a 70%+ confidence estimate and delivery is scheduled to occur within a year, but this time I chose to relax the rules a little to give you a view stretching two major releases and 18-months into the future. We wanted to reward your virtual attention with more access by providing a little more food for thought into what is coming with Red Hat Ceph Storage 5 and 6 this Summer and the next.

Not to be outdone, Sage walked us through...

Putting noisy neighbors in their place with “RBD Top” and QoS

with Jason Dillaman (Red Hat)

Prior to Red Hat Storage 4, Ceph storage administrators have not had access to built-in RBD performance monitoring and metrics gathering tools. While a storage administrator could monitor high-level cluster or OSD I/O metrics, oftentimes this was too coarse-grained to determine the source of noisy neighbor workloads running on top of RBD images. The best available workaround, assuming the storage administrator had access to all client nodes, was to poll the metrics from the client nodes via some kind of homegrown makeshift external tooling.

Ceph Storage 4 now incorporates a generic metrics gathering framework within the OSDs and MGRs to provide built-in monitoring, and new RBD performance monitoring tools are built on top of this framework to translate individual RADOS object metrics into...