[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: CephFS external storage


CephFS works on OKD 3.11 -- and probably earlier.
Kubernetes docs are pretty much accurate dealing with CephFS on OpenShift.
You'ld need a specific provisioner, with admin privileges over your Ceph cluster (creating CephFS volumes would create a Ceph keyring with limited privileges, such as your Pods may not access shares they're not meant to use)

One small detail though: I couldn't manage to make it work using the cephfs kernel driver.
If you end up unable to read/write your shares, try installing ceph-fuse instead (https://github.com/openshift/origin/issues/21778)

Other alternatives, glusterfs obviously. Viable: I wouldn't go there.
Stuff like CIFS shares. Though can break some applications (eg: jenkins, low posix compliance)
Haven't had much time to extensively test cephfs. I seem to remember the MDS leak (13.x)
All in all, there's no perfect solution.

Depending on your application, if you're lucky, another take on it could be to look into s3 (radosgw) integration.


On Tue, Oct 8, 2019 at 9:32 AM Benjamin Guillon <benjamin guillon cc in2p3 fr> wrote:
Hey there,

I'm investigating possible alternatives to NFS for ReadWriteMany capable external storage.
We're running OKD 3.11 and we are already using Ceph RBD succesfully.

However, since we would like RWM capabilities we were wondering if it was possible to use CephFS (which supports RWM and is available in Kubernetes Vanilla since v1.5+).
Anyone knows if that's possible or will be possible in the near future? Ideally we'd like to be able to create a storage class for dynamic provisioning but manual provisioning would be a good start.
Any other viable alternatives suggestions to NFS?

Thanks a lot!
Benjamin Guillon
CNRS/IN2P3 Computing Center
21 Avenue Pierre de Coubertin, CS70202
69627 Villeurbanne Cedex, France
users mailing list
users lists openshift redhat com

Samuel Martín Moro
{EPITECH.} 2011

"Nobody wants to say how this works.
 Maybe nobody knows ..."

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]