How install Ceph on a kubernetes cluster - kubernetes

Question

we want to use Ceph but we want to use Docker and Kubernetes to
deploy new instances of Ceph quickly.
I tried to use the default ceph docker hub: ceph/daemon-base. But I didn't work.
I tried to use the ceph-container. Seems like it doesn't work.
This is my last deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ceph3-deployment
spec:
replicas: 1
selector:
matchLabels:
app: ceph3
template:
metadata:
labels:
app: ceph3
spec:
containers:
- name: ceph
image: ceph/daemon-base:v3.0.5-stable-3.0-luminous-centos-7
resources:
limits:
memory: 512Mi
cpu: "500m"
requests:
memory: 256Mi
cpu: "250m"
volumeMounts:
- mountPath: /etc/ceph
name: etc-ceph
- mountPath: /var/lib/ceph
name: lib-ceph
volumes:
- name: etc-ceph
hostPath:
path: /etc/ceph
- name: lib-ceph
hostPath:
path: /var/lib/ceph
Does someone already install a ceph instance on Kubernetes?
I tried to follow the tutorial here
But pods not working:
pod/ceph-mds-7b49574f48-vhvtl 0/1 Pending 0 81s
pod/ceph-mon-75c49c4fd5-2cq2r 0/1 CrashLoopBackOff 3 81s
pod/ceph-mon-75c49c4fd5-6nprj 0/1 Pending 0 81s
pod/ceph-mon-75c49c4fd5-7vrp8 0/1 Pending 0 81s
pod/ceph-mon-check-5df985478b-d87rs 1/1 Running 0 81s

Answer 1

The common practice for deploying stateful systems on Kubernetes is to use an Operator to manage and codify the lifecycle management of the application. Rook is an operator that provides Ceph lifecycle management on Kubernetes clusters.
Documentation for using Rook to deploy Ceph clusters can be found at https://rook.io/docs/rook/v1.1/ceph-storage.html
For a basic introduction, you can use the Rook Storage Quickstart guide
The core Ceph team is highly involved in working on Rook and with the Rook community, and Rook is widely deployed within the Kubernetes community for distributed storage applications, and the Ceph Days event now has added Rook] explicitly to become Ceph + Rook Days

Answer 2

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:ceph:default" cannot list resource "pods" in API group "" in the namespace "ceph".
This error means Your default user doesn't have an access to resources in default namespace, look into documentation how to deal with rbac and how to grant user accesses. There is the same error on github.
Install Ceph
To install ceph I would recommend You to use helm
Install helm
There are few ways to install helm, You can find them there.
Personally i am using helm github releases to download latest version. When answering this question latest release was 2.15.0. That's how You can install it.
1.Download helm (If you want to install current version of helm, check above helm github releases link, there might be a new version of it. Everything you need to do then is change v2.15.0 in wget and tar command to new, for example v2.16.0)
wget https://get.helm.sh/helm-v2.15.0-linux-amd64.tar.gz
2.Unpack it
tar -zxvf helm-v2.15.0-linux-amd64.tar.gz
3.Find the helm binary in the unpacked directory, and move it to its desired destination
mv linux-amd64/helm /usr/local/bin/helm
4.Install tiller
The easiest way to install tiller into the cluster is simply to run helm init. This will validate that helm’s local environment is set up correctly (and set it up if necessary). Then it will connect to whatever cluster kubectl connects to by default (kubectl config view). Once it connects, it will install tiller into the kube-system namespace.
And then follow official documentation to install Ceph.