K8s 1.16: Mounting an existing directory in an image to a pv - kubernetes

Question

tl;dr: How do we mount an existing directory in a pod to a PV allowing us to be persistent with our data that will be generated?
We are running K8s 1.16.7 at the moment, with Azure Disk and Azure File integration. We have an image that contains some directories we would like to have stored on a PV for persistency. In Docker, this could be easily handled since the container would write the data to a hostmount. Does anyone know how to solve this issues in Kubernetes? When we do this now, the container boots but the directory (for example: /etc/nginx/conf.d/ as a mount into PV) is empty and there for the pod crashes.
Example:
In the container below, the /usr/src/app is filled with the hello-world application. After deployment of the file below, the container crashes due it not being able to find anything in /usr/src/app (directory is empty due to PV mount).
---
apiVersion: v1
kind: Namespace
metadata:
name: testwebsite
labels:
environment: development
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: normal
namespace: testwebsite
provisioner: disk.csi.azure.com
parameters:
storageaccounttype: Standard_LRS
kind: Managed
resourceGroup: resourcegroup
cachingmode: None
mountOptions:
- dir_mode=0777
- file_mode=0777
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-azurefile
namespace: testwebsite
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: normal
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-kubernetes
namespace: testwebsite
spec:
replicas: 1
selector:
matchLabels:
app: hello-kubernetes
template:
metadata:
labels:
app: hello-kubernetes
spec:
containers:
- name: hello-kubernetes
image: paulbouwer/hello-kubernetes:1.8
ports:
- containerPort: 8080
volumeMounts:
- name: azurefile01
mountPath: "/usr/src/app"
volumes:
- name: azurefile01
persistentVolumeClaim:
claimName: pvc-azurefile
Goal: Have the data thats in /usr/src/app within the container written to the PV.
Thx in advance!

Answer 1

As far as I understand your requirement, each time your Pod is created, you want its /usr/src/app to contain both the data, generated so far by your app and stored permanently in PersistentVolume as well as the original content of the /usr/src/app being the integral part of your paulbouwer/hello-kubernetes:1.8 image, available under /usr/src/app directory.
You can achieve it in kubernetes by using the init container, which would copy the original content of /usr/src/app directory during Pod startup process to the PersistentVolume which may already contain some data, previously generated by your app. After such volume initialization, the main container will mount the PersistentVolume containing both the data previously generated by your app (if any) as well as the original content of /usr/src/app directory from your image.
Your Deployment may look as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-kubernetes
namespace: testwebsite
spec:
replicas: 1
selector:
matchLabels:
app: hello-kubernetes
template:
metadata:
labels:
app: hello-kubernetes
spec:
initContainers:
- name: init-hello-kubernetes
image: paulbouwer/hello-kubernetes:1.8
command: ['sh', '-c', 'cp -a /usr/src/app/* /mnt/pv-content/']
volumeMounts:
- name: azurefile01
mountPath: "/mnt/pv-content"
containers:
- name: hello-kubernetes
image: paulbouwer/hello-kubernetes:1.8
ports:
- containerPort: 8080
volumeMounts:
- name: azurefile01
mountPath: "/usr/src/app"
volumes:
- name: azurefile01
persistentVolumeClaim:
claimName: pvc-azurefile
In order to get the original data from /usr/src/app/ of the paulbouwer/hello-kubernetes:1.8 image, your init container must be also based on that image.
One caveat: paulbouwer/hello-kubernetes:1.8 image must contain cp binary to be able to perform the operation.
As you can see it's not a very "elegant" solution. Well, in fact it isn't. And that's why it is not recommended to mount your PersistentVolume under the directory which already contains some important files, required by your app to run properly. But there is no way to mount a volume under certain mount point and preserve its original content at the same time. It simply doesn't work this way in Linux or other nix-based systems. You either mount the whole volume or you don't mount it at all and preserve the original content of a specific directory. The original content isn't even overwritten. It's still there. It simply remains unavailable while this specific path is used as a mount point for a different volume.