I want to use the Prometheus image to deploy a container as part of the local deployment. Usually one has to run the container with volume and bind-mount to get the configuration file (prometheus.yml) into the container:
docker run \
-p 9090:9090 \
-v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml \
prom/prometheus
How can I achieve this with Kubernetes when using Skaffold?
Your Kubernetes configuration will be something like this,
You can specify the port number and volume mounts. The important sections for mounting are volumeMounts and volumes.
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-prom
spec:
selector:
matchLabels:
app: my-prom
template:
metadata:
labels:
app: my-prom
spec:
containers:
- name: my-prom
image: prometheus:latest
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 9090
volumeMounts:
- name: prom-config
mountPath: /path/to/prometheus.yml
volumes:
- name: prom-config
hostPath:
path: /etc/prometheus/prometheus.yml
---
apiVersion: v1
kind: Service
metadata:
name: my-prom
spec:
selector:
app: my-prom
ports:
- port: 9090
targetPort: 9090
save the Kubernetes config file in a folder and add below config to skaffold.yaml with the path to K8s config file,
deploy:
kubectl:
manifests:
- k8s/*.yaml
Related
I am trying to run a k8s job with 2 pods in which one pod will try to connect to other pod.
I cannot connect to other pod using hostname of the pod as suggested in the doc - https://kubernetes.io/docs/concepts/workloads/controllers/job/#completion-mode.
I have created a service and trying to access the pod as k8s-train-0.default.svc.cluster.local as mentioned in the document.
apiVersion: batch/v1
kind: Job
metadata:
name: k8s-train
spec:
parallelism: 2
completions: 2
completionMode: Indexed
manualSelector: true
selector:
matchLabels:
app.kubernetes.io/name: proxy
template:
metadata:
labels:
app.kubernetes.io/name: proxy
spec:
containers:
- name: k8s-train
image: pytorch/pytorch:1.11.0-cuda11.3-cudnn8-runtime
command: ["/bin/sh","-c"]
args:
- echo starting;
export MASTER_PORT=54321;
export MASTER_ADDR=k8s-train-0.trainsvc.default.svc.cluster.local;
export WORLD_SIZE=8;
pip install -r /data/requirements.txt;
export NCCL_DEBUG=INFO;
python /data/bert.py --strategy=ddp --num_nodes=2 --gpus=4 --max_epochs=3;
echo done;
env:
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
ports:
- containerPort: 54321
name: master-port
resources:
requests:
nvidia.com/gpu: 4
limits:
nvidia.com/gpu: 4
volumeMounts:
- mountPath: /data
name: data
volumes:
- name: data
persistentVolumeClaim:
claimName: efs-claim
restartPolicy: Never
backoffLimit: 0
---
apiVersion: v1
kind: Service
metadata:
name: trainsvc
spec:
selector:
app.kubernetes.io/name: proxy
ports:
- name: master-svc-port
protocol: TCP
port: 54321
targetPort: master-port
clusterIP: None
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
I am looking to establish communication between pod either using the hostname or to assign svc only to one pod slected with job-index.
Please let me know if i'm missing something here.
Thanks.
I am trying to create a StatefulSet. I want to create a file on the attached volume so i am using this command touch /data/test.txt but it seems like the container crashes because of that. Why would it do that? If i don't use the command everything works fine. What are the properties of the /data directory mounted to volume? Like read/write permissions.
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: web
spec:
selector:
matchLabels:
app: nginx # has to match .spec.template.metadata.labels
serviceName: "nginx"
replicas: 3 # by default is 1
template:
metadata:
labels:
app: nginx # has to match .spec.selector.matchLabels
spec:
terminationGracePeriodSeconds: 10
containers:
- name: nginx
image: k8s.gcr.io/nginx-slim:0.8
ports:
- containerPort: 80
name: web
volumeMounts:
- name: www
mountPath: /data
args:
- /bin/sh
- -c
- touch /data/test.txt
volumeClaimTemplates:
- metadata:
name: www
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
Because the default ENTRYPOINT of k8s.gcr.io/nginx-slim:0.8 would be nginx start or something likely.
So, if you want to inject the image, you need to set command
command: ["/bin/sh","-c"]
args:
- |
touch /data/test.txt
And you can kubectl describe or kubectl logs to see what's wrong with your pod/deployment.
Hi I am trying to mount a directory from pod where couchdb is running . directory is /opt/couchdb/data and for mounting in kubernetes I am using this config for deployment .
apiVersion: v1
kind: Service
metadata:
name: couchdb0-peer0org1
spec:
ports:
- port: 5984
targetPort: 5984
type: NodePort
selector:
app: couchdb0-peer0org1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: couchdb0-peer0org1
spec:
selector:
matchLabels:
app: couchdb0-peer0org1
strategy:
type: Recreate
template:
metadata:
labels:
app: couchdb0-peer0org1
spec:
containers:
- image: hyperledger/fabric-couchdb
imagePullPolicy: IfNotPresent
name: couchdb0
env:
- name: COUCHDB_USER
value: admin
- name: COUCHDB_PASSWORD
value: admin
ports:
- containerPort: 5984
name: couchdb0
volumeMounts:
- name: datacouchdbpeer0org1
mountPath: /opt/couchdb/data
subPath: couchdb0
volumes:
- name: datacouchdbpeer0org1
persistentVolumeClaim:
claimName: worker1-incoming-volumeclaim
so by applying this deployments . I always gets result for the pods .
couchdb0-peer0org1-b89b984cf-7gjfq 0/1 CrashLoopBackOff 1 9s
couchdb0-peer0org2-86f558f6bb-jzrwf 0/1 CrashLoopBackOff 1 9s
But now the strange thing if I changed mounted directory from /opt/couchdb/data to /var/lib/couchdb then it works fine . But the issue is that I have to store the data for couchdb database in statefull manner .
Edit your /etc/exports with following content
"path/exported/directory *(rw,sync,no_subtree_check,no_root_squash)"
and then restart NFS server:
sudo /etc/init.d/nfs-kernel-server restart*
no_root_squash is used, remote root users are able to change any file on the shared file. This a quick solution but have some security concerns
kubectl logs nfs-685944f556-r2pjr
Serving /exports
Serving /
rpcinfo: can't contact rpcbind: : RPC: Unable to receive; errno = Connection refused
Starting rpcbind
exportfs: / does not support NFS export
NFS started
nfs.deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs
labels:
app: nfs
spec:
replicas: 1
selector:
matchLabels:
app: nfs
template:
metadata:
labels:
app: nfs
spec:
containers:
- name: nfs-server
image: gcr.io/google_containers/volume-nfs:0.8
ports:
- name: nfs
containerPort: 2049
- name: mountd
containerPort: 20048
- name: rpcbind
containerPort: 111
securityContext:
privileged: true
volumeMounts:
- mountPath: /exports
name: mypvc
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: nfs-data
What does exportfs refer to? How can I diagnose this further?
Within the nfs pod, not too sure why it's exporting /?
[root#nfs-685944f556-r2pjr /]# cat /etc/exports
/exports *(rw,fsid=0,insecure,no_root_squash)
/ *(rw,fsid=0,insecure,no_root_squash)
not too sure why it's exporting /
It is done by run_nfs.sh script, running with two arguments:
/bin/bash /usr/local/bin/run_nfs.sh /exports /
There's an issue with the image gcr.io/google_containers/volume-nfs, so it is suggested to use jsafrane/nfs-data image instead
See the corresponding github discussion
I need to provide access to the file /var/docker.sock on the Kubernetes host (actually, a GKE instance) to a container running on that host.
To do this I'd like to mount the directory into the container, by configuring the mount in the deployment.yaml for the container deployment.
How would I specify this in the deployment configuration?
Here is the current configuration, I have for the deployment:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: appd-sa-agent
spec:
replicas: 1
template:
metadata:
labels:
run: appd-sa-agent
spec:
containers:
- name: appd-sa-agent
image: docker.io/archbungle/appd-sa-agent:latest
ports:
- containerPort: 443
env:
- name: APPD_HOST
value: "https://graffiti201707132327203.saas.appdynamics.com"
How would I specify mounting the localhost file path to a directory mountpoint on the container?
Thanks!
T.
You need to define a hostPath volume.
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: appd-sa-agent
spec:
replicas: 1
template:
metadata:
labels:
run: appd-sa-agent
spec:
volumes:
- name: docker-socket
hostPath:
path: /var/run/docker.sock
containers:
- name: appd-sa-agent
image: docker.io/archbungle/appd-sa-agent:latest
volumeMounts:
- name: docker-socket
mountPath: /var/run/docker.sock
ports:
- containerPort: 443
env:
- name: APPD_HOST
value: "https://graffiti201707132327203.saas.appdynamics.com"
you need to use the hostPath option. Here is the sample yaml file.
https://kubernetes.io/docs/concepts/storage/volumes/#hostpath