Is there any way to run super-privileged containers using Kubernetes? - kubernetes

I want the all processes within the pod see the same network and process table, as well as share any IPCs with the host processes.
I know it possible when we use docker by leveraging the following command.
docker run -it --privileged --ipc=host --net=host --pid=host \
-v /:/host -v /run:/run -v /etc/localtime:/etc/localtime \
--name privcontainer centos7 /bin/bash
On the other hand, is there any way to run super-privileged containers using Kubernetes?
If possible, I would like to know the way to write pod yaml file.

There is a privileged flag on the SecurityContext of the container spec.
Check out documentation for more details.
I could only find an example from the v1.4 docs:
apiVersion: v1
kind: Pod
metadata:
name: hello-world
spec:
containers:
- name: hello-world-container
# The container definition
# ...
securityContext:
privileged: true ###Here is what you are looking for
seLinuxOptions:
level: "s0:c123,c456"
Even more infos here
I'm sure you're aware, but as a general word of caution, the privileged will remove all container security settings and open up the cluster to potential security vulnerabilities.

To disable the namespacing of a container PIDs, and thus allowing this container to view all processes on a host, you need to specify hostPID: true in the pod specs.
You might find this manifest useful if you want to inspect a Kubernetes host from within a pod:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: debug
spec:
selector:
matchLabels:
app: debug
template:
metadata:
labels:
app: debug
name: debug
spec:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
hostNetwork: true
hostPID: true
containers:
- name: linux
image: alpine
args:
- sleep
- "3600"
securityContext:
privileged: true
runAsGroup: 0
runAsUser: 0
volumeMounts:
- mountPath: /mnt/host
name: host
volumes:
- hostPath:
path: /
type: ""
name: host
This will instantiate a "debug" pod on each node on your cluster (including control-plane node if they are visible to you). This pod will have access to all PIDs from the host, will see all its networks, and the node filesystem will be browseable at /mnt/host.

Related

K8S cronjob scheduling on existing pod?

I have my application running in K8S pods. my application writes logs to a particular path for which I already have volume mounted on the pod. my requirement is to schedule cronjob which will trigger weekly once and read the logs from that pod volume and generate a report base on my script (which is basically filtering the logs based on some keywords). and send the report via mail.
unfortunately I am not sure how I will proceed on this as I couldn't get any doc or blog which talks about integrating conrjob to existing pod or volume.
apiVersion: v1
kind: Pod
metadata:
name: webserver
spec:
volumes:
- name: shared-logs
emptyDir: {}
containers:
- name: nginx
image: nginx
volumeMounts:
- name: shared-logs
mountPath: /var/log/nginx
- name: sidecar-container
image: busybox
command: ["sh","-c","while true; do cat /var/log/nginx/access.log /var/log/nginx/error.log; sleep 30; done"]
volumeMounts:
- name: shared-logs
mountPath: /var/log/nginx
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: "discovery-cronjob"
labels:
app.kubernetes.io/name: discovery
spec:
schedule: "*/5 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: log-report
image: busybox
command: ['/bin/sh']
args: ['-c', 'cat /var/log/nginx/access.log > nginx.log']
volumeMounts:
- mountPath: /log
name: shared-logs
restartPolicy: Never
I see two things here that you need to know:
Unfortunately, it is not possible to schedule a cronjob on an existing pod. Pods are ephemeral and job needs to finish. It would be impossible to tell if the job completed or not. This is by design.
Also in order to be able to see the files from one pod to another you must use a PVC. The logs created by your app have to be persisted if your job wants to access it. Here you can find some examples of how to Create ReadWriteMany PersistentVolumeClaims on your Kubernetes Cluster:
Kubernetes allows us to provision our PersistentVolumes dynamically
using PersistentVolumeClaims. Pods treat these claims as volumes. The
access mode of the PVC determines how many nodes can establish a
connection to it. We can refer to the resource provider’s docs for
their supported access modes.

How get consistent names for Pods in Kubernetes

I want to run this docker image in kubernetes:
https://hub.docker.com/_/rabbitmq
This is not a problem, and it is running. The problem is that I need to send through switches do the "docker run" command. For this image, when starting the container in docker you would run this:
docker run -d --hostname my-rabbit --name some-rabbit rabbitmq:3
The yaml file will then look something like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: rb
namespace: rabbittest
spec:
selector:
matchLabels:
app: rb
replicas: 1
template:
metadata:
labels:
app: rb
spec:
containers:
- name: rb-container
env:
- name: HOSTNAME
value: "rbnode001"
image: rabbitmq:3-management
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
volumeMounts:
- name: azure
mountPath: /mnt/azure
ports:
- containerPort: 5672
volumes:
- name: azure
azureFile:
secretName: azure-secret
shareName: rabbittest
readOnly: false
My question is, how do I get kubernetes to apply the --name and --hostname when kubernetes executes the "docker run" commands?
First of all, you need to create StatefulSets to RabbitMQ.
In your StatefulSet, add this ENV.
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: K8S_SERVICE_NAME
value: rabbitmq-headless
- name: RABBITMQ_NODENAME
value: $(POD_NAME).$(K8S_SERVICE_NAME).$(POD_NAMESPACE).svc.cluster.local
Here, K8S_SERVICE_NAME is headless service required by statefulset.
Finally, RABBITMQ_NODENAME will hold the HOSTNAME.
Kubernetes's options are just different from Docker's options. Many options have equivalents in the pod spec object; some options don't have direct equivalents, or are split into multiple layers (volume mounts, published ports).
For the two options you mention:
docker run --name sets the name of the container. In Kubernetes you don't usually directly interact with containers, so you can't set their names. The metadata: {name: } sets the name of the Deployment, and generated Pods have names derived from that; the containers within a Pod also have names but it's very rare to use these.
docker run --hostname sets the name a container believes its hostname is. You almost never need this option; RabbitMQ is the notable exception. The pod spec has a hostname: option that can set this.
spec:
template:
spec:
hostname: my-rabbit
containers: [...]
As #Shahriar's answer suggests, a StatefulSet is a better match for deployments that need some backing persistent state. In this case the StatefulSet automatically sets the host name based on the pod identity, so you don't need to do anything.
(The important detail for RabbitMQ is that the hostname must be consistent across recreating the container, and if the hostname is always rb-0 from a StatefulSet that satisfies this rule. An autogenerated Docker container ID, or a variable Kubernetes Deployment pod name, will be different on every restart, and Rabbit will lose track of its state.)

Kubernetes pod definition file issue

I can run the Docker container for wso2 ei with following command.
docker run -it -p 8280:8280 -p 8243:8243 -p 9443:9443 -v wso2ei:/home/wso2carbon --name integrator wso2/wso2ei-integrator
I'm trying to create the pod definition file for the same. I don't know how to do port mapping and volume mapping in pod definition file. The following is the file I have created up to now. How can I complete the rest?
apiVersion: v1
kind: Pod
metadata:
name: ei-pod
labels:
type: ei
version: 6.6.0
spec:
containers:
- name: integrator
image: wso2/wso2ei-integrator
Here is YAML content which might work:
apiVersion: v1
kind: Pod
metadata:
name: ei-pod
labels:
type: ei
version: 6.6.0
spec:
containers:
- name: integrator
image: wso2/wso2ei-integrator
ports:
- containerPort: 8280
volumeMounts:
- mountPath: /wso2carbon
name: wso2ei
volumes:
- name: wso2ei
hostPath:
# directory location on host
path: /home/wso2carbon
While the above YAML content is just a basic example, it's not recommended for production usage because of two reasons:
Use deployment or statefulset or daemonset instead of pods directly.
hostPath volume is not sharable between nodes. So use external volumes such as NFS or Block and mount it into the pod. Also look at dynamic volume provisioning using storage class.

Write to Secret file in pod

I define a Secret:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
stringData:
config.yaml: |-
apiUrl: "https://my.api.com/api/v1"
username: Administrator
password: NewPasswdTest11
And then creating volume mount in Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-webapp-test
labels:
name: k8s-webapp-test
version: 1.0.4
spec:
replicas: 2
selector:
matchLabels:
name: k8s-webapp-test
version: 1.0.4
template:
metadata:
labels:
name: k8s-webapp-test
version: 1.0.4
spec:
nodeSelector:
kubernetes.io/os: windows
volumes:
- name: secret-volume
secret:
secretName: string-data-secret
containers:
- name: k8s-webapp-test
image: dockerstore/k8s-webapp-test:1.0.4
ports:
- containerPort: 80
volumeMounts:
- name: secret-volume
mountPath: "/secrets"
readOnly: false
So, after the deployment, I have 2 pods with volume mounts in C:\secrets (I do use Windows nodes). When I try to edit config.yaml that is located in C:\secrets folder, I get following error:
Access to the path 'c:\secrets\config.yaml' is denied.
Although I marked file as readOnly false I cannot write to it. How can I modify the file?
As you can see here it is not possible by intention:
Secret, configMap, downwardAPI and projected volumes will be mounted as read-only volumes. Applications that attempt to write to these volumes will receive read-only filesystem errors. Previously, applications were allowed to make changes to these volumes, but those changes were reverted at an arbitrary interval by the system. Applications should be re-configured to write derived files to another location
You can look into using an init container which maps the secret and then copies it to the desired location where you might be able to modify it.
As an alternative to the init container you might also use a container lifecycle hook i.e. a PostStart-hook which executes immediately after a container is created.
lifecycle:
postStart:
exec:
command:
- "/bin/sh"
- "-c"
- >
cp -r /secrets ~/secrets;
You can create secrets from within a Pod but it seems you need to utilize the Kubernetes REST API to do so:
https://kubernetes.io/docs/concepts/overview/kubernetes-api/

Does kubernetes pod restart on failure on ImgPullBackOff

I have a kubernetes cluster working perfectly fine. I have 10 worker nodes and 1 master device. I have a below deployment.yaml file type as DaemonSet for pods and containers.
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: deployment
namespace: mynamespace
spec:
replicas: 2
selector:
matchLabels:
name: deployment
template:
metadata:
labels:
name: deployment
spec:
#List of all the containers
containers:
- name: container1
image: CRname/container1
imagePullPolicy: Always
volumeMounts:
- mountPath: /share
name: share-files
securityContext:
privileged: true
- name: container2
image: CRname/container2
imagePullPolicy: Always
volumeMounts:
- mountPath: /share
name: share-files
securityContext:
privileged: true
volumes:
- name: share-files
hostPath:
path: /home/user/shared-folder
imagePullSecrets:
- name: Mysecret
nodeSelector:
NodeType: ALL
On starting above, these two containers start running on all the worker nodes and runs perfectly fine. But I have observed that sometimes, few nodes show error as ImagePullBackOff which means that due to some network or any other issues, it wasn't able to download the image. I did use describe command to check which image failed. But the problem is it didn't try to automatically redownload the image. I had to delete the pod and thus it is automatically created and then it works fine.
I just want to know why the pod shows this error and do not try to redownload the image. Is there anything which I can add in the yaml file so that it delete and recreate the pod automatically on any type of error.
EDIT I would also like to ask when the deployment is created, node starts to pull the image from the container registry initially for the first time. Once they are downloaded locally on the nodes, why does it has to pull the image again when the image is locally present.?
Please suggest some good options. Thanks.
imagePullPolicy: Always cause you download the image everytime pod restart.
And the pod is always restarted by daemonset, so just wait a bit more time.