Jenkins kubernetes build user permission problem - kubernetes

Question

I am trying to do some ssh + git sub module builds, and initially I was getting this error when trying to pull code:
Submodule 'xxx' (git#host:thing/subdir/repo.git) registered for path 'repo'
Submodule 'yyy' (git#host:thing/subdir/yyy.git) registered for path 'yyy'
Cloning into '/home/jenkins/agent/workspace/thing/AAA/releaser/xxx'...
No user exists for uid 1000
fatal: Could not read from remote repository.
Which led me to this solution:
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- all
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 65534
Now, this worked, but I was seeing some non critical errors about not being able to save the host key to /.ssh, but it still checked out the code and so I didn't care since every run is like it's the first time.
However, I am now running into a different problem that really requires me to solve this user issue permanently:
+ mc alias set jenkins-user ****
mc: <ERROR> Unable to save new mc config. mkdir /.mc: permission denied.
I need to run the minio mc client - and when I give it creds it tries to save them to /.mc and it can't and fails the aliasing command, and thus the whole build.
So, I need to figure out a better way to solve the initial user problem, or tell it that, when running in this pod, to use a non privileged directory like /tmp.
So, I am open to a solution to either issue if someone can assist

Answer 1

Ok, I ended up solving it with a mix of Kubernetes changes and Dockerfile changes.
First, I had to add the Jenkins user with the right user id to a custom docker image that contained what I needed
FROM minio/mc
RUN \
microdnf update --nodocs && \
microdnf install git zip findutils --nodocs && \
microdnf clean all
RUN adduser --home-dir /home/jenkins/ --shell /bin/sh --uid 1000 jenkins
USER jenkins
WORKDIR /home/jenkins
ENTRYPOINT ["sh"]
Then, I had to update the pod definition to run under the same user that it was complaining about before:
apiVersion: v1
kind: Pod
metadata:
annotations: {}
spec:
securityContext:
allowPrivilegeEscalation: false
runAsUser: 1000
containers:
-
name: releaser
image: docker.io/chb0docker/releaser:latest
imagePullPolicy: Always
command:
- cat
tty: true