If I do
POD=$($KUBECTL get pod -lsvc=app,env=production -o jsonpath="{.items[0].metadata.name}")
kubectl debug -it --image=mpen/tinker "$POD" -- zsh -i
I can get into a shell running inside my pod, but I want access to the filesystem for a container I've called "php". I think this should be at /proc/1/root/app but that directory doesn't exist. For reference, my Dockerfile has:
WORKDIR /app
COPY . .
So all the files should be in the root /app directory.
If I add --target=php then I get permission denied:
❯ cd /proc/1/root
cd: permission denied: /proc/1/root
How do I get access to the files?
Reading through the documentation, using kubectl debug won't give you access to the filesystem in another container.
The simplest option may be to use kubectl exec to start a shell inside an existing container. There are some cases in which this isn't an option (for example, some containers contain only a single binary, and won't have a shell or other common utilities avaiable), but a php container will typically have a complete filesystem.
In this case, you can simply:
kubectl exec -it $POD -- sh
You can replace sh by bash or zsh depending on what shells are available in the existing image.
The linked documentation provides several other debugging options, but all involve working on copies of the pod.
I am trying out OWASP/ZAP to see if it is something we can use for our project, but I cannot make it work I don't know what I am doing wrong and the documentation really does not help. What I am trying is to run a scan on my api running in a docker container locally on my windows machine so I run the command:
docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable zap-baseline.py -t http://172.21.0.2:8080/swagger.json -g gen.conf -r testreport.html the ip 172.21.0.2 is the IPAddress of my api container even tried with localhost and 127.0.0.1
but it just hangs in the following log message:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 1:43:31 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Nothing happens and my zap docker container is in a unhealthy state, after some time it just crashes and ends up with a bunch of NullPointerExceptions. Is zap docker only working for linux, something specifically I need to do when running it on a windows machine? I don't get why this is not working even when I am following specifically the guideline in https://github.com/zaproxy/zaproxy/wiki/Docker
Edit 1
My latest try where I am trying to target my host ip address directly and the port that I am exposing my api to gives me the following error:
_XSERVTransmkdir: ERROR: euid != 0,directory /tmp/.X11-unix will not be created.
Feb 14, 2019 2:12:07 PM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
Total of 3 URLs
ERROR Permission denied
2019-02-14 14:12:57,116 I/O error(13): Permission denied
Traceback (most recent call last):
File "/zap/zap-baseline.py", line 347, in main
with open(base_dir + generate, 'w') as f:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
Found Java version 1.8.0_151
Available memory: 3928 MB
Setting jvm heap size: -Xmx982m
213 [main] INFO org.zaproxy.zap.DaemonBootstrap
When you run docker with: docker run -v $(pwd):/zap/wrk/:rw ...
you are mapping the /zap/wrk/ directory in the docker image to the current working directory (cwd) of the machine in which you are running docker.
I think the problem is that your current user doesn't have write access to the cwd.
Try below command, hope it resolves issue.
$docker run --user $(id -u):$(id -g) -v $(pwd):/zap/wrk/:rw --rm -t owasp/zap2docker-stable zap-baseline.py -t https://your_url -g gen.conf -r testreport.html
The key error here is:
IOError: [Errno 13] Permission denied: '/zap/wrk/gen.conf'
This means that the script cannot write to the gen.conf file that you have mounted on /zap/wrk
Do you have write access to the cwd when its not mounted?
The reason for that is, if you use -r parameter, zap will attempt to generate the file report.html at location /zap/wrk/. In order to make this work, we have to mount a directory to this location /zap/wrk.
But when you do so, it is important that the zap container is able to perform the write operations on the mounted directory.
So, below is the working solution using gitlab ci yml. I started with this approach of using image: owasp/zap2docker-stable however then had to go to the vanilla docker commands to execute it.
test_site:
stage: test
image: docker:latest
script:
# The folder zap-reports created locally will be mounted to owasp/zap2docker docker container,
# On execution it will generate the reports in this folder. Current user is passed so reports can be generated"
- mkdir zap-reports
- cd zap-reports
- docker pull owasp/zap2docker-stable:latest || echo
- docker run --name zap-container --rm -v $(pwd):/zap/wrk -u $(id -u ${USER}):$(id -g ${USER}) owasp/zap2docker-stable zap-baseline.py -t "https://example.com" -r report.html
artifacts:
when: always
paths:
- zap-reports
allow_failure: true
So the trick in the above code is
Mount local directory zap-reports to /zap/wrk as in $(pwd):/zap/wrk
Pass the current user and group on the host machine to the docker container so the process is using the same user / group. This allows writing of reports on the directory mounted from local host. This is done by -u $(id -u ${USER}):$(id -g ${USER})
Below is the working code with image: owasp/zap2docker-stable
test_site:
variables:
GIT_STRATEGY: none
stage: test
image:
name: owasp/zap2docker-stable:latest
before_script:
- mkdir -p /zap/wrk
script:
- zap-baseline.py -t "https://example.com" -g gen.conf -I -r testreport.html
- cp /zap/wrk/testreport.html testreport.html
artifacts:
when: always
paths:
- zap.out
- testreport.html
I have a docker Image that basically runs a one time script. That scripts takes 3 arguments. My docker file is
FROM <some image>
ARG URL
ARG USER
ARG PASSWORD
RUN apt update && apt install curl -y
COPY register.sh .
RUN chmod u+x register.sh
CMD ["sh", "-c", "./register.sh $URL $USER $PASSWORD"]
When I spin up the contianer using docker run -e URL=someUrl -e USER=someUser -e PASSWORD=somePassword -itd <IMAGE_ID> it works perfectly fine.
Now I want to deploy this as a job.
My basic Job looks like:
apiVersion: batch/v1
kind: Job
metadata:
name: register
spec:
template:
spec:
containers:
- name: register
image: registeration:1.0
args: ["someUrl", "someUser", "somePassword"]
restartPolicy: Never
backoffLimit: 4
But this the pod errors out on
Error: failed to start container "register": Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"someUrl\": executable file not found in $PATH"
Looks like it is taking my args as commands and trying to execute them. Is that correct ? What can I do to fix this ?
In the Dockerfile as you've written it, two things happen:
The URL, username, and password are fixed in the image. Anyone who can get the image can run docker history and see them in plain text.
The container startup doesn't take any arguments; it just runs the single command with its fixed set of arguments.
Especially since you're planning to pass these arguments in at execution time, I wouldn't bother trying to include them in the image. I'd reduce the Dockerfile to:
FROM ubuntu:18.04
RUN apt update \
&& DEBIAN_FRONTEND=noninteractive \
apt install --assume-yes --no-install-recommends \
curl
COPY register.sh /usr/bin
RUN chmod u+x /usr/bin/register.sh
ENTRYPOINT ["register.sh"]
When you launch it, the Kubernetes args: get passed as command-line parameters to the entrypoint. (It is the same thing as the Docker Compose command: and the free-form command at the end of a plain docker run command.) Making the script be the container entrypoint will make your Kubernetes YAML work the way you expect.
In general I prefer using CMD to ENTRYPOINT. (Among other things, it makes it easier to docker run --rm -it ... /bin/sh to debug your image build.) If you do that, then the Kubernetes args: need to include the name of the script it's running:
args: ["./register.sh", "someUrl", "someUser", "somePassword"]
Use:
args: ["sh", "-c", "./register.sh someUrl someUser somePassword"]
I'm trying to copy files from Kubernetes Pods to my local system. I am getting the below error while running following command:
kubectl cp aks-ssh2-6cd4948f6f-fp9tl:/home/azureuser/test.cap ./test.cap
Output:
tar: home/azureuser/test: Cannot stat: No such file or directory tar:
Exiting with failure status due to previous errors error:
home/azureuser/test no such file or directory
I could see the file under above given path. I am really confused.
Could you please help me out?
As stated inkubectl help:
kubectl cp --help
Copy files and directories to and from containers.
Examples:
# !!!Important Note!!!
# Requires that the 'tar' binary is present in your container
# image. If 'tar' is not present, 'kubectl cp' will fail.
# Copy /tmp/foo_dir local directory to /tmp/bar_dir in a remote pod in the default namespace
kubectl cp /tmp/foo_dir <some-pod>:/tmp/bar_dir
# Copy /tmp/foo local file to /tmp/bar in a remote pod in a specific container
kubectl cp /tmp/foo <some-pod>:/tmp/bar -c <specific-container>
# Copy /tmp/foo local file to /tmp/bar in a remote pod in namespace <some-namespace>
kubectl cp /tmp/foo <some-namespace>/<some-pod>:/tmp/bar
# Copy /tmp/foo from a remote pod to /tmp/bar locally
kubectl cp <some-namespace>/<some-pod>:/tmp/foo /tmp/bar
Options:
-c, --container='': Container name. If omitted, the first container in the pod will be chosen
Usage:
kubectl cp <file-spec-src> <file-spec-dest> [options]
Use "kubectl options" for a list of global command-line options (applies to all commands).
You can also login to your Containter and check if file is there:
kubectl exec -it aks-ssh2-6cd4948f6f-fp9tl /bin/bash
ls -la /home/azureuser/test.cap
If this still doesn't work, try:
You may try to copy your files to workdir and then retry to copy them using just their names. It's weird, but it works for now.
Consider advice of kchugalinskiy here #58692.
Let's say you are copying file from bin folder to local system. The command is
kubectl cp default/POD_NAME:bin/FILE_NAME /Users/username/FILE_NAME
You can connect to POD to verify if you are specifying correct file name
kubectl exec -ti POD_NAME bash
According to https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands
kubectl cp <file-spec-src> <file-spec-dest> is equivalent to using
kubectl exec -n <some-namespace> <some-pod> -- tar cf - <src-file> | tar xf - -C <dest-file>
So technically if you do not have tar installed on the pod, you can do kubectl exec -n <some-namespace> <some-pod> -- cat <src-file> > <dest-file>
Assuming the file is small or already compressed, the effect should be the same, except you cannot use cat on a directory or a set of files.
The command in the question posted is absolutely right. As answered before, this particular issue seems to be a missing tar binary in the container. I actually did not know it was needed, but confirmed that the pod has it:
# find / -name tar
/bin/tar
/usr/lib/mime/packages/tar
/usr/share/doc/tar
My error was using . to copy to the current directory (works with cp and scp) because it needs the full path, as shown in the original question:
kubectl cp pod-name-shown-in-get-pods:path/to/filename /local/dir/filename
But not:
kubectl cp pod-name-shown-in-get-pods:path/to/filename .
Which gives:
error: open .: is a directory
tar: Removing leading `/' from member names
Now the tar in the error message makes sense!
Note that if there is a leading / in the source path, as in the following example:
kubectl cp pod-name-shown-in-get-pods:/etc/resolv.conf /local/dir/resolv.conf
You would also see:
tar: Removing leading `/' from member names
However, the warning can be ignored, as the file would still copied. Use etc/resolv.conf instead of /etc/resolv.conf in the above example, to copy without the warning.
"kubectl cp" command is used to copy files from pods to local path and vice versa
Copying file from pod to local
kubectl cp <pod_name>:<file_path> <destination_path>
Copying file from specific container of pod to local
kubectl cp <pod_name>:<file_path> <destination_path> -c specific_container
Copying file from local to pod
kubectl cp <local_source_path> <pod_name>:<destination_path>
kubectl cp command is already mentioned by some of the users on this thread.
kubectl cp <pod-id>:<path> <local-path> -n <namespace> -c <specific_container>
Note that to run this command tar utility should already be installed on the pod.
However I have come across few errors while running this command on windows PowerShell.
PS P:\Users\nstty\Downloads\k8s-diags> kubectl cp dremio-master-0:/var/log/dremio/server.log P:\Users\nstty\Downloads\k8s-diags\server-logs\
error: one of src or dest must be a local file specification
error: one of src or dest must be a local file specification
When running this command on windows, don't use the full path of the local system. Use relative path instead (. or ..). Now using relative path in the below command but getting a different error.
PS P:\Users\nstty\Downloads\k8s-diags> kubectl cp dremio-master-0:/var/log/dremio/server.log .
tar: Removing leading `/' from member names
error: open .: is a directory
error: open .: is a directory
If you are copying a file, then in the local path use the relative path along with the file name that you want for the copied file. kubectl will first create this file and then copy the contents to this file. Below is the working command.
PS P:\Users\nstty\Downloads\k8s-diags> kubectl cp dremio-master-0:/var/log/dremio/server.log .\server-logs\server.log
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
This message is just a warning from tar utility in your pod. The file should be copied to your local system.
Alternate option: If you want to avoid kubectl cp, here is another approach which we use.
kubectl cp <pod-id>:<path> <destination-path> -n <namespace>
Worked for me.
You can mount a local directory into the pod.
Update your aks-ssh yaml file:
spec:
...
containers:
...
volumeMounts:
- name: test-dir
mountPath: /home/azureuser
...
volumes:
- name: test-dir
hostPath:
path: /path/to/your/local/dir
Now you can access your files in the local directory.
For people working on a Windows machine there is an additional gotcha: As at October 2021 you cannot include a drive letter in your local path.
So if you were to try a command like:
kubectl cp aks-ssh2-6cd4948f6f-fp9tl:/home/azureuser/test.cap C:/Temp/Test
you would get this error because kubectl cp sees the colon in the Windows path as the separator between a pod name and the path within the pod.
So it would see C:/Temp/Test as a pod named "C" with a path "/Temp/Test"
The way to get around this is to use a relative Windows path instead of an absolute path. It will need to be relative to your current working directory.
So if my current working directory is C:\Users\JoeBloggs and I wanted to copy down to C:\Temp\Test I'd need to use the command:
kubectl cp aks-ssh2-6cd4948f6f-fp9tl:/home/azureuser/test.cap ../../Temp/Test
Note that this issue looks like it may be fixed soon. See https://github.com/kubernetes/kubernetes/pull/94165
Maybe someone could met this error
tar: removing leading '/' from member names
error: open .: is a directory
Which induced by the following command
kc cp -n monitoring <pod name>:/usr/share/grafana/conf/defaults.ini ./
kc cp -n monitoring <pod name>:/usr/share/grafana/conf/defaults.ini ./default.ini
To solve it, we should add a destination folder per doc
kc cp -n monitoring <pod name>:/usr/share/grafana/conf/defaults.ini ./tmp/default.ini
This works for me:
kubectl cp "namespace"/"pod_name":"path_in_pod" "local_path"
Example:
kubectl cp mynamespace/mypod:var/www/html/index.html \Users\myuser\Desktop\index.html
I resolve this problem by set the source folder to be relative path.
If the file location is /home/azureuser/test.cap, and working dir is /home/azureuser/, the cmd is
kubectl cp aks-ssh2-6cd4948f6f-fp9tl:test.cap ./test.cap
If anyone uses windows pods, it may be hard to get files copied to the pods from local machine with those linux paths for kubectl cp command:
Procedure to copy files from local machine to kubernetes pod: (especially windows container)
I want to copy node.aspx from my local machine to
podname:\c:\inetpub\wwwroot
First upload Node.aspx to your cloud drive, path will be
/home/{your_username} in my case /home/pranesh
Then find out the pod name, in my case its
aspx-deployment-84597d88f5-pk5nh, follow below command
PS /home/pranesh> kubectl cp /home/pranesh/Node.aspx aspx-deployment-84597d88f5-pk5nh:/Node.aspx
This copies the file to c drive of container,
then move file from c drive to required path with powershell
PS /home/pranesh> kubectl exec aspx-deployment-84597d88f5-pk5nh powershell "Copy-Item "C:\Node.aspx" -Destination "C:\inetpub\wwwroot""
Use the reverse procedure for copying from container to cloud drive and download.
kubectl cp will not work if your container does not have tar command in the PATH. From your error it sees like tar command is not available on your container.
https://github.com/kubernetes/kubernetes/issues/58512
Please explore other options
Kubernetes gives a file not found error when user does not have permissions to a pod. That was my problem.
On my side the issue was with having multiple containers inside the pod:
kubectl cp -c grafana \
metrics/grafana-5c4f76b49b-p88lc:/etc/grafana \
./grafana/etc
so set the container name with -c grafana and properly prefix the namespace of the pod in my case metrics/
I tried this method on Azure and it worked:
kubectl cp 'POD NAME':xyz.json test
kubectl cp is the command for copying
POD NAME = vote-app
xyz.json is the file that needs to be copied from pod
the test is the file created in the drive of the AZURE directory...
So final command would be:
kubectl cp vote-app:xyz.json test
test will get generated in Azure Directory and later u can download test file from the download option of Azure
I couldn't get kubectl to work for this. Was getting error:
The connection to the server localhost:8080 was refused - did you specify the right host or port?
This worked for me instead:
docker cp CONTAINERID:FILEWITHPATH DESTFILENAME
where "CONTAINERID" was retrieved by calling docker ps.
you need to mention the namespace in which your pod is available.
I have a Dockerfile with below code
FROM microsoft/mssql-server-windows-express
COPY ./create-db.sql .
ENV ACCEPT_EULA=Y
ENV sa_password=##$wo0RD!
CMD sqlcmd -i create-db.sql
and I can create image but when I run container with the image I don't see created database on the SQL Server because the script is executed before SQL Server was started.
Can I do that the script will be execute after start the service with SQL Server?
RUN gets used to build the layers in an image. CMD is the command that is run when you launch an instance (a "container") of the built image.
Also, if your script depends on those environment variables, if it's an older version of Docker, it might fail because those variables are not defined the way you want them defined!
In older versions of docker the Dockerfile ENV command uses spaces instead of "="
Your Dockerfile should probably be:
FROM microsoft/mssql-server-windows-express
COPY ./create-db.sql .
ENV ACCEPT_EULA Y
ENV SA_PASSWORD ##$wo0RD!
RUN sqlcmd -i create-db.sql
This will create an image containing the database with your password inside it.
(If the SQL file somehow uses the environment variables, this wouldn't make sense as you might as well update the SQL file before you copy it over.) If you want to be able to override the password between the docker build and docker run steps, by using docker run --env sa_password=##$wo0RD! ..., you will need to change the last line to:
CMD sqlcmd -i create-db.sql && .\start -sa_password $env:SA_PASSWORD \
-ACCEPT_EULA $env:ACCEPT_EULA -attach_dbs \"$env:attach_dbs\" -Verbose
Which is a modified version of the CMD line that is inherited from the upstream image.
You can follow this link https://github.com/microsoft/mssql-docker/issues/11.
Credits to Robin Moffatt.
Change your docker-compose.yml file to contain the following
mssql:
image: microsoft/mssql-server-windows-express
environment:
- SA_PASSWORD=##$wo0RD!
- ACCEPT_EULA=Y
volumes:
# directory with sql script on pc to /scripts/
# - ./data/mssql:/scripts/
- ./create-db.sql:/scripts/
command:
- /bin/bash
- -c
- |
# Launch MSSQL and send to background
/opt/mssql/bin/sqlservr &
# Wait 30 seconds for it to be available
# (lame, I know, but there's no nc available to start prodding network ports)
sleep 30
# Run every script in /scripts
# TODO set a flag so that this is only done once on creation,
# and not every time the container runs
for foo in /scripts/*.sql
do /opt/mssql-tools/bin/sqlcmd -U sa -P $$SA_PASSWORD -l 30 -e -i $$foo
done
# So that the container doesn't shut down, sleep this thread
sleep infinity