I am trying to learn Kubernetes and installed Minikube on my local. I created a sample python app and its corresponding image was pushed to public docker registry. I started the cluster with
kubectl apply -f <<my-app.yml>>
It got started as expected. I stopped Minikube and deleted all the containers and images and restarted my Mac.
My Questions
I start my docker desktop and as soon as I run
minikube start
Minikube goes and pulls the images from public docker registry and starts the container. Is there a configuration file that Minikube looks into to start my container that I had deleted from my local? I am not able to understand from where is Minikube picking up my-app's configurations which was defined in manifest folder.
I have tried to look for config files and did find cache folder. But it does not contain any information about my app
I found this is expected behavior:
minikube stop command should stop the underlying VM or container, but keep user data intact.
So when I manually delete already created resources it does not automatically starts.
More information :
https://github.com/kubernetes/minikube/issues/13552
Related
I have a VM with kubernetes installed using kubeadm (NOT minikube). The VM acts a the single node of the cluster, with taints removed to allow it to act as both Master and Worker node (as shown in the kubernetes documentation).
I have saved, transfered and loaded a my app:test image into it. I can easily run a container with it using docker run.
It shows up when I run sudo docker images.
When I create a deployment/pod that uses this image and specify Image-PullPolicy: IfNotPresent or Never, I still have the ImagePullBackoff error. The describe command shows me it tries to pull the image from dockerhub...
Note that when I try to use a local image that was pulled as the result of creating another pod, the ImagePullPolicies seem to work, no problem. Although the image doesn't appear when i run sudo docker images --all.
How can I use a local image for pods in kubernetes? Is there a way to do it without using a private repository?
image doesn't appear when i run sudo docker images --all
Based on your comment, you are using K8s v1.22, which means it is likely your cluster is using containerd container runtime instead of docker (you can check with kubectl get nodes -o wide, and see the last column).
Try listing your images with crictl images and pulling with crictl pull <image_name> to preload the images on the node.
One can do so with a combination of crictl and ctr, if using containerd.
TLDR: these steps, which are also described in the crictl github documentation:
1- Once you get the image on the node (in my case, a VM), make sure it is in an archive (.tar). You can do that with the docker save or ctr image export commands.
2- Use sudo ctr -n=k8s.io images import myimage.tar while in the same directory as thearchived image to add it to containerd in the namespace that kubernetes uses to track it's images. It should now appear when you run sudo crictl images.
As suggested, I tried listing images with crictl and my app:test did not appear. However, trying to import my local image through crictl didn't seem to work either. I used crictl pull app:test and it showed the following error message:
FATA[0000] pulling image failed: rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/app:test": failed to resolve reference "docker.io/library/app:test": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed.
However, when following these steps, my image is finally recognized as an existing local image in kubernetes. They are actually the same as suggested in the crictl github documentation
How does one explain this? How do images get "registered" in the kubernetes cluster? Why couldn't crictl import the image? I might post another issue to ask that...
Your cluster is bottled inside of your VM, so what you call local will always be remote for that cluster in that VM. And the reason that kubernetes is trying to pull those images, is because it can't find them in the VM.
Dockerhub is the default place to download containers from, but you can set kubernetes to pull from aws (ECR) from azure (ACR), from github packages (GCR) and from your own private server.
You've got about 100 ways to solve this, none of them are easy or will just work.
1 - easiest, push your images to Dockerhub and let your cluster pull from it.
2 - setup a local private container registry and set your kubernetes VM to pull from it (see this)
3 - setup a private container registry in your kubernetes cluster and setup scripts in your local env to push to it (see this)
I am trying to learn Kubernetes and so I installed Minikube on my local Windows 10 Home machine and then I tried installing the kubectl. However so far I have been unsuccessful in getting it right.
So this what I have done so far:
Downloaded the kubectl.exe file from https://storage.googleapis.com/kubernetes-release/release/v1.18.0/bin/windows/amd64/kubectl.exe
Then I added the path of this exe in the path environment variable as shown below:
However this didn't work when I executed kubectl version on the command prompt or even on pwoershell (in admin mode)
Next I tried using the curl command as given in the docs - https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-with-curl-on-windows
However that too didn't work as shown below:
Upon searching for answers to fix the issue, I stumbled upon this StackOverfow question which explained how to create a .kube config folder because it didn't exist on my local machine. I followed the instructions, but that too failed.
So right now I am completely out of ideas and not sure whats the issue here. FYI, I was able to install everything in a breeze on my Mac, however Windows is just acting crazy.
Any help would be really helpful.
As user #paltaa mentioned:
did you do a minikube start ? – paltaa 2 days ago
The fact that you did not start the minikube is the most probable cause why you are getting this error.
Additionally this error message shows when the minikube is stopped as stopping will change the current-context inside the config file.
There is no need to create a config file inside of a .kube directory as the minikube start will create appropriate files and directories for you automatically.
If you run minikube start command successfully you should get below message at the end of configuration process which will indicate that the kubectl is set for minikube automatically.
Done! kubectl is not configured to use "minikube"
Additionally if you invoke command $ kubectl config you will get more information how kubectl is looking for configuration files:
The loading order follows these rules:
1. If the --kubeconfig flag is set, then only that file is loaded. The flag may only be set once and no merging takes
place.
2. If $KUBECONFIG environment variable is set, then it is used as a list of paths (normal path delimiting rules for
your system). These paths are merged. When a value is modified, it is modified in the file that defines the stanza. When
a value is created, it is created in the first file that exists. If no files in the chain exist, then it creates the
last file in the list.
3. Otherwise, ${HOME}/.kube/config is used and no merging takes place.
Please take a special look on part:
Otherwise, ${HOME}/.kube/config is used
Even if you do not set the KUBECONFIG environment variable kubectl will default to $USER_DIRECTORY (for example C:\Users\yoda\.
If for some reason your cluster is running and files got deleted/corrupted you can:
minikube stop
minikube start
which will recreate a .kube/config
Steps for running minikube on Windows in this case could be:
Download and install Kubernetes.io: Install minikube using an installer executable
Download, install and configure a Hypervisor (for example Virtualbox)
Download kubectl
OPTIONAL: Add the kubectl directory to Windows environment variables
Run from command line or powershell from current user: $ minikube start --vm-driver=virtualbox
Wait for configuration to finish and invoke command like $ kubectl get nodes.
I am trying to configure the kube-apiserver so that it uses encryption to configure secrets in my minikube cluster.
For that, I have followed the documentation on kubernetes.io but got stuck at step 3 that says
Set the --encryption-provider-config flag on the kube-apiserver to point to the location of the config file.
I have discovered the option --extra-config on minikube start and have tried starting my setup using
minikube start --extra-config=apiserver.encryption-provider-config=encryptionConf.yaml
but naturally it doesn't work as encryptionConf.yaml is located in my local file system and not in the pod that's spun up by minikube. The error minikube log gives me is
error: error opening encryption provider configuration file "encryptionConf.yaml": open encryptionConf.yaml: no such file or directory
What is the best practice to get the encryption configuration file onto the kube-apiserver? Or is minikube perhaps the wrong tool to try out these kinds of things?
I found the solution myself in this GitHub issue where they have a similar issue for passing a configuration file. The comment that helped me was the slightly hacky solution that made use of the fact that the directory /var/lib/localkube/certs/ from the minikube VM is mounted into the apiserver.
So my final solution was to run
minikube mount .:/var/lib/minikube/certs/hack
where in the current directory I had my encryptionConf.yaml and then start minikube like so
minikube start --extra-config=apiserver.encryption-provider-config=/var/lib/minikube/certs/hack/encryptionConf.yaml
Based on drivers used some directories are mounted on to your minikube VM.
Check this link - https://kubernetes.io/docs/setup/minikube/#mounted-host-folders
Also ~/.minikube/files is also mounted into the VM at /files. So you can keep your files there and use that path for API server config
I had similar issues in windows regarding filepath location
since C:\Users\%USERNAME%\ is by default mounted in minikube VM
so i copied the files to Desktop folder( any folder under C drive )
minikube --extra-config=apiserver.encryption-provider-config=/c/Users/%USERNAME%/.../<file-name>
hope this is helpful for folks facing this issues on windows platform.
I was trying to deploy my local docker image on kubernetes, but doesn't work for me.
I loaded image into docker and tagged it as app:v1, then I ran image by use kubectl this way kubectl run app --image=app:v1 --port=8080.
If I want to lookup my pods I see error "Failed to pull image "app:v1": rpc error: code = 2 desc = Error: image library/app not found".
What am I doing wrong?
In normal case your Kubernetes cluster runs on a different machine than your docker build was run on, hence it has no access to your local image (unless you are using minikube and you eval minikubes environment to actually run your docker commands against docker daemon powering the minikube install).
To get it working you need to push the image to a registry available to kubernetes cluster.
By running your command you actually tell kubernetes to pull app:v1 from official docherhub hosted images.
I have an instance group of Container VMs running my app on a docker container.
I am trying to find a good strategy to manage the application logs for docker + MEAN + Google Cloud Compute Machines.
I can see the logs on individual containers running docker logs [container_id].
However, if I stop and start the VM I lose those logs. I also have VMs dynamically added by Auto scaler and would like to have a convenient way to access the logs.
Stack is MEAN and Logging tool is bunyan.
Is is possible to centralize or combine the logs from all VMS in one persistent location?
any suggestions?
UPDATES:
I installed fluentd agent and now I can see logs when I manually run thins on the shell: logger "some message for testing"
However, the logs from my container vm from my docker container never shows up on logs.
I still don't know how to get those docker logs to turn up on google cloud logs. It is supposed to be automatically collected.
cheers
Leo
Here is a yaml, Dockerfile and conf for a fluentd pod inside kubernetes.
Adjust the yaml to mount a disk:
https://github.com/GoogleCloudPlatform/kubernetes/tree/master/contrib/logging/fluentd-sidecar-gcp
Then adjust the config to log to the disk.
Build the container with the new configuration.
Deploy the new container.