determine the Reason for Pod Failure - kubernetes

I have a broken pod in default namespace. I want to store associated error events to a file error.txt. I need to use the -o wide output specifier with my command.
I saw command like this
- kubectl get pod termination-demo -o go-template="{{range
.status.containerStatuses}}{{.lastState.terminated.message}}{{end}}"
but i need to use -o wide command.

If you want to achieve the pod state :
kubectl get pod -n <NAMESPACE> <POD> --output=wide -o go-template="{{range.status.containerStatuses}}{{.lastState.terminated.message}}{{end}}" > pod.err
Verify by :
$ cat pod.err

not sure what you looking to achieve:
if your goal is to get the content into a file you can redirect the output to a file:
kubectl get pod termination-demo -o go-template="{{range.status.containerStatuses}}{{.lastState.terminated.message}}{{end}} > error.txt

Related

How to grep older logs from all pods in K8 (-l and --since=10m not working simultaneously)

I have been trying to find a solution to this among the previously asked questions, but I can't find one that works for my use case (which to me seems like a general use case)
So I have a load balancer service and 5 pods in a namespace that share a label app=abc_application. So when I want to follow logs in all pods simultaneously, I use this
kubectl logs -f -l app=abc_application -c abc_application_container
Now my use case looks like this. I have a request that failed an hour back and I want to check the logs. I wanted to use the --since=60m argument but that doesn't work with the above command.
Is there any alternative than getting logs of individual pods? Can this command not be integrated?
I tried this with kubectl tail and got it working
To install kubectl tail -> kubectl krew install tail
kubectl tail -n <namespace> -l app=abc_application --since=2h
you can also do the same with logs
kubectl logs <POD name> -n <Namespace name> --since-time='2021-09-21T10:00:00Z'
Using simple since with logs
kubectl logs <POD name> -n <Namespace name> --since=60h (5s, 2m, or 3h)
If you want to tail logs by a few line
kubectl logs <POD name> -n <Namespace name> --tail=200
If want to grep anything from logs
kubectl logs <POD name> -n <Namespace name> | grep <string>
With the above command, you can pass the container name with -c & -l for label.
Reference : https://jamesdefabia.github.io/docs/user-guide/kubectl/kubectl_logs/

combine multiple column output in kubectl using custom-columns or jsonpath

This is the output with custom-columns
$ kubectl -n web get pod -ocustom-columns="Name:.metadata.name,Image:.spec.containers[0].image"
Name Image
mysql-0 myrepo.mydomain.com/mysql:5.7
mysql-1 myrepo.mydomain.com/mysql:5.7
mysql-2 myrepo.mydomain.com/mysql:5.7
This is the output with jsonpath for single pod
$ kubectl -n web get pod -o jsonpath="{..metadata.name}:{..spec.containers[0].image}" mysql-0
mysql-0:myrepo.mydomain.com/mysql:5.7
This is the output with jsonpath for multiple pods
$ kubectl -n web get pod -o jsonpath="{..metadata.name}:{..spec.containers[0].image}"
mysql-0 mysql-1 mysql-2:myrepo.mydomain.com/mysql:5.7 myrepo.mydomain.com/mysql:5.7 myrepo.mydomain.com/mysql:5.7
Now how to combine this into a single column or word, something like this, using -ocustom-columns or -ojsonpath
mysql-0=myrepo.mydomain.com/mysql:5.7
mysql-1=myrepo.mydomain.com/mysql:5.7
mysql-2=myrepo.mydomain.com/mysql:5.7
Using kubectl using plain jsonpath:
kubectl get pod -n <namespace> -o jsonpath='{range .items[*]}{.metadata.name}={.spec.containers[*].image}{"\n"}{end}'
Example:
kubectl get pod -n default -o jsonpath='{range .items[*]}{.metadata.name}={.spec.containers[*].image}{"\n"}{end}'
nginx-0=nginx
nginx-1=nginx
nginx-2=nginx
Here range feature is used to loop over all the pods:
{range items[*]} ...<LOGIC HERE>... {end}
Between the range block(As described above), use the jsonpath, notice the = sign is used as per our requirement.
{.metadata.name}={.spec.containers[*].image}{"\n"}

Retriving the pods name , the associated images and a label attribute

I know I can get the pods using:
kubectl get pods -n "namespace", and also to retrieve a json output
I'm trying to expand to get the pods name, the associated images, and a label attribute called 'base'. Also the date when I retrieve this information.
You can try this using yaml output.
kubectl get pods --all-namespaces -o yaml| egrep "name:|image:"
This will give you name of pod and image which is there for running the pod.
OR
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}"
This command will give you all images which are there in all pods.
If you found this is difficult then use,
kubectl get pod --all-namespaces
Check which pods image you need to find then use,
kubectl describe pod <pod_name> -n <namespace>
For reference use Link
Here you can find the description of kubectl get command.
What are you looking for is this:
output o Output format. One of:
json|yaml|wide|name|custom-columns=...|custom-columns-file=...|go-template=...|go-template-file=...|jsonpath=...|jsonpath-file=...
See custom columns
[http://kubernetes.io/docs/user-guide/kubectl-overview/#custom-columns],
golang template [http://golang.org/pkg/text/template/#pkg-overview]
and jsonpath template [http://kubernetes.io/docs/user-guide/jsonpath].
For example:
List a pod identified by type and name specified in "pod.yaml" in JSON
output format:
kubectl get -f pod.yaml -o json
Adjust by using the flags that you need from there.
Please let me know if that helped.
You can try jsonpath to retrieve the values for json output.
kubectl get po --all-namespaces -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.containers[0].image}{"\t"}{.metadata.labels.k8s-app}{"\n"}{end}'
probably you can write shell script and achieve this. first try to get all running pods across all namespaces using:
kubectl get pods -all-namespaces
and then iterate over each pod and execute following command:
kubectl describe pods <name of pod>
In the describe command you can get all information that you are looking for.

How to correctly export kubernetes resources?

I've created several resources using k8s Ansible module. Now I'd like to export the resources into Kubernetes manifests (so I don't need to use Ansible anymore). I've started by exporting a Service:
$ kubectl get svc myservice -o yaml --export > myservice.yaml
$ kubectl apply -f myservice.yaml
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
service/myservice configured
Why do I get the the warning? And why there's service/myservice configured and not service/myservice unchanged? Is there a better way to export resources?
You're doing right, don't worry about the warning.
If you want to get rid of it, you have to delete all the generation, selfLink and so on keys.
Yes, you are doing it right.
Let me show you small trick:
kubectl get svc myservice -o yaml --export | sed -e '/status:/d' -e '/creationTimestamp:/d' -e '/selfLink: [a-z0-9A-Z/]\+/d' -e '/resourceVersion: "[0-9]\+"/d' -e '/phase:/d' -e '/uid: [a-z0-9-]\+/d' > myservice.yaml
Will generate proper yaml file without status, creationTimestamp, selfLink, resourceVersion, phase and uid.
You get the warning because you're using kubectl apply on a resource that you previously created with kubectl create. If you create the initial Service with kubectl apply, you shouldn't get the warning.
The configured instead of unchanged might be because of some metadata or generated data that is also included in the output of kubectl get svc myservice -o yaml --export.

How do you cleanly list all the containers in a kubernetes pod?

I am looking to list all the containers in a pod in a script that gather's logs after running a test. kubectl describe pods -l k8s-app=kube-dns returns a lot of info, but I am just looking for a return like:
etcd
kube2sky
skydns
I don't see a simple way to format the describe output. Is there another command? (and I guess worst case there is always parsing the output of describe).
Answer
kubectl get pods POD_NAME_HERE -o jsonpath='{.spec.containers[*].name}'
Explanation
This gets the JSON object representing the pod. It then uses kubectl's JSONpath to extract the name of each container from the pod.
You can use get and choose one of the supported output template with the --output (-o) flag.
Take jsonpath for example,
kubectl get pods -l k8s-app=kube-dns -o jsonpath={.items[*].spec.containers[*].name} gives you etcd kube2sky skydns.
Other supported output output templates are go-template, go-template-file, jsonpath-file. See http://kubernetes.io/docs/user-guide/jsonpath/ for how to use jsonpath template. See https://golang.org/pkg/text/template/#pkg-overview for how to use go template.
Update: Check this doc for other example commands to list container images: https://kubernetes.io/docs/tasks/access-application-cluster/list-all-running-container-images/
Quick hack to avoid constructing the JSONpath query for a single pod:
$ kubectl logs mypod-123
a container name must be specified for pod mypod-123, choose one of: [etcd kubesky skydns]
I put some ideas together into the following:
Simple line:
kubectl get po -o jsonpath='{range .items[*]}{"pod: "}{.metadata.name}{"\n"}{range .spec.containers[*]}{"\tname: "}{.name}{"\n\timage: "}{.image}{"\n"}{end}'
Split (for readability):
kubectl get po -o jsonpath='
{range .items[*]}
{"pod: "}
{.metadata.name}
{"\n"}{range .spec.containers[*]}
{"\tname: "}
{.name}
{"\n\timage: "}
{.image}
{"\n"}
{end}'
How to list BOTH init and non-init containers for all pods
kubectl get pod -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name"
Output looks like this:
NAME INIT-CONTAINERS CONTAINERS
helm-install-traefik-sjts9 <none> helm
metrics-server-86cbb8457f-dkpqm <none> metrics-server
local-path-provisioner-5ff76fc89d-vjs6l <none> local-path-provisioner
coredns-6488c6fcc6-zp9gv <none> coredns
svclb-traefik-f5wwh <none> lb-port-80,lb-port-443
traefik-6f9cbd9bd4-pcbmz <none> traefik
dc-postgresql-0 init-chmod-data dc-postgresql
backend-5c4bf48d6f-7c8c6 wait-for-db backend
if you want a clear output of which containers are from each Pod
kubectl get po -l k8s-app=kube-dns \
-o=custom-columns=NAME:.metadata.name,CONTAINERS:.spec.containers[*].name
To get the output in the separate lines:
kubectl get pods POD_NAME_HERE -o jsonpath='{range .spec.containers[*]}{.name}{"\n"}{end}'
Output:
base-container
sidecar-0
sidecar-1
sidecar-2
If you use json as output format of kubectl get you get plenty details of a pod. With json processors like jq it is easy to select or filter for certain parts you are interested in.
To list the containers of a pod the jq query looks like this:
kubectl get --all-namespaces --selector k8s-app=kube-dns --output json pods \
| jq --raw-output '.items[].spec.containers[].name'
If you want to see all details regarding one specific container try something like this:
kubectl get --all-namespaces --selector k8s-app=kube-dns --output json pods \
| jq '.items[].spec.containers[] | select(.name=="etcd")'
Use below command:
kubectl get pods -o=custom-columns=PodName:.metadata.name,Containers:.spec.containers[*].name,Image:.spec.containers[*].image
To see verbose information along with configmaps of all containers in a particular pod, use this command:
kubectl describe pod/<pod name> -n <namespace name>
Use below command to see all the information of a particular pod
kubectl get pod <pod name> -n <namespace name> -o yaml
For overall details about the pod try following command to get the container details as well
kubectl describe pod <podname>
I use this to display image versions on the pods.
kubectl get pods -o=jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t"}{range .spec.containers[*]}{.image}{end}{end}' && printf '\n'
It's just a small modification of script from here, with adding new line to start next console command on the new line, removed commas at the end of each line and listing only my pods, without service pods (e.g. --all-namespaces option is removed).
There are enough answers here but sometimes you want to see a deployment object pods' containers and initContainers. To do that;
1- Retrieve the deployment name
kubectl get deployment
2- Retrieve containers' names
kubectl get deployment <deployment-name> -o jsonpath='{.spec.template.spec.containers[*].name}'
3- Retrieve initContainers' names
kubectl get deployment <deployment-name> -o jsonpath='{.spec.template.spec.initContainers[*].name}'
Easiest way to know the containers in a pod:
kubectl logs -c -n