I have a file test.txt
---
kind: ClusterRole
metadata:
kind: ClusterRoleBinding
metadata:
subjects:
roleRef:
kind: ClusterRole
name: bla
And need this output with sed (just the line that matches the exact pattern)
kind: ClusterRole
metadata:
sed -n '/kind: ClusterRole/,/metadata/p' test.txt
kind: ClusterRole
metadata:
kind: ClusterRoleBinding
metadata:
kind: ClusterRole
name: bla
is showing ClusterRole, ClusterRoleBinding and additional indented ClusterRole
sed -n '/kind: ClusterRole\b/,/metadata/p' test.txt
sed -n '/\<kind: ClusterRole\>/,/metadata/p' test.txt
Both of the above output to nothing what am I doing wrong?
just so it's clear and I'm not getting any grep -B 2 suggestions ;-) this is an example file, the original file is a lot bigger and has hundreds of ClusterRoles so I need to figure out how to match the exact pattern.
Thank you!
lema
You should include start-of-line (^) and end-of-line ($) in your pattern.
If you just need to search the exact multiple lines, then you can achieve it using pcregrep
pcregrep -M 'kind: ClusterRole(\n)metadata:' test.txt
where -M, --multiline allow patterns to match more than one line.
More details can be found in this SO post.
Edit:
You can also use grep
grep -Pazo 'kind: ClusterRole\nmetadata:' test.txt
Reference: this SO post
This might work for you (GNU sed):
sed -n 'N;/^kind: ClusterRole\nmetadata:$/p;D' file
Append the next line and print the lines if it matches. Delete the first line and repeat.
If you only want the first match, use:
sed -n 'N;/^kind: ClusterRole\nmetadata:$/{p;q};D' file
Related
When using Kubernetes .yml files, I can do the following:
$ cat configmap.yml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
foo: ${FOO}
bar: ${BAR}
static: doesNotChange
$ export FOO=myFooVal
$ export BAR=myBarVal
$ cat configmap.yml | envsubst | kubectl apply -f -
This would replace ${FOO} and ${BAR} in the configmap.yml file before actually applying the file to the cluster.
How could I achieve the very same behavior with a Kubernetes secret which has it's data values base64 encoded?
I would need to read all the keys in the data: field, decode the values, apply the environment variables and encode it again.
A tool to decode and encode the data: values inplace would be much appreciated.
It is actually possible, to store the secret.yml with stringData instead of data which allows to keep the files in plain text (SOPS encryption is still possible and encouraged)
$ cat secret.yml
apiVersion: v1
kind: Secret
metadata:
name: test-secret
namespace: default
type: Opaque
stringData:
dotenv: |
DATABASE_URL="postgresql://test:test#localhost:5432/test?schema=public"
API_PORT=${PORT}
FOO=${FOO}
BAR=${BAR}
$ export PORT=80
$ export FOO=myFooValue
$ export BAR=myBarValue
$ cat secret.yml | envsubst | kubectl apply -f -
A plus is for sure, that this not only allows for creation of the secret, but updating is also possible.
Just for documentation, here would be the full call with SOPS:
$ sops --decrypt secret.enc.yml | envsubst | kubectl apply -f -
I am trying to generate yaml output with kubectl --dry-run.
When I run this
$ kubectl create configmap my-config-map -o yaml --dry-run=client | kubectl annotate -f- --dry-run=client -o yaml --local this-is-my-target-directory='{{ template "This.is.long.path.for.some.value" . }}'
I get the following output where annotations are splitting into 2 lines.
apiVersion: v1
kind: ConfigMap
metadata:
annotations:
this-is-my-target-directory: '{{ template "This.is.long.path.for.some.value" .
}}'
creationTimestamp: null
name: my-config-map
my desired output is that annotations should be in one line. like below
apiVersion: v1
kind: ConfigMap
metadata:
annotations:
this-is-my-target-directory: '{{ template "This.is.long.path.for.some.value" . }}'
creationTimestamp: null
name: my-config-map
If I reduce the size of the string then it becomes 1 line. I could not find anywhere in the documentation anything about line length. Can anyone guide me on how to fix this?
I think you can't find anything because this is a totally valid yaml. So I guess you can use it as it's without the need to put the curly brackets at the same line.
I have a file with three configmaps in it, like the one below.
apiVersion: v1
data:
TEST: "one"
kind: ConfigMap
metadata:
name: test-config-one
---
apiVersion: v1
data:
TEST: "two"
kind: ConfigMap
metadata:
name: test-config-two
---
apiVersion: v1
data:
TEST: "three"
kind: ConfigMap
metadata:
name: test-config-three
I'm trying to apply only test-config-three to the cluster. I know I can break that out into its own file and run kubectl apply -f test-config-three.yaml, but is there a way to do that without having to create a new file?
I was hoping to be able to do something like:
cat file.yml | yq <get only test-config-three> | kubectl apply -f -
But yq doesn't seem to support finding a single resource in a file. I also looked at tools like kubesplit but they tend to output all resources to separate files.
Is there a way to isolate and output a single resource from a yaml file containing multiple resources without creating a new file?
Update
Thanks to #Inian's answer below, I was able to get this full command working.
cat file.yml | yq e 'select(.data.TEST == "three")' - | kubectl apply -f -
There are two versions of yq implemented, one in Python and one in Go as I've highlighted in my answer at How can I parse a YAML file from a Linux shell script?
Using the Python version - kislyuk/yq
yq -y 'select(.data.TEST == "three")' yaml
Go version - mikefarah/yq
yq e 'select(.data.TEST == "three")' yaml
If you have python you can try following
export MYFILE=file.yml
export DOC_NUMBER=2
python3 -c "import yaml; print(yaml.dump(list(yaml.safe_load_all(open('"$MYFILE"')))["$DOC_NUMBER"]))" | kubectl apply -f -
yaml.safe_load_all(open('"$MYFILE"')) loads all the documents in the yaml file into a list. Then you are selecting "$DOC_NUMBER" document with ["$DOC_NUMBER"].
yaml.dump would dump the loaded object back into yaml format which is then printed.
I have a text file with the below text on multiple places:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: harbor-exporter
labels:
release: mgmt
I want to have:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: harbor-exporter
namespace: test
labels:
release: mgmt
Because this text is present on multiple places in the file, I do need to search for:
metadata:\n name:harbor-exporter and append the line namespace: test. Next to that, the metadata: should be present at the beginning of the line. What is the best way of achieving this with sed?
I have tried:
cat harbor-exporter.yaml | sed ':a;N;$!ba;s|metadata:\n name: harbor-exporter|NewString|g'
But I do need the word metadata: at the start of a line. So I put a ^ in front of metadata: and that failing.
Command which is failing:
cat harbor-exporter.yaml | sed ':a;N;$!ba;s|^metadata:\n name: harbor-exporter|NewString|g'
Thanks
This might work for you (GNU sed):
sed -E ':a;/metadata:/{n;/name: harbor-export/!ba;p;s/\S.*/namespace: test/}' file
Match on a line containing metadata, print it and fetch the next line. If that line does not contain name: harbor-export, start again. Otherwise, print that line and replace it with namespace:test.
Tested on GNU sed, not sure if this requires changes on other implementations: If metadata: cannot occur in consecutive lines, the logic can be simplified:
sed '/^metadata:$/{n; s/^ name: harbor-exporter$/&\n namespace: test/}'
/^metadata:$/ if whole line content is metadata:
n fetch next line (pattern space is replace with this new line)
s/^ name: harbor-exporter$/&\n namespace: test/ add required line if the next line so fetched has the content name: harbor-exporter
Also, you might be better off with a tool that understands YAML such as https://github.com/TomWright/dasel
How can I get an ApiVersion for some resource kind, using the kubectl.
Example:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1 ( complete string )
I tried kubectl api-resources, and kubectl api-versions but could not find some complete mapping. Is there any way to concatenate the output of those commands and get the complete string for each resource kind? Or maybe there is some other command.
I'm not aware of a kubectl command that can provide mapping between all resources in a cluster - built-in and custom, used or not - and their API Groups and group versions (apiVersion: $GROUP_NAME/$VERSION).
If using curl and jq is an option, the following one-liner will provide such mapping:
for v in `curl -ks https://<k8s-master>:<port>/apis | jq -r .groups[].versions[].groupVersion`; do for r in `curl -ks "https://<k8s-master>:<port>/apis/${v}" | jq -r '.resources[]?.kind' | sort -u`; do echo ${r} - ${v}; done ; done
Few explanations:
replace <k8s-master>:<port> with the name/IP and port of the Kubernetes API
Server master.
break it into multiple lines for better readability.
-k in the curl command is to trust any cert.
-s in the curl command is for silent output. E.g. no progress output.
-r in the jq command is for raw output. E.g. no double quotes will be output for strings.
change echo ${r} - ${v} in a way to output the mapping as desired.
Notice that the above doesn't handle api/v1. It is a legacy API Group and its resources are now also under named groups - see https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups
Partial output for a Kubernetes cluster:
APIService - apiregistration.k8s.io/v1
APIService - apiregistration.k8s.io/v1beta1
DaemonSet - extensions/v1beta1
Deployment - extensions/v1beta1
DeploymentRollback - extensions/v1beta1
...
Role - rbac.authorization.k8s.io/v1
RoleBinding - rbac.authorization.k8s.io/v1
ClusterRole - rbac.authorization.k8s.io/v1beta1
ClusterRoleBinding - rbac.authorization.k8s.io/v1beta1
Role - rbac.authorization.k8s.io/v1beta1
RoleBinding - rbac.authorization.k8s.io/v1beta1
...
You can try
kubectl get roles --all-namespaces -o jsonpath='{.items[*].apiVersion}'