Dynamically create the configmap yaml file - kubernetes

Background : I have close to 15 *.properties files in different location. I need to create configmap for each properties files.
Currently I am manually creating configmap yaml files using
kubectl create configmap app-properties --from-file= /path/app.properties.
Mount and everything thing is working fine.
Requirement : As soon we add any new key/value to properties file, it should get reflected in configmap yaml file. Can I dynamically create configmap yaml using some "include files".

You could watch the properties files for modifications and recreate the ConfigMap whenever they change.
To do so, there are different tools for macOS and Linux.
Linux
On Linux, you can watch files for changes with inotifywait. You could do something along the following lines:
Create the file monitor.sh:
#!/bin/bash
FILE=$1
inotifywait -m -e modify "$FILE" |
while read; do
kubectl create configmap "$(basename $FILE)" --from-file="$FILE" --dry-run -o yaml | kubectl apply -f -
done
Then execute it for each properties file:
./monitor.sh /path/app.properties
This will generate an updated ConfigMap YAML manifest with kubectl create and apply it with kubectl apply every time the /path/app.properties file is modified.
You can install inotifywait with:
sudo apt-get install inotify-tools
macOS
On macOS, you can use fswatch to watch for file modifications.
Create the file monitor.sh:
#!/bin/bash
FILE=$1
fswatch "$FILE" |
while read; do
kubectl create configmap "$(basename $FILE)" --from-file="$FILE" --dry-run -o yaml | kubectl apply -f -
done
Then execute it for each properties file:
./monitor.sh /path/app.properties
This will generate an updated ConfigMap YAML manifest with kubectl create and apply it with kubectl apply every time the /path/app.properties file is modified.
You can install fswatch with:
brew install fswatch
Note
fswatch might also be available on Linux (sudo apt-get install fswatch), in which case you can use the monitor.sh script for macOS on Linux too. However, you might need to use fswatch -o (with the -o option) to ensure only a single output line.

Related

Deploy separate k8s manifest files

I have a Spring boot application and I deploy the application to Kubernetes using a single k8s.yml manifest file via Github actions. This k8s.yml manifest contains Secrets, Service, Ingress, Deployment configurations. I was able to deploy the application successfully as well. Now I plan to separate the Secrets, Service, Ingress, Deployment configurations into a separate file as secrets.yml, service.yml, ingress.yml and deployment.yml.
Previously I use the below command for deployment
kubectl: 1.5.4
command: |
sed -e 's/$SEC/${{ secrets.SEC }}/g' -e 's/$APP/${{ env.APP_NAME }}/g' -e 's/$ES/${{ env.ES }}/g' deployment.yml | kubectl apply -f -
Now after the separation I use the below commands
kubectl: 1.5.4
command: |
kubectl apply -f secrets.yml
kubectl apply -f service.yml
sed -e 's/$ES/${{ env.ES }}/g' ingress.yml | kubectl apply -f -
sed -e 's/$SEC/${{ secrets.SEC }}/g' -e 's/$APP/${{ env.APP_NAME }}/g' deployment.yml | kubectl apply -f -
But some how the application is not deploying correctly, I would like to know if the command which I am using is correct or not
You can consider perform the sed first, then apply all files kubectl apply -f . instead of going one by one. Append --recursive if you have files in sub folder to apply, too.
Like the other answer says, you can ask kubectl to apply all files recursively in a directory.
Now the sed replaces are soon going to become overwhelming as the resources and configuration grow.
That is why kubctl comes with integrated kustomize support:
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
In short:
Break up all kubernetes resources into smaller files/components, put them in a directory.
Place a file called kustomization.yaml in same directory.
In the kustomization.yaml configure which file you want to apply, in which order, and also do some on-the-fly edits.
And apply with -k flag:
kubectl apply -k <kustomization_directory>

no matches for kind "Kustomization" in version "kustomize.config.k8s.io/v1beta1"

Getting the below error for the command kubectl apply -n prod -f kustomize/kustomization.yaml
error: unable to recognize "kustomize/kustomization.yaml": no matches for kind "Kustomization" in version "kustomize.config.k8s.io/v1beta1"
Please advise.
Firstly I recommend to read official doc: kubernetes-kustomization.
To solve the problem use -k flag instead of -f flag in command:
$ kubectl apply -k <kustomization_directory>
If in your kustomize directory will be only one manifest file (kustomization.yaml) then use $ kubectl apply -k kustomize/ from this directory. Otherwise create new empty directory and put your kustomization.yaml there then execute following command from parent directory $ kubectl apply -k new-directory/
Take a look: kustomize-no-matches, kustomize-kubernetes-no-matches.

How to replace JSON value in kubectl output using go-template?

I have a configMap and I want to create a backup configMap by using the last applied configuration from that.
I use the following command to get the last applied configuration:
kubectl get cm app-map -n app-space \
-o go-template \
--template='{{index .metadata "annotations" "kubectl.kubernetes.io/last-applied-configuration"}}' > backup.json
It returns something like this [the content of backup.json]:
{"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"app-map","creationTimestamp":null},"data":{"app.yml":"xxxxxxxxx","config.yml":"yyyyyyyyy"}}
Now, I want my backup configMap to have a different name. So, I want to change the .metadata.name from app-map to app-map-backup.
Is there a way I can achieve that with kubectl and -o go-template? I want to have the name changed before I write it to the backup.json file.
I know I can do that using jq but I do not have permission to install jq on the server where I am using kubectl.
you could use kubectl bulk plugin. The below command will replicate your config map
# get resource(s) and create with field(name) change
kubectl bulk configmap app-map -n app-space create name app-mapp-backup
Kubectl bulk is very powerful to use, I suggest to check samples.
You cannot do this just using kubectl. But there are other ways.
You can download statically linked jq binary from official jq website:
wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
chmod +x jq-linux64
and then you can use this binary like following:
kubectl -o go-template [...] | ./jq-linux64 ...
or you can use sed:
kubectl -o go-template [...] | sed 's/"name":"app-map"/"name":"app-map-backup"/'

How to merge kubectl config file with ~/.kube/config?

Is there a simple kubectl command to take a kubeconfig file (that contains a cluster+context+user) and merge it into the ~/.kube/config file as an additional context?
Do this:
export KUBECONFIG=~/.kube/config:~/someotherconfig
kubectl config view --flatten
You can then pipe that out to a new file if needed.
If you find yourself doing this a lot... There is now also the krew plugin package manager for kubectl.
The krew plugin "konfig" can help you manage your ~/.kube/config file.
Using the konfig plugin the syntax will be:
kubectl konfig import -s new.yaml
To install krew: https://github.com/kubernetes-sigs/krew
To install konfig: kubectl krew install konfig
Using multiple kubeconfigs at once
Sometimes you have a bunch of small kubeconfig files (e.g. one per cluster) but you want to use them all at once, with tools like kubectl or kubectx that work with multiple contexts at once.
To do that, you need a “merged” kubeconfig file. In the section "Merging kubeconfig files" below, we explain how you can merge the kubeconfigs into a single file, but you can also merge them in-memory.
By specifying multiple files in KUBECONFIG environment variable, you can temporarily stitch kubeconfig files together and use them all in kubectl .
#
# Kubeconfig in-memory merge
#
export KUBECONFIG=file1:file2
kubectl get pods --context=cluster-1
kubectl get pods --context=cluster-2
#
# For your example
# merging your kubeconfig file w/ $HOME/.kube/config (w/ cp backup)
#
cp $HOME/.kube/config $HOME/.kube/config.backup.$(date +%Y-%m-%d.%H:%M:%S)
KUBECONFIG= $HOME/.kube/config:file2: kubectl config view --merge --flatten > \
~/.kube/merged_kubeconfig && mv ~/.kube/merged_kubeconfig ~/.kube/config
kubectl get pods --context=cluster-1
kubectl get pods --context=cluster-2
Merging kubeconfig files
Since kubeconfig files are structured YAML files, you can’t just append them to get one big kubeconfig file, but kubectl can help you merge these files:
#
# Merging your kubeconfig file w/ $HOME/.kube/config (w/ cp backup)
#
cp $HOME/.kube/config $HOME/.kube/config.backup.$(date +%Y-%m-%d.%H:%M:%S)
KUBECONFIG=$HOME/.kube/config:file2:file3 kubectl config view --merge --flatten > \
~/.kube/merged_kubeconfig && mv ~/.kube/merged_kubeconfig ~/.kube/config
kubectl get pods --context=cluster-1
kubectl get pods --context=cluster-2
Extracting a context from a kubeconfig file
Let’s say you followed the before merging kubeconfig files and have a merged kubeconfig file in $HOME/.kube/config. Now you want to extract a cluster’s information to a portable kubeconfig file that only has the parts you need to connect to that cluster.
Run:
KUBECONFIG=$HOME/.kube/config kubectl config view \
--minify --flatten --context=context-1 > $HOME/.kube/config-context-1
#
# using --kubeconfig flag
#
kubectl get pods --kubeconfig=$HOME/.kube/config-context-1
#
# or
# using `KUBECONFIG` environment variable
#
KUBECONFIG=$HOME/.kube/config-context-1 kubectl get pods
#
# or
# keep using kubeconfig file at $HOME/.kube/config (which has the merged context)
#
kubectl get pods --context=cluster-1
In this command, we extract data about context-1 from $HOME/.kube/config to config-context-1 file. The --minify flag allows us to extract only info about that context, and the --flatten flag allows us to keep the credentials unredacted.
ref article: https://ahmet.im/blog/mastering-kubeconfig/
If you want to merge two config files in a single one
I found this way (not sure if this is the simplest)
# Add the two config files to the env var
export KUBECONFIG=~/.kube/config:~/Desktop/configFile2.yaml
# Review that you have two configurations in one view
kubectl config view
# View the raw config and output to a new file
kubectl config view --raw > /tmp/config
Then copy the new config file where you want, also do not forget to unset KUBECONFIG the env variable
It's possible, follow the steps:
create a backup from your config file:
cp ~/.kube/config-bkp
create a file with your new config file:
vi ~/.kube/new-config
merge them into config:
KUBECONFIG=~/.kube/config:~/.kube/new-config kubectl config view --flatten > ~/.kube/config
To see the available contexts use:
kubectl config get-contexts
To change the context use:
kubectl config use-context YOUR-CONTEXT-NAME
You can follow these instruction if you want to have some structure in your ~/.kube directory.
Add the following snippet to your ~/.bashrc
Add config files under ~/.kube/config.d separately.
call update_kubeconfigs, or open new terminal
update_kubeconfigs just looks at ~/.kube/config.d directory and if there were any files newer than the current config file under ~/.kube/config, it updates it.
function update_kubeconfigs() {
[ ! -d "$HOME/.kube/config.d" ] && mkdir $HOME/.kube/config.d -p -v
# Will run only if there are new files in the config directory
local new_files=$(find $HOME/.kube/config.d/ -newer $HOME/.kube/config -type f | wc -l)
if [[ $new_files -ne "0" ]]; then
local current_context=$(kubectl config current-context) # Save last context
local kubeconfigfile="$HOME/.kube/config" # New config file
cp -a $kubeconfigfile "${kubeconfigfile}_$(date +"%Y%m%d%H%M%S")" # Backup
local kubeconfig_files="$kubeconfigfile:$(ls $HOME/.kube/config.d/* | tr '\n' ':')"
KUBECONFIG=$kubeconfig_files kubectl config view --merge --flatten > "$HOME/.kube/tmp"
mv "$HOME/.kube/tmp" $kubeconfigfile && chmod 600 $kubeconfigfile
export KUBECONFIG=$kubeconfigfile
kubectl config use-context $current_context --namespace=default
fi
}
# This will call each time you source .bashrc, remove it if you want to call it manually each time
update_kubeconfigs
Removing files form ~/.kube/confing.d will not invoke the script again. Also as #rafaelrezend pointed out, checkout for name conflicts in the config files, that might cause issues.
Github gist includes a fix for updating credentials
Since I have many Kubernetes conf files in the ~/.kube directory, I simply chain them tot the KUBECONFIG env variable in the ~/.zshrc file:
export KUBECONFIG=$HOME/.kube/config
for conf in ~/.kube/*.conf; do
export KUBECONFIG=$KUBECONFIG:$conf
done
Going forward, I do not recommend merging kubeconfig files using kubectl.
it is a manual effort as seen above (set environment variables etc.)
has the disadvantage that when using this context via "kubectl
config use-context" it WRITES the current context to the kubeconfig file. Thus,
this influences other terminal sessions that use a context from the same
kubeconfig file (they all suddenly point to the same context).
Instead, I would recommend using a tool that circumvents such issues by e.g recursively search & displays all available contexts for kubeconfig files and works on a temporary copy.
Checkout kubeswitch (the tool I wrote to deal with > 1000 kubeconfig files) and this section explaining how it works.
If you look for a tool that also does namespace switching and other related things, take a look at "kubie".
As mentioned in the comment by #talarczykco on the top answer, piping back to the same ~/.kube/config will only write the second file and you will lose the original content!
Here is a safer way to first capture the full output then pipe.
Note must surround the variable kubeconfig with " otherwise you lose all newlines!
konfig=$(KUBECONFIG=~/.kube/config:new-config.yaml kubectl config view --flatten)
echo "$konfig" > ~/.kube/config
If you prefer a CLI tool I can highly recommend: KubeCM, which is also able to merge, switch, add...
kubecm add -f ./your_new_config
You would be asked either to merge into ~/.kube/config or create a .yml file in your current folder.
https://github.com/sunny0826/kubecm
To dynamically merge multiple config files in you .bashrc:
export KUBECONFIG=/Users/<user>/.kube/config:/Users/<user>/.kube/other.config
source <(kubectl completion bash)
After fresh source, verify:
kubectl config view
If you use bash you can use this to simply add the configs:
function kmerge() {
DATE=$(date +"%Y%m%d%H%M")
KUBECONFIG=~/.kube/config:$1 kubectl config view --flatten > ~/.kube/mergedkub && mv ~/.kube/config ~/.kube/config-$DATE && mv ~/.kube/mergedkub ~/.kube/config
}
Then just use "kmerge $newConfigfile" to add this.
Be aware the clusternames etc. should be different from existing config entries!
I keep the yaml files for each cluster separate and then combine them with this python script:
import argparse
import yaml
parser = argparse.ArgumentParser()
parser.add_argument('files', metavar='YAMLFILES', type=argparse.FileType('r'), nargs='*')
args = parser.parse_args()
y = {'apiVersion': 'v1', 'kind': 'Config', 'clusters': [],'contexts': [],
'current-context': None, 'preferences': {}, 'users': []}
for a in args.files:
f = yaml.load(a, Loader=yaml.Loader)
y['clusters'].append(f['clusters'][0])
y['contexts'].append(f['contexts'][0])
y['users'].append(f['users'][0])
y['current-context'] = f['contexts'][0]['name']
print(yaml.dump(y, Dumper=yaml.Dumper))
Zsh users can use a =(...) process substitution to generate a temporary merged configuration file, and copy it to ~/.kube/config all in one line:
cp =(KUBECONFIG=~/.kube/config:~/.kube/config.other kubectl config view --flatten) ~/.kube/config
Going a step further, we can hold the configuration file in our system clipboard and use a nested process substitution that reads it. Just make sure that the clipboard content is the actual configuration file before pressing Enter:
cp =(KUBECONFIG=~/.kube/config:<(pbpaste) kubectl config view --flatten) ~/.kube/config

How to configure kubectl with cluster information from a .conf file?

I have an admin.conf file containing info about a cluster, so that the following command works fine:
kubectl --kubeconfig ./admin.conf get nodes
How can I config kubectl to use the cluster, user and authentication from this file as default in one command? I only see separate set-cluster, set-credentials, set-context, use-context etc. I want to get the same output when I simply run:
kubectl get nodes
Here are the official documentation for how to configure kubectl
http://kubernetes.io/docs/user-guide/kubeconfig-file/
You have a few options, specifically to this question, you can just copy your admin.conf to ~/.kube/config
The best way I've found was to use an environment variable:
export KUBECONFIG=/path/to/admin.conf
I just alias the kubectl command into separate ones for my dev and production environments via .bashrc
alias k8='kubectl'
alias k8prd='kubectl --kubeconfig ~/.kube/config_prd.conf'
I prefer this method as it requires me to define the environment for each command.. whereas using an environment variable could potentially lead you to running a command within the wrong environment
Before answers have been very solid and informative, I will try to add
my 2 cents here
Configure kubeconfig file knowing its precedence
If you’re using kubectl, here’s the preference that takes effect while determining which kubeconfig file is used.
use --kubeconfig flag, if specified
use KUBECONFIG environment variable, if specified
use $HOME/.kube/config file
With this, you can easily override kubeconfig file you use per the kubectl command:
#
# using --kubeconfig flag
#
kubectl get pods --kubeconfig=file1
kubectl get pods --kubeconfig=file2
#
# or
# using `KUBECONFIG` environment variable
#
KUBECONFIG=file1 kubectl get pods
KUBECONFIG=file2 kubectl get pods
#
# or
# merging your kubeconfig file w/ $HOME/.kube/config (w/ cp backup)
#
cp $HOME/.kube/config $HOME/.kube/config.backup.$(date +%Y-%m-%d.%H:%M:%S)
KUBECONFIG= $HOME/.kube/config:file2:file3 kubectl config view --merge --flatten > \
~/.kube/merged_kubeconfig && mv ~/.kube/merged_kubeconfig ~/.kube/config
kubectl get pods --context=cluster-1
kubectl get pods --context=cluster-2
NOTE: The --minify flag allows us to extract only info about that context, and the --flatten flag allows us to keep the credentials unredacted.
For your example
kubectl get pods --kubeconfig=/path/to/admin.conf
#
# or:
#
KUBECONFIG=/path/to/admin.conf kubectl get pods
#
# or:
#
cp $HOME/.kube/config $HOME/.kube/config.backup.$(date)
KUBECONFIG= $HOME/.kube/config:/path/to/admin.conf kubectl config view --merge --flatten > \
~/.kube/merged_kubeconfig && mv ~/.kube/merged_kubeconfig ~/.kube/config
kubectl get pods --context=cluster-1
kubectl get pods --context=cluster-2
Although this precedence list not officially specified in the documentation it is codified here. If you’re developing client tools for Kubernetes, you should consider using cli-runtime library which will bring the standard --kubeconfig flag and $KUBECONFIG detection to your program.
ref article: https://ahmet.im/blog/mastering-kubeconfig/
I name all cluster configs as .kubeconfig and this lives in project directory.
Then in .bashrc or .bash_profile I have the following export:
export KUBECONFIG=.kubeconfig:$HOME/.kube/config
This way when I'm in the project directory kubectl will load local .kubeconfig.
Hope that helps
kubectl uses ~/.kube/config as the default configuration file. So you could just copy your admin.conf over it.
Because there is no built-in kubectl config merge command at the moment (follow this) you can add this function to your .bashrc (or .zshrc):
function kmerge() {
if [ $# -eq 0 ]
then
echo "Please pass the location of the kubeconfig you wish to merge"
fi
KUBECONFIG=~/.kube/config:$1 kubectl config view --flatten > ~/.kube/mergedkub && mv ~/.kube/mergedkub ~/.kube/config
}
Then you can just run from termial:
kmerge /path/to/admin.conf
and the config file will be merged to ~/.kube/config.
You can now switch to the new context with:
kubectl config use-context <new-context-name>
Or if you're using kubectx (recommended) you can run: kubectx <new-context-name>.
(The kmerge function is based on #MichaelSp answer at this post).
Kubernetes keeps the path to search for config files in $KUBECONFIG
If you want to add one more config path on top of the existing KUBECONFIG without overriding it (and keeping ~/.kube/config as the default path to search).
Just run the following each time you want to add a conf file to the KUBECONFIG path
export KUBECONFIG=${KUBECONFIG:-~/.kube/config}:/path/to/admin.conf
You can check it worked by listing the available contexts
kubectl config get-contexts
Then select the one you want to use
kubectl config use-context <context-name>
Manage your config files proper,place below in your profile file, source the .profile / .bash_profile
for kconfig in $HOME/.kube/config $(find $HOME/.kube/ -iname "*.config")
do
if [ -f "$kconfig" ];then
export KUBECONFIG=$KUBECONFIG:$kconfig
fi
done
switch the contexts from kubectl
When you type kubectl I guess you prefer to know which cluster you are pointing. Maybe it's worth creating an alias for that?
alias kube-mycluster='kubectl --kubeconfig ~/.kube/mycluster.conf'
This is possible:
export KUBECONFIG=~/.kube/config:~/.kube/cluster0:~/.kube/cluster1:~/.kube/cluster3
and:
kubectl config use-context cluster0