I need to pass dynamic env variable to kubectl create. Something like this
kubectl create -f app.yaml --Target=prod
Based on Target code deploys on different servers.
If you want to avoid installing 3rd party plugin then you can replace the text using sed "s/orginal/change/". It worked. I used this in Jenkins shell.
cat app.yaml | sed "s/l3-apps/l2-apps/" | kubectl create -f -
You can achieve this in two ways:
Use Helm. It is a "package manager" for Kubernetes and is built exactly for your use case (dynamic variables to configure behaviour of your resources). If it is only a single variable, "converting" your deployment is as simple as creating a new Helm chart, copy your files into templates/, modify values.yaml and use {{ .Values.target }} in your templates. See the quickstart guide for a more in-depth introduction to Helm.
If you consider Helm to be over the top for a single variable, use kubectl's capability to read from standard input. You'll need an additional templating tool (for example mustache). Rewrite your deployment to fit your templating tool. Create a dynamic data.yml in your deployment process (e.g. a simple bash script that reads from environment variables) and run something like mustache data.yml deployment.mustache | kubectl apply -f -.
kubectl config set-context allows you to configure cluster, namespace, user credentials and more and save it as a "context" in your ~/.kube/config.
The you can use --context option of kubectl exactly in a way that you used --Target in your example.
I've published a command-line tool ysed that also performs what you need.
Related
I want to intercept the helm YAML and customize it using a Python script, and then install it. I have been doing something like helm template | python3 script... | kubectl apply -f - but of course this doesn't create a helm release in my cluster, so I lose out on helm rollback etc.
I have considered using Kustomize but it doesn't have the features that I'd like.
Is there a way to take pre-generated YAML, like that from helm template or helm install --dry-run and then install/upgrade that using helm?
Isn't that what post-renderers are for?
See https://helm.sh/docs/topics/advanced/#post-rendering
A post-renderer can be any executable that accepts rendered Kubernetes manifests on STDIN and returns valid Kubernetes manifests on STDOUT. It should return an non-0 exit code in the event of a failure. This is the only "API" between the two components. It allows for great flexibility in what you can do with your post-render process.
A post renderer can be used with install, upgrade, and template. To use a post-renderer, use the --post-renderer flag with a path to the renderer executable you wish to use:
$ helm install mychart stable/wordpress --post-renderer ./path/to/executable
I haven't used it myself yet, but it looks interesting if you want to run your own alternative kustomize.
See https://github.com/vmware-tanzu/carvel-ytt/tree/develop/examples/helm-ytt-post-renderer for an example that is not kustomize.
I have 'deployment.yaml' & 'service.yaml' files to deploy application in kubernetes (minikube).
To do a manual deployment I can use kubectl apply -f <file.yaml> to create resources but I am looking for a way to automate deployment using scripts. So I can trigger a script and all the needed yaml can be triggered and pods created accordingly.
Any suggestion or guidance will be helpful.
It sounds to me like you need some simple bash script where you put all commands you need
#!/bin/bash
NAMESPACE='mebbns'
DEPL_DIR='~/mebb_depl'
SVC_DIR='~/svc_depl'
kubectl create namespace $NAMESPACE
kubectl apply -n $NAMESPACE -f "$DEPL_DIR/deployment.yaml"
kubectl apply -n $NAMESPACE -f "$SVC_DIR/service.yaml"
kubectl get all -n $NAMESPACE
For more complex tasks you can check examples like setup-aks-vnet-acr.sh
kubectl Cheat Sheet should help you with basic commands you may require.
You can use a script to call kubectl for simple scenarious.
For more complex setups have a look at Argo cd https://argoproj.github.io/
There are many ways to automate your task. Any scripting language would be alright; such as Bash, Ansible, etc. If you are looking for sophisticated deployment tools, you can have a look at the common Kubernetes CI/CD tools
Kubernetes automation doesn't end with the CI/CD. When you have multiple applications with each of them has specific environment variables, security credentials, specific network and security requirements etc., then you will also need tools for organizing Kubernetes configuration, such as Helm and Kustomize.
I am setting up kubernetes for an application with 8 microservices,activemq,postgres,redis and mongodb.
After the entire configuration of pods and deployment ,is there any way to create a single master deployment yaml file which will create the entire set of services,replcas etc for the entire application.
Note:I will be using multiple deployment yaml files,statefulsets etc for all above mentioned services.
You can use this script:
NAMESPACE="your_namespace"
RESOURCES="configmap secret daemonset deployment service hpa"
for resource in ${RESOURCES};do
rsrcs=$(kubectl -n ${NAMESPACE} get -o json ${resource}|jq '.items[].metadata.name'|sed "s/\"//g")
for r in ${rsrcs};do
dir="${NAMESPACE}/${resource}"
mkdir -p "${dir}"
kubectl -n ${NAMESPACE} get -o yaml ${resource} ${r} > "${dir}/${r}.yaml"
done
done
Remember to specify what resources you want exported in the script.
More info here
Is there any way to create a single master deployment yaml file which will create the entire set of services,replicas etc for the entire application.
Since you already mentioned kubernetes-helm why don't you actually used it for that exact purpose? In short helm is sort of package manager for Kubernetes, some say similar to yum or apt. It deploys charts which you can actually refer to as packed application. Its pack of all your pre-configured applications which can be deploy as one unit. It's not entirely one file but more collection of files that build so called helm chart.
What are the helm charts?
Well they are basically K8s yaml manifest combined into a single package that can be installed to your cluster. And installing the package is just as simple as running single command such as helm install. Once done the charts are highly reusable which reduces the time for creating dev, test and prod environments.
As an example of a complex helm chart deploying multiple resources you many want to check Stackstorm.
Basically once deployed without any custom config this chart will deploy 2 replicas for each component of StackStorm as well as backends like RabbitMQ, MongoDB and Redis.
I have one kubernetes operator (ex: kubectl get oracle_ctrl). Now I want to provide custom arguments for the kubectl command.
ex: kubectl apply oracle_ctrl --auto-discover=true --name=vcn1
I can write one more controller to do the same job. But I don't want to write one more controller and make use of existing controller.
Is it possible to use operator-sdk to provide custom args to kubectl?
No, this isn't possible.
kubernetes/kubectl#914 has a little bit further discussion of this, but its essential description is "we should start the proposal and design process to eventually write something better than kubectl create to support it". Your CRD can define additional columns to be shown in kubectl get but this is really the only kubectl-related extension point. You could potentially create a kubectl plugin or another CLI tool that does what you need.
Rather than using the kubectl imperative tools, it's often a better practice to directly write YAML artifacts and commit them to source control. You can parameterize these using tools like Helm or Kustomize. If kubectl apply -f or helm install is your primary way of loading things into the cluster, then you don't need custom CLI options to make this work.
I would like to have a slightly different deployment configuration in different invironments. That is, in Prod and Ver, I don't want all containers to be deployed.
With docker-compose we solve that by having incremental docker-compose files that we combine, like: docker-compose up -f docker-compose.yml -f docker-compose-prod.yml
How can that be done using Helm charts?
We have a structure with Chart.yaml and values.yaml in the top, and then one yaml file per container in a subfolder. The naive solution would be to copy that structure and leave out some of the chart files, but I would prefer to have only one file (at most one file!) per service.
We deploy to AKS using CircleCI.
To summarize:
Today, each service has it's own yaml file, and on every deploy, all of them gets deployed. I want to configure my charts so that only a subset of the services gets deployed in certain environments.
EDIT:
kubectl has the the possibility to use selectors, like kubectl create cfg.yaml --selector=tier=frontend or kubectl create cfg.yaml --selector=environment=prod and I already tag my containers, so that would have been simple. But helm install does not have the possibility to accept a similar flag and pass it to kubectl.
just create one values file for each environment and target those:
helm install . -f values.production.yaml
helm install . -f values.development.yaml
you can use condition to toggle deployments, imagine you have something,yaml which you want conditionally deployed:
{{ if .Values.something}}
something.yaml original content goes here
{{ end }}