We would like to have some recommendations, since we want to integrate helmfile in our deployment process...
Our infrastructure has following details:
we have many customers
all customers have the same installed services
(each customer get's it's own services, no sharing between customers)
credentials are different for each customer
we prefer a seperate
deployment process (we dont want to upgrade all customers at the same
time)
all customer-config data is seperated into seperate config
files, like:
config/customer1.yaml
config/customer2.yaml
config/customer3.yaml
So I'm wondering, if we should use "Environment" with the customer name, to upgrade it.. or would you recommend another variable?
And do you think it's better to create multiple helmfiles for this process, or just one?
Thank you!
do you think it's better to create multiple helmfiles for this process, or just one?
Using one helmfile for multiple environemnts is quite practical and it saves you writing multiple helmfiles.
we should use "Environment" with the customer name?
For a similar setup (deploying to multiple environements with different values and configurations), I have in Helmfile:
- name: my-app
namespace: "{{ .Namespace }}"
chart: k4r-distance-matrix-api
values:
- my-app/values.yaml ## here go the common values if any exists
- my-app/values.{{ .Environment.Name }}.yaml ## here goes the environment specific values
In the deploy step in my CI I have:
.deploy:
stage: deploy
variables:
ENVIRONMENT: ""
CONTEXT: ""
NAMESPACE: ""
before_script:
- kubectl config use-context $CONTEXT
script:
- helmfile -e "$ENVIRONMENT" --namespace "$NAMESPACE" sync
Related
I have 2 Kubernetes instances:
Production
Testing
Both with a gitlab runner for CI/CD pipelines.
Since some jobs are only for production and others only for testing I tagged the runners(in values.yaml).
helm get values gitlab-runner -n gitlab-runner for the testing runner shows this:
USER-SUPPLIED VALUES:
gitlabUrl: https://...
runnerRegistrationToken: ...
tags: testing
This is not working however and I have to manually set the tag in the UI (Group > CI/CD > Runners).
The problem with this is that the servers frequently reboot, which resets tags requiring a lot of manual upkeep.
Why is the setting in values.yaml not working? And is there a way to set tags which persist after reboots?
I am testing role differences right now so I have a context for each role setup.
Terminal session Admin, I want to be able to use context Admin in one session so I can update the rules as needed.
In terminal session User, I want to be able to test that role via its context.
(Note: I am on EKS so roles map to IAM roles)
Well, I am an idiot.
Natively, there is no answer in the --help output for kubectl; however, there is output for this in the man page.
All one has to do is throw the --context flag into their command.
However, the below-mentioned kubectx tool is what I use day to day now.
Here are some tips for managing multiple kubectl contexts:
Use asdf to manage multiple kubectl versions
Set the KUBECONFIG env var to change between multiple kubeconfig files
Use kube-ps1 to keep track of your current context/namespace
Use kubectx and kubens to change fast between clusters/namespaces
Use aliases to combine them all together
Take a look at this article, it explains how to accomplish this: Using different kubectl versions with multiple Kubernetes clusters (Disclaimer: I wrote the mentioned article)
I also recommend this reads: Mastering the KUBECONFIG file and Configure Access to Multiple Clusters
Now, there is kubie.
https://github.com/sbstp/kubie
it does it all.
You can create a copy of your context file that is located under ~/.kube/config, and in 2 different shells, point to 2 different config files using export KUBECONFIG=/path/to/kubeconfig1 on the first and export KUBECONFIG=/path/to/kubeconfig2 on the second. You can edit those files to have 2 different context selected.
To easily select contexts/switch between them, you can use kubectx, as suggested by Blokje5.
I always like kubectx as a way to quickly switch context. If you correctly setup your contexts with the aws-iam-authenticator, like so:
users:
- name: kubernetes-admin
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "<cluster_id>"
- "-r"
- "<admin_role_arn>"
- name: kubernetes-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "<cluster_id>"
- "-r"
- "<user_role_arn>"
This should allow you to easily switch contexts. (Note: This assumes an assume-role type situation. You can also pass AWS_PROFILE to the aws-iam-authenticator instead.)
I am trying to setup a module to deploy resources in the cloud (it could be any cloud provider). I don't see the advantages of using templates (ie. the deploy manager) over direct API calls :
Creation of VM using a template :
# deployment.yaml
resources:
- type: compute.v1.instance
name: quickstart-deployment-vm
properties:
zone: us-central1-f
machineType: f1-micro
...
# bash command to deploy yaml file
gcloud deployment-manager deployments create vm-deploy --config deployment.yaml
Creation of VM using a API call :
def addInstance(http, listOfHeaders):
url = "https://www.googleapis.com/compute/v1/projects/[PROJECT_ID]/zones/[ZONE]/instances"
body = {
"name": "quickstart-deployment-vm",
"zone": " us-central1-f",
"machineType": "f1-micro",
...
}]
bodyContentURLEncoded = urllib.urlencode(bodyContent)
http.request(uri=url, method="POST", body=body)
Can someone explain to me what benefits I get using templates?
readability\easy of use\authentication handled for you\no need to be a coder\etc. There can be many advantages, it really depends on how you look at it. It depends on your background\tools you use.
It might be more beneficial to use python all the way for you specifically.
It's easier to use templates and you get a lot of builtin functionality such as running a validation on your template to scan for possible security vulnerabilities and similar. You can also easily delete your infra using the same template as you create it. FWIW, I've gone all the way with templates and do as much as I can with templates and in smaller units. It makes it easy to move out a part of the infra or duplicate it to another project, using a pipeline in GitLab to deploy it for example.
The reason to use templates over API calls is that templates can be used in use cases where a deterministic outcome is required.
Both Template and API call has its own benefits. There is always a tradeoff between the two options. If you want more flexibility in the deployment, then the API call suits you better. On the other hand, if the security and complete revision is your priority, then Template should be your choice. Details can be found in this online documentation.
When using a template, orchestration of the deployment is handled by the platform. When using API calls (or other imperative approaches) you need to handle orchestration.
It's not clear for me from the documentation if it's even possible to pass one job's output to the another job (not from task to task, but from job to job).
I don't know if conceptually I'm doing the right thing, maybe it should be modeled differently in Concourse, but what I'm trying to achieve is having pipeline for Java project split into several granular jobs, which can be executed in parallel, and triggered independently if I need to re-run some job.
How I see the pipeline:
First job:
pulls the code from github repo
builds the project with maven
deploys artifacts to the maven repository (mvn deploy)
updates SNAPSHOT versions of the Maven project submodules
copies artifacts (jar files) to the output directory (output of the task)
Second job:
picks up jar's from the output
builds docker containers for all of them (in parallel)
Pipeline goes on
I was unable to pass the output from job 1 to job 2.
Also, I am curious if any changes I introduce to the original git repo resource will be present in the next job (from job 1 to job 2).
So the questions are:
What is a proper way to pass build state from job to job (I know, jobs might get scheduled on different nodes, and definitely in different containers)?
Is it necessary to store the state in a resource (say, S3/git)?
Is the Concourse stateless by design (in this context)?
Where's the best place to get more info? I've tried the manual, it's just not that detailed.
What I've found so far:
outputs are not passed from job to job
Any changes to the resource (put to the github repo) are fetched in the next job, but changes in working copy are not
Minimal example (it fails if commented lines are uncommented with error: missing inputs: gist-upd, gist-out):
---
resources:
- name: gist
type: git
source:
uri: "git#bitbucket.org:snippets/foo/bar.git"
branch: master
private_key: {{private_git_key}}
jobs:
- name: update
plan:
- get: gist
trigger: true
- task: update-gist
config:
platform: linux
image_resource:
type: docker-image
source: {repository: concourse/bosh-cli}
inputs:
- name: gist
outputs:
- name: gist-upd
- name: gist-out
run:
path: sh
args:
- -exc
- |
git config --global user.email "nobody#concourse.ci"
git config --global user.name "Concourse"
git clone gist gist-upd
cd gist-upd
echo `date` > test
git commit -am "upd"
cd ../gist
echo "foo" > test
cd ../gist-out
echo "out" > test
- put: gist
params: {repository: gist-upd}
- name: fetch-updated
plan:
- get: gist
passed: [update]
trigger: true
- task: check-gist
config:
platform: linux
image_resource:
type: docker-image
source: {repository: alpine}
inputs:
- name: gist
#- name: gist-upd
#- name: gist-out
run:
path: sh
args:
- -exc
- |
ls -l gist
cat gist/test
#ls -l gist-upd
#cat gist-upd/test
#ls -l gist-out
#cat gist-out/test
To answer your questions one by one.
All build state needs to be passed from job to job in the form of a resource which must be stored on some sort of external store.
It is necessary to store on some sort of external store. Each resource type handles this upload and download itself, so for your specific case I would check out this maven custom resource type, which seems to do what you want it to.
Yes, this statelessness is the defining trait behind concourse. The only stateful element in concourse is a resource, which must be strictly versioned and stored on an external data store. When you combine the containerization of tasks with the external store of resources, you get the guaranteed reproducibility that concourse provides. Each version of a resource is going to be backed up on some sort of data store, and so even if the data center that your ci runs on is to completely fall down, you can still have strict reproducibility of each of your ci builds.
In order to get more info I would recommend doing a tutorial of some kind to get your hands dirty and build a pipeline yourself. Stark and wayne have a tutorial that could be useful. In order to help understand resources there is also a resources tutorial, which might be helpful for you specifically.
Also, to get to your specific error, the reason that you are seeing missing inputs is because concourse will look for directories (made by resource gets) named each of those inputs. So you would need to get resource instances named gist-upd and gist-out prior to to starting the task.
Once I have done my deploy I need to update the database structure if any patches need to be applied.
My hosts are:
[qa]
qa1
qa2
[prod]
prod1
prod2
I only want this to be run once per environment based on which environments are being deployed to.
Scenarios:
- All : db patches should be applied once for each environment e.g. qa1 + prod1
- Prod : db patches should be applied to just production e.g.prod1
- QA : db patches should be applied to just qa e.g.qa1
I can use the delegate_to option but how would I cover all scenarios above?
For example if I write: delegate_to: "{{ groups['prod'][0] }}" then qa wouldn't get updated etc.
Thanks
You can write separate playbooks to cover the updates of different environments, specifying which one(s) you want in the hosts variable. For example you would have three playbooks, to cover each permutation of update, each with the following hosts directive set: hosts: qa, hosts: qa:prod, hosts: prod
The other option is to make one playbook to target all groups, hosts: qa:prod, and then use the limit option (--limit/-l) for ansible-playbook to decide which groups to target.