I have the following helmfile
releases:
- name: whoami-mn
namespace: whoami-mn-{{ .Environment.Name }}
chart: tons/whoami-mn
version: {{ .Values.chartVersions.whoami }}
installed: {{ .Values.installed }}
values:
- env/{{ .Environment.Name }}-values.yaml
environments:
dev:
values:
- installed: true
- chartVersions:
whoami: 0.3.0
otherApp: 0.2.0
prod:
values:
- installed: true
- chartVersions:
whoami: 0.2.0
otherApp: 0.1.0
But when running helmfile -e dev template I get the following error
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:5:23: executing "stringTemplate" at <.Values.chartVersions.whoami>: map has no entry for key "chartVersions"
I can do something like the following
environments:
dev:
values:
- installed: true
- chartVersionWhoami: 0.3.0
That works but I'd prefer having all chart version grouped under a parent key. Any suggestions about how to get the former working?
UPDATE:
The following works
...
version: {{ (index .Values "chartVersions").whoamiMn }}
...
With the environment defined like the following
environments:
dev:
values:
- installed: true
- chartVersions:
whoamiMn: 0.3.0
But I really feel like my first attempt should work as well!
After experimenting and discussing further here. I figured out that I can fix this issue by simply listing my environments before my releases. In my pedantic point of view I'd appreciate having my releases listed first but it works.
Related
Background
I have a small elixir project and I am trying to use GithubWorkflow to have a nice pipeline and automate some actions.
Problem
However I can't even get it started as I get the following error:
Setup elixir
1s
AUCTION_HOUSE_TOKEN: ***
Run actions/setup-elixir#v1
with:
elixir-version: 1.10-otp-22
otp-version: 22.1
install-hex: true
install-rebar: true
env:
COOKIE: ***
TOKEN: ***
##[error]Cannot read property 'includes' of undefined
Code
I think my workflow might be incorrect, could someone help figure out what's wrong?
name: build
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
env:
AUCTION_HOUSE_COOKIE: ${{ secrets.MARKET_MANAGER_WM_COOKIE }}
AUCTION_HOUSE_TOKEN: ${{ secrets.MARKET_MANAGER_WM_XCSRFTOKEN }}
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Setup elixir
uses: actions/setup-elixir#v1
with:
elixir-version: '1.10-otp-22' # Define the elixir version [required]
otp-version: '22.1' # Define the OTP version [required]
- name: Install Dependencies
run: mix deps.get
- name: Run Tests
run: mix test
- name: Run credo code analyser
run: mix credo --strict
- name: Run typechecks
run: mix dialyzer
- name: Build executable
run: mix escript.build
env:
MIX_ENV: prod
Summary
The elixir version in the file was wrong. It should have:
with:
elixir-version: '1.10.4 # Define the elixir version [required]
otp-version: '22.1' # Define the OTP version [required]
Instead of the current content.
RCA
So this one was interesting. When it comes to elixir versions, because I use asdf as my version manager, I am allowed to pick the best version (the most optimized) for the erlang setup I have.
How do I know which elixir version is the best for, lets say, erlang 22.1?
Well, I use bob's list:
https://bobs-list.kobrakai.de/#version_v1.10
And while this is fine and it does work on my machine and in docker files and so on, Github Action wont accept it. Github Actions takes its elixir version's from here:
https://github.com/elixir-lang/elixir/releases
And as you can see, there no official release called "1.10-otp-22"
So, instead of the current content, the file should instead have:
with:
elixir-version: '1.10.4' # Define the elixir version [required]
otp-version: '22.1' # Define the OTP version [required]
Given a yaml file (helmfile) like the following
releases:
- chart: ../charts/foo
name: foo
namespace: '{{ .Values.stack }}'
values:
- ../config/templates/foo-values.yaml.gotmpl
set:
- name: image.tag
value: 22
- name: replicas
value: 1
- chart: ../charts/bar
name: bar
namespace: '{{ .Values.stack }}'
values:
- ../config/templates/bar-values.yaml.gotmpl
set:
- name: image.bar_proxy.tag
value: 46
- name: image.bar.tag
value: 29
- name: replicas
value: 1
I'm trying to figure out a clean way to update a specific image tag. For example, I'd like to update image.bar_proxy.tag from 46 to 51.
I have the following, which does the job, but it requires that you know the exact index of the array item:
yq -y '.releases[] |= if .name=="bar" then .set[0].value |= 51 else . end' helmfile-example.yaml
So if the array order were to change at some point this would break.
A preferred solution would be: "update image.bar_proxy.tag value from 46 to 51 where set[].name==image.bar_proxy.tag". Any ideas on how to achieve a more specific conditional selection like this?
FYI our yq version:
$ yq --version
yq 2.10.0
You can use the following filter to make it work. It works by dynamically selecting the index of the object where your tag exists. On the selected object .value=51 will update the value as you wanted. You can also use the -i flag to do in-place modification of the original file.
yq -y '.releases[].set |= map(select(.name == "image.bar_proxy.tag").value=51)' yaml
See the underlying jq filter acting on the JSON object at jq-playground
Given the context of using Helmfile, there are a couple of ways you can approach this without necessarily editing the helmfile.yaml. Helmfile allows using the Go text/template language in many places, similarly to the underlying Helm tool, and has some other features that can help.
One of the easiest things you can do is take advantage of values: being a list, and of unknown values generally being ignored. You (or your CI/CD system) can write a separate YAML file that contains just the tags (JSON may be easier to write and is valid YAML)
# tags.yaml
image:
tag: 22
bar: {tag: 29}
bar_proxy: {tag: 46}
and then include this file as an additional file in the helmfile.yaml. (This would be equivalent to using helm install -f with multiple local values files, rather than helm install --set individual values.)
releases:
- name: foo
values:
- ../config/templates/foo-values.yaml.gotmpl
- tags.yaml
# no `set:`
- name: bar
values:
- ../config/templates/bar-values.yaml.gotmpl
- tags.yaml
- replicas: 1
# no `set:`
Helmfile's templating extensions also include env and requiredEnv to read ordinary environment variables from the host system. Helm proper does not have these to try to minimize the number of implicit inputs to a chart, but for Helmfile it's a possible way to provide values at deploy time.
releases:
- name: bar
set:
- name: image.bar_proxy.tag
value: {{ env "BAR_PROXY_TAG" | default "46" }}
- name: image.bar.tag
value: {{ requiredEnv "BAR_TAG" }}
How do you override values in a Helm list with --set param in Azure DevOps?
Simple use case in values.yaml:
environment:
- name: foo
value: override_me
- name: bar
value: override_me
- name: baz
value: override_me
In the deployment.yaml file I use it like so:
env:
{{ toYaml .Values.environment | indent 10}}
One thing that kind of works, but not really, is:
environment[0].name=foo,environment[0].value=hello,{...}
The problem with this override is that it will override the entire list, even if I only want to replace value [0], not [1] and [2].
Also I get parsing errors when I pass url:s or int's (not on localhost, only AZ DevOps) - to overcome that paring error, you can escape it with \" - but then the chart is messed up - even though it passes the validation.
So, is it possible to override the env list in my case in Azure DevOps helm deployment? Or do I need to restructure the list to individual key=value pairs?
I've got weird experience when doing this, in 2 similar cases in one case it replaces them, in one overrides the whole array. so in the second case what I had to do is this:
environment:
- name: v1
value: keep_me
- name: v2
value: keep_me
- name: v3
value: keep_me
- name: foo
value: override_me
- name: bar
value: override_me
and I was doing this in the Azure Devops:
--set environment[3].name=foo,environment[03.value=xxx
for the other one I didnt have to do that, it would gladly overwrite only the values I've input. no idea why it did that.
Get yourself some variables defined in the task:
Use a standard set:
I was using a bash task in a release pipeline pointed at a deploy.sh file which existed as a published artifact. You need to chmod +x the file for this to work properly.
Installed AWX docker from here - https://github.com/ansible/awx. I am trying to add a callback-plugin for a specific project as written here - https://docs.ansible.com/ansible-tower/latest/html/administration/tipsandtricks.html#using-callback-plugins-with-tower. Does not work. I add to Template-> EXTRA VARIABLES lines
---
bin_ansible_callbacks: true
callback_plugins: /callback_plugins
stdout_callback: selective
Does not work.
I add the directory /var/lib/awx/projects/test/callback_plugins/ to SETTINGS-> JOBS-> ANSIBLE CALLBACK PLUGINS - it doesn't work either.
Tell me, please, how to do it correctly, so that another (custom) plugin picks up and earns.
I'm issuing the same problem, after some debugs the problem I've open a issue on AWX project https://github.com/ansible/awx/issues/4149
In the meantime I've applied a workaround that consists in create a symlinks for each callback plugin you want to use in the callback_plugins folder of your roles project
For example, if you are using the ara project
- name: Research for callbacks in virtualenv libs
find:
path: '{{ ansible_playbook_python|dirname|dirname }}/{{ item }}'
file_type: file
depth: 1
patterns: '*.py'
excludes: '__init__*'
register: _internal__callbacks
with_items:
- lib/python3.6/site-packages/ara/plugins/callbacks
# TODO : prevent existing callbacks to be overwritten
- name: Create symlinks from virtualenv lib directory to local callback_plugins/
file:
src: '{{ item }}'
dest: '{{ playbook_dir }}/callback_plugins/{{ item|basename }}'
state: link
with_items: "{{ _internal__callbacks.results|map(attribute='files')|flatten|map(attribute='path')|list }}"
seems like you have to use callbacks_enabled instead of callback_plugins. put this example configuration in the /var/lib/awx/ansible.cfg file:
[defaults]
callback_whitelist = profile_tasks
--- works on AWX 17.x
I have a folder where I place unmaintained python packages, so I installed them from the zip rather than from their repository.
I am using Ansible 2.0, so the find command seems to be the way to do it
So far I was doing the following:
- name: Install unmaintained dependencies
pip:
name: "{{ my_project_app }}/requirements/{{ item }}"
virtualenv: "{{ my_project_venv }}"
with_items:
- django-hijack-2.0.0.zip
- django-image-cropping-django-19.zip
- pisa-3.0.33.zip
Now I'm playing with the find command
- name: Loading unmaintained dependencies
find:
paths: "{{ my_project_app }}/requirements"
patterns: "*.zip"
register: unmaintained_dependencies
- debug:
var: unmaintained_dependencies
If I run the playbook above, I get the following output
ok: [192.168.1.212] => {
"files_found": {
"changed": false,
"examined": 3,
"files": [
{
...
"path": "/data/my_project/requirements/pisa-3.0.33.zip",
...
},
...
],
"matched": 3,
"msg": ""
}
}
I guess that there must be a way to put everything together but here is where I'm stuck.
I still don't get what are you trying to achieve here yet.. Are you using the find module just because pip module doesn't install packages from zip files?
For your find workaround, you can create a task that iterates over the results of the find task, using with_items: files_found.files, and using {{ item.path }} whenever you need the path:
- name: Install unmaintained dependencies
pip:
name: "{{ item.path }}"
virtualenv: "{{ my_project_venv }}"
with_items: "{{files_found.files}}"
Also, instead of using file, you can try to make a loop using with_fileglob:
- pip:
name: {{ item }}
virtualenv: "{{ my_project_venv }}"
with_fileglob:
- "{{ my_project_app }}"/requirements/*.zip
Note, I didn't have time to test any of these solutions, or to ask more about what were you trying to achieve, but I hope they help with your problem.