I have the following map of strings and I would like to change the value of "image.tag" key .
I tried the following but it does not work as I expected. The problem here is that image.tag is a string but I am not sure how to express that. Thanks
yq eval --inplace ".spec.chart.values.\"image.tag\": \"$TAG\"" values.yaml
spec:
chart:
values:
image.tag: master
You don't have to use double quotes for reading variables from shell. mikefarah/yq provides a method strenv to load variables (also environment) form the shell.
Also by using single quotes, you can just wrap image.tag under double quotes to let it be treated as a single word.
Use the style method to set quotes for the updated value style="double" reflects the updated tag value to be treated as a string.
newtag="foo" yq e --inplace '.spec.chart.values."image.tag" |= strenv(newtag) | ..style="double"' values.yaml
or if the new tag is defined as a shell variable say TAG
newtag="$TAG" yq e --inplace '.spec.chart.values."image.tag" |= strenv(newtag) | ..style="double"' values.yaml
Note that, if you are using yq version above 4.18.1, the eval action e is the default one and can be skipped altogether.
Related
In a shell scrip I want to assigning a variable what to use in a value in a deployment. For the life of me I can not figure out how to get it to work.
My helm deploy script file has the following in order to set the value to use my variable :
--set AuthConfValue=$AUTH_CONF_VALUE
And I have this in the deployment.yaml file in order to use the variable :
- name: KONG_SETTING
value: "{ {{ .Values.AuthConfValue }} }"
If I assign the variable in my shell script like the following :
AUTH_CONF_VALUE="ernie"
It will work and the value in the deployment will show up like so:
value: '{ ernie }'
Now if I try to assign the variable like this:
AUTH_CONF_VALUE="\\\"ernie\\\":\\\"123\\\""
I will then get the error error converting YAML to JSON: yaml: line 118: did not find expected key when the helm deploy runs.
I was hoping that this would give me the following value in the deployment :
value: "{ "ernie":"123" }"
If I hardcode the value into the deployment.yaml with this:
- name: KONG_SETTING
value: "{ \"ernie\": \"123\" }"
and then run the helm deploy it will work and populate the value in the deployment with this -
value: "{ "ernie":"123" }"
Can someone show me if/how I might be able to do this?
The Helm --set option also uses backslash escaping. So in your example, the $AUTH_CONF_VALUE variable in the host shell contains a single backslash before each quote, which is consumed by --set, so .Values.AuthConfValue contains no backslashes at all, and you get invalid YAML.
If you want to keep this as close to the existing form as you can, let's construct a string with no backslashes at all (and hopefully no commas or brackets either, since those also have special meaning to --set)
AUTH_CONF_VALUE='"ernie":"123"'
helm install --set AuthConfValue="$AUTH_CONF_VALUE" .
When Helm expands a template it doesn't know anything about the context where it might be used. In your case, you know
.Values.AuthConfValue is the body of a JSON object
If you surround it in curly braces { ... } then it should be a valid JSON object
You need to turn that into a correctly-escaped YAML string
Helm contains a lightly-documented toJson function that takes an arbitrary object and converts it to JSON; any valid JSON is also valid YAML. So the closest-to-what-you-have approach might look like
- name: KONG_SETTING
value: {{ printf "{%s}" .Values.AuthConfValue | toJson }}
If you're willing to modify your deploy process a little more, you can have less escaping and more certainty. In the sequence above, we have a string that happens to be a JSON object; what if we had an actual object? Imagine settings like
# kong-auth.yaml
authConf:
ernie: "123"
You could provide this file at install time with a helm install -f option. Since valid JSON is valid YAML, again, you could also provide a JSON file here without changing anything.
helm install -f kong-auth.yaml .
Now with this setup .Values.authConf is an object; the only escaping you need to do is standard YAML/JSON escaping (for example quoting "123" so it's a string and not a number). Now we can use toJson twice, once to get the {"ernie":"123"} JSON object string, and a second time to escape that string as a value "{\"ernie\":\"123\"}".
- name: KONG_SETTING
value: {{ .Values.authConf | toJson | toJson }}
Setting this up would require modifying your deployment script, but it would be much safer against quoting and escaping concerns.
I have an ansible playbook, which first initializes a fact using set_fact, and then a task that consumes the fact and generates a YAML file from it.
The playbook looks like this
- name: Test yaml output
hosts: localhost
become: true
tasks:
- name: set config
set_fact:
config:
A12345: '00000000000000000000000087895423'
A12352: '00000000000000000000000087565857'
A12353: '00000000000000000000000031200527'
- name : gen yaml file
copy:
dest: "a.yaml"
content: "{{ config | to_nice_yaml }}"
Actual Output
When I run the playbook, the output in a.yaml is
A12345: 00000000000000000000000087895423
A12352: 00000000000000000000000087565857
A12353: '00000000000000000000000031200527'
Notice only the last line has the value in quotes
Expected Output
The expected output is
A12345: '00000000000000000000000087895423'
A12352: '00000000000000000000000087565857'
A12353: '00000000000000000000000031200527'
All values should be quoted.
I cannot, for the life of me, figure out why only the last line has the value printed in single-quotes.
I've tried this with Ansible version 2.7.7, and version 2.11.12, both running against Python 3.7.3. The behavior is the same.
It's because 031200527 is an octal number, whereas 087895423 is not, thus, the octal scalar needs quoting but the other values do not because the leading zeros are interpreted in yaml exactly the same way 00hello would be -- just the ascii 0 followed by other ascii characters
If it really bothers you that much, and having quoted scalars is obligatory for some reason, to_nice_yaml accepts the same kwargs as does pyyaml.dump:
- debug:
msg: '{{ thing | to_nice_yaml(default_style=quote) }}'
vars:
quote: "'"
thing:
A1234: '008123'
A2345: '003123'
which in this case will also quote the keys, but unconditionally quotes the scalars
I'm trying to use yq https://github.com/mikefarah/yq v4.3.2 to add a yaml value in a CloudFormation template like so:
Mappings:
RegionMap:
us-east-1:
AMI: 'ami-YeahRight'
Instead, what I'm getting is:
Mappings:
RegionMap:
us-east-1:
AMI: ami-YeahRight
The style bits in the documentation and from this SO answer yq processing a string with quotation marks made me think that this portion of a bash script would work however the style portion is ignored.
region="us-east-1"
ami="ami-YeahRight"
echo Inserting $ami into $region
yq eval '.Mappings.RegionMap.'"$region"'.AMI='"$ami"' style="single"' -i temp.yaml
I've tried a whole bunch of similar bits but can't seem to crack this nut. Any help here would be greatly appreciated!
mikefaraq/yq is going through major leap of changes starting from v4 and I'm not surprised that things are breaking in-between.
On v4.4 I can make this work, but using env() function to look-up the variables and use the ..style attribute to set the quoting style
region="us-east-1" ami="ami-YeahRight" yq e '.Mappings.RegionMap.[env(region)] = env(ami) | ..style="single"' yaml
I have a yaml file that looks something like this:
a:
desc:
value: 1
b:
desc:
value: 2
# ...
I want to convert it to this:
a: 1
b: 2
# ...
In yq v2, I used the command .[] |= .value' to update each element of the array to the value in the .value field. Is there a way to do this with yq v3?
cat config.yaml | yq w - "*" "*.value"
yields
a: *.value
b: *.value
# ...
``
Your claim seems to be contradicting each other. There are two versions of yq implementations out there. A python implementation as a wrapper over jq and other written in Go.. See my answer that covers in details about those versions.
When you said you used .[] |= .value in yq v2, that's actually not the Go version, but the version with the Python wrapper over jq, since that syntax matches its DSL. But the other attempt yq w - "*" "*.value" seems to be the actual Go version.
Since there is an uncertainty around which version of yq in installed for you, I'll try to provide my view in both the versions
kislyuk's yq
yq -y '.[] |= .value' yaml
mikefarah's yq
The Go version does not have dynamic transformational capabilities like its Python version and does not support this type of update directly. Because the write/new field creations syntax is simply
yq w <yaml_file> <path_expression> <new value>
where the new value is not an expression but a literal value. Had it supported expressions, we could have conjured up a way to do the transformation. The Go version is otherwise good, but lacking support in some key transformational capabilities.
P.S. I've raised a GitHub feature request to allow such transformations. See https://github.com/mikefarah/yq/issues/602
As of today Dec 21st, 2020, yq v4 is in beta and supports this transformation. Download the v4 version and try
yq eval '.[] |= .value' test.yml
I have a downloaded file through helm inspect called sftp.yaml
I have a parameter in that sftp.yaml file:-
sftp:
allowedMACs: "hmac-sha2-512"
allowedCiphers: aes256-ctr
Now if i install the corresponding helm chart after commenting out the entire line of "allowedMACs" from custom values files i.e. "sftp.yaml", then K8s takes the delta of sftp.yaml and the actual values.yaml and then use values.yaml's "allowedMACs".
However What i want is if "allowedMACs" line is commented in "sftp.yaml" custom values file, then it should not set the env variable at all, or sets it as null.
presently my deployment file's env section looks like
- name: MACs
value: {{ default "" .Values.sftp.allowedMACs | quote }}
You need to either override (with new value) or unset the value, if you only comment out the section you are not doing any of the above and the default value is going to be used.
Basically you are looking to unset a default value. As per banzaicloud example this can be done like so:
helm install stable/chart-name --set sftp.allowedMACs=null
You can also use override value file in a similar way:
sftp:
allowedMACs: null
allowedCiphers: aes256-ctr
This is available in Helm since version 2.6. If you like in-depth information you can review the issue and the subsequent PR that introduced the feature.
yeah I think helm would retrieve values from all values files, so if allowedMACs is in one of those it'll get populated. If this parameter is affected only by sftp.yaml file should it really belong only to it and would i make sense to remove it from main values.yaml?