Motivation
I have a simple docker-compose.yaml file which is of this structure
services:
foo:
image: docker.registry.url:version
foo2:
image: docker.registry.url2:version
foo3:
image: docker.registry.url3:version
And I can easly do:
GET
yq '.services.foo.image' docker-compose.yaml
docker.registry.url:version
SET
yq -i '.services.foo.image = "foo"' docker-compose.yaml
Wish
I don't know how many services I'll have but I want to loop over all of them and fix the URL of the registry in case it comes from my registry and needs some updates.
Basically I would like to extract all keys in a way they can be used in a query again - similar to what is in the Motivation as an SET example.
yq <<magic command>> docker-compose.yaml
.services.foo.image .services.foo2.image .services.foo3.image
And using these keys I can then loop over it using:
for key in .asdf.asdf. .asdf.; do echo "Some query using $key"; done
What I tried
yq '.services.*.image | path' docker-compose.yaml
- services
- foo
- image
- services
- foo2
- image
- services
- foo3
- image
yq '.services.*.image | path | .[-2]' docker-compose.yaml
foo
foo2
foo3
Maybe with some query and merging this can print the path in a way it can later be used for a query again.
You were almost there with the path filter; you just have to convert it to a dotpath:
yq eval '.services.*.image | path | "." + join(".")' docker-compose.yaml
.services.foo.image
.services.foo2.image
.services.foo3.image
Is the output as expected?
yq '.services | to_entries | .[] | .key + " - " + .value.image' docker-compose.yaml
Output
foo - docker.registry.url:version
foo2 - docker.registry.url2:version
foo3 - docker.registry.url3:version
I'm actually quite happy with:
for key in $(yq '.services.*.image | path | .[-2]' docker-compose.yaml); do echo "$key - $(y
q ".services.$key.image" docker-compose.yaml)"; done
But is there some better concept?
Related
Given the following jq pipeline of expressions:
echo '{"foo": 1}' | jq '.foo + 2 as $bar | {$bar}'
I would expect the output:
{
"bar": 3
}
What I get is:
jq: error (at <stdin>:1): number (1) and object ({"bar":2}) cannot be added
What is this object and why is jq trying to add to it?
I can resolve this issue with parentheses but I'm still unclear as to what was happening in the original statement:
echo '{"foo": 1}' | jq '(.foo + 2) as $bar | {$bar}'
{
"bar": 3
}
Your first filter was executed as if it had been parenthesized like this
echo '{"foo": 1}' | jq '.foo + (2 as $bar | {$bar})'
Thus, jq tried to add a number (here 1) to an object (here {"bar":2}).
This is because the syntax for a variable binding, as noted in the manual's corresponding section, takes on the form ... as $identifier | .... It "includes" the pipe and the following expression. This is reflected by the fact that a binding without the following pipe and expression cannot stand alone.
I have sample.yaml file that looks like the following:
a:
b:
- "x"
- "y"
- "z"
and I have another file called toadd.yaml that contains the following
- "first to add"
- "second to add"
I want to modify sample.yaml file so that it looks like:
a:
b:
- "x
- "y"
- "z"
- "first to add"
- "second to add"
Also, I dont want redundant naming! so if there is "x" already in toadd.yaml than I dont want it to be added two times in sample.yaml/a.b
Please note that I have tried the following:
while read line; do
yq '.a.b += ['$line']' sample.yaml
done <toadd.yaml
and I fell on:
Error: Bad expression, could not find matching `]`
If the files are relatively smaller, you could just directly load the second file on to the first. See Merging two files together
yq '.a.b += load("toadd.yaml")' sample.yaml
Tested on mikefarah/yq version 4.25.1
To solve the redundancy requirement, do a unique operation before forming the array again.
yq 'load("toadd.yaml") as $data | .a.b |= ( . + $data | unique )' sample.yaml
which can be further simplified to just
yq '.a.b |= ( . + load("toadd.yaml") | unique )' sample.yaml
I'm trying to use yq to find if a key-value pair exists in a yaml.
Here's an example yaml:
houseThings:
- houseThing:
thingType: chair
- houseThing:
thingType: table
- houseThing:
thingType: door
I just want an expression that evaluates to true (or any value, or exits with zero status) if the key-value pair of thingType: door exists in the yaml above.
The best I can do so far is find if the value exists by recursively walking all nodes and checking their value:
yq eval '.. | select(. == "door")' my_file.yaml which returns door. But I also want to make sure thingType is its key.
You could use the select statement under houseThing as
yq e '.houseThings[].houseThing | select(.thingType == "door")' yaml
or do a recursive look for it
yq e '.. | select(has("thingType")) | select(.thingType == "door")' yaml
I have a dictionary with a few lists(contains a # of strings).
Example List:
hosts = ['199.168.1.100:1000', '199.168.1.101:1000']
When I try to print this out using ruamel.yaml, the elements show up as
hosts:
- 199.168.1.100:1000
- 199.168.1.101:1000
I want the results to be
hosts:
- '199.168.1.100:1000'
- '199.168.1.101:1000'
So I traversed through the list and created a new list with each element being a ruamel SingleQuotedString
S = ruamel.yaml.scalarstring.SingleQuotedScalarString
new_list = []
for e in hosts:
new_list.append(S(e))
hosts = new_list
When I print this out, I still end up printing the "hosts" list without any quotes. What am I doing wrong here?
In the following I assume you mean dumping to YAML when you indicate printing.
Your approach is in principle correct, as using the "global"
yaml.default_style = "'"
would also get the key hosts quoted, and that is not what you
want. Maybe you are not reassigning hosts to the actual datastructure that
you are dumping, because hosts is just the value of the key value pair you
are dumpiong.
The following:
import sys
import ruamel.yaml
S = ruamel.yaml.scalarstring.SingleQuotedScalarString
yaml = ruamel.yaml.YAML()
data = dict(hosts = [S(x) for x in ['199.168.1.100:1000', '199.168.1.101:1000']])
yaml.dump(data, sys.stdout)
will give what you want without problem:
hosts:
- '199.168.1.100:1000'
- '199.168.1.101:1000'
I'm getting the output of a command on the remote system and storing it in a variable. It is then used to fill in a file template which gets placed on the system.
- name: Retrieve Initiator Name
command: /usr/sbin/iscsi-iname
register: iscsiname
- name: Setup InitiatorName File
template: src=initiatorname.iscsi.template dest=/etc/iscsi/initiatorname.iscsi
The initiatorname.iscsi.template file contains:
InitiatorName={{ iscsiname.stdout_lines }}
When I run it however, I get a file with the following:
InitiatorName=[u'iqn.2005-03.org.open-iscsi:2bb08ec8f94']
What I want:
InitiatorName=iqn.2005-03.org.open-iscsi:2bb08ec8f94
What am I doing wrong?
I realize I could write this to the file with an "echo "InitiatorName=$(/usr/sbin/iscsi-iname)" > /etc/iscsi/initiatorname.iscsi" but that seems like an un-Ansible way of doing it.
Thanks in advance.
FWIW, if you really do have an array:
[u'string1', u'string2', u'string3']
And you want your template/whatever result to be NOT:
ABC=[u'string1', u'string2', u'string3']
But you prefer:
ABC=["string1", "string2", "string3"]
Then, this will do the trick:
ABC=["{{ iscsiname.stdout_lines | list | join("\", \"") }}"]
(extra backslashes due to my code being in a string originally.)
Use a filter to avoid unicode strings:
InitiatorName = {{ iscsiname.stdout_lines | to_yaml }}
Ansible Playbook Filters
To avoid the 80 symbol limit of PyYAML, just use the to_json filter instead:
InitiatorName = {{ iscsiname.stdout_lines | to_yaml }}
In my case, I'd like to create a python array from a comma seperated list. So a,b,c should become ["a", "b", "c"]. But without the 'u' prefix because I need string comparisations (without special chars) from WebSpher. Since they seems not to have the same encoding, comparisation fails. For this reason, I can't simply use var.split(',').
Since the strings contains no special chars, I just use to_json in combination with map(trim). This fixes the problem that a, b would become "a", " b".
restartApps = {{ apps.split(',') | map('trim') | list | to_json }}
Since JSON also knows arrays, I get the same result than python would generate, but without the u prefix.