I wish to check if my text Options exists between opening and closing LocationMatch tags. Below sed command gives me the desired result.
$ sed -n '/^<LocationMatch "^\/+$">/,/^<\/LocationMatch/p' httpd.conf | grep -i 'Options '
Options -Indexes
However, I'm getting syntax error when executing the same command from ansible.
- name: Check if Options exists between Location Match tags
shell: "sed -n '/^<LocationMatch \"^\/+$\">/,/^<\/LocationMatch/p' {{ httpd_home }}/conf/httpd.conf | grep -i 'Options '"
register: Optionsexist
Output:
shell: "sed -n '/^<LocationMatch \"^\/+$\">/,/^<\/LocationMatch/p' {{ httpd_home }}/conf/httpd.conf | grep -i 'Options '"
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
I did keep ansible's escape charecter before the double quotes in the sed command.
Can you please suggest a working syntax.
With User #Zeitounators suggestion the below formatted code with escape charecters helped overcome the syntax error.
- name: Check if Options exists between Location Match tags
shell: "sed -n '/^<LocationMatch \"^\\/+$\\">/,/^<\/LocationMatch/p' {{ httpd_home }}/conf/httpd.conf | grep -i 'Options '"
register: Optionsexist
Related
My goal is to ensure that a simple text file has a newline at EOF, so that for example
[localhost]# cat my-file.txt
content[localhost]#
results in
[localhost]# cat my-file.txt
content
[localhost]#
Since my host configuration is done by Ansible (2.9) I wanted to make use of sed so that the newline is inserted only if necessary, as described here https://unix.stackexchange.com/a/31955.
Ansible Task:
- name: Ensure newline at end of file
command: "sed -i -e '$a\' {{ item }}"
loop:
- /path/to/my/first/file
- /second/file
Which results in following error:
TASK [iot-edge : Ensure newline at end of file] ******************************************
task path: ...yml:148
[WARNING]: Consider using the replace, lineinfile or template module rather than running 'sed'. If you need to use command because replace, lineinfile or template is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
failed: [localhost] (item=/path/to/file) => {"ansible_loop_var": "item", "changed": true, "cmd": ["sed", "-i", "-e", "$a", "/path/to/file"], "delta": "0:00:00.008481", "end": "2023-01-04 14:46:47.681080", "item": "/path/to/file", "msg": "non-zero return code", "rc": 1, "start": "2023-01-04 14:46:47.672599", "stderr": "sed: -e expression #1, char 2: expected \\ after `a', `c' or `i'", "stderr_lines": ["sed: -e expression #1, char 2: expected \\ after `a', `c' or `i'"], "stdout": "", "stdout_lines": []}
The problem is that Ansible removes the \ as you can see in the cmd property.
Of course, there are also some other solutions, but I'm interested to understand the underlying problem. Thank you!
You're running into a problem with layered parsing. Technically, Ansible does not remove the \, you just have not included a \ in the command you asked Ansible to run.
The first problem that you're running into is that YAML has multiple ways to represent strings, and you don't understand how the one that you've chosen (double-quoted flow scalar) works. Inside of a YAML double-quoted scalar, a \ is the start of an escape sequence, and the escape sequence \' is a representation of the character '. To represent a \. you should instead use \\.
- name: Ensure newline at end of file
command: "sed -i -e '$a\\' {{ item }}"
loop:
- /path/to/my/first/file
This will, however, still fail, because Ansible has to do weird internal things to make command: foo work. There are a number of ways to get this command working, but in my opinion the cleanest is to use the cmd parameter with a plain scalar:
# Avoid the need for escapes
- name: Ensure newline at end of file
command:
cmd: sed -i -e '$a\' {{ item }}
loop:
- /path/to/my/first/file
# Escape it until all levels of Ansible's parsing are happy (for esoteric reasons this also requires removing the '')
- name: Ensure newline at end of file
command: "sed -i -e $a\\\\ {{ item }}"
loop:
- /path/to/my/first/file
# Remove YAML escaping, only the required escaping for task parsing
- name: Ensure newline at end of file
command: sed -i -e $a\\ {{ item }}
loop:
- /path/to/my/first/file
My goal is to ensure that a simple text file has a newline at EOF
You can do the following to solve your use-case solve in Ansible:
- shell: cat /path/to/file
register: cat_file
- name: debug file with a newline
debug:
msg: "{{ cat_file.stdout }}\n"
- name: copy the cat_file to a place with a newline
copy:
content: "{{ cat_file.stdout }}\n"
dest: /tmp/file_newline
Note that there are multiple solutions to this use-case, you might even solve it in bash itself.
I would recommend you to use | yaml blocks with shell scripts as these allow you to avoid the need to add any quoting for the strings.
Read https://yaml-multiline.info/ well and you will understand how this can help you remove the yaml quote escaping from the equation.
I'm installing Docker on some hosts and using Ansible playbook to do this. However, we have a startup script for Consul that breaks when docker is installed, as Docker adds a virtual NIC and adds an extra value to the variable.
Original Variables
NODEIP=`hostname -I`
NODE=`hostname -I |sed 's/[.]/-/g'`
I can manually change them to the following and this works.
NODEIP=$(hostname -I | grep -o "[^ ]\\+" | awk /^10\./"{print $1}")
NODE=$(hostname -I | grep -o "[^ ]\+" | awk /^10\./"{print $1}" |sed "s/[.]/-/g")
However, I need to add these to an Ansible playbook. I've modified the variable for NODE and it gets updated in the script, but NODEIP does not. See sample playbook code below.
name: Fix consul startup script for Docker virtual network interface
shell: sed -i 's/NODEIP=`hostname -I`/s_.*_NODEIP=$(hostname -I | grep -o \"[^ ]\\+\" | awk /^10\./\"{print \$1}\")' filename
shell: sed -i '/NODE=`hostname -I |sed/s_.*_NODE=$(echo $NODEIP|sed 's/[.]/-/g')_' filename
I'm going insane trying to get this to work properly. Can anyone help?
Whenever you run an ansible-playbook, it will gather_facts by default. This task will populate your ansible-playbook execution with the variables listed here: Ansible facts
In your case, you are looking for:
NODEIP={{ ansible_default_ipv4.address }}
NODE={{ ansible_hostname }}
Below code should solve your requirement
- name: Command for NodeIP
shell: hostname -I | grep -o "[^ ]\\+" | awk /^10\./"{print $1}"
register: NODEIP
- name: Command for NodeName
shell: hostname -I | grep -o "[^ ]\+" | awk /^10\./"{print $1}" |sed "s/[.]/-/g"
register: NODE
Above code will store command output to NODEIP and NODE variables respectively.
To learn more about usage of ansible return values, refer https://docs.ansible.com/ansible/latest/reference_appendices/common_return_values.html
Thanks to all for the input. Here's what I ended up coming up with (had to escape some characters with backslashes). Delighted it works
- name: replace NODEIP variable in consul startup script
lineinfile:
path: filename
regexp: '^NODEIP='
line: "NODEIP=$(hostname -I | grep -o \"[^ ]\\+\" | awk /^10\\./\"{print $1}\")"
backrefs: yes
- name: replace NODE variable in consul startup script
lineinfile:
path: filename
regexp: '^NODE='
line: "NODE=$(hostname -I | grep -o \"[^ ]\\+\" | awk /^10\\./\"{print $1}\" |sed \"s/[.]/-/g\")"
backrefs: yes
I am trying to separate out parts of a path as follows. My input path takes the following possible forms:
bucket
bucket/dir1
bucket/dir1/dir2
bucket/dir1/dir2/dir3
...
I want to separate the first part of the path (bucket) from the rest of the string if present (dir1/dir2/dir3/...), and store both in separate variables.
The following gives me something close to what I want:
❯ BUCKET=$(echo "bucket/dir1/dir2" | sed 's#\(^[^\/]*\)[\/]\(.*\)#\1#')
❯ EXTENS=$(echo "bucket/dir1/dir2" | sed 's#\(^[^\/]*\)[\/]\(.*\)#\2#')
echo $BUCKET $EXTENS
❯ bucket dir1/dir2
HOWEVER, it fails if I only have bucket as input (without a slash):
❯ BUCKET=$(echo "bucket" | sed 's#\(^[^\/]*\)[\/]\(.*\)#\1#')
❯ EXTENS=$(echo "bucket" | sed 's#\(^[^\/]*\)[\/]\(.*\)#\2#')
echo $BUCKET $EXTENS
❯ bucket bucket
... because, in the absence of the first '/', no capture happens, so no substitution takes place. When the input is just 'bucket' I would like $EXTENS to be set to the empty string "".
Thanks!
For something so simple you could use bash built-in instead of launching sed:
$ path="bucket/dir1/dir2"
$ bucket="${path%%/*}"
$ extens="${path#$bucket}"
$ printf '|%s|%s|\n' "$bucket" "$extens"
|bucket|/dir1/dir2|
$ path="bucket"
$ bucket="${path%%/*}"
$ extens="${path#$bucket}"
$ printf '|%s|%s|\n' "$bucket" "$extens"
|bucket||
But if you really want to use sed and capture groups:
$ declare -a bucket_extens
$ mapfile -td '' bucket_extens < <(printf '%s' "bucket/dir1/dir2" | sed -E 's!([^/]*)(.*)!\1\x00\2!')
$ printf '|%s|%s|\n' "${bucket_extens[#]}"
|bucket|/dir1/dir2|
$ mapfile -td '' bucket_extens < <(printf '%s' "bucket" | sed -E 's!([^/]*)(.*)!\1\x00\2!')
$ printf '|%s|%s|\n' "${bucket_extens[#]}"
|bucket||
We use the extended regex (-E) to simplify a bit, and ! as separator of the substitute command. The first capture group is simply anything not containing a slash and the second is everything else, including nothing if there's nothing else.
In the replacement string we separate the two capture groups with a NUL character (\x00). We then use mapfile to assign the result to bash array bucket_extens.
The NUL trick is a way to deal with file names containing spaces, newlines... NUL is the only character that cannot be part of a file name. The -d '' option of mapfile indicates that the lines to map are separated by NUL instead of the default newline.
Don't capture anything. Instead, just match what you don't want and replace it with nothing:
BUCKET=$(echo "bucket" | sed 's#/.*##'). # bucket
BUCKET=$(echo "bucket/dir1/dir2" | sed 's#/.*##') # bucket
EXTENS=$(echo "bucket" | sed 's#[^/]*##') # blank
EXTENS=$(echo "bucket/dir1/dir2" | sed 's#[^/]*##') # /dir1/dir2
As you are putting a slash in the regex. the string with no slashes will not
match. Let's make the slash optional as /\?. (A backslash before ?
is requires due to the sed BRE.) Then would you please try:
#!/bin/bash
#path="bucket/dir1/dir2"
path="bucket"
bucket=$(echo "$path" | sed 's#\(^[^/]*\)/\?\(.*\)#\1#')
extens=$(echo "$path" | sed 's#\(^[^/]*\)/\?\(.*\)#\2#')
echo "$bucket" "$extens"
You don't need to prepend a backslash to a slash.
By convention, it is recommended to use lower cases for user variables.
I would like to have the following command integrated in an Ansible playbook task:
cut -f 1 -d: /etc/passwd | xargs -n 1 -I {} bash -c ' echo -e "\n{}" ; chage -l {}'.
Any quote inside breaks the whole command. How I can avoid it to make it run the whole string?
Many thanks in advance.
You can simply use the YAML literal block string syntax. In that way you don't need to escape any quotes. Instead, you can pass your shell command as is.
Example:
- name: test task
shell:
cmd: |
cut -f 1 -d: /etc/passwd | xargs -n 1 -I {} bash -c ' echo -e "\n{}" ; chage -l {}'
tags: test
You can escape them with \”
example: "hello=\"hi\""
In the following example I am looking to include the * metacharacter in the string replacement. If I understand correctly, I should be able to escape the character using \ but after doing that this is the result I am seeing:
❯ echo 'foo(*)' | sed s/foo\(\*/bar/g
bar*)
I was expecting to see the following:
❯ echo 'foo(*)' | sed s/foo\(\*/bar/g
bar)
You need to quote your sed command:
echo 'foo(*)' | sed 's/foo(\*/bar/g'
bar)