Escape backslash in ansible command module when executing sed - sed

Question

My goal is to ensure that a simple text file has a newline at EOF, so that for example
[localhost]# cat my-file.txt
content[localhost]#
results in
[localhost]# cat my-file.txt
content
[localhost]#
Since my host configuration is done by Ansible (2.9) I wanted to make use of sed so that the newline is inserted only if necessary, as described here https://unix.stackexchange.com/a/31955.
Ansible Task:
- name: Ensure newline at end of file
command: "sed -i -e '$a\' {{ item }}"
loop:
- /path/to/my/first/file
- /second/file
Which results in following error:
TASK [iot-edge : Ensure newline at end of file] ******************************************
task path: ...yml:148
[WARNING]: Consider using the replace, lineinfile or template module rather than running 'sed'. If you need to use command because replace, lineinfile or template is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message.
failed: [localhost] (item=/path/to/file) => {"ansible_loop_var": "item", "changed": true, "cmd": ["sed", "-i", "-e", "$a", "/path/to/file"], "delta": "0:00:00.008481", "end": "2023-01-04 14:46:47.681080", "item": "/path/to/file", "msg": "non-zero return code", "rc": 1, "start": "2023-01-04 14:46:47.672599", "stderr": "sed: -e expression #1, char 2: expected \\ after `a', `c' or `i'", "stderr_lines": ["sed: -e expression #1, char 2: expected \\ after `a', `c' or `i'"], "stdout": "", "stdout_lines": []}
The problem is that Ansible removes the \ as you can see in the cmd property.
Of course, there are also some other solutions, but I'm interested to understand the underlying problem. Thank you!

Answer 1

You're running into a problem with layered parsing. Technically, Ansible does not remove the \, you just have not included a \ in the command you asked Ansible to run.
The first problem that you're running into is that YAML has multiple ways to represent strings, and you don't understand how the one that you've chosen (double-quoted flow scalar) works. Inside of a YAML double-quoted scalar, a \ is the start of an escape sequence, and the escape sequence \' is a representation of the character '. To represent a \. you should instead use \\.
- name: Ensure newline at end of file
command: "sed -i -e '$a\\' {{ item }}"
loop:
- /path/to/my/first/file
This will, however, still fail, because Ansible has to do weird internal things to make command: foo work. There are a number of ways to get this command working, but in my opinion the cleanest is to use the cmd parameter with a plain scalar:
# Avoid the need for escapes
- name: Ensure newline at end of file
command:
cmd: sed -i -e '$a\' {{ item }}
loop:
- /path/to/my/first/file
# Escape it until all levels of Ansible's parsing are happy (for esoteric reasons this also requires removing the '')
- name: Ensure newline at end of file
command: "sed -i -e $a\\\\ {{ item }}"
loop:
- /path/to/my/first/file
# Remove YAML escaping, only the required escaping for task parsing
- name: Ensure newline at end of file
command: sed -i -e $a\\ {{ item }}
loop:
- /path/to/my/first/file

Answer 2

My goal is to ensure that a simple text file has a newline at EOF
You can do the following to solve your use-case solve in Ansible:
- shell: cat /path/to/file
register: cat_file
- name: debug file with a newline
debug:
msg: "{{ cat_file.stdout }}\n"
- name: copy the cat_file to a place with a newline
copy:
content: "{{ cat_file.stdout }}\n"
dest: /tmp/file_newline
Note that there are multiple solutions to this use-case, you might even solve it in bash itself.

Answer 3

I would recommend you to use | yaml blocks with shell scripts as these allow you to avoid the need to add any quoting for the strings.
Read https://yaml-multiline.info/ well and you will understand how this can help you remove the yaml quote escaping from the equation.