Create systemd unit in cloud-init and enable it - ubuntu-16.04

Question

I've created the following systemd unit in the cloud-init file:
- path: /etc/systemd/system/multi-user.target.wants/docker-compose.service
owner: root:root
permissions: '0755'
content: |
[Unit]
Description=Docker Compose Boot Up
Requires=docker.service
After=docker.service
[Service]
Type=simple
ExecStart=/usr/local/bin/docker-compose -f /opt/data/docker-compose.yml up -d
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target
When I try to run
sudo systemctl enable docker-compose.service
to create the symlink I get this:
Failed to execute operation: No such file or directory
However I'm sure that the file is under /etc/systemd/system/multi-user.target.wants

Answer 1

I had the same need, but I was working from a recipe that said to create /etc/systemd/system/unit.service and then do systemctl enable --now unit.
So I created the unit file with write_files and did the reload and enable in a text/x-shellscript part and that worked fine. (User scripts run last and in order, while I don't think there are guarantees about when the write_files key in the user-data is processed. I found out the hard way that it's before the user key so you can't set ownership to users that cloud-init creates).
I think runcmd entries are converted to user scripts and run in list order (either before or after the other user scripts), so if you don't like x-shellscript parts you can do the reload and enable that way. /var/log/cloud-init.log is where I check the order, there is probably a config file too.
Full disclosure: I forgot the systemctl daemon-reload command but it still worked. Actually there is a caveat against systemd manipulations from cloud-init because it's running under systemd itself and some systemd commands may wait for cloud-init to finish -- deadlock!

Answer 2

After unit file creation but before any manipulations with it systemd should be notified about the changes:
systemctl daemon-reload
So cloud-init YAML block creating docker-compose.service file should be followed by:
runcmd:
- systemctl daemon-reload

Answer 3

Check that every file involved is present and valid:
ls -l /etc/systemd/system/multi-user.target.wants/docker-compose.service
ls -l /usr/local/bin/docker-compose
ls -l /opt/data/docker-compose.yml
systemd-analyze verify /etc/systemd/system/multi-user.target.wants/docker-compose.service
Also consider the timing. Even if the files exist once they are fully booted, would /etc/systemd/system/multi-user.target.wants/ exist when cloud-init runs?