ModuleNotFoundError: No module named 'azure.mgmt.network.version' [duplicate] - azure-devops

After upgrading ansible to version 2.10.5 and python3.8.10 my playbook.yml fails with this error.
ModuleNotFoundError: No module named 'azure.mgmt.monitor.version'
fatal: [localhost]: FAILED! => {"attempts": 1, "changed": false, "msg": "Failed to import the required Python library (ansible[azure] (azure >= 2.0.0)) on certrenewplay's Python /usr/bin/python3`
The module is there if I run python3 -c "import azure.mgmt.monitor" and if I run pip3 list I see it installed as azure-mgmt-monitor==2.0.0
The exact part of the playbook code that is erroring is this:
- name: Create _acme-challenge record for zone "{{ env_name_dot }}"
azure_rm_dnsrecordset:
subscription_id: "{{ mgmt_subscription }}"
client_id: "{{ mgmt_vault_azure_client_id }}"
tenant: "{{ mgmt_vault_azure_tenant_id }}"
secret: "{{ mgmt_vault_azure_client_secret }}"
resource_group: "{{ mgmt_rg }}"
relative_name: "_acme-challenge.{{ env_name }}"
zone_name: "{{ dns_zone_name }}.{{ dns_zone_domain }}"
record_type: TXT
state: present
records:
- entry: "{{ cn_challenge_data }}"
time_to_live: 60
when: dns_zone_name != 'activedrop'
register: add_record
retries: 1
delay: 10
until: add_record is succeeded
I'm not sure what I'm doing wrong-can anyone advise please or help me on this please?
Thanks

This same issue happened to me because Ansible now ships with its own version of the Azure collection and it was conflicting with the version I had manually installed in my own playbook using the "ansible-galaxy collection" command.
What I suggest you do is only use the version that ships with Ansible and then install its requirements like so:
pip install -r /usr/lib/python3/dist-packages/ansible_collections/azure/azcollection/requirements-azure.txt
It is easier to setup correctly on a freshly installed system (e.g in Docker) than it is to fix a broken system.

I think that you did not follow the instruction about installing azure collection from https://github.com/ansible-collections/azure
Installing collection itself does not install python dependencies, these are installed using python pip and you need to be sure you are installing them inside the same python (v)env where ansible is installed, or ansible will give you the error that you seen when trying to load the module.

Unfortunately the azure-mgmt-monitor package is bugged, even on 3.0.0 to not properly create a version submodule. Haven't been able to track down exactly where in the code it's busted, but it is and there is a direct import of that submodule in the Ansible Galaxy module causing it to fail. Unfortunately you should use the Azure CLI at this time and forget about using azure_rm

Related

List all my hetzner server names and their IPv4 addresses - getting error

I'm trying to receive a list of all my server names and their IPs on Hetzner using ansible and hcloud module however I'm receiving the following error;
ERROR! couldn't resolve module/action 'hcloud'. This often indicates a misspelling, missing collection, or incorrect module path.
The error appears to be in '/home/melvmagr/repos/ansible/server-content/server-content.yml': line 8, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
tasks:
- name: List servers
^ here
Here is my YAML file;
---
- name: List Hetzner server names and IP addresses
hosts: servername_example
gather_facts: false
vars:
hcloud_token: "MY_HETZNER_API_TOKEN"
tasks:
- name: List servers
hcloud:
api_token: "MY_HETZNER_API_TOKEN"
state: present
command: server_list
register: server_list
- name: Print server names and IP addresses
debug:
msg: "Server {{ item.name }} has IP address {{ item.public_net.ipv4.ip }}"
loop: "{{ server_list.servers }}"
More info that could prove to be helpful:
❯ ansible --version
ansible [core 2.12.10]
config file = /home/melvmagr/repos/ansible/ansible.cfg
configured module search path = ['/home/melvmagr/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /home/melvmagr/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.8.10 (default, Nov 14 2022, 12:59:47) [GCC 9.4.0]
jinja version = 2.10.1
libyaml = True
This is my ansible.cfg file;
[defaults]
inventory=./inventories
host_key_checking = False
log_path=/var/tmp/ansible_history
timeout=300
I've also installed the following collections;
ansible-galaxy collection install hetzner.hcloud
❯ ls ~/.ansible/collections/ansible_collections/hetzner/hcloud/
CHANGELOG.rst COPYING FILES.json MANIFEST.json README.md changelogs meta plugins tests
Any help would be greatly appreciated. Thank you!
Managed to solve it by simplifying my playbook the following way;
---
- name: Get server information from Hetzner Cloud
hosts: myservername
gather_facts: false
tasks:
- name: Print server names and IP addresses
debug:
msg: "{{ inventory_hostname }} IP is {{ ansible_ssh_host }}"

Is it possible to configure Azure Windows VMs using Ansible on Azure DevOps Microsoft Hosted Ubuntu agents?

We try to configure an Azure VM using an Azure DevOps pipeline. We first create the machine using Terraform and then we need to configure it. Right now the pipeline is functional when we use a customized Ubuntu Azure DevOps agent (a VM we setup ourselves in Azure).
We prefer to use a Microsoft Hosted Ubuntu Agent. When we try to run our pipeline using the Microsoft Hosted Ubuntu agent we fail with a message "winrm or requests is not installed".
We have done a lot of research and attempts to install the needed components, but none have been fruitful.
All the examples and documentation on the internet we can find don't mention our specific use case. Ansible configuration of Windows VMs in Azure from a Microsoft Hosted Ubuntu agent. Isn't it possible for some reason?
If it is, any pointers in the right direction will be much appreciated!
The error we see in the Azure DevOps pipeline is this:
ansible-playbook -vvvv -i inventory/hosts.cfg main.yml --extra-vars '{"customer_name": "<REMOVED>" }'
ansible-playbook [core 2.12.5]
config file = None
configured module search path = ['/home/vsts/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/vsts/.local/lib/python3.8/site-packages/ansible
ansible collection location = /home/vsts/.ansible/collections:/usr/share/ansible/collections
executable location = /home/vsts/.local/bin/ansible-playbook
python version = 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0]
jinja version = 2.10.1
libyaml = True
No config file found; using defaults
setting up inventory plugins
host_list declined parsing /home/vsts/work/1/s/ansible/inventory/hosts.cfg as it did not pass its verify_file() method
auto declined parsing /home/vsts/work/1/s/ansible/inventory/hosts.cfg as it did not pass its verify_file() method
yaml declined parsing /home/vsts/work/1/s/ansible/inventory/hosts.cfg as it did not pass its verify_file() method
Parsed /home/vsts/work/1/s/ansible/inventory/hosts.cfg inventory source with ini plugin
Loading collection ansible.windows from /home/vsts/.local/lib/python3.8/site-packages/ansible_collections/ansible/windows
Loading collection community.windows from /home/vsts/.local/lib/python3.8/site-packages/ansible_collections/community/windows
redirecting (type: modules) ansible.builtin.win_service to ansible.windows.win_service
redirecting (type: modules) ansible.builtin.win_service to ansible.windows.win_service
redirecting (type: modules) ansible.builtin.win_service to ansible.windows.win_service
redirecting (type: modules) ansible.builtin.win_service to ansible.windows.win_service
redirecting (type: modules) ansible.builtin.win_service to ansible.windows.win_service
Loading callback plugin default of type stdout, v2.0 from /home/vsts/.local/lib/python3.8/site-packages/ansible/plugins/callback/default.py
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.
PLAYBOOK: main.yml *************************************************************
Positional arguments: main.yml
verbosity: 4
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/home/vsts/work/1/s/ansible/inventory/hosts.cfg',)
extra_vars: ('{"customer_name": "<REMOVED>"}',)
forks: 5
1 plays in main.yml
PLAY [windows:pro] *********************************************************
TASK [Gathering Facts] *********************************************************
task path: /home/vsts/work/1/s/ansible/main.yml:1
redirecting (type: modules) ansible.builtin.setup to ansible.windows.setup
Using module file /home/vsts/.local/lib/python3.8/site-packages/ansible_collections/ansible/windows/plugins/modules/setup.ps1
Pipelining is enabled.
**fatal: [51.144.125.149]: FAILED! => {
"msg": "winrm or requests is not installed: No module named 'winrm'"
}**
PLAY RECAP *********************************************************************
51.144.125.149 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
We tried to fix the problem by installing various potentially relevant components in the pipeline just before running the ansible-playbook command, for instance this one
pip3 install pywinrm
Later, based on input on this SO question we tried this in the pipeline:
python3 -m pip install --ignore-installed pywinrm
find / -name winrm.py
ansible-playbook -vvv -i inventory/hosts.cfg main.yml
The find command finds winrm.py here:
/opt/pipx/venvs/ansible-core/lib/python3.8/site-packages/ansible/plugins/connection/winrm.py
The ansible-playbook configuration we are using is:
ansible-playbook [core 2.12.5]
config file = None
configured module search path =
['/home/vsts/.ansible/plugins/modules',
'/usr/share/ansible/plugins/modules']
ansible python module location = /opt/pipx/venvs/ansible-
core/lib/python3.8/site-packages/ansible
ansible collection location =
/home/vsts/.ansible/collections:/usr/share/ansible/collections
executable location = /opt/pipx_bin/ansible-playbook
python version = 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC
9.4.0]
jinja version = 3.1.2
libyaml = True
No config file found; using defaults
The error we get is:
task path: /home/vsts/work/1/s/ansible/main.yml:1
redirecting (type: modules) ansible.builtin.setup to
ansible.windows.setup
Using module file /opt/pipx/venvs/ansible-
core/lib/python3.8/site-
packages/ansible_collections/ansible/windows/plugins/modules/
setup.ps1
Pipelining is enabled.
fatal: [13.73.148.141]: FAILED! => {
"msg": "winrm or requests is not installed: No module named
'winrm'"
}
you can try solution in RedHat knowledgebase
https://access.redhat.com/solutions/3356681
Last comment suggestion (replace yum with apt commands)
I was getting this error even if python2-winrm version 0.3.0 is
already installed via yum
yum list installed | grep winrm python2-winrm.noarch
0.3.0-1.el7 #epel
pip install "pywinrm>=0.2.2" only resulted in "Requirement already
satisfied"
I ran this to resolve the error -
yum autoremove python2-winrm.noarch
pip install "pywinrm>=0.2.2"
Then ping: pong worked just fine over https, port=5986
ram#thinkred1cartoon$ ansible all -i hosts.txt -m win_ping
172.16.96.135 | SUCCESS => {
"changed": false,
"ping": "pong" }
conversely, if you don't want to run command 1, then command 2 won't
work for you. In that case, run command 3
3 ) pip install --ignore-installed "pywinrm>=0.2.2"

How to deploy custom rpms on to salt-minion?

I'm working on salt-stack for setting up multiple machines, I wanted to ask how can we deploy rpms(placed at a custom location in master) on to the minions? I already have an idea of how can we install packages using top.sls file and name of the package that needs to be installed on minions but what I'm looking for is to deploy my custom rpms on to the minions from master.
There are two ways to approach this:
Option 1:
Define the list of RPMs in a pillar file:
package_names:
- custom-rpm1: custom-rpm1-2.6.1-2.el7.x86_64.rpm
- custom-rpm2: custom-rpm2-release-el7-3.noarch.rpm
- custom-rpm3: custom-rpm3-latest.noarch.rpm
Then in an SLS file:
install-rpm:
pkg.installed:
- sources: {{ pillar['package_names'] }}
Option 2:
Copy the directory containing the RPMs (salt://rpms in below example is relative to file_roots) to target machine and use rpm command to install (with wildcard):
copy-rpms-dir:
file.recurse:
- name: /tmp/rpms
- source: salt://rpms
install-rpms:
cmd.run:
- name: rpm -ivh /tmp/rpms/*.rpm
- success_retcodes:
- 2
Installing with rpm command requires extra check for return codes as it returns non-zero (2) when RPM is already installed.

How to stop 'import psycopg2' from causing an Exception when starting an Azure Container?

I am trying to deploy a Django REST API using Azure App Service on Linux. I am using a postgresql Database and deploy via pipeline. Azure has postgresql 9.6. After running my pipeline, the Website shows an Server Error (500).
The AppLogs show, that the Container couldn't be started due an failed import of psycopg2.
[ERROR] Exception in worker process
Traceback (most recent call last):
File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 25, in
import psycopg2 as Database
File "/home/site/wwwroot/antenv/lib/python3.7/site-packages/psycopg2/__init__.py", line 50, in
from psycopg2._psycopg import ( # noqa
ImportError: /home/site/wwwroot/antenv/lib/python3.7/site-packages/psycopg2/_psycopg.cpython-37m-x86_64-linux-gnu.so: undefined symbol: PQencryptPasswordConn
In the Build-stage of my pipeline, I set up my environment (python3.7) like this:
- script: |
python -m venv antenv
source antenv/bin/activate
python -m pip install --upgrade pip
pip install setup
pip install -r requirements.txt
Where requirements.txt looks like this:
Django==3.0.2
djangorestframework==3.11.0
psycopg2-binary==2.8.4
pandas==0.25.3
pytest==5.3.5
pytest-django==3.8.0
pytest-mock==2.0.0
python-dateutil==2.8.1
sqlparse==0.3.0
whitenoise==5.0.1
BuildJob and DeploymentJob seem to run flawless. the Build-logs indicate that psycopg2_binary-2.8.4-cp37-cp37m-manylinux1_x86_64.whl was correctly downloaded and installed.
Also the App runs fine on my machine when using the database on azure by configuring in the settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'databasename',
'USER': 'user#postgresqlserver',
'PASSWORD': 'Password',
'HOST': 'postgresqlserver.postgres.database.azure.com',
'PORT': '',
'OPTIONS': {'sslmode': 'require'}
}
} # Of course the info is actually saved in environment variables
This gives me the feeling, that something with the psycopg2 installation is not working... For others the *psycopg2-binary seemed to do the trick but unfortunateley not for me.
Am I right to assume that on azure I'm nether able to install postgresql10 as suggested here https://github.com/psycopg/psycopg2/issues/983 nor can install from source like suggested here https://github.com/psycopg/psycopg2/issues/1018?
There must be something I am missing, I would be grateful for any advice!
EDIT:
Taking a look at the library (as suggested here https://stackoverflow.com/a/59652816/13183775) I found that I don't have a PQencryptPasswordConn function but only a PQencryptPassword function. I have the feeling that this is expected for Postgresql9.6 (https://github.com/psycopg/psycopg2/blob/cb3353be1f10590cdc2a894ada42c3b4c171feb7/psycopg/psycopgmodule.c#L466).
To check, whether there are multiple versions libpq:
/>find . -name "libpq*"
./var/lib/dpkg/info/libpq5:amd64.symbols
./var/lib/dpkg/info/libpq5:amd64.shlibs
./var/lib/dpkg/info/libpq5:amd64.list
./var/lib/dpkg/info/libpq5:amd64.triggers
./var/lib/dpkg/info/libpq-dev.list
./var/lib/dpkg/info/libpq5:amd64.md5sums
./var/lib/dpkg/info/libpq-dev.md5sums
./usr/share/doc/libpq5
./usr/share/doc/libpq-dev
./usr/share/locale/ko/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/it/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/pl/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/zh_TW/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/tr/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/cs/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/de/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/ru/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/sv/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/pt_BR/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/fr/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/es/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/zh_CN/LC_MESSAGES/libpq5-9.6.mo
./usr/share/locale/ja/LC_MESSAGES/libpq5-9.6.mo
./usr/lib/x86_64-linux-gnu/pkgconfig/libpq.pc
./usr/lib/x86_64-linux-gnu/libpq.so.5
./usr/lib/x86_64-linux-gnu/libpq.so
./usr/lib/x86_64-linux-gnu/libpq.so.5.9
./usr/lib/x86_64-linux-gnu/libpq.a
./usr/include/postgresql/libpq-events.h
./usr/include/postgresql/libpq-fe.h
./usr/include/postgresql/libpq
./usr/include/postgresql/libpq/libpq-fs.h
./usr/include/postgresql/internal/libpq
./usr/include/postgresql/internal/libpq-int.h>
Sadly I'm not able to see here wether there are multiple libpq versions...

Using Jinja template "ssh-rsa {{ v1.public_ssh_keys }}" in cloud-config gives a warning "Could not find supported sub-handler for type text/jinja2"

I've started to use cloud-config to initialize my Ubuntu 16.04 VM instances on Cloudstack.
In my cloud-config, I'm trying to use the public key of a key pair created in Cloudstack with this Jinja template :
## template: jinja
#cloud-config
cloud_init_modules:
- users-groups
users:
- default
- name: xyz
ssh-authorized-keys:
- ssh-rsa {{ v1.public_ssh_keys }}
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
I receive the following error message :
2019-07-16 19:27:35,737 - jinja_template.py[WARNING]: Ignoring jinja template for part-001. Could not find supported sub-handler for type text/jinja2
No 'init' modules to run under section 'cloud_init_modules'
The key is however correctly passed in the instance data :
# cloud-init query v1.public_ssh_keys
[
"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0EyQVA+A+CnrVBo7psi4XuZIbS6Zxq3JtiJK/wCZoPB3vKdvtDtwYkrFf1gBg4rzLDJiTTbbYZaPDa0IP82jdwSBBTzjJbF9mbeJfG70DbWqgzxzU40/fJibD1GqsEmFGh9jDAyyGLDuYo5o6LEprr3n6WIDbcsIlZFSr648RhpvTT8RmAcBj26/fkLsG2lSsQlNkWelSFj4kTn5FRcEbFb1QwwAwMza4gCqz4x3pmoqB/tSc81YRVDosik972j3+Np7Unz16PBbe6Sri5JYZaT5muS7baY9M87Nu/cOKZkr6kkxpGTHIKv6tW71f/CQVcTLEz9IG55xghAjVq9Kv"
]
I've tried with the package python-jinja2 installed (apt-get install -y python-jinja2) but it doesn't fix my issue.