Having issues with rs.add() in ansible playbook for mongo - mongodb

I am using below tasks in my playbook to initialize cluster and add secondary to primary:
- name: Initialize replica set
run_once: true
delegate_to: host1
shell: >
mongo --eval 'printjson(rs.initiate())'
- name: Format secondaries
run_once: true
local_action:
module: debug
msg: '"{{ item }}:27017"'
with_items: ['host2', 'host3']
register: secondaries
- name: Add secondaries
run_once: true
delegate_to: host1
shell: >
/usr/bin/mongo --eval 'printjson(rs.add({{ item.msg }}))'
with_items: secondaries.results
I am getting below error:
TASK [mongodb-setup : Add secondaries] *******************************
fatal: [host1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'msg'\n\nThe error appears to have been in '/var/lib/awx/projects/_dev/roles/mongodb-setup/tasks/users.yml': line 15, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Add secondaries\n ^ here\n"}
Thanks for the response, I have amended my code as below
- name: Add secondaries
run_once: true
delegate_to: host-1
shell: >
/usr/bin/mongo --eval 'printjson(rs.add({{ item }}:27017))'
with_items:
- host2
- host3
but getting below error
failed: [host-2 -> host-1] (item=host-2) => {"changed": true, "cmd": "/usr/bin/mongo --eval 'printjson(rs.add(host-2:27017))'", "delta": "0:00:00.173077", "end": "2019-08-06 13:29:09.422560", "item": "host-2", "msg": "non-zero return code", "rc": 252, "start": "2019-08-06 13:29:09.249483", "stderr": "", "stderr_lines": [], "stdout": "MongoDB shell version: 3.2.22\nconnecting to: test\n2019-08-06T13:29:09.419-0500 E QUERY [thread1] SyntaxError: missing ) after argument list #(shell eval):1:37", "stdout_lines": ["MongoDB shell version: 3.2.22", "connecting to: test", "2019-08-06T13:29:09.419-0500 E QUERY [thread1] SyntaxError: missing ) after argument list #(shell eval):1:37"]}

You issue is not with rs.add() but with the data you loop over. In your last task, your item list is a single string.
# Wrong #
with_items: secondaries.results
You want to pass an actual list form your previously registered result:
with_items: "{{ secondaries.results }}"
That being said, registering the result of a debug task is rather odd. You should use set_fact to register what you need in a var, or better directly loop other your list of hosts in your task. It also looks like the rs.add funcion is exepecting a string so you should quote the argument in your eval. Something like:
- name: Add secondaries
shell: >
/usr/bin/mongo --eval 'printjson(rs.add("{{ item }}:27017"))'
with_items:
- host2
- host3
And the way you use delegation seems rather strange to me in this context but it's hard to give any valid clues without a complete playbook example of what you are trying to do (that you might give in a new question if necessary).

Related

Ansible: How to read file and push results to templates

I have task to read data from csv file and push result to templates and copy those templates to different servers. however, i am getting error while writing to template. below are details -
main.yml
- name: Print return information from the previous task
vars:
test_csv: "{{ lookup('file', '/u00/ansible/Playbooks/files/newrelic_test.csv', wantlist=True) }}"
ansible.builtin.debug:
var: test_csv
- name: copy template
template:
src: /u00/ansible/Playbooks/files/infra-config.yml_template
dest: /u00/app/monitor/infra-config.yml
with_items: test_csv
notify: confirm copy done
- name: Start the New Relic Service
ansible.builtin.systemd:
name: infra.service
state: started
become: yes
become_user: root
infra-config.yml_template -
custom_attributes:
application : {{ item.Application }}
env : {{ item.env }}
datacenter : {{ item.Datacenter }}
log:
file: /u00/app/monitor/infra.log
csv file content
Application,Host,env,Datacenter
Microsoft,testserver1,TEST,DC1
Apple,testserver2,TEST,DC2
error -
> TASK [config-update : copy template]
> ******************************************* [0;31mAn exception occurred during task execution. To see the full traceback, use -vvv.
> The error was: ansible.errors.AnsibleUndefinedVariable:
> 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute
> 'db_name'[0m [0;31mfailed: [testserver1]
> (item=test_csv) => {"ansible_loop_var": "item", "changed": false,
> "item": "test_csv", "msg": "AnsibleUndefinedVariable:
> 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute
> 'db_name'"}
Expectation is to read csv file and use variables in template in different servers.
testserver1 -
> custom_attributes: application : Microsoft env : Test datacenter : DC1
> log: file: /u00/app/monitor/infra.log
testserver2 -
> custom_attributes: application : Apple env : Test datacenter : DC1
> log: file: /u00/app/monitor/infra.log
There are few things to fix in your playbook. First, you are defining your test_csv variable inside a task and it cannot be accessible by other tasks. You can use register instead. However, the first task returns a list but with one string like this "test_csv": ["Application,Host,env,Datacenter\nMicrosoft,testserver1,TEST,DC1 \nApple,testserver2,TEST,DC2"] which only results one item in the test_csv list.
You can achieve this by using read_csv module as well. Below I demonstrate how:
Note that I have added a condition using inventory_hostname on the copy template task since you might want to target each csv line according to its hostname. You can modify this according to your needs.
Csv file content:
Application,Host,env,Datacenter
Microsoft,localhost,TEST,DC1
Apple,testserver2,TEST,DC2
Example of playbook for testing:
- name: Check status
hosts: localhost
gather_facts: no
tasks:
- name: read csv file and return a list
read_csv:
path: test.csv
register: applications
- name: Ouput applications from previous task
debug:
msg: "{{ item.Application }}"
loop: "{{ applications.list }}"
- name: copy template
template:
src: src.yml_template ##I would recommendr to use .j2 jinja template instead.
dest: dest.yml
loop: "{{ applications.list }}"
when: inventory_hostname == item.Host
src.yml_template content:
custom_attributes:
application : {{ item.Application }}
env : {{ item.env }}
datacenter : {{ item.Datacenter }}
log:
file: /u00/app/monitor/infra.log
Gives in dest.yml:
custom_attributes:
application : Microsoft
env : TEST
datacenter : DC1
log:
file: /u00/app/monitor/infra.log
Cli output:
PLAY [Check status] **********************************************************************************************************************************************************
TASK [read csv file and return a list] ***************************************************************************************************************************************
ok: [localhost]
TASK [Ouput applications from previous task] *********************************************************************************************************************************
ok: [localhost] => (item={'Application': 'Microsoft', 'Host': 'localhost', 'env': 'TEST', 'Datacenter': 'DC1 '}) => {
"msg": "Microsoft"
}
ok: [localhost] => (item={'Application': 'Apple', 'Host': 'testserver2', 'env': 'TEST', 'Datacenter': 'DC2'}) => {
"msg": "Apple"
}
TASK [copy template] *********************************************************************************************************************************************************
changed: [localhost] => (item={'Application': 'Microsoft', 'Host': 'localhost', 'env': 'TEST', 'Datacenter': 'DC1 '})
skipping: [localhost] => (item={'Application': 'Apple', 'Host': 'testserver2', 'env': 'TEST', 'Datacenter': 'DC2'})

ERROR Configuring mongoDB using Ansible (MongoNetworkError: connect ECONNREFUSED)

I'm trying to configure a replicaset of mongodb using ansible,
I succeeded to install mongoDB on the primary server and created the replica-set configuration file except when I launch the playbook, I get an error of type: MongoNetworkError: connect ECONNREFUSED 3.142.150.62:28041
Does anyone have an idea please how to solve this?
attached, the playbook and the error on the Jenkins console
Playbook:
---
- name: Play1
hosts: hhe
#connection: local
become: true
#remote_user: ec2-user
#remote_user: root
tasks:
- name: Install gnupg
package:
name: gnupg
state: present
- name: Import the public key used by the package management system
shell: wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -
- name: Create a list file for MongoDB
shell: echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list
- name: Reload local package database
command: sudo apt-get update
- name: Installation of mongodb-org
package:
name: mongodb-org
state: present
update_cache: yes
- name: Start mongodb
service:
name: mongod
state: started
enabled: yes
- name: Play2
hosts: hhe
become: true
tasks:
- name: create directories on all the EC2 instances
shell: mkdir -p replicaset/member
- name: Play3
hosts: secondary1
become: true
tasks:
- name: Start mongoDB with the following command on secondary1
shell: nohup mongod --port 28042 --bind_ip localhost,ec2-18-191-39-71.us-east-2.compute.amazonaws.com --replSet replica_demo --dbpath replicaset/member &
- name: Play4
hosts: secondary2
become: true
tasks:
- name: Start mongoDB with the following command on secondary2
shell: nohup mongod --port 28043 --bind_ip localhost,ec2-18-221-31-81.us-east-2.compute.amazonaws.com --replSet replica_demo --dbpath replicaset/member &
- name: Play5
hosts: arbiter
become: true
tasks:
- name: Start mongoDB with the following command on arbiter
shell: nohup mongod --port 27018 --bind_ip localhost,ec2-13-58-35-255.us-east-2.compute.amazonaws.com --replSet replica_demo --dbpath replicaset/member &
- name: Play6
hosts: primary
become: true
tasks:
- name: Start mongoDB with the following command on primary
shell: nohup mongod --port 28041 --bind_ip localhost,ec2-3-142-150-62.us-east-2.compute.amazonaws.com --replSet replica_demo --dbpath replicaset/member &
- name: Create replicaset initialize file
copy:
dest: /tmp/replicaset_conf.js
mode: "u=rw,g=r,o=rwx"
content: |
var cfg =
{
"_id" : "replica_demo",
"version" : 1,
"members" : [
{
"_id" : 0,
"host" : "3.142.150.62:28041"
},
{
"_id" : 1,
"host" : "18.191.39.71:28042"
},
{
"_id" : 2,
"host" : "18.221.31.81:28043"
}
]
}
rs.initiate(cfg)
- name: Pause for a while
pause: seconds=20
- name: Initialize the replicaset
shell: mongo /tmp/replicaset_conf.js
The error on Jenkins Consol:
PLAY [Play6] *******************************************************************
TASK [Gathering Facts] *********************************************************
ok: [primary]
TASK [Start mongoDB with the following command on primary] *********************
changed: [primary]
TASK [Create replicaset initialize file] ***************************************
ok: [primary]
TASK [Pause for a while] *******************************************************
Pausing for 20 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [primary]
TASK [Initialize the replicaset] ***********************************************
fatal: [primary]: FAILED! => {"changed": true, "cmd": "/usr/bin/mongo 3.142.150.62:28041 /tmp/replicaset_conf.js", "delta": "0:00:00.146406", "end": "2022-08-11 09:46:07.195269", "msg": "non-zero return code", "rc": 1, "start": "2022-08-11 09:46:07.048863", "stderr": "", "stderr_lines": [], "stdout": "MongoDB shell version v5.0.10\nconnecting to: mongodb://3.142.150.62:28041/test?compressors=disabled&gssapiServiceName=mongodb\nError: couldn't connect to server 3.142.150.62:28041, connection attempt failed: SocketException: Error connecting to 3.142.150.62:28041 :: caused by :: Connection refused :\nconnect#src/mongo/shell/mongo.js:372:17\n#(connect):2:6\nexception: connect failed\nexiting with code 1", "stdout_lines": ["MongoDB shell version v5.0.10", "connecting to: mongodb://3.142.150.62:28041/test?compressors=disabled&gssapiServiceName=mongodb", "Error: couldn't connect to server 3.142.150.62:28041, connection attempt failed: SocketException: Error connecting to 3.142.150.62:28041 :: caused by :: Connection refused :", "connect#src/mongo/shell/mongo.js:372:17", "#(connect):2:6", "exception: connect failed", "exiting with code 1"]}
You start the service already with
service:
name: mongod
state: started
enabled: yes
thus shell: nohup mongod ... & is pointless. You cannot start the mongod service multiple times, unless you use different port and dbPath. You should prefer to start the mongod as service, i.e. systemctl start mongod or similar instead of nohup mongod ... &. I prefer to use the configuration file (typically /etc/mongod.conf) rather than command line options.
Plain mongo command uses the default port 27017, i.e. it does not connect to the MongoDB instances you started in above task.
You should wait till replica set is initated. You can do it like this:
content: |
var cfg =
{
"_id" : "replica_demo",
"version" : 1,
"members" : [
{
"_id" : 0,
"host" : "3.142.150.62:28041"
},
{
"_id" : 1,
"host" : "18.191.39.71:28042"
},
{
"_id" : 2,
"host" : "18.221.31.81:28043"
}
]
}
rs.initiate(cfg)
while (! db.hello().isWritablePrimary ) { sleep(1000) }
You configured an ARBITER. However, an arbiter node is useful only with an even number of Replica Set members. With 3 members it does not make much sense. Anyway, you don't add the arbiter to your Replica Set, so what is the reason to define it?
Just a note, you don't have to create a temp file, you can execute script directly, e.g. similar to this:
shell:
cmd: mongo --eval '{{ script }}'
executable: /bin/bash
vars:
script: |
var cfg =
{
"_id" : "replica_demo",
...
}
rs.initiate(cfg)
while (! db.hello().isWritablePrimary ) { sleep(1000) }
print(rs.status().ok)
register: ret
failed_when: ret.stdout_lines | last != "1"
Be aware of correct quoting.

Ansible failed_when statement

In the following task I want to make ansible not to fail when stderr contains the object has been modified or have a resource type.
- name: List nodes
delegate_to: localhost
shell: kubectl get nodes
run_once: true
register: output_error
failed_when: (output_error.stderr_lines | length > 0) and ("'the object has been modified' not in output_error.stderr or 'have a resource type' not in output_error.stderr")
However I am reveiving the following error message:
kubectl get nodes", "delta": "0:00:00.498169", "end": "2022-08-05 11:08:01.386024", "failed_when_result": true, "item": "nodes", "rc": 0, "start": "2022-08-05 11:08:00.887855", "stderr": "error: the server doesn't have a resource type \"nodes\"", "stderr_lines": ["error: the server doesn't have a resource type \"nodes\""], "stdout": "", "stdout_lines": []}
Use a regex which includes both alternatives:
- name: List nodes
delegate_to: localhost
shell: kubectl get nodes
run_once: true
register: output_error
failed_when:
- output_error.stderr_lines | length > 0
- output_error.stderr is not regex('the object has been modified|have a resource type')
For easier reading and maintenance you can set your positive matches in a list and join them together in an expression.
- name: List nodes
vars:
acceptable_err_matches:
- the object has been modified
- have a resource type
match_regex: acceptable_err_matches | map('regex_escape') | join('|')
delegate_to: localhost
shell: kubectl get nodes
run_once: true
register: output_error
failed_when:
- output_error.stderr_lines | length > 0
- output_error.stderr is not regex(match_regex)

How can I use '--extra-vars' for replicas in Ansible playbooks?

I am trying to set a default value of 1 replicas for pod deployment but also I would like to have the option to change the value by using --extra-vars="pod_replicas=2". I have tried the following but it doesn't work for me.
vars:
- pod_replicas: 1
spec:
replicas: "{{ pod_replicas }}"
ERROR:
TASK [Create a deployment]
fatal: [localhost]: FAILED! => {"changed": false, "error": 422, "msg": "Failed to patch object: b'{\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"m essage\":\" \\\\\"\\\\\" is invalid: patch: Invalid value: \\\\\"{\\\\\\\\\\\\\"apiVersion\\\\\\\\\\\\\":\\\\\\\\\\\\\"apps/v1\\\\\\\\\\\\\",\\\\\\\\\\\\\"kind\\\\\\\\\\\\\":\\\\\\\\\ \\\\"Deployment\\\\\\\\\\\\\",\\\\\\\\\\\\\"metadata\\\\\\\\\\\\\":{\\\\\\\\\\\\\"annotations\\\\\\\\\\\\\":{\\\\\\\\\\\\\"deployment.kubernetes.io/revision\\\\\\\\\\\\\":\\\\\\\\\\\\ \"1\\\\\\\\\\\\\"},\\\\\\\\\\\\\
(...)
\\"2022-02-14T12:13:38Z\\\\\\\\\\\\\",\\\\\\\\\\\\\"lastTransitionTime\\\\\\\\\\\\\":\\\\\\\\\\\\\"2022-02-14T12:13:33Z\\\\\\\\\\\\\",\\\\\\\\\\\\\"reason\\\\\\\\\\\\\":\\\\\\\\\\ \\\"NewReplicaSetAvailable\\\\\\\\\\\\\",\\\\\\\\\\\\\"message\\\\\\\\\\\\\":\\\\\\\\\\\\\"ReplicaSet \\\\\\\\\\\\\\\\\\\\\\\\\\\\\"ovms-deployment-57c9bbdfb8\\\\\\\\\\\\\\\\\\\\\\\\\ \\\\" has successfully progressed.\\\\\\\\\\\\\"},{\\\\\\\\\\\\\"type\\\\\\\\\\\\\":\\\\\\\\\\\\\"Available\\\\\\\\\\\\\",\\\\\\\\\\\\\"status\\\\\\\\\\\\\":\\\\\\\\\\\\\"True\\\\\\\\ \\\\\",\\\\\\\\\\\\\"lastUpdateTime\\\\\\\\\\\\\":\\\\\\\\\\\\\"2022-02-14T14:18:33Z\\\\\\\\\\\\\",\\\\\\\\\\\\\"lastTransitionTime\\\\\\\\\\\\\":\\\\\\\\\\\\\"2022-02-14T14:18:33Z\\\ \\\\\\\\\\",\\\\\\\\\\\\\"reason\\\\\\\\\\\\\":\\\\\\\\\\\\\"MinimumReplicasAvailable\\\\\\\\\\\\\",\\\\\\\\\\\\\"message\\\\\\\\\\\\\":\\\\\\\\\\\\\"Deployment has minimum availabili ty.\\\\\\\\\\\\\"}]}}\\\\\": v1.Deployment.Spec: v1.DeploymentSpec.Replicas: readUint32: unexpected character: \\\\ufffd, error found in #10 byte of ...|eplicas\\\\\":\\\\\"1\\\\\",\\ \\\"revisi|..., bigger context ...|\\\\\"spec\\\\\":{\\\\\"progressDeadlineSeconds\\\\\":600,\\\\\"replicas\\\\\":\\\\\"1\\\\\",\\\\\"revisionHistoryLimit\\\\\":10,\\\\\"selector\\\\\ ":{\\\\\"matchLab|...\",\"field\":\"patch\"}]},\"code\":422}\\n'", "reason": "Unprocessable Entity", "status": 422}
Any idea how I can fix this?? Thank you!
Regarding your question
How can I use --extra-vars in Ansible playbooks?
you may have a look into Understanding variable precedence, Using -e extra variables at the command line and the following small test setup
---
- hosts: localhost
become: false
gather_facts: false
vars:
REPLICAS: 1
tasks:
- name: Show value
debug:
msg: "{{ REPLICAS }} in {{ REPLICAS | type_debug }}"
which will for a run with
ansible-playbook vars.yml
result into an output of
TASK [Show value] ******
ok: [localhost] =>
msg: 1 in int
and for a run with
ansible-playbook --extra-vars="REPLICAS=2" vars.yml
into an output of
TASK [Show value] ******
ok: [localhost] =>
msg: 2 in unicode
Because of the error message
v1.Deployment.Spec: v1.DeploymentSpec.Replicas: readUint32: unexpected character: \\\\ufffd, error found in #10 byte of ...|eplicas\\\\\":\\\\\"1\\\\\"
I've introduced the type_debug filter. Maybe it will be necessary to cast the data type to integer.
- name: Show value
debug:
msg: "{{ REPLICAS }} in {{ REPLICAS | int | type_debug }}"
Further Occurences
When I've been tying numeric values from a variable file, they've been resolved as string not numbers
I have found a solution. Using a json object as an argumet seems to work:
ansible-playbook --extra-vars '{ "pod_replicas":2 }' <playbook>.yaml

Is it possible to change a variable's value in ansible?

I wrote a playbook that read a content of two files. The first one is responsible for holding switches interfaces dynamically that have the protocol CDP.
example.cdp:
0/0
14/0
The second one (.cfg), is a file the contains also dynamically a bunch of interfaces that I need to push to a device using the cisco command "shutdown" to test my master/backup environment. If the interfaces of the example.cdp are here, I need to remove them because I cannot lose the communication with this device since the management is in-band.
example.cfg:
interface FastEthernet0/0
shutdown
interface FastEthernet1/0
shutdown
interface FastEthernet2/0
shutdown
interface FastEthernet2/1
shutdown
...
interface FastEthernet14/0
shutdown
playbook:
- name: Looping file
debug:
msg: "{{ item }}"
register: items
with_file:
- ~/ANSIBLE/{{ inventory_hostname }}.cfg
- debug: var=items.results[0].item
- name: capturing interfaces with cdp
raw: egrep '[0-9]+\/[0-9]+ ' -o ~/ANSIBLE/{{ inventory_hostname }}.cdp
register: cdp
- debug: var=cdp.stdout_lines
- set_fact:
cdp: "{{cdp.stdout_lines}}"
- debug: var=cdp
- name: Removing interfaces with cdp
raw: sed 's/interface FastEthernet{{item}}//' ~/ANSIBLE/{{ inventory_hostname }}.cfg
with_items:
- "{{cdp}}"
register: items
- debug: var=items
- name: Applying The Shutdown Template
ios_config:
lines:
- "{{ items.results[0].item }}"
provider: "{{cli}}"
register: shut1
- debug: var=shut1
tags: shut1
running the playbook:
<169.255.0.1> EXEC sed 's/interface FastEthernet0/0 //' ~/ANSIBLE /169.255.0.1.cfg
failed: [169.255.0.1] (item=0/0 ) => {
"changed": true,
"failed": true,
"item": "0/0 ",
"rc": 1,
"stderr": "sed: -e expression #1, char 30: unknown option to `s'\n",
"stdout": "",
"stdout_lines": []
}
<169.255.0.1> EXEC sed 's/interface FastEthernet14/0 //' ~/ANSIBLE/169.255.0.1.cfg
failed: [169.255.0.1] (item=14/0 ) => {
"changed": true,
"failed": true,
"item": "14/0 ",
"rc": 1,
"stderr": "sed: -e expression #1, char 31: unknown option to `s'\n",
"stdout": "",
"stdout_lines": []
}
As you can see, the problem is the content of the var "cdp". The interfaces have the symbol "/", wich is use in "sed" command and I should backslashed this one to solve my problem using ansible. Is there a way to open a variable and make some regsub on it?
sed can use any character as the regex tokenizer, so solve your issue quickly, turn it into (for instance using # character):
sed 's#interface FastEthernet{{item}}##' ~/ANSIBLE/{{ inventory_hostname }}.cfg
I have the impression templating would be a better way to write your tasks though.