Mongodb Lookup error in Ansible playbook - mongodb

I am using mongoDB with ansible where I am trying to query to mongodb collection using ansible playbook.
main.yaml(vars file where i have declared all variables)
source_dir_qry_res: "/home/dpatel/Desktop/query_output/"
mongodb_parameters:
- { collection: '1.1.1.1-mx', filter: {"config.groups.interfaces.interface.name" : "xyz"} , qry_res_file: 'allconfig' }
In above file, 'filter' is the query run against mongodb collection. I used filter that way by referring "Mongo Lookup inside this Ansible lookup doc http://docs.ansible.com/ansible/playbooks_lookups.html"
#main_run.yaml
- name: query from mongodb collection
connection: local
gather_facts: no
vars_files:
- 'vars/main.yml'
tasks:
- name: "query to db"
query_config:
collection: "{{ item.collection }}"
filter: "{{ item.filter}}"
qry_res_file: "{{ item.qry_res_file}}"
source_dir: "{{ source_dir_qry_res }}"
with_items: "{{ mongodb_parameters }}"
tags: "save-query-result"
query_config (custom ansible module)
def main():
module = AnsibleModule(
argument_spec=dict(
#host=dict(required=True),
collection=dict(required=False),
qry_res_file=dict(required=False),
filter = dict(required=False),
source_dir=dict(required=True),
logfile=dict(required=False, default=None)),
supports_check_mode=True)
m_args = module.params
m_results = dict(changed=False)
try:
conn = pymongo.MongoClient()
#db = conn.m_args['dtbase']
db = conn.mydb
coll_name = m_args['collection']
print "Connected successfully!!!"
except Exception as ex:
module.fail_json(msg='mongodb connection error: %s' % ex.message)
try:
query = db[coll_name].find(m_args['filter'])
if(query):
target_json_file = m_args['source_dir']+m_args['collection'] + m_args['qry_res_file'] + ".json"
for ele in query:
del ele['_id']
with open(target_json_file, 'a') as the_file:
json.dump(ele, the_file)
except Exception as ex:
module.fail_json(msg="failed to get query result: %s" %ex.message)
module.exit_json(**m_results)
from ansible.module_utils.basic import *
if __name__ == '__main__':
main()
mongo collection (mydb[1.1.1.1-mx].find())
{"config": { "groups": {"name": "123", "system": {"host-name": "something"}, "interfaces": {"interface": {"name": "xyz"}}}}}
when I run below query from mongo shell, it's working fine.
mydb[1.1.1.1-mx].find({"config.groups.interfaces.interface.name" : "xyz"})
but when I tried to run same query though the code it gaves me below error
$ ansible-playbook -i hosts main_run.yml
error msg:"failed to get query result: filter must be an instance of dict, bson.son.SON, or other type that inherits from collections.Mapping"
please see these screenshot for detail error message. If anyone have any idea to solve this problem please share your thoughts that would be really helpful.

I would try to set required type in your argument_spec
argument_spec=dict(
collection = dict(required=False),
qry_res_file = dict(required=False),
filter = dict(required=False, type='dict'),
source_dir = dict(required=True),
logfile = dict(required=False, default=None))
because, by default argument type is string, so you end up with your filter object converted to string (see incovation.module_args in your output).

Related

mongo shell command not accept use db command

i am trying to pull records form mongo data base with json format to do that i am running below js :
conatin of get_DRMPhysicalresources_Data.js
cursor = db.physicalresources.find().limit(10)
while ( cursor.hasNext() ){
print( JSON.stringify(cursor.next()) );
}
the command i run to get the records :
mongo host_name:port/data_base_name -u user -p 'password' --eval "load(\"get_DRMPhysicalresources_Data.js\")" > DRMPhysicalresources.json
and i am able to get the result as josn formate inside DRMPhysicalresources.json , now i want to switch to other data base using use command i try add use db as below :
conatin of get_DRMPhysicalresources_Data.js
use db2_test
cursor = db.physicalresources.find().limit(10)
while ( cursor.hasNext() ){
print( JSON.stringify(cursor.next()) );
}
the command i run to get the records :
mongo host_name:port/data_base_name -u user -p 'password' --eval "load(\"get_DRMPhysicalresources_Data.js\")" > DRMPhysicalresources.json
but i am getting below errors :
MongoDB shell version v4.2.3
connecting to: "some data base info"
Implicit session: session { "id" : UUID("8c85c6af-ebed-416d-9ab8-d6739a4230cb") }
MongoDB server version: 4.4.11
WARNING: shell and server versions do not match
2022-04-11T13:39:30.121+0300 E QUERY [js] uncaught exception: SyntaxError: unexpected token: identifier :
#(shell eval):1:1
2022-04-11T13:39:30.122+0300 E QUERY [js] Error: error loading js file: get_DRMPhysicalresources_Data.js :
#(shell eval):1:1
2022-04-11T13:39:30.122+0300 E - [main] exiting with code -4
is there are any way to add use db2_test without break it ?
You can try this:
db = new Mongo().getDB("myOtherDatabase");
or
db = db.getSiblingDB('myOtherDatabase')
( this is the exact js equivalent to the 'use database' helper)
docs

Having issues with rs.add() in ansible playbook for mongo

I am using below tasks in my playbook to initialize cluster and add secondary to primary:
- name: Initialize replica set
run_once: true
delegate_to: host1
shell: >
mongo --eval 'printjson(rs.initiate())'
- name: Format secondaries
run_once: true
local_action:
module: debug
msg: '"{{ item }}:27017"'
with_items: ['host2', 'host3']
register: secondaries
- name: Add secondaries
run_once: true
delegate_to: host1
shell: >
/usr/bin/mongo --eval 'printjson(rs.add({{ item.msg }}))'
with_items: secondaries.results
I am getting below error:
TASK [mongodb-setup : Add secondaries] *******************************
fatal: [host1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'msg'\n\nThe error appears to have been in '/var/lib/awx/projects/_dev/roles/mongodb-setup/tasks/users.yml': line 15, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Add secondaries\n ^ here\n"}
Thanks for the response, I have amended my code as below
- name: Add secondaries
run_once: true
delegate_to: host-1
shell: >
/usr/bin/mongo --eval 'printjson(rs.add({{ item }}:27017))'
with_items:
- host2
- host3
but getting below error
failed: [host-2 -> host-1] (item=host-2) => {"changed": true, "cmd": "/usr/bin/mongo --eval 'printjson(rs.add(host-2:27017))'", "delta": "0:00:00.173077", "end": "2019-08-06 13:29:09.422560", "item": "host-2", "msg": "non-zero return code", "rc": 252, "start": "2019-08-06 13:29:09.249483", "stderr": "", "stderr_lines": [], "stdout": "MongoDB shell version: 3.2.22\nconnecting to: test\n2019-08-06T13:29:09.419-0500 E QUERY [thread1] SyntaxError: missing ) after argument list #(shell eval):1:37", "stdout_lines": ["MongoDB shell version: 3.2.22", "connecting to: test", "2019-08-06T13:29:09.419-0500 E QUERY [thread1] SyntaxError: missing ) after argument list #(shell eval):1:37"]}
You issue is not with rs.add() but with the data you loop over. In your last task, your item list is a single string.
# Wrong #
with_items: secondaries.results
You want to pass an actual list form your previously registered result:
with_items: "{{ secondaries.results }}"
That being said, registering the result of a debug task is rather odd. You should use set_fact to register what you need in a var, or better directly loop other your list of hosts in your task. It also looks like the rs.add funcion is exepecting a string so you should quote the argument in your eval. Something like:
- name: Add secondaries
shell: >
/usr/bin/mongo --eval 'printjson(rs.add("{{ item }}:27017"))'
with_items:
- host2
- host3
And the way you use delegation seems rather strange to me in this context but it's hard to give any valid clues without a complete playbook example of what you are trying to do (that you might give in a new question if necessary).

> show dbs 2017-04-25T12:02:12.277-0400 E QUERY Error: listDatabases failed:{

I created a one node Mongo database, everything went fine and I'm able to enter mongo shell and execute commands like "db","use test" , but that's about it , I'm not able to execute any other commands like "show dbs" or insert anything. I googled as there are lot of threads on this error with some suggesting to run rs.slaveOk() and some others asking to create user, I tried creating user as below:
db.createUser(
{
user: "okkadu",
pwd: "abc123",
roles: [ { role: "root", db: "test" } ]
}
)
However this also failed with below error:
Error: couldn't add user: not authorized on test to execute command
Could someone help me out what else can I try ?, please find the copy if my config file as well:
##
**### Basic Defaults
##
port = 27017
dbpath = /opt/mongodb/data
logappend = true
logpath = /var/log/mongodb/mongodb.log
rest = true
fork = true
#bind_ip = 127.0.0.1
pidfilepath = /var/log/mongodb/mongodb.pid
journal = true
httpinterface = true
storageEngine = wiredTiger
auth=true**

Mongodb authentication using elixir-mongo

I have just started using Elixir, so I figure I have some basic misunderstanding going on here. Here is the code...
defmodule Mdb do
def connect(collection, this_db \\ "db-test") do
{:ok, mongo} = Mongo.connect("db-test.some-mongo-server.com", 12345)
db = mongo |> Mongo.db(this_db)
db |> Mongo.auth("user", "secretpassword")
db
end
end
I start with iex -S mix
and when I try db = Mdb.connect("users") I get
** (UndefinedFunctionError) undefined function: Mongo.auth/3
Mongo.auth(%Mongo.Db{auth: nil, mongo: %Mongo.Server{host: 'db-test.some-mongo-server.com', id_prefix: 12641, mode: :passive, opts: %{}, port: 12345, socket: #Port<0.5732>, timeout: 6000}, name: "db-stage", opts: %{mode: :passive, timeout: 6000}}, "user", "secretpassword")
(mdb_play) lib/mdb.ex:7: Mdb.connect/2
I looks like Mongo.auth/3 is undefined, but that makes no sense to me. Can any one point me towards my error?
thanks for the help
I just played around it, and faced the same error. As in the error message, Mongo.auth seems not defined, and it might be Mongo.Db.auth instead. However, I faced another error (ArgumentError) on Mongo.Db.auth too. It may be certain issue in the library.
** (ArgumentError) argument error
:erlang.byte_size
...
(mongo) lib/mongo_request.ex:43: Mongo.Request.cmd/3
(mongo) lib/mongo_db.ex:44: Mongo.Db.auth/1
I'm not familiar with the library, but after small change in Mongo.Db.auth, normal call seems started working.
I tried with the following sequence.
mongo = Mongo.connect!(server, port)
db = mongo |> Mongo.db(db_name)
db |> Mongo.Db.auth(user_name, password)
collection = db |> Mongo.Db.collection(collection_name)
collection |> Mongo.Collection.count()
The change I tried is in the following fork-repo.
https://github.com/parroty/elixir-mongo

Pass a .js file to mongo db.eval()

I am trying to get variety working with db.eval() instead of commandline --eval. This is because I get some problems with authentication.
The normal use is from the commandline:
$ mongo test --eval "var collection = 'users', maxDepth = 3" /path/to/variety.js
What I am trying to do is this:
$ mongo
>> use admin
>> db.auth("foo", "bar")
>> use anotherColl
>> db.eval("/path/to/variety.js", "var collection = 'users', maxDepth = 3")
>> Thu Mar 7 13:19:10 uncaught exception: {
"errmsg" : "compile failed: JS Error: SyntaxError: invalid flag after regular expression nofile_a:0",
"ok" : 0
}
Is there a way to have db.eval() eat an javascript file instead of an string?
According to #Sammaye and my gut feeling you can't put in .js files in db.eval(). However for variety there is an solution described here.
mongo -u USER-p PASS admin --eval "var collection = 'COL', db_name='DB_NAME'" variety.js
And now the authentication problem :(