I am using the new ReplicaSetConnection method for making a connection to my mongodb cluster. The change really comes down to replacing pymongo.Connection with pymongo.ReplicaSetConnection. I use the connection for my purposes and then I call end_request on the connection to make sure I flush the connection before I call disconnect() on the connection. This ensures that I dont have a large collection of half-connected sockets after a long run. This works great when I use Connection, but when I use ReplicaSetConnection pymongo complains that I'm trying to run end_request() on a database object despite the fact that i am most definitely calling this against the ReplicaSetConnection object. Is this something new in pymongo or is this an error in the driver? Below is a manual run through of the problem I'm experiencing.
>>> import pymongo
>>> s = pymongo.ReplicaSetConnection("192.168.1.1:27017, 192.168.1.2:27017", replicaSet='rep1', safe=True)
>>> s
ReplicaSetConnection([u'192.168.1.1:27017', u'192.168.1.2:27017'])
>>> s.read_preference = pymongo.ReadPreference.SECONDARY
>>> s
ReplicaSetConnection([u'192.168.1.1:27017', u'192.168.1.2:27017'])
>>> type(s)
<class 'pymongo.replica_set_connection.ReplicaSetConnection'>
>>> d = s['test']
>>> s.end_request()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.linux-x86_64/egg/pymongo/database.py", line 696, in __call__
TypeError: 'Database' object is not callable. If you meant to call the 'end_request' method on a 'Connection' object it is failing because no such method exists.
>>> s.disconnect()
>>> s
ReplicaSetConnection([u'192.168.1.1:27017', u'192.168.1.2:27017'])
ReplicaSetConnection in PyMongo 2.1 doesn't support end_request(); it will in version 2.2 to be released in the next couple weeks. Meanwhile, there's no need to call end_request() before disconnect. Disconnect will close all sockets.
Related
I created a MongoDB replica set and I am testing the failover scenarios.
I am using pymongo as my driver and I am connecting using an mongodb+srv:// connection string and I set my readPreference to primaryPreferred.
However when I shut down my primary server I get the following error from pymongo (prior to the secondary assuming the role of the primary, once the secondary is primary it works as expected), my question is that since I am using a readPreference of primaryPreferred shouldn't the read be directed to the secondary and not error out?
Traceback (most recent call last):
File ".\main.py", line 86, in <module>
main()
File ".\main.py", line 18, in main
commands[command](*args)
File ".\main.py", line 26, in print_specs
for db in client.list_databases():
File "C:\Python38\lib\site-packages\pymongo\mongo_client.py", line 1890, in list_databases
res = admin._retryable_read_command(cmd, session=session)
File "C:\Python38\lib\site-packages\pymongo\database.py", line 748, in _retryable_read_command
return self.__client._retryable_read(
File "C:\Python38\lib\site-packages\pymongo\mongo_client.py", line 1453, in _retryable_read
server = self._select_server(
File "C:\Python38\lib\site-packages\pymongo\mongo_client.py", line 1253, in _select_server
server = topology.select_server(server_selector)
File "C:\Python38\lib\site-packages\pymongo\topology.py", line 233, in select_server
return random.choice(self.select_servers(selector,
File "C:\Python38\lib\site-packages\pymongo\topology.py", line 192, in select_servers
server_descriptions = self._select_servers_loop(
File "C:\Python38\lib\site-packages\pymongo\topology.py", line 208, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: No replica set members match selector "Primary()"
You have either:
Not specified the read preference at all, or specified it incorrectly, or
Have issued an operation that doesn't respect read preference, or
Are attempting a write, to which read preference doesn't apply.
Going by the stack trace, you are listing databases which is a read operation so it's one of the first two reasons.
I understood mongodb show dbslist only the dbs if they are not
empty. Why this rationale and why its implemented that way to confuse
a typical db users who want to shift from other db world to this mongo
world. Why don't they simply show all the dbs.
Rephrasing:
Once I installed mongodb instance, I logged into mongo shell and issue the command show dbs but its not listing the default db i.e. test. Why?
MongoDB lazily evaluates databases. If every time a user said db =client["someDBName"] a full database instance sprang to life then there would be a lot of empty databases obscuring the ones with data in them. Instead the client and server delay creation of the database until a collection is created. In Python you can force database and collection creation by using the create_collection command as follows:
>>> import pymongo
>>> c=pymongo.MongoClient()
>>> c["dummydb"].create_collection("dummycol")
Collection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'dummydb'), 'dummycol')
>>> c["dummydb"].list_collection_names()
['dummycol']
>>> c.list_database_names()
['admin', 'bonkers', 'census', 'config', 'dummydb', 'dw', 'local', 'logdb', 'test']
>>> import pymongo
>>> c.list_database_names()
['admin', 'census', 'config', 'dw', 'local', 'logdb', 'test']
>>> c=pymongo.MongoClient()
>>> c["dummydb"].create_collection("dummycol")
Collection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'dummydb'), 'dummycol')
>>> c.list_database_names()
['admin', 'census', 'config', 'dummydb', 'dw', 'local', 'logdb', 'test']
>>>
I'm currently working on a simple python CRUD script to check MongoDB out. Turns out I'm liking it a lot, but I have found myself being unable to work with MongoDB transactions. Everytime I try to start a transaction an exception is thrown saying:
This MongoDB deployment does not support retryable writes. Please add retryWrites=false to your connection string.
And, eventhough I've already added that option to my connection string:
self._client = MongoClient('mongodb://localhost/?retryWrites=false')
self._db = self._client.workouts
self._collection = self._db.workouts
That error is still popping up when running the following lines of code:
with self._client.start_session() as s:
with s.start_transaction():
self._collection.delete_one({'_id': id}, session=s)
next = self._collection.find_one({'_id': next_id}, session=s)
return next
What can I do?
I'm running python 3.7.3, pymongo 3.9.0 and MongoDB 4.0.12.
I try to insert and pull some data from MongoDB.
The connection was setup correctly follow there instruction on mongodb.com
try:
client = MongoClient(
'mongodb+srv://user:pw!#cluster0-nghj0.gcp.mongodb.net/test?retryWrites=true',
ssl=True)
print("connected")
except:
print('failed')
I manually create a Database: messager.messager and put some json file in it
when I try to use collection.find() or collection.insert_one(...)
db = client.messager
collection = db.messager
for i in collection.find():
print(i)
It returns Timeout error:
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/cursor.py", line 1225, in next
if len(self.__data) or self._refresh():
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/cursor.py", line 1117, in _refresh
self.__session = self.__collection.database.client._ensure_session()
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1598, in _ensure_session
return self.__start_session(True, causal_consistency=False)
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1551, in __start_session
server_session = self._get_server_session()
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/mongo_client.py", line 1584, in _get_server_session
return self._topology.get_server_session()
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/topology.py", line 434, in get_server_session
None)
File "/Users/anhnguyen/Documents/GitHub/GoogleCloud_Flask/comming soon/env/lib/python3.7/site-packages/pymongo/topology.py", line 200, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed
Where did it goes wrong ?
Here is my Mongodb.com setup:
From the pymongo documentation for errors you have the following issue.
exception pymongo.errors.ServerSelectionTimeoutError(message='', errors=None)
Thrown when no MongoDB server is available for an operation
If there is no suitable server for an operation PyMongo tries for serverSelectionTimeoutMS (default 30 seconds) to find one, then throws this exception. For example, it is thrown after attempting an operation when PyMongo cannot connect to any server, or if you attempt an insert into a replica set that has no primary and does not elect one within the timeout window, or if you attempt to query with a Read Preference that the replica set cannot satisfy.
You have to check whether your network has good connectivity to the network where your MongoDB server resides.
There might be a case, where the Primary Node of the Replica Set becomes unresponsive. In such cases you need to restart your cluster(if you have the access permissions).
Also, create the connections as follows:
mongo_conn = MongoClient('mongodb+srv://cluster0-nghj0.gcp.mongodb.net/test?retryWrites=true', username=your_username, password=pwd, authSource='admin', authMechanism='SCRAM-SHA-1')
above is the best practice to follow. mongodb+srv urls do not need the ssl=True mention.
I created the following mongo replica sets by using mongo cli:
> config = { _id:"repset", members:[{_id:0,host:"192.168.0.1:27017"},{_id:1,host:"192.168.0.2:27017"},{_id:2,host:"192.168.0.3:27017"}]}
> rs.initiate(config);
All the mongo servers run properly.
>>> import pymongo
>>> from pymongo import MongoClient
>>> servers = ["192.168.0.1:27017", "192.168.0.2:27017", "192.168.0.3:27017"]
>>> MongoClient(servers)
>>> xc = MongoClient()
>>> print xc
MongoClient('localhost', 27017)
>>> print xc.database_names()
[u'test_repsets', u'local', u'admin', u'test']
After I kill the local mongodb server, it shows me connection timeout error:
pymongo.errors.ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused
It seems there is no auto fail over, although I defined the mongodb servers.
I am wondering if pymongo handles fail over automatically, or how this situation is handled properly?
Thank you in advance.
in Pymongo 3.x you may want to explicitly state what replica set you are connecting to. I know Pymongo 3.x switched up some of the ways it handles being given an array of servers. I got this off the Pymongo API about connections to replicas and auto failover
In your code :
MongoClient(servers)
Above line is not assigned to any variable. It should assign to variable (in your case you again created instance which causes error.)
Please add following line
>>> #MongoClient(servers) # remove this line
>>> #xc = MongoClient() # remove this line
>>> xc = MongoClient(servers) # add this line