I am using Motor driver to connect to Mongo DB.
Below is the code to insert data to the collection
client = motor.MotorClient('mongodb://localhost:27017').open_sync()
conn = client['database']['collection']
result = conn.insert({'foo': 'bar'})
print 'result:', result
The insert statement always returns None.
This is not a Tornado application.
Can motor be only used with Tornado?
If not why is the insert returning none?
You use motor just like pymongo. But motor is asynchronous: it means that when your print is executed, maybe the db request is not finished yet.
Furthermore, motor insert does not return anything, and you need to use a callback function as second argument with it. Cf. the differences between pymongo and motor, and the motor tutorial on how to insert a document.
In your case, the good way of soing this would be:
client = motor.MotorClient('mongodb://localhost:27017').open_sync()
conn = client['database']['collection']
result = conn.insert({'foo': 'bar'}, callback=once_done)
def once_done(result, error):
if error: print 'error:', error
else:
print 'result:', result
I guess, the WriteConcern is not set from the client-driver.
If you set it as safe=true, then you will get the status of the insert operation. Otherwise with safe=false, the insert operation is fire and forget.
You can try:
motor.MotorClient('mongodb://localhost:27017/?safe=true')
Related
For performance optimisation we are trying to read data from Mongo secondary server for selected scenarios. I am using the inline query using "withReadPreference(ReadPreference.secondaryPreferred())" to read the data, PFB the code snippet.
What I want to confirm the data we are getting is coming from secondary server after executing the inline query highlighted, is there any method available to check the same from Java or Springboot
public User read(final String userId) {
final ObjectId objectId = new ObjectId(userId);
final User user = collection.withReadPreference(ReadPreference.secondaryPreferred()).findOne(objectId).as(User.class);
return user;
}
Pretty much the same way in Java. Note we use secondary() not secondaryPrefered(); this guarantees reads from secondary ONLY:
import com.mongodb.ReadPreference;
{
// This is your "regular" primaryPrefered collection:
MongoCollection<BsonDocument> tcoll = db.getCollection("myCollection", BsonDocument.class);
// ... various operations on tcoll, then create a new
// handle that FORCES reads from secondary and will timeout and
// fail if no secondary can be found:
MongoCollection<BsonDocument> xcoll = tcoll.withReadPreference(ReadPreference.secondary());
BsonDocument f7 = xcoll.find(queryExpr).first();
}
I want to use an asynchronous connection to a postgres database for inserting a realtime incomming data. I am using Twisted for the TCP communication and i am giving txpostgres a shot for the interaction with the database.I am stacked with a weird message when i try to add a callback for my asynchronous insert.Here my code:
try:
conn = txpostgres.ConnectionPool(50,params)
d = conn.start()
def save_conn(c):
self.list_cnx.append(c)
print str(c),"Connect OK"
def print_err(m):
print m
d.addCallbacks(lambda _: save_conn(conn),lambda __: print_err('Connect NO'))
except Exception as e:
print "Cannot connect to database!!"
I am adding the refrence of the connction pool in a list for future query.
def insert_data(self,dic):
try:
insArRq="""INSERT INTO test_pool(date_msg, msg) VALUES ('%s','%s')"""%(dic['date'],dic['msg'])
for c in self.list_cnx:
def insert_finich():
print "insert finich"
def insert_error():
print "insert error"
d = c.runOperation(insArRq) # return a deferred as mentioned in the documentation
print d # for debug
d.addCallBack(insert_finich) # error mesage
except Exception as ee:
print "Insert error : ",ee
When i try to add a callback for the deferred returned by the runOperation this error appear:
<Deferred at 0x8d9782c waiting on Deferred at 0x8d9786c>
Insert error : Deferred instance has no attribute 'addCallBack'
and sometimes:
<Deferred at 0x8d97a0c>
Insert error : Deferred instance has no attribute 'addCallBack'
Help me please, i'am new to defrred concepts so i think i'am missing something.Thanks
The method is named addCallback; case is important.
In ruby standard library, popen return a object which has pid method,
In eventmachine, popen return a Connection object which contain no process id information.
Demo code here:
require 'eventmachine'
def work
s = EM.popen('ls')
puts s.pid # how to get new process's pid?
end
EventMachine.run {
work
EM.stop
}
As EM.popen returns a standard a EM Connection object, you can use s.get_pid to get the pid of the subprocess.
http://eventmachine.rubyforge.org/EventMachine/Connection.html#get_pid-instance_method
Our application is build upon mongodb replica set.
I'd like to catch all exceptions thrown among the time frame when replica set is in process of automatic failover.
I will make application retry or wait for failover completes.
So that the failover won't influence user.
I found document describing the behavior of java driver here: https://jira.mongodb.org/browse/DOCS-581
I write a test program to find all possible exceptions, they are all MongoException but with different message:
MongoException.Network: "Read operation to server /10.11.0.121:27017 failed on database test"
MongoException: "can't find a master"
MongoException: "not talking to master and retries used up"
MongoException: "No replica set members available in [ here is replica set status ] for { "mode" : "primary"}"
Maybe more...
I'm confused and not sure if it is safe to determine by error message.
Also I don't want to catch all MongoException.
Any suggestion?
Thanks
I am now of the opinion that Mongo in Java is particularly weak in this regards. I don't think your strategy of interpreting the error codes scales well or will survive driver evolution. This is, of course, opinion.
The good news is that the Mongo driver provides a way get the status of a ReplicaSet: http://api.mongodb.org/java/2.11.1/com/mongodb/ReplicaSetStatus.html. You can use it directly to figure out whether there is a Master visible to your application. If that is all you want to know, the http://api.mongodb.org/java/2.11.1/com/mongodb/Mongo.html#getReplicaSetStatus() is all you need. Grab that kid and check for a not-null master and you are on your way.
ReplicaSetStatus rss = mongo.getReplicaSetStatus();
boolean driverInFailover = rss.getMaster() == null;
If what you really need is to figure out if the ReplSet is dead, read-only, or read-write, this gets more difficult. Here is the code that kind-of works for me. I hate it.
#Override
public ReplSetStatus getReplSetStatus() {
ReplSetStatus rss = ReplSetStatus.DOWN;
MongoClient freshClient = null;
try {
if ( mongo != null ) {
ReplicaSetStatus replicaSetStatus = mongo.getReplicaSetStatus();
if ( replicaSetStatus != null ) {
if ( replicaSetStatus.getMaster() != null ) {
rss = ReplSetStatus.ReadWrite;
} else {
/*
* When mongo.getReplicaSetStatus().getMaster() returns null, it takes a a
* fresh client to assert whether the ReplSet is read-only or completely
* down. I freaking hate this, but take it up with 10gen.
*/
freshClient = new MongoClient( mongo.getAllAddress(), mongo.getMongoClientOptions() );
replicaSetStatus = freshClient.getReplicaSetStatus();
if ( replicaSetStatus != null ) {
rss = replicaSetStatus.getMaster() != null ? ReplSetStatus.ReadWrite : ReplSetStatus.ReadOnly;
} else {
log.warn( "freshClient.getReplicaSetStatus() is null" );
}
}
} else {
log.warn( "mongo.getReplicaSetStatus() returned null" );
}
} else {
throw new IllegalStateException( "mongo is null?!?" );
}
} catch ( Throwable t ) {
log.error( "Ingore unexpected error", t );
} finally {
if ( freshClient != null ) {
freshClient.close();
}
}
log.debug( "getReplSetStatus(): {}", rss );
return rss;
}
I hate it because it doesn't follow the Mongo Java Driver convention of your application only needs a single Mongo and through this singleton you connect to the rest of the Mongo data structures (DB, Collection, etc). I have only been able to observe this working by new'ing up a second Mongo during the check so that I can rely upon the ReplicaSetStatus null check to discriminate between "ReplSet-DOWN" and "read-only".
What is really needed in this driver is some way to ask direct questions of the Mongo to see if the ReplSet can be expected at this moment to support each of the WriteConcerns or ReadPreferences. Something like...
/**
* #return true if current state of Client can support readPreference, false otherwise
*/
boolean mongo.canDoRead( ReadPreference readPreference )
/**
* #return true if current state of Client can support writeConcern; false otherwise
*/
boolean mongo.canDoWrite( WriteConcern writeConcern )
This makes sense to me because it acknowledges the fact that the ReplSet may have been great when the Mongo was created, but conditions right now mean that Read or Write operations of a specific type may fail due to changing conditions.
In any event, maybe http://api.mongodb.org/java/2.11.1/com/mongodb/ReplicaSetStatus.html gets you what you need.
When Mongo is failing over, there are no nodes in a PRIMARY state. You can just get the replica set status via the replSetGetStatus command and look for a master node. If you don't find one, you can assume that the cluster is in a failover transition state, and can retry as desired, checking the replica set status on each failed connection.
I don't know the Java driver implementation itself, but I'd do catch all MongoExceptions, then filter them on getCode() basis. If the error code does not apply to replica sets failures, then I'd rethrow the MongoException.
The problem is, to my knowledge there is no error codes reference in the documentation. Well there is a stub here, but this is fairly incomplete. The only way is to read the code of the Java driver to know what code it uses…
Trying to insert into a mongodb database from scala. the below codes dont create a db or collection. tried using the default test db too. how do i perform CRUD operations?
object Store {
def main(args: Array[String]) = {
def addMongo(): Unit = {
var mongo = new Mongo()
var db = mongo.getDB("mybd")
var coll = db.getCollection("somecollection")
var obj = new BasicDBObject()
obj.put("name", "Mongo")
obj.put("type", "db")
coll.insert(obj)
coll.save(obj)
println("Saved") //to print to console
}
}
On a first glance things look OK in your code although you have that stray def addMongo(): Unit = {
code at the top. I'll defer to a suggestion on looking for errors here.... Two items of note:
1) save() and insert() are complementary operations - you only need one. insert() will always attempt to create a new document ... save() will create one if the _id field isn't set, and update the represented _id if it does.
2) Mongo clients do not wait for an answer to a write operation by default. It is very possible & likely that an error is occurring within MongoDB causing your write to fail. the getLastError() command will return the result of the last write operation on the current connection. Because MongoDB's Java driver uses connection pools you have to tell it to lock you onto a single connection for the duration of an operation you want to run 'safely' (e.g. check result). This is the easiest way from the Java driver (in Scala, sample code wise, though):
mongo.requestStart() // lock the connection in
coll.insert(obj) // attempt the insert
getLastError.throwOnError() // This tells the getLastError command to throw an exception in case of an error
mongo.requestDone() // release the connection lock
Take a look at this excellent writeup on MongoDB's Write Durability, which focuses specifically on the Java Driver.
You may also want to take a look at the Scala driver I maintain (Casbah) which wraps the Java driver and provides more scala functionality.
We provide among other things an execute-around-method version of the safe write concept in safely() which makes things a lot easier for testing for writes' success.
You just missed the addMongo call in main. The fix is trivial:
object Store {
def main(args: Array[String]) = {
def addMongo(): Unit = {
var mongo = new Mongo()
var db = mongo.getDB("mybd")
var coll = db.getCollection("somecollection")
var obj = new BasicDBObject()
obj.put("name", "Mongo")
obj.put("type", "db")
coll.insert(obj)
coll.save(obj)
println("Saved") //to print to console
}
addMongo // call it!
}