mongodb grails simple application times out - mongodb

I'm having an issue with mongodb 2.6.5 and grails 2.4.4 that I can't resolve. For the sake of isolating the problem I created a simple 2.4.4 grails app, installed the grails mongodb plugin (compile ":mongodb:3.0.2"), commented out the hibernate dependencies, added my mongodb datasource, and set up a simple domain class (com.nerds.Nerd). When I generate-all and then start the app and navigate to the NerdController CRUD page I get the following error every time:
MongoTimeoutException occurred when processing request: [GET] /MONGO/nerd/index
Timed out while waiting to connect after 10000 ms. Stacktrace follows:
com.mongodb.MongoTimeoutException: Timed out while waiting to connect after 10000 ms
I can access mongo via http using http://localhost:28017/
I have also tested manually adding data and querying from mongo. This all works fine.
In the debug log prior to the timeout it looks like GORM aquired a mongo session and then tried rolling back a transaction.
DatastoreTransactionManager:128 - Found thread-bound Session [org.grails.datastore.mapping.mongo.MongoSession#e47ee6] for Datastore transaction
DatastoreTransactionManager:128 - Creating new transaction with name [null]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT,readOnly
DatastoreTransactionManager:128 - Initiating transaction rollback
DatastoreTransactionManager:128 - Rolling back Datastore transaction on Session [org.grails.datastore.mapping.mongo.MongoSession#e47ee6]
DatastoreTransactionManager:128 - Resuming suspended transaction after completion of inner transaction
Any insight would be helpful. Thanks
edit: The mongo datasource is pretty simple. I'm using the correct port.
From the mongo log:
014-11-18T13:10:13.388-0900 [initandlisten] MongoDB starting : pid=17275 port=27017 dbpath=/var/lib/mongodb 32-bit host=enterprise
from DataSource.groovy
grails { mongo { host = 'localhost' port = 27017 databaseName = 'mydb' } }

I'm fairly certain the issue was on the mongod side. I stopped the mongo daemon, put it into high verbose debug mode (using mongod -vvvv command), and when I tried to replicate the issue while watching the console output, the issue did not happen. I'm not entirely sure what the exact cause of the timeout was, but its not happening now. Thanks for the responses.

Related

mongo MongoCursorNotFoundException in long-running query loop

I have a simple query loop that gets a MongoCursorNotFoundException after processing about 44,000 of 96,945 documents in around 93 minutes.
MongoIterable<MasterDocument> query = masterCollection.find().noCursorTimeout(true);
for (MasterDocument masterDocument : query) { ... do some stuff ... }
The "do some stuff" part takes a while, which is why the entire loop takes so long.
My problem is that I get this exception after handling maybe half of the documents in the collection.
I am running both the client application and the mongod server locally on my Windows 10 laptop, accessing the server via localhost.
The server log shows lots of messages like this:
{"t":{"$date":"2021-01-04T20:21:35.510-08:00"},"s":"I", "c":"COMMAND", "id":51803, "ctx":"conn27","msg":"Slow query","attr":{"type":"command","ns":"master_database.MasterCollection","command":{"find":"MasterCollection","filter":{"hashCode":1753339282},"$db":"master_database","lsid":{"id":{"$uuid":"6a252f51-2c6e-4c01-ae03-1a80aab109e0"}}},"planSummary":"COLLSCAN","keysExamined":0,"docsExamined":96944,"cursorExhausted":true,"numYields":96,"nreturned":0,"queryHash":"DBC59907","planCacheKey":"DBC59907","reslen":121,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":97}},"Global":{"acquireCount":{"r":97}},"Database":{"acquireCount":{"r":97}},"Collection":{"acquireCount":{"r":97}},"Mutex":{"acquireCount":{"r":1}}},"storage":{},"protocol":"op_msg","durationMillis":147}}
The last of these messages is followed by:
{"t":{"$date":"2021-01-04T20:21:35.521-08:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn27","msg":"Connection ended","attr":{"remote":"127.0.0.1:58990","connectionId":27,"connectionCount":14}}
{"t":{"$date":"2021-01-04T20:21:35.522-08:00"},"s":"I", "c":"NETWORK", "id":22944, "ctx":"conn26","msg":"Connection ended","attr":{"remote":"127.0.0.1:58989","connectionId":26,"connectionCount":13}}
{"t":{"$date":"2021-01-04T20:21:35.922-08:00"},"s":"I", "c":"-", "id":20883, "ctx":"conn25","msg":"Interrupted operation as its client disconnected","attr":{"opId":310196}}
I have tried:
Using "noCursorTimeout(true)" on the query cursor (as shown above)
Starting the server with "mongod --setParameter localLogicalSessionTimeoutMinutes=240". This last seems to have caused additional log messages that say "error":"Location13111: wrong type for field (expireAfterSeconds) long != int"
I am using mongod 4.4 and the latest mongo java api.
You may need to increase the default cursor idle timeout to bigger value in all shards and mongos:
check the parameter(default is 10 min = 600000 ms ):
use admin
db.runCommand({getParameter:1, cursorTimeoutMillis: 1})
and update to bigger value:
use admin
db.runCommand({setParameter:1, cursorTimeoutMillis: 600000000 })
also the COLSCAN in your logs indicate that you dont use indexes in your query , maybe you need to create one on "hashCode" ...
Thanks for the response.
It turned out that my application ran to completion once I started mongod with "--setParameter localLogicalSessionTimeoutMinutes=240, despite the error message that I saw in the console log.
You are absolutely right that I should have an index on "hashCode". (I had one before but forgot to recreate it after recreating the collection.)

pymongo - MongoClient retryWrites=false is not working

I'm currently working on a simple python CRUD script to check MongoDB out. Turns out I'm liking it a lot, but I have found myself being unable to work with MongoDB transactions. Everytime I try to start a transaction an exception is thrown saying:
This MongoDB deployment does not support retryable writes. Please add retryWrites=false to your connection string.
And, eventhough I've already added that option to my connection string:
self._client = MongoClient('mongodb://localhost/?retryWrites=false')
self._db = self._client.workouts
self._collection = self._db.workouts
That error is still popping up when running the following lines of code:
with self._client.start_session() as s:
with s.start_transaction():
self._collection.delete_one({'_id': id}, session=s)
next = self._collection.find_one({'_id': next_id}, session=s)
return next
What can I do?
I'm running python 3.7.3, pymongo 3.9.0 and MongoDB 4.0.12.

MongoDB "no SNI name sent" error from heroku java application

I followed this tutorial to deploy a sample application to Heroku. I just added the below method in MyResource class and returned the result from it instead of "Hello World" from getIt() method. I'm connecting to an atlas free tier cluster:
static String getMessage() {
MongoClient mongoClient = new MongoClient(new MongoClientURI("mongodb://<USER>:<PASSWORD>#cluster0-shard-00-00-2lbue.mongodb.net:27017,cluster0-shard-00-01-2lbue.mongodb.net:27017,cluster0-shard-00-02-2lbue.mongodb.net:27017/test?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin"));
DB database = mongoClient.getDB("mastery");
DBCollection collection = database.getCollection("summary");
DBObject query = new BasicDBObject("_id", new ObjectId("5c563fa2645d6b444c018dcb"));
DBCursor cursor = collection.find(query);
return (String)cursor.one().get("message");
}
This is the driver I'm using:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver</artifactId>
<version>3.9.1</version>
</dependency>
This is my import:
import com.mongodb.*;
The application works fine from my local system. But I face the below error when I deploy the application to Heroku and hit the service:
INFO: Exception in monitor thread while connecting to server cluster0-shard-00-01-2lbue.mongodb.net:27017
com.mongodb.MongoCommandException: Command failed with error 8000 (AtlasError): 'no SNI name sent, make sure using a MongoDB 3.4+ driver/shell.' on server cluster0-shard-00-01-2lbue.mongodb.net:27017. The full response is { "ok" : 0, "errmsg" : "no SNI name sent, make sure using a MongoDB 3.4+ driver/shell.", "code" : 8000, "codeName" : "AtlasError" }
at com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:179)
at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:299)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:255)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:106)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:63)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:127)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:117)
at java.lang.Thread.run(Thread.java:748)
What is this SNI name? I can understand that the drivers are able to pick it from my machine, but not from Heroku machine. But I'm clueless on how to go about solving this! Is there a way to configure Heroku to reveal the SNI name when the driver asks for it? Can we get this value manually from somewhere in Heroku and directly feed it to the MongoDB drivers? Any help is appreciated.
EDIT:
It turned out that the client mentions the SNI name of the server it wishes to connect to as part of TLS security. And there seems to be a way to manually indicate the name in python driver. Is there any way to do this from java? Still puzzled why this is not an issue when running the app locally.
The code I was using to connect to the cluster turned out to be wrong. I followed the directions from the docs and it mentioned this:
To connect to an Atlas M0 (Free Tier) cluster, you must use Java
version 8 or greater and use a Java driver version that supports
MongoDB 3.4.
So I changed the java version to 1.8 in system.properties file:
java.runtime.version=1.8
It was earlier set to 1.7. I was also getting a deprecation warning on one of the methods I had used. So I again followed the docs to use the latest code and it worked like a charm.
The real takeaway here is to refer to the official docs everytime :)

Error in connecting to mongodb (mongodb-linux-x86_64-2.8.0-rc5) : auth failed, when starting nodejs appserver

I am new to mongodb, I am able to get my nodejs appserver started with mongodb 2.4 and 2.6. I have a requirement to test the latest mongodb, which is 2.8 with our app. I am able to start the mongodb 2.8 but when I try to start my nodejs appserver, I get following error.
2015-01-16T18:43:51.115Z <warn> globals.js:45 () MONGO URI = mongodb://myuser:password#localhost:27017/test
2015-01-16T18:43:51.178Z <error> globals.js:45 () #### Error in connecting to mongodb
2015-01-16T18:43:51.180Z <error> globals.js:45 () { [MongoError: auth failed] name: 'MongoError', ok: 0, errmsg: 'auth failed', code: 18 }
2015-01-16T18:43:51.182Z <error> globals.js:45 () MongoError: auth failed
.........
Also, I have created myuser in the test db and authenticated it successfully. Please help, I have tried all possible ways and with no success. Thank you very much in advance. (my environment is Ubantu Linux 14.04, virtualbox) One more thing that I forgot to mention is that I am using wiredTiger storageEngine. I tried with MMAPv1 as well and same result.
I had the same issue. It worked for me on Mac, but on Linux I hit this failure until I upgraded module mongoose from 3.6.x to 3.8.x (3.8.25). That solved the problem for me. You might consider doing the same, at least to rule out this problem.

Can't Connect to MongoDB from play app with salat: command failed [listDatabases]

I am trying to get started with the salat plugin in playframework. I have configured the database in application.conf and added all the dependencies to Build.scala and added salat to the play.plugins file. I haven't actually added any code to the project yet, I just followed the instructions on the github page, and then tried to run the project. I am getting the following error message
(Server started, use Ctrl+D to stop and go back to the console...)
[info] play - mongodb [default] connected at heroku_app4620908#ds031907.mongolab.com:31907/heroku_app4620908
[error] application -
! #6bchnaacn - Internal server error, for request [GET /] ->
play.api.Configuration$$anon$1: Configuration error [couldn't connect to [ds031907.mongolab.com/107.21.153.211:31907]]
at play.api.Configuration$.play$api$Configuration$$configError(Configuration.scala:71) ~[play_2.9.1-2.0.3.jar:2.0.3]
at play.api.Configuration.reportError(Configuration.scala:258) ~[play_2.9.1-2.0.3.jar:2.0.3]
at se.radley.plugin.salat.SalatPlugin$$anonfun$onStart$1.apply(SalatPlugin.scala:105) ~[play-plugins-salat_2.9.1-1.0.8.jar:1.0.8]
at se.radley.plugin.salat.SalatPlugin$$anonfun$onStart$1.apply(SalatPlugin.scala:98) ~[play-plugins-salat_2.9.1-1.0.8.jar:1.0.8]
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194) ~[scala-library.jar:0.11.3]
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194) ~[scala-library.jar:0.11.3]
Caused by: com.mongodb.CommandResult$CommandFailure: command failed [listDatabases]: { "serverUsed" : "db-uri" , "errmsg" : "need to login" , "ok" : 0.0}
at com.mongodb.CommandResult.getException(CommandResult.java:88) ~[mongo-java-driver-2.8.0.jar:na
at com.mongodb.CommandResult.throwOnError(CommandResult.java:134) ~[mongo-java-driver-2.8.0.jar:na]
at com.mongodb.Mongo.getDatabaseNames(Mongo.java:356) ~[mongo-java-driver-2.8.0.jar:na]
at com.mongodb.casbah.MongoConnection.getDatabaseNames(MongoConnection.scala:190) ~[casbah-core_2.9.1-2.4.1.jar:2.4.1]
at se.radley.plugin.salat.SalatPlugin$$anonfun$onStart$1.apply(SalatPlugin.scala:103) ~[play-plugins-salat_2.9.1-1.0.8.jar:1.0.8]
at se.radley.plugin.salat.SalatPlugin$$anonfun$onStart$1.apply(SalatPlugin.scala:98) ~[play-plugins-salat_2.9.1-1.0.8.jar:1.0.8]
I am stumped because I added my password and everything to the conf file. From the log it looks as though whatever is trying to connect to the database for me, is not logging in first, using the info I provided.
As you say, it looks like the root issue is that MongoDB is rejecting the command "listDatabases". This command requires administrator access to the MongoDB process since it reveals information about the other databases hosted there.
Unfortunately, the message it returns, "need to login", is a little misleading. You have logged in! You just don't have permission to list the databases.
Here's a simple experiment you can try yourself with the MongoDB shell. See that "show dbs" fails with the same error message you got in your app, but "show collections", which doesn't require access to any other databases outside your own, succeeds?
% mongo ds031907.mongolab.com:31907/heroku_app4620908 -u heroku_app4620908 -p your_password
MongoDB shell version: 2.0.7
connecting to: ds031907.mongolab.com:31907/heroku_app4620908
> show dbs
Fri Aug 17 13:12:10 uncaught exception: listDatabases failed:{ "errmsg" : "need to login", "ok" : 0 }
> show collections
system.indexes
system.users
(Note: I did this with my own MongoLab account and modified the text when copying and pasting here so you could just copy it into your terminal.)
Is there a way to avoid making the listDatabases call? I'm not familiar with the framework you're using.
I've the problem as well...
it should come with the SalatPlugin's onStart method that is requesting all database names: source._2.connection.getDatabaseNames().
This code is just testing the aliveness of the server...
I'm gonna check with leon how we could this differently! Sadly, you won't be able to connect until this will be fixed!
Stay tuned on this issue https://github.com/leon/play-salat/issues/23
This is fixed in the latest version of play-salat, it now uses getCollectionNames instead
use admin
db.addUser('userName', 'userPassword')
db.auth('userName', 'userPassword')
show dbs
java:
DB db = mongo.getDB("YouDBName") ;
db.authenticate("userName", "userPassword".toCharArray()) ;
System.out.println(mongo.getDatabaseNames()) ;