ReactiveMongo/Play: `LastError: DatabaseException['<none>']`, while db still updates - mongodb

weird problem: While my play application tries to insert/update records from some mongoDB collections while using reactivemongo, the operation seems to fail with a mysterious message, but the record does, actually, gets inserted/updated.
More info:
Insert to problematic collections from the mongo console works well
reading from all collections works well
reading and writing to other collections in the same db works well
writing to the problematic collections used to work.
Error message is:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[LastError: DatabaseException['<none>']]]
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:280)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:206)
at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:100)
at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:99)
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:344)
at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:343)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:70)
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
Caused by: reactivemongo.api.commands.LastError: DatabaseException['<none>']
Using ReactiveMongo 0.11.14, Play 2.5.4, Scala 2.11.7, MongoDB 3.4.0.
Thanks!
UPDATE - The mystery thickens!
Based on #Yaroslav_Derman's answer, I added a .recover clause, like so:
collectionRef.flatMap( c =>
c.update( BSONDocument("_id" -> publicationWithId.id.get), publicationWithId.asInstanceOf[PublicationItem], upsert=true))
.map(wr => {
Logger.warn("Write Result: " + wr )
Logger.warn("wr.inError: " + wr.inError)
Logger.warn("*************")
publicationWithId
}).recover({
case de:DatabaseException => {
Logger.warn("DatabaseException: " + de.getMessage())
Logger.warn("Cause: " + de.getCause())
Logger.warn("Code: " + de.code)
publicationWithId
}
})
The recover clause does get called. Here's the log:
[info] application - Saving pub t3
[warn] application - *************
[warn] application - Saving publication Publication(Some(BSONObjectID("5848101d7263468d01ff390d")),t3,2016-12-07,desc,auth,filename,None)
[info] application - Resolving database...
[info] application - Resolving database...
[warn] application - DatabaseException: DatabaseException['<none>']
[warn] application - Cause: null
[warn] application - Code: None
So no cause, no code, message is "'<none>'", but still an error. What gives?
I tried to move to 0.12, but that caused some compilation errors across the app, plus I'm not sure that would solve the problem. So I'd like to understand what's wrong first.
UPDATE #2:
Migrated to reactive-mongo 0.12.0. Problem persists.

Problem solved by downgrading to MongoDB 3.2.8. Turns out reactiveMongo 0.12.0 is not compatible with mongoDB 3.4.
Thanks everyone who looked into this.

For play reactivemongo 0.12.0 you can do like this
def appsDb = reactiveMongoApi.database.map(_.collection[JSONCollection](DesktopApp.COLLECTION_NAME))
def save(id: String, user: User, body: JsValue) = {
val version = (body \ "version").as[String]
val app = DesktopApp(id, version, user)
appsDb.flatMap(
_.insert(app)
.map(_ => app)
.recover(processError)
)
}
def processError[T]: PartialFunction[Throwable, T] = {
case ex: DatabaseException if ex.code.contains(10054 | 10056 | 10058 | 10107 | 13435 | 13436) =>
//Custom exception which processed in Error Handler
throw new AppException(ResponseCode.ALREADY_EXISTS, "Entity already exists")
case ex: DatabaseException if ex.code.contains(10057 | 15845 | 16550) =>
//Custom exception which processed in Error Handler
throw new AppException(ResponseCode.ENTITY_NOT_FOUND, "Entity not found")
case ex: Exception =>
//Custom exception which processed in Error Handler
throw new InternalServerErrorException(ex.getMessage)
}
Also you can add logs in processError method

LastError was deprecated in 0.11, replaced by WriteResult.
LastError does not, actually, means error, it could mean successful result, you need to check inError property of the LastError object to detect if it's real error. As I see, the '<none>' error message give a good chance that this is not error.
Here is the example "how it was in 0.10": http://reactivemongo.org/releases/0.10/documentation/tutorial/write-documents.html

Related

Moto testing not raising proper exception

I have the following function I wish to test:
def download(self):
s3 = boto3.client('s3')
try:
with open(self.flow_cells +'.tar', 'wb') as data:
s3.download_fileobj(
self.source_s3_bucket,
self.source_key,
data
)
return True
except botocore.exceptions.ClientError as error:
print(error.response['Error']['Code'])
I am using pytest to test code with moto. All other tests and botocore exceptions are getting flagged except for this one. I am capturing in standard out that it is getting to the exception function and printing the correct code, but moto is not flagging it as an Exception
Here is my testing code.
def test_download(parse_args, file_test):
with moto.mock_s3():
s3 = boto3.resource('s3')
s3.create_bucket(Bucket=parse_args.glacier_s3_bucket, CreateBucketConfiguration={
'LocationConstraint': 'us-east-1'
})
s3.create_bucket(Bucket=parse_args.output_s3_bucket, CreateBucketConfiguration={
'LocationConstraint': 'us-east-1'
})
bucket_version = s3.BucketVersioning(parse_args.glacier_s3_bucket)
bucket_version.enable()
s3.Object(parse_args.glacier_s3_bucket, 'flowcells/flowcell-testing.tar').put\
(Body=open(file_test, 'rb'))
glacier_client = GlacierRestoreClient(parse_args)
assert glacier_client.download() is True
s3.Object(glacier_client.source_s3_bucket, glacier_client.source_key).delete()
with pytest.raises(Exception) as error:
glacier_client.download()
assert 'Error' in error
Inside except clause the exception is silenced and not propagated, so you need to re-raise it:
except botocore.exceptions.ClientError as error:
print(error.response['Error']['Code'])
raise
Bare raise re-raises exception that was just caught.
PS. Shameless plug: I was one of those who asked Guido 20 years ago to add bare raise! :-)

java.lang.NoClassDefFoundError: Could not initialize class - spark/scala

I am new to spark/scala development. I am using maven to build my project and IDE is intelliJ. I am trying to query a hive table and then iterate over the resulting dataframe(using foreach). Here's my code:
try{
val DF_1 = hiveContext.sql("select distinct(address) from
test_table where trim(address)!=''")
println("number of rows: "+DF_1.count)
DF_1.foreach(x => {
val y =hiveContext.sql("select place from test_table where address='"+x(0).toString+"'")
if(y.count > 1){
println("Multiple place values for address: "+x(0).toString)
y.foreach(r => println(r))
println("*************")
}
})}
catch {case e: Exception => e.printStackTrace()}
With each iteration, I am Querying the same table to get another column, trying to see if there are multiple values of places for each address in test_table. I have no compilation errors and the application builds successfully. But, when I run the above code, I get the following error:
java.lang.NoClassDefFoundError: Could not initialize class xxxxxxxx
the application launches successfully, prints the count of rows in DF_1 and then fails with the above error at the foreach loop. I did a jar xvf on my jar and can see the main class - driver.class:
com/.../driver$$anonfun$1$$anonfun$apply$1.class
com/.../driver$$anonfun$1.class
com/.../driver$$anonfun$2.class
com/.../driver$$anonfun$3.class
com/.../driver$$anonfun$4.class
com/.../driver$$anonfun$5.class
com/.../driver$$anonfun$main$1$$anonfun$apply$1.class
com/.../driver$$anonfun$main$1$$anonfun$apply$2.class
com/.../driver$$anonfun$main$1$$anonfun$apply$3.class
com/.../driver$$anonfun$main$1.class
com/.../driver$$anonfun$main$10$$anonfun$apply$9.class
com/.../driver$$anonfun$main$10.class
com/.../driver$$anonfun$main$11.class
com/.../driver$$anonfun$main$12.class
com/.../driver$$anonfun$main$13.class
com/.../driver$$anonfun$main$14.class
com/.../driver$$anonfun$main$15.class
com/.../driver$$anonfun$main$16.class
com/.../driver$$anonfun$main$17.class
com/.../driver$$anonfun$main$18.class
com/.../driver$$anonfun$main$19.class
com/.../driver$$anonfun$main$2$$anonfun$apply$4.class
com/.../driver$$anonfun$main$2$$anonfun$apply$5.class
com/.../driver$$anonfun$main$2$$anonfun$apply$6.class
com/.../driver$$anonfun$main$2.class
com/.../driver$$anonfun$main$20.class
com/.../driver$$anonfun$main$21.class
com/.../driver$$anonfun$main$22.class
com/.../driver$$anonfun$main$23.class
com/.../driver$$anonfun$main$3$$anonfun$apply$7.class
com/.../driver$$anonfun$main$3$$anonfun$apply$8.class
com/.../driver$$anonfun$main$3.class
com/.../driver$$anonfun$main$4$$anonfun$apply$9.class
com/.../driver$$anonfun$main$4.class
com/.../driver$$anonfun$main$5.class
com/.../driver$$anonfun$main$6$$anonfun$apply$1.class
com/.../driver$$anonfun$main$6$$anonfun$apply$2.class
com/.../driver$$anonfun$main$6$$anonfun$apply$3.class
com/.../driver$$anonfun$main$6$$anonfun$apply$4.class
com/.../driver$$anonfun$main$6$$anonfun$apply$5.class
com/.../driver$$anonfun$main$6.class
com/.../driver$$anonfun$main$7$$anonfun$apply$1.class
com/.../driver$$anonfun$main$7$$anonfun$apply$2.class
com/.../driver$$anonfun$main$7$$anonfun$apply$3.class
com/.../driver$$anonfun$main$7$$anonfun$apply$4.class
com/.../driver$$anonfun$main$7$$anonfun$apply$5.class
com/.../driver$$anonfun$main$7$$anonfun$apply$6.class
com/.../driver$$anonfun$main$7$$anonfun$apply$7.class
com/.../driver$$anonfun$main$7$$anonfun$apply$8.class
com/.../driver$$anonfun$main$7.class
com/.../driver$$anonfun$main$8$$anonfun$apply$10.class
com/.../driver$$anonfun$main$8$$anonfun$apply$4.class
com/.../driver$$anonfun$main$8$$anonfun$apply$5.class
com/.../driver$$anonfun$main$8$$anonfun$apply$6.class
com/.../driver$$anonfun$main$8$$anonfun$apply$7.class
com/.../driver$$anonfun$main$8$$anonfun$apply$8.class
com/.../driver$$anonfun$main$8$$anonfun$apply$9.class
com/.../driver$$anonfun$main$8.class
com/.../driver$$anonfun$main$9$$anonfun$apply$11.class
com/.../driver$$anonfun$main$9$$anonfun$apply$7.class
com/.../driver$$anonfun$main$9$$anonfun$apply$8.class
com/.../driver$$anonfun$main$9$$anonfun$apply$9.class
com/.../driver$$anonfun$main$9.class
com/.../driver$.class
com/.../driver.class
I am not facing the error when I launch the job in local mode instead of yarn. What is causing the issue and how can it be corrected?
Any help would be appreciated, Thank you.
Looks like your your jar or some dependencies aren't distributed between worker nodes. In local mode it works because you have the jars in the place. In yarn mode you need to build a fat-jar with all dependencies include hive and spark libraries in it.

Play-Framework: 2.3.x: play - Cannot invoke the action, eventually got an error: java.lang.IllegalArgumentException:

I am using play-framework 2.3.x with reactivemongo-extension JSON type. following is my code for fetch the data from db as below:
def getStoredAccessToken(authInfo: AuthInfo[User]) = {
println(">>>>>>>>>>>>>>>>>>>>>>: BEFORE"); //$doc("clientId" $eq authInfo.user.email, "userId" $eq authInfo.user._id.get)
var future = accessTokenService.findRandom(Json.obj("clientId" -> authInfo.user.email, "userId" -> authInfo.user._id.get));
println(">>>>>>>>>>>>>>>>>>>>>>: AFTER: "+future);
future.map { option => {
println("*************************** ")
println("***************************: "+option.isEmpty)
if (!option.isEmpty){
var accessToken = option.get;println(">>>>>>>>>>>>>>>>>>>>>>: BEFORE VALUE");
var value = Crypto.validateToken(accessToken.createdAt.value)
println(">>>>>>>>>>>>>>>>>>>>>>: "+value);
Some(scalaoauth2.provider.AccessToken(accessToken.accessToken, accessToken.refreshToken, authInfo.scope,
Some(value), new Date(accessToken.createdAt.value)))
}else{
Option.empty
}
}}
}
When i using BSONDao and BsonDocument for fetching the data, this code successfully run, but after converting to JSONDao i getting the following error:
Note: Some time this code will run but some it thrown an exception after converting to JSON
play - Cannot invoke the action, eventually got an error: java.lang.IllegalArgumentException: bound must be positive
application -
Following are the logs of application full exception strack trace as below:
>>>>>>>>>>>>>>>>>>>>>>: BEFORE
>>>>>>>>>>>>>>>>>>>>>>: AFTER: scala.concurrent.impl.Promise$DefaultPromise#7f4703e3
play - Cannot invoke the action, eventually got an error: java.lang.IllegalArgumentException: bound must be positive
application -
! #6m1520jff - Internal server error, for (POST) [/oauth2/token] ->
play.api.Application$$anon$1: Execution exception[[IllegalArgumentException: bound must be positive]]
at play.api.Application$class.handleError(Application.scala:296) ~[play_2.11-2.3.8.jar:2.3.8]
at play.api.DefaultApplication.handleError(Application.scala:402) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$3$$anonfun$applyOrElse$4.apply(PlayDefaultUpstreamHandler.scala:320) [play_2.11-2.3.8.jar:2.3.8]
at scala.Option.map(Option.scala:146) [scala-library-2.11.6.jar:na]
Caused by: java.lang.IllegalArgumentException: bound must be positive
at java.util.Random.nextInt(Random.java:388) ~[na:1.8.0_40]
at scala.util.Random.nextInt(Random.scala:66) ~[scala-library-2.11.6.jar:na]
The problem is solve, but i am not sure, why this produce, I think there is problem with reactivemongo-extension JSONDao library. because when i use findOne instead of findRandom the code is run successfully, but the findRandom is run good on BSON dao. Still not found what the exact problem is that, but following is the resolved code.
def getStoredAccessToken(authInfo: AuthInfo[User]) = {
println(authInfo.user.email+" ---- "+authInfo.user._id.get)
var future = accessTokenService.findOne($doc("clientId" $eq authInfo.user.email, "userId" $eq authInfo.user._id.get)); //user findOne instead of findRandom in JsonDao
future.map { option => {
if (!option.isEmpty){
var accessToken = option.get;
var value = Crypto.validateToken(accessToken.createdAt.value)
Some(scalaoauth2.provider.AccessToken(accessToken.accessToken, accessToken.refreshToken, authInfo.scope,
Some(value), new Date(accessToken.createdAt.value)))
}else{
Option.empty
}
}}
}

Play 2.1 Scala SQLException Connection Timed out waiting for a free available connection

I have been working on this issue for quite a while now and I cannot find a solution...
A web app built with play framework 2.2.1 using h2 db (for dev) and a simple Model package.
I am trying to implement a REST JSON endpoint and the code works... but only once per server instance.
def createOtherModel() = Action(parse.json) {
request =>
request.body \ "name" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match name =(")).as("application/json")
case name: JsValue =>
request.body \ "value" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match value =(")).as("application/json")
case value: JsValue =>
// this breaks the secod time
val session = ThinkingSession.dummy
val json = Json.obj(
"content" -> value,
"thinkingSession" -> session.id,
)
)
Ok(Json.obj("content" -> json)).as("application/json")
}
} else {
BadRequest(Json.obj("error" -> true,
"message" -> "Name was not content =(")).as("application/json")
}
}
}
so basically I read the JSON, echo the "value" value, create a model obj and send it's id.
the ThinkingSession.dummy function does this:
def all(): List[ThinkingSession] = {
// Tried explicitly closing connection, no difference
//val conn = DB.getConnection()
//try {
// DB.withConnection { implicit conn =>
// SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
// }
//} finally {
// conn.close()
//}
DB.withConnection { implicit conn =>
SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
}
}
def dummy: ThinkingSession = {
(all() head)
}
So this should do a SELECT * FROM thinking_session, create a model obj list from the result and return the first out of the list.
This works fine the first time after server start but the second time I get a
play.api.Application$$anon$1: Execution exception[[SQLException: Timed out waiting for a free available connection.]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at scala.Option.map(Option.scala:145) [scala-library.jar:na]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: java.sql.SQLException: Timed out waiting for a free available connection.
at com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal(DefaultConnectionStrategy.java:88) ~[bonecp.jar:na]
at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection(AbstractConnectionStrategy.java:90) ~[bonecp.jar:na]
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:553) ~[bonecp.jar:na]
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:131) ~[bonecp.jar:na]
at play.api.db.DBApi$class.getConnection(DB.scala:67) ~[play-jdbc_2.10.jar:2.2.1]
at play.api.db.BoneCPApi.getConnection(DB.scala:276) ~[play-jdbc_2.10.jar:2.2.1]
My application.conf (db section)
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:file:database/[my_db]"
db.default.logStatements=true
db.default.idleConnectionTestPeriod=5 minutes
db.default.connectionTestStatement="SELECT 1"
db.default.maxConnectionAge=0
db.default.connectionTimeout=10000
Initially the only thing set in my config was the connection and the error occurred. I added all the other stuff while reading up on the issue on the web.
What is interesting is that when I use the h2 in memory db it works once after server start and after that it fails. when I use the h2 file system db it only works once, regardless of the server instances.
Can anyone give me some insight on this issue? Have found some stuff on bonecp problem and tried upgrading to 0.8.0-rc1 but nothing changed... I am at a loss =(
Try to set a maxConnectionAge and idle timeout
turns out the error was quite somewhere else... it was a good ol' stack overflow... have not seen one in a long time. I tried down-voting my question but it's not possible^^

Getting The entire node set is unreachable error when trying to connect to MongoDB with reactivemongo in my play2 application

When working locally I have no problem testing my application by calling:
http://localhost:9000/r/123
method:
def showSurvey(id: String) = Action {
implicit val reader = Review.ReviewReader
Async {
val cursor = reviews.find(BSONDocument("_id" -> BSONObjectID(id))).cursor[Review]
cursor.headOption.map(maybeReview =>
maybeReview.map(review => {
// fill form
Ok(views.html.review.desktopSurvey(Some(id), surveyForm.fill(SurveyForm(review.grade, review.text, REGULAR_AUTH, false, "", "")), grades))
}
).getOrElse {
//NotFound Temporary below:
val review = Review(Some(new BSONObjectID(id)), 3, "bla", Some(new DateTime()), Some(new DateTime()), Some("0.0.0.0"), ReviewStatus.NOT_CLAIMED, Some(1), Some(1L))
Ok(views.html.review.desktopSurvey(Some(id), surveyForm.fill(SurveyForm(review.grade, review.text, REGULAR_AUTH, false, "", "")), grades))
}
).recover {
case e => InternalServerError(e.getMessage())
}
}
}
But when running the app in production by doing clean compile stage and then going to the
url I get:
[error] r.c.a.MongoDBSystem - The entire node set is unreachable, is there a network problem?
I find this very strange since the plugin seems to work properly:
[info] application - ReactiveMongoPlugin successfully started with db 'blala'! Servers:
[87.238.57.140:27017]
By checking the mongodb log i found that the connection was ok but the bson sent in was invalid:
Tue May 21 11:18:11.257 [conn531] Assertion: 10307:Client Error: bad object in message: invalid bson
0xdcf361 0xd90a1b 0xd90f5c 0x75b289 0x75b3fb 0x9f4367 0x9f57e2 0x6e747a 0xdbbb7e 0x7fa22e96d9ca 0x7fa22dd14cdd
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdcf361]
/usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0xd90a1b]
/usr/bin/mongod() [0xd90f5c]
/usr/bin/mongod(_ZN5mongo9DbMessage9nextJsObjEv+0x249) [0x75b289]
/usr/bin/mongod(_ZN5mongo12QueryMessageC1ERNS_9DbMessageE+0x8b) [0x75b3fb]
/usr/bin/mongod() [0x9f4367]
/usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x392) [0x9f57e2]
/usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9a) [0x6e747a]
/usr/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x42e) [0xdbbb7e]
/lib/libpthread.so.0(+0x69ca) [0x7fa22e96d9ca]
/lib/libc.so.6(clone+0x6d) [0x7fa22dd14cdd]
Tue May 21 11:18:11.261 [conn531] AssertionException handling request, closing client connection: 10307 Client Error: bad object in message: invalid bson
This made me look at the code and the problem was that I was sending in an invalid ObjectId, in my case 123 . Somehow this error got swallowed by my application. So when there is a possibility that the ObjectId sent in might be corrupt check it with:
val maybeOID: Try[BSONObjectID] = BSONObjectID.parse(id)
if (maybeOID.isSuccess) {
// Do stuff
}