I am writing an Action in Play which should add a Document in MongoDB if it doesn't exist already. I have tried two approaches but none of them are working. I am stuck around how to handle Future returned by reactivemongoplugin
The Action gets JSON data. It valides it. If JSON is OK, I check if the user exists (look at firstname). If it doesnt, I want to add it else return an error. I am unable to combine ReactiveMongoPlugin's 'find' and 'insert' methods in the same action
Approach 1** - This doesn't compile because the code returns scala.concurrent.Future[play.api.mvc.SimpleResult] while it requires play.api.mvc.SimpleResult. I know that it is because I am using a map inside a map.
def registrationRequest = Action.async(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr: JsResult[User2] = request.body.validate[User2]
Logger.debug("jr is " + jr)
jr match {
case s: JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
val query = BSONDocument("user.firstname" -> user.firstName)
Logger.debug("query is:" + query)
//result is of type Future[Option[BSONDocument]]
val findFuture:Future[Option[BSONDocument]] = collection.find(query).one
findFuture.map(option => option match {
case None => {
//no record. Can add
Logger.debug("No record from mongo: Can add")
val newUserDoc = BSONDocument (
"id" -> user.id,
"user" -> BSONDocument (
"firstname" -> user.firstName,
"lastname" -> user.lastName,
"email" -> BSONArray (user.email (0) ),
"password" -> user.password,
"address" -> BSONDocument (
"street" -> user.address.street,
"country" -> user.address.country
)
)
)
//addResult is of type Future[LastError]
//this code is problamatic. I am calling a map within a map which creates a Future[Future[Result]]. I need only Future[Result]
val insertResult = collection.insert (newUserDoc)
insertResult.map(le=>{
if(le.ok) {
Logger.debug("insertFuture map")
val ack = Acknowledgment(0, "insert success: ")
Logger.debug("insert success:" + Json.toJson(ack))
Ok(Json.toJson(ack))
}else {
Logger.debug("not inserting")
val ack = Acknowledgment (0, "duplicate: ")
Logger.debug ("fail ack:" + Json.toJson (ack) )
BadRequest (Json.toJson (ack) )
}
})}
case Some(x) => {
//duplicae record
Logger.debug("error from Mongo. Duplicate:" + x)
val ack = Acknowledgment(0, "duplicate: " + x.toString())
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
})
//findFutureResult is a Future[Int]
case f: JsError => Future.successful({
Logger.debug("error: " + JsError.toFlatJson(f))
val ack = Acknowledgment(0, JsError.toFlatJson(f).toString())
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
})
}
}
}
Approach 2 - In this approach, I broke the steps to avoid calling map within map. Following code is for JsSuccess part
case s: JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
val query = BSONDocument("user.firstname" -> user.firstName)
Logger.debug("query is:" + query)
//result is of type Future[Option[BSONDocument]]
val findFuture:Future[Option[BSONDocument]] = collection.find(query).one
//findFutureResult is a Future[Int]
//to avoid calling map within map, I am creating single Futures which would convey result of one Future to another.
val findFutureResult:Future[Int] = findFuture.map(option => option match {
case None => {
//no record. Can add
Logger.debug("No record from mongo: Can add")
1 //return of 1 means record can be added
}
case Some(x) => {
//duplicae record
Logger.debug("error from Mongo. Duplicate:" + x)
2 //return of 2 means record cannot be added.
}
})
//this code would return LastError. the value of LastError can be used to check if insert was performed or not. Accordingly, I want to send OK or BadRequest
val insertFuture:Future[Future[LastError]] = findFutureResult.map(r => {r match {
case 1 => {
Logger.debug("findFutureResult map. Adding doc")
val newUserDoc = BSONDocument (
"id" -> user.id,
"user" -> BSONDocument (
"firstname" -> user.firstName,
"lastname" -> user.lastName,
"email" -> BSONArray (user.email (0) ),
"password" -> user.password,
"address" -> BSONDocument (
"street" -> user.address.street,
"country" -> user.address.country
)
)
)
//addResult is of type Future[LastError]
collection.insert (newUserDoc)
}
case 2 => Future.successful({
Logger.debug("findFutureResultmap. Not adding a duplicate")
LastError(false,None, None, None, None, 0,false )
})
}
})
//this is the problematic code. How do i check value of LastError? insertFuture is Future[Future[LastError]] and not Future[LastError]
insertFuture.map(lef=>{ lef.map(le=>{ // I cannot call map inside map as explained in approach 1
if(le.ok) {
Logger.debug("insertFuture map")
val ack = Acknowledgment(0, "insert success: ")
Logger.debug("insert success:" + Json.toJson(ack))
Ok(Json.toJson(ack))
}
else {
Logger.debug("not inserting")
val ack = Acknowledgment (0, "duplicate: ")
Logger.debug ("fail ack:" + Json.toJson (ack) )
BadRequest (Json.toJson (ack) )
}
})})
}
}
I know what the problem is in the code. I do not know how to solve it. I assume my approach is not bad - I want to check database before inserting in it but I am unable to fit it around reactivemongo apis and Futures
flatMap worked. Thank you Cyrille Corpet and cchantep
def registrationRequest = Action.async(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr: JsResult[User2] = request.body.validate[User2]
Logger.debug("jr is " + jr)
jr match {
case s: JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
val query = BSONDocument("user.firstname" -> user.firstName)
Logger.debug("query is:" + query)
//result is of type Future[Option[BSONDocument]]
val findFuture: Future[Option[BSONDocument]] = collection.find(query).one
val insertFuture: Future[Future[LastError]] = findFuture.map(option => option match {
case None => {
//no record. Can add
Logger.debug("No record from mongo: Can add")
//1
Logger.debug("findFutureResult map. Adding doc")
val newUserDoc = BSONDocument(
"id" -> user.id,
"user" -> BSONDocument(
"firstname" -> user.firstName,
"lastname" -> user.lastName,
"email" -> BSONArray(user.email(0)),
"password" -> user.password,
"address" -> BSONDocument(
"street" -> user.address.street,
"country" -> user.address.country
)
)
)
//addResult is of type Future[LastError]
collection.insert(newUserDoc)
}
case Some(x) => {
//duplicae record
Logger.debug("error from Mongo. Duplicate:" + x)
//2
Future.successful({
Logger.debug("findFutureResultmap. Not adding a duplicate")
LastError(false, None, None, None, None, 0, false)
})
}
})
insertFuture.flatMap(lef => {
lef.map(le => {
if (le.ok) {
Logger.debug("insertFuture map")
val ack = Acknowledgment(0, "insert success: ")
Logger.debug("insert success:" + Json.toJson(ack))
Ok(Json.toJson(ack))
}
else {
Logger.debug("not inserting")
val ack = Acknowledgment(0, "duplicate: ")
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
})
//findFutureResult is a Future[Int]
})
}
case f: JsError => Future.successful({
Logger.debug("error: " + JsError.toFlatJson(f))
val ack = Acknowledgment(0, JsError.toFlatJson(f).toString())
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
})
}
}
}
Related
I have a problem to make Transaction via JSONCollection, I getting the following error:
JsResultException(errors:List((,List(JsonValidationError(List(CommandError[code=14, errmsg=BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long', doc: {"operationTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"ok":0,"errmsg":"BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long'","code":14,"codeName":"TypeMismatch","$clusterTime":{"clusterTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"signature":{"hash":{"$binary":"0000000000000000000000000000000000000000","$type":"00"},"keyId":0}}}]),WrappedArray())))))
I tried to change my project to BSONCollection but got some troubles, maybe there solution to overcome the above error with JSONCollection.
Also the exceptions occurs on testing update method, but checking the insertOneViaTransaction and setRuleAsInactiveViaTransaction is finished with success
This is my code for Transaction:
Update:
def update(oldRule: ExistRuleDto): Future[UpdateResult] = {
val transaction = (collection: JSONCollection) => for {
newRule <- dao.insertOneViaTransaction(collection,oldRule.toUpdatedRule) // insert new with ref to old
oldRule <- dao.setRuleAsInactiveViaTransaction(collection,oldRule.id)
} yield UpdateResult(oldRule, newRule)
makeTransaction[UpdateResult](transaction)
}
makeTransaction:
def makeTransaction[Out](block: JSONCollection => Future[Out]): Future[Out] = for {
dbWithSession <- dao.collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
coll = dbWithTx.collection[JSONCollection](dao.collection.name)
// Operations:
res <- block(coll)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield res
insertOneViaTransaction:
def insertOneViaTransaction(collection: JSONCollection, rule: Rule): Future[Rule] = {
collection.insert.one(rule).map {
case DefaultWriteResult(true, 1, _, _, _, _) => rule
case err => throw GeneralDBError(s"$rule was not inserted, something went wrong: $err")
}.recover {
case WriteResult.Code(11000) => throw DuplicationError(s"$rule exist on DB")
case err => throw GeneralDBError(err.getMessage)
}
}
setRuleAsInactiveViaTransaction:
def setRuleAsInactiveViaTransaction(collection: JSONCollection, ruleId: BSONObjectID): Future[Rule] = {
collection.findAndUpdate(
Json.obj(s"${Rule.ID}" -> ruleId),
Json.obj(
"$set" -> Json.obj(s"${Rule.Metadata}.${Metadata.Active}" -> false),
"$unset" -> Json.obj(s"${Rule.Metadata}.${Metadata.LastVersionExists}" -> "")),
fetchNewObject = true, upsert = false, sort = None, fields = None, bypassDocumentValidation = false, writeConcern = WriteConcern.Acknowledged, maxTime = None, collation = None, arrayFilters = Nil
).map(el => el.result[Rule].getOrElse {
val msg = s"Operation fail for updating ruleId = $ruleId"
logger.error(msg)
throw GeneralUpdateError(msg)
})
}
I'm using the following dependencies:
Play:
"com.typesafe.play" % "sbt-plugin" % "2.7.2
Reactivemongo:
"org.reactivemongo" %% "play2-reactivemongo" % "0.18.8-play27"
Solve it. (Not with compact)
Serializers:
implicit object JsValueHandler extends BSONHandler[BSONValue, JsValue] {
implicit override def read(bson: BSONValue): JsValue = BSONFormats.toJSON(bson)
implicit override def write(j: JsValue): BSONValue = BSONFormats.toBSON(j).get
}
asTransaction:
def asTransaction[Out](block: BSONCollection => Future[Out]): Future[Out] = {
for {
dbWithSession <- collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
collectionWithTx = dbWithTx.collection[BSONCollection](collection.name)
out <- block(collectionWithTx)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield out
}.recover {
case ex: Exception =>
logger.warn(s"asTransaction failed with ex: ${ex.getMessage}, rollback to previous state...")
throw GeneralDBErrorOnTx(ex.getMessage)
}
transaction example:
def `change visibility of ExistsRules and insert UpdateEvents`(oldRules: List[Rule], active: Boolean): Future[Unit] = {
ruleDao.asTransaction { collectionTx =>
for {
// (1) - $active old Rules
_ <- ruleDao.updateManyWithBsonCollection(
collectionTx,
filter = BSONDocument(s"${Rule.ID}" -> BSONDocument("$in" -> oldRules.map(_._id))),
update = BSONDocument("$set" -> BSONDocument(s"${Rule.Metadata}.${Metadata.Active}" -> active)))
// (2) - Sync Cache with Update Events
_ <- eventsService.addEvents(oldRules.map(rule => RuleEvent(rule.metadata.cacheKey, Update)))
} yield ()
}
}
Enjoy!
Pls suggest here I have an upload service in Akka HTTP micro service it's working fine. now I need to write the test cases for below code
path( "file-upload") {
extractClientIP { ip =>
optionalHeaderValueByName(Constants.AUTH) { auth =>
(post & extractRequestContext) { request =>
extractRequestContext {
ctx => {
implicit val materializer = ctx.materializer
implicit val ec = ctx.executionContext
val currentTime = TimeObject.getCurrentTime()
fileUpload("fileUpload") {
case (fileInfo, fileStream) =>
val localPath = Configuration.excelFilePath
val uniqueidString = "12345"
val filename = uniqueidString + fileInfo.fileName
val sink = FileIO.toPath(Paths.get(localPath) resolve filename)
val writeResult = fileStream.runWith(sink)
onSuccess(writeResult) { result =>
result.status match {
case Success(_) =>
var excelPath = localPath + File.separator + uniqueidString + fileInfo.fileName
var doc_count = itemsExcelParse(excelPath, companyCode, subCompanyId, userId)
val currentTime2 = TimeObject.getCurrentTime()
var upload_time = currentTime2 - currentTime
val resp: JsValue = Json.toJson(doc_count)
complete {
val json: JsValue = Json.obj("status" -> Constants.SUCCESS,
"status_details" -> "null", "upload_details" -> resp)
HttpResponse(status = StatusCodes.OK, entity = HttpEntity(ContentType(MediaTypes.`application/json`), json.toString))
}
case Failure(e) =>
complete {
val json: JsValue = Json.obj("status" -> Constants.ERROR, "status_details" -> Constants.ERROR_445)
HttpResponse(status = StatusCodes.BandwidthLimitExceeded, entity = HttpEntity(ContentType(MediaTypes.`application/json`), json.toString))
}
}
}
}
}
}
}
}
}
}
I have tried test cases but it's not working
" File upload " should "be able to upload file" in {
val p: Path = Paths.get("E:\\Excel\\Tables.xlsx")
val formData = Multipart.FormData.fromPath("fileUpload", ContentTypes.NoContentType, p, 1000)
Post(s"/file-upload", formData) -> route -> check {
println(" File upload - file uploaded successfully")
status shouldBe StatusCodes.OK
responseAs[String] contains "File successfully uploaded"
}
}
I have changed content type also into application/octet-stream. File not uploaded to the server please suggest here how can I write the test case for file uploading.
I am creating a user registration module. On submission (using JSON), I want to check if the JSON parsed correctly. If there is a problem in JSON, I want to return error. If the JSON is correct, I want to check if the user already exists (look at firstname). Data is in MongoDB. I am using ReactiveMongoPlugin 0.10. I will use 'one' method which returns Future[Option[BSONDocument]]. How do I wait for this Future to finish before the Action completes?
Approach 1 - use Action and try to handle result of Future myself. Code doesn't compile and dont know how to wait for the Future to finish
def registrationRequest = Action(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr:JsResult[User2] = request.body.validate[User2]
Logger.debug( "jr is "+jr)
jr match {
case s:JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
// the data from client is a JSON of type {user:{firstname:"name"}}. I have created code to parse the JSON
val query = BSONDocument("user"-> BSONDocument("firstname"->user.firstname))
Logger.debug("query is:"+query)
val result = collection.find(query).one
I want to now wait for result and return either an Ok(Json.toJson(ack)) or BadRequest(Json.toJson(ack)). How do i do that? I have written following code but I am stuck at two points (a) will the code wait for future to finish (b) onComplete returns Unit but Play's Action requires play.api.mvc.Result. How do I do that?
//I guess data would be Success or Failure
result onComplete ( data =>
data match {
//If Success, value would be Some or None
case Success(value) => {
value match {
case None => { //no record. Can add
Logger.debug("No record from mongo: Can add")
val ack = Acknowledgment (1, "Welcome " + user.firstName + " " + user.lastName)
Logger.debug ("success ack Json:" + Json.toJson (ack) )
Ok (Json.toJson (ack) )
}
case Some(x) => { //duplicae record
Logger.debug("error from Mongo. Duplicate:"+x)
val ack = Acknowledgment(0,"duplicate: "+x.toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
}
case Failure (e)=> {
Logger.debug("error from Mongo."+e)
val ack = Acknowledgment(0,"MongoDB Error: "+e.toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}) //onComplete returns Unit. Action needs play.api.mvc.Result
case f:JsError => {
Logger.debug("error: "+JsError.toFlatJson(f))
val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
}
Approach 2 - I read that I should use Action.async but I am unable to fit the pieces together. The 2nd approach I followed was to use Action.Async but the code didnt compile because it expects Future[SimpleResult]
def registrationRequest = Action.async(parse.json) { request => {
Logger.debug("received message:" + request)
Logger.debug("received message:" + request.body)
val jr:JsResult[User2] = request.body.validate[User2]
Logger.debug( "jr is "+jr)
jr match {
case s:JsSuccess[User2] => {
val user = s.get
Logger.debug("opening database connection")
val driver = new MongoDriver()
val connection = driver.connection(List("localhost"))
val db = connection.db("website-db")
val collection = db.collection[BSONCollection]("users")
val query = BSONDocument("user"-> BSONDocument("firstname"->user.firstName))
Logger.debug("query is:"+query)
//result is of type Future[Option[BSONDocument]]
val result = collection.find(query).one
result.map(option => option match {
case None => {
//no record. Can add
Logger.debug("No record from mongo: Can add")
val ack = Acknowledgment(1, "Welcome " + user.firstName + " " + user.lastName)
Logger.debug("success ack Json:" + Json.toJson(ack))
Ok(Json.toJson(ack))
}
case Some(x) => {
//duplicae record
Logger.debug("error from Mongo. Duplicate:" + x)
val ack = Acknowledgment(0, "duplicate: " + x.toString())
Logger.debug("fail ack:" + Json.toJson(ack))
BadRequest(Json.toJson(ack))
}
}
)
}
case f:JsError => {
Logger.debug("error: "+JsError.toFlatJson(f))
val ack = Acknowledgment(0,JsError.toFlatJson(f).toString())
Logger.debug("fail ack:"+Json.toJson(ack))
BadRequest(Json.toJson(ack)) //Action.async expect scala.concurrent.Future[play.api.mvc.SimpleResult]
}
}
}
The solution is to use Action.async which returns Future[SimpleResult]. Inside the code use map and flatMap on Future to return Future[SimpleResult].
So I've tried seemingly countless things to get this to work. When I call queueWrite, the println statements give me this:
{ "uuid" : "49f2-0b64-4bf3-49f2a35b-bbe8-4954f742d88b" }
and this:
{ "uuid" : "49f2-0b64-4bf3-49f2a35b-bbe8-4954f742d88b", "name" : "personName", "key" : "3E6A" }
Which (I'm pretty sure) is just fine. However, after it prints, I get this:
java.lang.IllegalArgumentException: Invalid BSON field name uuid
Afaik, the field name uuid is fine, the only things about an improper name I could really find is to just make sure there are no '.' symbols in it (which there aren't)
def queueWrite(collection: String, filter: Map[String, () => String], data: Map[String, () => String]) {
val col = collections.get(collection).get
val filterDoc = new BsonDocument
filter.foreach(f => { filterDoc append (f._1, new BsonString(f._2.apply)) })
val filterBson = Document(filterDoc)
println("filter: \n" + filterBson.toJson)
val dataDoc = new BsonDocument
data.foreach(f => { dataDoc append (f._1, new BsonString(f._2.apply)) })
val dataBson = Document(dataDoc)
println("data: \n" + dataBson.toJson)
val options = new FindOneAndUpdateOptions
options.returnDocument(ReturnDocument.AFTER)
options.upsert(true)
val observer = new Observer[Document] {
override def onSubscribe(s: Subscription) = s.request(1)
override def onNext(doc: Document) = println(doc.toJson)
override def onError(e: Throwable) = e.printStackTrace
override def onComplete = println("onComplete")
}
val observable: Observable[Document] = col.findOneAndUpdate(filterBson, dataBson, options)
observable.subscribe(observer)
}
Any ideas / suggestions are greatly appreciated as always :)
I'm trying to write some basic queries with Slick for SQLite database
Here is my code:
class MigrationLog(name: String) {
val migrationEvents = TableQuery[MigrationEventTable]
lazy val db: Future[SQLiteDriver.backend.DatabaseDef] = {
val db = Database.forURL(s"jdbc:sqlite:$name.db", driver = "org.sqlite.JDBC")
val setup = DBIO.seq(migrationEvents.schema.create)
val createFuture = for {
tables <- db.run(MTable.getTables)
createResult <- if (tables.length == 0) db.run(setup) else Future.successful()
} yield createResult
createFuture.map(_ => db)
}
val addEvent: (String, String) => Future[String] = (aggregateId, eventType) => {
val id = java.util.UUID.randomUUID().toString
val command = DBIO.seq(migrationEvents += (id, aggregateId, None, eventType, "CREATED", System.currentTimeMillis, None))
db.flatMap(_.run(command).map(_ => id))
}
val eventSubmitted: (String, String) => Future[Unit] = (id, batchId) => {
val q = for { e <- migrationEvents if e.id === id } yield (e.batchId, e.status, e.updatedAt)
val updateAction = q.update(Some(batchId), "SUBMITTED", Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val eventMigrationCompleted: (String, String, String) => Future[Unit] = (batchId, id, status) => {
val q = for { e <- migrationEvents if e.batchId === batchId && e.id === id} yield (e.status, e.updatedAt)
val updateAction = q.update(status, Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val allEvents = () => {
db.flatMap(_.run(migrationEvents.result))
}
}
Here is how I'm using it:
val migrationLog = MigrationLog("test")
val events = for {
id <- migrationLog.addEvent("aggregateUserId", "userAccessControl")
_ <- migrationLog.eventSubmitted(id, "batchID_generated_from_idam")
_ <- migrationLog.eventMigrationCompleted("batchID_generated_from_idam", id, "Successful")
events <- migrationLog.allEvents()
} yield events
events.map(_.foreach(event => event match {
case (id, aggregateId, batchId, eventType, status, submitted, updatedAt) => println(s"$id $aggregateId $batchId $eventType $status $submitted $updatedAt")
}))
The idea is to add event first, then update it with batchId (which also updates status) and then update the status when the job is done. events should contain events with status Successful.
What happens is that after running this code it prints events with status SUBMITTED. If I wait a while and do the same allEvents query or just go and check the db from command line using sqlite3 then it's updated correctly.
I'm properly waiting for futures to be resolved before starting the next operation, auto-commit should be enabled by default.
Am I missing something?
Turns out the problem was with db.map(_.run(updateAction)) which returns Future[Future[Int]] which means that the command was not finished by the time I tried to run another query.
Replacing it with db.flatMap(_.run(updateAction)) solved the issue.