I am trying to run this simple code, which insert a document to "reports" collection under transaction.
This is the code:
def runWithTransaction(): Future[Unit] = {
for {
dbWithSession <- db.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
reportsCollection = dbWithTx.collection[JSONCollection]("reports")
_ <- reportsCollection.insert.one(BSONDocument("id" -> "123123"))
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield ()
}
I am getting the following error:
play.api.libs.json.JsResultException: JsResultException(errors:List((,List(JsonValidationError(List(CommandError[code=14, errmsg=BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long', doc: {"operationTime":{"$time":1573135213,"$i":1,"$timestamp":{"t":1573135213,"i":1}},"ok":0,"errmsg":"BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long'","code":14,"codeName":"TypeMismatch","$clusterTime":{"clusterTime":{"$time":1573135213,"$i":1,"$timestamp":{"t":1573135213,"i":1}},"signature":{"hash":{"$binary":"0000000000000000000000000000000000000000","$type":"00"},"keyId":0}}}]),WrappedArray())))))
at reactivemongo.play.json.JSONSerializationPack$.deserialize(JSONSerializationPack.scala:61)
at reactivemongo.play.json.JSONSerializationPack$.deserialize(JSONSerializationPack.scala:33)
at reactivemongo.api.commands.Command$$anon$2.$anonfun$one$6(commands.scala:141)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:658)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:92)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:92)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:41)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:49)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
I am using:
scala 2.12
"org.reactivemongo" % "play2-reactivemongo_2.12" % "0.18.8-play27"
MongoDB version 4.2.1 (its actually a replica set with 1 primary)
Any help is appreciated, thanks.
Use this code to Insert:
def insert(coll: BSONCollection, document: BSONDocument): Future[Unit] = {
val writeRes: Future[WriteResult] = coll.insert.one(document)
writeRes.onComplete { // Dummy callbacks
case Failure(e) => e.printStackTrace()
case Success(writeResult) =>
println(s"successfully inserted document with result: $writeResult")
}
writeRes.map(_ => {}) // in this example, do nothing with the success
}
And If you want to update, use this:
def update(collection: BSONCollection, item_id: Long, modifier: BSONDocument) = {
val selector = BSONDocument("_id" -> item_id)
println("selector ===" + selector)
val futureUpdate = collection.update.one(
q = selector, u = modifier,
upsert = true, multi = false)
futureUpdate.onComplete { // Dummy callbacks
case Failure(e) => e.printStackTrace()
case Success(writeResult) =>
println(s"successfully updated document with result: $writeResult")
}
}
Related
I have a problem to make Transaction via JSONCollection, I getting the following error:
JsResultException(errors:List((,List(JsonValidationError(List(CommandError[code=14, errmsg=BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long', doc: {"operationTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"ok":0,"errmsg":"BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long'","code":14,"codeName":"TypeMismatch","$clusterTime":{"clusterTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"signature":{"hash":{"$binary":"0000000000000000000000000000000000000000","$type":"00"},"keyId":0}}}]),WrappedArray())))))
I tried to change my project to BSONCollection but got some troubles, maybe there solution to overcome the above error with JSONCollection.
Also the exceptions occurs on testing update method, but checking the insertOneViaTransaction and setRuleAsInactiveViaTransaction is finished with success
This is my code for Transaction:
Update:
def update(oldRule: ExistRuleDto): Future[UpdateResult] = {
val transaction = (collection: JSONCollection) => for {
newRule <- dao.insertOneViaTransaction(collection,oldRule.toUpdatedRule) // insert new with ref to old
oldRule <- dao.setRuleAsInactiveViaTransaction(collection,oldRule.id)
} yield UpdateResult(oldRule, newRule)
makeTransaction[UpdateResult](transaction)
}
makeTransaction:
def makeTransaction[Out](block: JSONCollection => Future[Out]): Future[Out] = for {
dbWithSession <- dao.collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
coll = dbWithTx.collection[JSONCollection](dao.collection.name)
// Operations:
res <- block(coll)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield res
insertOneViaTransaction:
def insertOneViaTransaction(collection: JSONCollection, rule: Rule): Future[Rule] = {
collection.insert.one(rule).map {
case DefaultWriteResult(true, 1, _, _, _, _) => rule
case err => throw GeneralDBError(s"$rule was not inserted, something went wrong: $err")
}.recover {
case WriteResult.Code(11000) => throw DuplicationError(s"$rule exist on DB")
case err => throw GeneralDBError(err.getMessage)
}
}
setRuleAsInactiveViaTransaction:
def setRuleAsInactiveViaTransaction(collection: JSONCollection, ruleId: BSONObjectID): Future[Rule] = {
collection.findAndUpdate(
Json.obj(s"${Rule.ID}" -> ruleId),
Json.obj(
"$set" -> Json.obj(s"${Rule.Metadata}.${Metadata.Active}" -> false),
"$unset" -> Json.obj(s"${Rule.Metadata}.${Metadata.LastVersionExists}" -> "")),
fetchNewObject = true, upsert = false, sort = None, fields = None, bypassDocumentValidation = false, writeConcern = WriteConcern.Acknowledged, maxTime = None, collation = None, arrayFilters = Nil
).map(el => el.result[Rule].getOrElse {
val msg = s"Operation fail for updating ruleId = $ruleId"
logger.error(msg)
throw GeneralUpdateError(msg)
})
}
I'm using the following dependencies:
Play:
"com.typesafe.play" % "sbt-plugin" % "2.7.2
Reactivemongo:
"org.reactivemongo" %% "play2-reactivemongo" % "0.18.8-play27"
Solve it. (Not with compact)
Serializers:
implicit object JsValueHandler extends BSONHandler[BSONValue, JsValue] {
implicit override def read(bson: BSONValue): JsValue = BSONFormats.toJSON(bson)
implicit override def write(j: JsValue): BSONValue = BSONFormats.toBSON(j).get
}
asTransaction:
def asTransaction[Out](block: BSONCollection => Future[Out]): Future[Out] = {
for {
dbWithSession <- collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
collectionWithTx = dbWithTx.collection[BSONCollection](collection.name)
out <- block(collectionWithTx)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield out
}.recover {
case ex: Exception =>
logger.warn(s"asTransaction failed with ex: ${ex.getMessage}, rollback to previous state...")
throw GeneralDBErrorOnTx(ex.getMessage)
}
transaction example:
def `change visibility of ExistsRules and insert UpdateEvents`(oldRules: List[Rule], active: Boolean): Future[Unit] = {
ruleDao.asTransaction { collectionTx =>
for {
// (1) - $active old Rules
_ <- ruleDao.updateManyWithBsonCollection(
collectionTx,
filter = BSONDocument(s"${Rule.ID}" -> BSONDocument("$in" -> oldRules.map(_._id))),
update = BSONDocument("$set" -> BSONDocument(s"${Rule.Metadata}.${Metadata.Active}" -> active)))
// (2) - Sync Cache with Update Events
_ <- eventsService.addEvents(oldRules.map(rule => RuleEvent(rule.metadata.cacheKey, Update)))
} yield ()
}
}
Enjoy!
I have a endpoint that takes in a JSON body. I have implicit reads and writes for this JSON format. In the endpoint, I do a validation of the JSON and fold on the result! Here it is:
def updatePowerPlant(id: Int) = Action.async(parse.tolerantJson) { request =>
request.body.validate[PowerPlantConfig].fold(
errors => {
Future.successful(
BadRequest(
Json.obj("message" -> s"invalid PowerPlantConfig ${errors.mkString(",")}")
).enableCors
)
},
success => {
dbService.insertOrUpdatePowerPlant(success).runAsync.materialize.map {
case Failure(ex) =>
InternalServerError(s"Error updating PowerPlant " +
s"Reason => ${ex.getMessage}").enableCors
case Success(result) =>
result match {
case Left(errorMessage) =>
BadRequest(Json.obj("message" -> s"invalid PowerPlantConfig $errorMessage")).enableCors
case Right(updatedConfig) =>
Ok(Json.prettyPrint(Json.toJson(updatedConfig))).enableCors
}
}
}
)
}
So as it can be seen that I fold on the error and I return a BadRequest. But when I tried writing a unit test, I do not get the HTTP status as BadRequest as I expect, but the test crashes with an exception as below:
JsResultException(errors:List((,List(ValidationError(List(error.expected.jsnumber),WrappedArray())))))
play.api.libs.json.JsResultException: JsResultException(errors:List((,List(ValidationError(List(error.expected.jsnumber),WrappedArray())))))
at play.api.libs.json.JsReadable$$anonfun$2.apply(JsReadable.scala:23)
at play.api.libs.json.JsReadable$$anonfun$2.apply(JsReadable.scala:23)
at play.api.libs.json.JsResult$class.fold(JsResult.scala:73)
at play.api.libs.json.JsError.fold(JsResult.scala:13)
at play.api.libs.json.JsReadable$class.as(JsReadable.scala:21)
at play.api.libs.json.JsDefined.as(JsLookup.scala:132)
at com.inland24.plantsim.models.package$$anon$1.reads(package.scala:61)
at play.api.libs.json.JsValue$class.validate(JsValue.scala:18)
at play.api.libs.json.JsObject.validate(JsValue.scala:76)
at com.inland24.plantsim.controllers.PowerPlantController$$anonfun$updatePowerPlant1$1.apply(PowerPlantController.scala:64)
at com.inland24.plantsim.controllers.PowerPlantController$$anonfun$updatePowerPlant1$1.apply(PowerPlantController.scala:63)
at play.api.mvc.Action$.invokeBlock(Action.scala:498)
at play.api.mvc.Action$.invokeBlock(Action.scala:495)
at play.api.mvc.ActionBuilder$$anon$2.apply(Action.scala:458)
at com.inland24.plantsim.controllers.PowerPlantControllerTest$$anonfun$4$$anonfun$apply$mcV$sp$11.apply(PowerPlantControllerTest.scala:313)
at com.inland24.plantsim.controllers.PowerPlantControllerTest$$anonfun$4$$anonfun$apply$mcV$sp$11.apply(PowerPlantControllerTest.scala:296)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.WordSpecLike$$anon$1.apply(WordSpecLike.scala:1078)
at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.withFixture(PowerPlantControllerTest.scala:40)
at org.scalatest.WordSpecLike$class.invokeWithFixture$1(WordSpecLike.scala:1075)
at org.scalatest.WordSpecLike$$anonfun$runTest$1.apply(WordSpecLike.scala:1088)
at org.scalatest.WordSpecLike$$anonfun$runTest$1.apply(WordSpecLike.scala:1088)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
at org.scalatest.WordSpecLike$class.runTest(WordSpecLike.scala:1088)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.runTest(PowerPlantControllerTest.scala:40)
at org.scalatest.WordSpecLike$$anonfun$runTests$1.apply(WordSpecLike.scala:1147)
at org.scalatest.WordSpecLike$$anonfun$runTests$1.apply(WordSpecLike.scala:1147)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:373)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:410)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
at org.scalatest.WordSpecLike$class.runTests(WordSpecLike.scala:1147)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.runTests(PowerPlantControllerTest.scala:40)
at org.scalatest.Suite$class.run(Suite.scala:1147)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.org$scalatest$WordSpecLike$$super$run(PowerPlantControllerTest.scala:40)
at org.scalatest.WordSpecLike$$anonfun$run$1.apply(WordSpecLike.scala:1192)
at org.scalatest.WordSpecLike$$anonfun$run$1.apply(WordSpecLike.scala:1192)
at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
at org.scalatest.WordSpecLike$class.run(WordSpecLike.scala:1192)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.org$scalatest$BeforeAndAfterAll$$super$run(PowerPlantControllerTest.scala:40)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:213)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:210)
at com.inland24.plantsim.controllers.PowerPlantControllerTest.run(PowerPlantControllerTest.scala:40)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1340)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1334)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1334)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1500)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
at org.scalatest.tools.Runner$.run(Runner.scala:850)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:138)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
Here is my unit test:
"not update for an invalid PowerPlantConfig JSON" in {
// We are updating the PowerPlant with id = 101, Notice that the powerPlantId is invalid
val jsBody =
"""
|{
| "powerPlantId":"invalidId",
| "powerPlantName":"joesan 1",
| "minPower":100,
| "maxPower":800,
| "rampPowerRate":20.0,
| "rampRateInSeconds":"2 seconds",
| "powerPlantType":"RampUpType"
|}
""".stripMargin
val result: Future[Result] =
controller.updatePowerPlant(101)
.apply(
FakeRequest().withBody(Json.parse(jsBody))
)
result.materialize.map {
case Success(succ) =>
assert(succ.header.status === BAD_REQUEST)
case Failure(_) =>
fail("Unexpected test failure when Updating a PowerPlant! Please Analyze!")
}
}
Any idea why I'm not getting the expected behavior? I'm expecting that I get a HTTP BadRequest back!
EDIT: To get rid of the unexpected exception, I had to wrap my code into a Try block and I do not want that. So this piece of code gets rid of the error:
def updatePowerPlant(id: Int) = Action.async(parse.tolerantJson) { request =>
scala.util.Try(request.body.validate[PowerPlantConfig]) match {
case Failure(fail) =>
Future.successful(InternalServerError(s"Error updating PowerPlant " +
s"Reason => ${fail.getMessage}").enableCors)
case Success(succ) =>
succ.fold(
errors => {
Future.successful(
BadRequest(
Json.obj("message" -> s"invalid PowerPlantConfig ${errors.mkString(",")}")
).enableCors
)
},
success => {
dbService.insertOrUpdatePowerPlant(success).runAsync.materialize.map {
case Failure(ex) =>
InternalServerError(s"Error updating PowerPlant " +
s"Reason => ${ex.getMessage}").enableCors
case Success(result) =>
result match {
case Left(errorMessage) =>
BadRequest(Json.obj("message" -> s"invalid PowerPlantConfig $errorMessage")).enableCors
case Right(updatedConfig) =>
Ok(Json.prettyPrint(Json.toJson(updatedConfig))).enableCors
}
}
}
)
}
}
But as it can be seen that there is this additional Try(....) block and I do not want this!
Here is my definition of PowerPlantConfig:
sealed trait PowerPlantConfig {
def id: Int
def name: String
def minPower: Double
def maxPower: Double
def powerPlantType: PowerPlantType
}
object PowerPlantConfig {
case class OnOffTypeConfig(
id: Int,
name: String,
minPower: Double,
maxPower: Double,
powerPlantType: PowerPlantType
) extends PowerPlantConfig
case class RampUpTypeConfig(
id: Int,
name: String,
minPower: Double,
maxPower: Double,
rampPowerRate: Double,
rampRateInSeconds: FiniteDuration,
powerPlantType: PowerPlantType
) extends PowerPlantConfig
case class UnknownConfig(
id: Int = -1,
name: String,
minPower: Double,
maxPower: Double,
powerPlantType: PowerPlantType
) extends PowerPlantConfig
// represents all the PowerPlant's from the database
case class PowerPlantsConfig(
snapshotDateTime: DateTime,
powerPlantConfigSeq: Seq[PowerPlantConfig]
)
}
Here is my JSON reads and writes:
implicit val powerPlantCfgFormat: Format[PowerPlantConfig] = new Format[PowerPlantConfig] {
def reads(json: JsValue): JsResult[PowerPlantConfig] = {
val powerPlantTyp = PowerPlantType.fromString((json \ "powerPlantType").as[String])
powerPlantTyp match {
case PowerPlantType.OnOffType =>
JsSuccess(OnOffTypeConfig(
id = (json \ "powerPlantId").as[Int],
name = (json \ "powerPlantName").as[String],
minPower = (json \ "minPower").as[Double],
maxPower = (json \ "maxPower").as[Double],
powerPlantType = powerPlantTyp
))
case PowerPlantType.RampUpType =>
JsSuccess(RampUpTypeConfig(
id = (json \ "powerPlantId").as[Int],
name = (json \ "powerPlantName").as[String],
minPower = (json \ "minPower").as[Double],
rampPowerRate = (json \ "rampPowerRate").as[Double],
rampRateInSeconds = Duration.apply((json \ "rampRateInSeconds").as[String]).asInstanceOf[FiniteDuration],
maxPower = (json \ "maxPower").as[Double],
powerPlantType = powerPlantTyp
))
case _ =>
JsSuccess(UnknownConfig(
id = (json \ "powerPlantId").as[Int],
name = (json \ "powerPlantName").as[String],
minPower = (json \ "minPower").as[Double],
maxPower = (json \ "maxPower").as[Double],
powerPlantType = powerPlantTyp
))
}
}
def writes(o: PowerPlantConfig): JsValue = {
if (o.powerPlantType == RampUpType) {
Json.obj(
"powerPlantId" -> o.id,
"powerPlantName" -> o.name,
"minPower" -> o.minPower,
"maxPower" -> o.maxPower,
"rampPowerRate" -> o.asInstanceOf[RampUpTypeConfig].rampPowerRate,
"rampRateInSeconds" -> o.asInstanceOf[RampUpTypeConfig].rampRateInSeconds.toString(),
"powerPlantType" -> PowerPlantType.toString(o.powerPlantType)
)
}
else {
Json.obj(
"powerPlantId" -> o.id,
"powerPlantName" -> o.name,
"minPower" -> o.minPower,
"maxPower" -> o.maxPower,
"powerPlantType" -> PowerPlantType.toString(o.powerPlantType)
)
}
}
}
According your stacktrace (line I marked)
JsResultException(errors:List((,List(ValidationError(List(error.expected.jsnumber),WrappedArray())))))
play.api.libs.json.JsResultException: JsResultException(errors:List((,List(ValidationError(List(error.expected.jsnumber),WrappedArray())))))
at play.api.libs.json.JsReadable$$anonfun$2.apply(JsReadable.scala:23)
at play.api.libs.json.JsReadable$$anonfun$2.apply(JsReadable.scala:23)
at play.api.libs.json.JsResult$class.fold(JsResult.scala:73)
at play.api.libs.json.JsError.fold(JsResult.scala:13)
--> at play.api.libs.json.JsReadable$class.as(JsReadable.scala:21)
at play.api.libs.json.JsDefined.as(JsLookup.scala:132)
at com.inland24.plantsim.models.package$$anon$1.reads(package.scala:61)
at play.api.libs.json.JsValue$class.validate(JsValue.scala:18)
at play.api.libs.json.JsObject.validate(JsValue.scala:76)
at com.inland24.plantsim.controllers.PowerPlantController$$anonfun$updatePowerPlant1$1.apply(PowerPlantController.scala:64)
at com.inland24.plantsim.controllers.PowerPlantController$$anonfun$updatePowerPlant1$1.apply(PowerPlantController.scala:63)
you used as[Int] in your Read format for id field for PowerPlantConfig.
When you call as[Int], you are trying to force the given json path to type Int. It throws an exception if it cannot (as in your test). You can read on difference betweem as, asOpt and validate here for example
Update
If you look into implementation of as, asOpt and validate you will see that all these three do at first the same thing, but than differs in a way:
validate - I do need result either result or info on failure wrapped (just call reads of implicit arg on json)
asOpt - I need either result or none, if reads under used for resolution return parse error it is ignored as not set at all
as - I need either result, or exception. In other words "I'm sure, that this is always such type, if not, than it is general error"
Both as and asOpt are "extended validate" with interpreting result.
Example
Example how to move from as to validate in hierarchy (two Formats - one as yours with as which will throw exception, and another with validate which will not throw exception):
sealed trait PowerPlantConfig {
def id: Int
}
case class RampUpTypeConfig(id: Int) extends PowerPlantConfig
implicit val powerPlantCfgFormat: Format[PowerPlantConfig] = new Format[PowerPlantConfig] {
def reads(json: JsValue): JsResult[PowerPlantConfig] = {
JsSuccess(RampUpTypeConfig(
id = (json \ "powerPlantId").as[Int]
))
}
def writes(o: PowerPlantConfig): JsValue = {
Json.obj(
"powerPlantId" -> o.id)
}
}
val powerPlantCfgFormatFixed: Format[PowerPlantConfig] = new Format[PowerPlantConfig] {
def reads(json: JsValue): JsResult[PowerPlantConfig] = {
for {
id <- (json \ "powerPlantId").validate[Int]
} yield {
RampUpTypeConfig(
id = id
)
}
}
def writes(o: PowerPlantConfig): JsValue = {
Json.obj(
"id" -> o.id)
}
}
Json.parse("""{"powerPlantId":"123"}""").validate[PowerPlantConfig](powerPlantCfgFormatFixed)
And output will be not an exception but JsFailure as expected
res1: play.api.libs.json.JsResult[PowerPlantConfig] = JsError(List((,List(ValidationError(error.expected.jsnumber,WrappedArray())))))
I have a function
`
def getFollowers(userId: Long): Try[Set[Long]] = {
Try({
val followerIds = mutable.Set[Long]()
var cursor = -1L
do {
val client = AkkaStreamTwitterClient()
val res = client.friendsFollowers().getFollowersIDs(userId, cursor, 5000)
res.getIDs.toList.foreach(x => followerIds.add(x))
if (res.hasNext) {
cursor = res.getNextCursor
}
else {
cursor = -1 // Exit the loop
}
} while (cursor > 0)
val immutableFollower = scala.collection.immutable.Set(followerIds.toSet.toArray:_*)
immutableFollower
})
}
`
The function is called from
`
Source(() =>
AkkaStreamTwitterHelper.getFollowers(userId).get.toIterable.iterator)
.grouped(100)
.map(x => AkkaStreamTwitterHelper.lookupUsers(x.toList))
.mapConcat(identity(_))
.runForeach(x => output.offer(x.getScreenName))
.onComplete({
case _ =>
Console.println(s"Fetched ${output.size()} profiles")
val endTime = System.nanoTime()
Console.println(s"Time taken: ${(endTime - startTime)/1000000000.00}s")
system.terminate()
Runtime.getRuntime.exit(0)
}) (ec)
}
`
The line "source" gives an error
Scala Error convert type mismatch; found : () ⇒ Iterator[Long] required: scala.collection.immutable.Iterable[?]
I code from
http://blog.abhinav.ca/blog/2015/02/19/scaling-with-akka-streams/
You might need Source.fromIterator depending on your version.
Source.fromIterator(() =>
AkkaStreamTwitterHelper.getFollowers(userId).get.toIterator)
By using getFollowers(userId).get.toIterable.iterator you've got a funtion that returns an interator: () => Iterator, not the iterator itself. You need to apply this function (simply change it to getFollowers(userId).get.toIterable.iterator())
I'm trying to write some basic queries with Slick for SQLite database
Here is my code:
class MigrationLog(name: String) {
val migrationEvents = TableQuery[MigrationEventTable]
lazy val db: Future[SQLiteDriver.backend.DatabaseDef] = {
val db = Database.forURL(s"jdbc:sqlite:$name.db", driver = "org.sqlite.JDBC")
val setup = DBIO.seq(migrationEvents.schema.create)
val createFuture = for {
tables <- db.run(MTable.getTables)
createResult <- if (tables.length == 0) db.run(setup) else Future.successful()
} yield createResult
createFuture.map(_ => db)
}
val addEvent: (String, String) => Future[String] = (aggregateId, eventType) => {
val id = java.util.UUID.randomUUID().toString
val command = DBIO.seq(migrationEvents += (id, aggregateId, None, eventType, "CREATED", System.currentTimeMillis, None))
db.flatMap(_.run(command).map(_ => id))
}
val eventSubmitted: (String, String) => Future[Unit] = (id, batchId) => {
val q = for { e <- migrationEvents if e.id === id } yield (e.batchId, e.status, e.updatedAt)
val updateAction = q.update(Some(batchId), "SUBMITTED", Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val eventMigrationCompleted: (String, String, String) => Future[Unit] = (batchId, id, status) => {
val q = for { e <- migrationEvents if e.batchId === batchId && e.id === id} yield (e.status, e.updatedAt)
val updateAction = q.update(status, Some(System.currentTimeMillis))
db.map(_.run(updateAction))
}
val allEvents = () => {
db.flatMap(_.run(migrationEvents.result))
}
}
Here is how I'm using it:
val migrationLog = MigrationLog("test")
val events = for {
id <- migrationLog.addEvent("aggregateUserId", "userAccessControl")
_ <- migrationLog.eventSubmitted(id, "batchID_generated_from_idam")
_ <- migrationLog.eventMigrationCompleted("batchID_generated_from_idam", id, "Successful")
events <- migrationLog.allEvents()
} yield events
events.map(_.foreach(event => event match {
case (id, aggregateId, batchId, eventType, status, submitted, updatedAt) => println(s"$id $aggregateId $batchId $eventType $status $submitted $updatedAt")
}))
The idea is to add event first, then update it with batchId (which also updates status) and then update the status when the job is done. events should contain events with status Successful.
What happens is that after running this code it prints events with status SUBMITTED. If I wait a while and do the same allEvents query or just go and check the db from command line using sqlite3 then it's updated correctly.
I'm properly waiting for futures to be resolved before starting the next operation, auto-commit should be enabled by default.
Am I missing something?
Turns out the problem was with db.map(_.run(updateAction)) which returns Future[Future[Int]] which means that the command was not finished by the time I tried to run another query.
Replacing it with db.flatMap(_.run(updateAction)) solved the issue.
I am trying to test some functionality in the app. I have other tests (with scalatest for reactivemongo) and are working, but with this I am getting this exception
[info] - should Persist and find a token * FAILED *
[info] The future returned an exception of type: reactivemongo.api.commands.bson.DefaultBSONCommandError, with message: CommandError[code=26, errmsg=ns not found, doc: {
[info] ok: BSONDouble(0.0),
[info] errmsg: "ns not found",
[info] code: BSONInteger(26)
[info] }]. (DaosApplicationSpecOneAppPerTest.scala:74)
This is the code for the tests (both are throwing the same error)
class UserTokenDaoMongoSpec extends DaosApplicationSpecOneAppPerTest {
"UserTokenDao" should {
"Persist and find a token" in withUserTokenDao { userTokenDao =>
val future = for {
_ <- userTokenDao.save(token)
maybeToken <- userTokenDao.find(token.id)
} yield {
maybeToken.map(_ == token)
}
whenReady (future) { result =>
result.get must be (true)
}
}
"Remove a token" in withUserTokenDao { userTokenDao =>
val future = for {
_ <- userTokenDao.save(token)
_ <- userTokenDao.remove(token.id)
maybeToken <- userTokenDao.find(token.id)
} yield {
maybeToken
}
whenReady (future) { result =>
result must be (None)
}
}
}
}
and for brevity, this is the method that inherits
def withUserTokenDao[T](t: UserTokenDao => T):T = running(app) {
val userTokenDao = new UserTokenDaoMongo
whenReady (userTokenDao.tokens.drop()) { result =>
t(userTokenDao)
}
}
The UserTokenDao implementation
class UserTokenDaoMongo extends UserTokenDao {
lazy val reactiveMongoApi = current.injector.instanceOf[ReactiveMongoApi]
val tokens = reactiveMongoApi.db.collection[JSONCollection]("tokens")
def find(id:UUID):Future[Option[UserToken]] =
tokens.find(Json.obj("id" -> id)).one[UserToken]
def save(token:UserToken):Future[UserToken] = for {
_ <- tokens.insert(token)
} yield token
def remove(id:UUID):Future[Unit] = for {
_ <- tokens.remove(Json.obj("id" -> id))
} yield ()
}
and this is the model of UserToken
class UserTokenDaoMongo extends UserTokenDao {
lazy val reactiveMongoApi = current.injector.instanceOf[ReactiveMongoApi]
val tokens = reactiveMongoApi.db.collection[JSONCollection]("tokens")
def find(id:UUID):Future[Option[UserToken]] =
tokens.find(Json.obj("id" -> id)).one[UserToken]
def save(token:UserToken):Future[UserToken] = for {
_ <- tokens.insert(token)
} yield token
def remove(id:UUID):Future[Unit] = for {
_ <- tokens.remove(Json.obj("id" -> id))
} yield ()
}
I am not sure what could be causing the error
Thank you
It turned out that the problem was that the collection tokens did not exist. I got into the mongo console, then I created the collection and the tests started working.