Testing reactivemongo with scalatest raise an exception - scala

I am trying to test some functionality in the app. I have other tests (with scalatest for reactivemongo) and are working, but with this I am getting this exception
[info] - should Persist and find a token * FAILED *
[info] The future returned an exception of type: reactivemongo.api.commands.bson.DefaultBSONCommandError, with message: CommandError[code=26, errmsg=ns not found, doc: {
[info] ok: BSONDouble(0.0),
[info] errmsg: "ns not found",
[info] code: BSONInteger(26)
[info] }]. (DaosApplicationSpecOneAppPerTest.scala:74)
This is the code for the tests (both are throwing the same error)
class UserTokenDaoMongoSpec extends DaosApplicationSpecOneAppPerTest {
"UserTokenDao" should {
"Persist and find a token" in withUserTokenDao { userTokenDao =>
val future = for {
_ <- userTokenDao.save(token)
maybeToken <- userTokenDao.find(token.id)
} yield {
maybeToken.map(_ == token)
}
whenReady (future) { result =>
result.get must be (true)
}
}
"Remove a token" in withUserTokenDao { userTokenDao =>
val future = for {
_ <- userTokenDao.save(token)
_ <- userTokenDao.remove(token.id)
maybeToken <- userTokenDao.find(token.id)
} yield {
maybeToken
}
whenReady (future) { result =>
result must be (None)
}
}
}
}
and for brevity, this is the method that inherits
def withUserTokenDao[T](t: UserTokenDao => T):T = running(app) {
val userTokenDao = new UserTokenDaoMongo
whenReady (userTokenDao.tokens.drop()) { result =>
t(userTokenDao)
}
}
The UserTokenDao implementation
class UserTokenDaoMongo extends UserTokenDao {
lazy val reactiveMongoApi = current.injector.instanceOf[ReactiveMongoApi]
val tokens = reactiveMongoApi.db.collection[JSONCollection]("tokens")
def find(id:UUID):Future[Option[UserToken]] =
tokens.find(Json.obj("id" -> id)).one[UserToken]
def save(token:UserToken):Future[UserToken] = for {
_ <- tokens.insert(token)
} yield token
def remove(id:UUID):Future[Unit] = for {
_ <- tokens.remove(Json.obj("id" -> id))
} yield ()
}
and this is the model of UserToken
class UserTokenDaoMongo extends UserTokenDao {
lazy val reactiveMongoApi = current.injector.instanceOf[ReactiveMongoApi]
val tokens = reactiveMongoApi.db.collection[JSONCollection]("tokens")
def find(id:UUID):Future[Option[UserToken]] =
tokens.find(Json.obj("id" -> id)).one[UserToken]
def save(token:UserToken):Future[UserToken] = for {
_ <- tokens.insert(token)
} yield token
def remove(id:UUID):Future[Unit] = for {
_ <- tokens.remove(Json.obj("id" -> id))
} yield ()
}
I am not sure what could be causing the error
Thank you

It turned out that the problem was that the collection tokens did not exist. I got into the mongo console, then I created the collection and the tests started working.

Related

Transactions With JSONCollection instead of BSONCollection

I have a problem to make Transaction via JSONCollection, I getting the following error:
JsResultException(errors:List((,List(JsonValidationError(List(CommandError[code=14, errmsg=BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long', doc: {"operationTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"ok":0,"errmsg":"BSON field 'OperationSessionInfo.txnNumber' is the wrong type 'int', expected type 'long'","code":14,"codeName":"TypeMismatch","$clusterTime":{"clusterTime":{"$time":1596894245,"$i":5,"$timestamp":{"t":1596894245,"i":5}},"signature":{"hash":{"$binary":"0000000000000000000000000000000000000000","$type":"00"},"keyId":0}}}]),WrappedArray())))))
I tried to change my project to BSONCollection but got some troubles, maybe there solution to overcome the above error with JSONCollection.
Also the exceptions occurs on testing update method, but checking the insertOneViaTransaction and setRuleAsInactiveViaTransaction is finished with success
This is my code for Transaction:
Update:
def update(oldRule: ExistRuleDto): Future[UpdateResult] = {
val transaction = (collection: JSONCollection) => for {
newRule <- dao.insertOneViaTransaction(collection,oldRule.toUpdatedRule) // insert new with ref to old
oldRule <- dao.setRuleAsInactiveViaTransaction(collection,oldRule.id)
} yield UpdateResult(oldRule, newRule)
makeTransaction[UpdateResult](transaction)
}
makeTransaction:
def makeTransaction[Out](block: JSONCollection => Future[Out]): Future[Out] = for {
dbWithSession <- dao.collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
coll = dbWithTx.collection[JSONCollection](dao.collection.name)
// Operations:
res <- block(coll)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield res
insertOneViaTransaction:
def insertOneViaTransaction(collection: JSONCollection, rule: Rule): Future[Rule] = {
collection.insert.one(rule).map {
case DefaultWriteResult(true, 1, _, _, _, _) => rule
case err => throw GeneralDBError(s"$rule was not inserted, something went wrong: $err")
}.recover {
case WriteResult.Code(11000) => throw DuplicationError(s"$rule exist on DB")
case err => throw GeneralDBError(err.getMessage)
}
}
setRuleAsInactiveViaTransaction:
def setRuleAsInactiveViaTransaction(collection: JSONCollection, ruleId: BSONObjectID): Future[Rule] = {
collection.findAndUpdate(
Json.obj(s"${Rule.ID}" -> ruleId),
Json.obj(
"$set" -> Json.obj(s"${Rule.Metadata}.${Metadata.Active}" -> false),
"$unset" -> Json.obj(s"${Rule.Metadata}.${Metadata.LastVersionExists}" -> "")),
fetchNewObject = true, upsert = false, sort = None, fields = None, bypassDocumentValidation = false, writeConcern = WriteConcern.Acknowledged, maxTime = None, collation = None, arrayFilters = Nil
).map(el => el.result[Rule].getOrElse {
val msg = s"Operation fail for updating ruleId = $ruleId"
logger.error(msg)
throw GeneralUpdateError(msg)
})
}
I'm using the following dependencies:
Play:
"com.typesafe.play" % "sbt-plugin" % "2.7.2
Reactivemongo:
"org.reactivemongo" %% "play2-reactivemongo" % "0.18.8-play27"
Solve it. (Not with compact)
Serializers:
implicit object JsValueHandler extends BSONHandler[BSONValue, JsValue] {
implicit override def read(bson: BSONValue): JsValue = BSONFormats.toJSON(bson)
implicit override def write(j: JsValue): BSONValue = BSONFormats.toBSON(j).get
}
asTransaction:
def asTransaction[Out](block: BSONCollection => Future[Out]): Future[Out] = {
for {
dbWithSession <- collection.db.startSession()
dbWithTx <- dbWithSession.startTransaction(None)
collectionWithTx = dbWithTx.collection[BSONCollection](collection.name)
out <- block(collectionWithTx)
_ <- dbWithTx.commitTransaction()
_ <- dbWithSession.endSession()
} yield out
}.recover {
case ex: Exception =>
logger.warn(s"asTransaction failed with ex: ${ex.getMessage}, rollback to previous state...")
throw GeneralDBErrorOnTx(ex.getMessage)
}
transaction example:
def `change visibility of ExistsRules and insert UpdateEvents`(oldRules: List[Rule], active: Boolean): Future[Unit] = {
ruleDao.asTransaction { collectionTx =>
for {
// (1) - $active old Rules
_ <- ruleDao.updateManyWithBsonCollection(
collectionTx,
filter = BSONDocument(s"${Rule.ID}" -> BSONDocument("$in" -> oldRules.map(_._id))),
update = BSONDocument("$set" -> BSONDocument(s"${Rule.Metadata}.${Metadata.Active}" -> active)))
// (2) - Sync Cache with Update Events
_ <- eventsService.addEvents(oldRules.map(rule => RuleEvent(rule.metadata.cacheKey, Update)))
} yield ()
}
}
Enjoy!

Download an archive file on the fly using play framework 2.3

I'am trying to create and download an archive file without relying on memory(to avoid out of memory exception for large file) I'am using for that play framework 2.3 with Scala, after some research I found an example: https://gist.github.com/kirked/412b5156f94419e71ce4a84ec1d54761, I made some modification on it. My problem is when I download the file and try to open it I get this exception: An error occurred while loading the archive.
here all the code:
def zip() = Action {
implicit request: Request[AnyContent] =>
val buffer = new ZipBuffer(10000)
val writeCentralDirectory = Enumerator.generateM(Future{
if (buffer.isClosed) {
None
}
else {
buffer.flush
buffer.close
Some(buffer.bytes)
}
})
val test = Enumerator.apply(ResolvedSource2("test.txt", "helllo"))
Ok.chunked(test &> zipeach2(buffer) andThen writeCentralDirectory >>> Enumerator.eof) as withCharset("application/zip") withHeaders(
CONTENT_DISPOSITION -> s"attachment; filename=aa.zip")
}
case class ResolvedSource2(filepath: String, stream: String)
def zipeach2(buffer: ZipBuffer)(implicit ec: ExecutionContext): Enumeratee[ResolvedSource2, Array[Byte]] = {
Enumeratee.mapConcat[ResolvedSource2] { source =>
buffer.zipStream.putNextEntry(new ZipEntry(source.filepath))
var done = false
def entryDone: Unit = {
done = true
buffer.zipStream.closeEntry
}
def restOfStream: Stream[Array[Byte]] = {
if (done) Stream.empty
else {
while (!done && !buffer.full) {
try {
val byte = source.stream
buffer.zipStream.write(byte.getBytes)
entryDone
}
catch {
case e: IOException =>
println(s"reading/zipping stream [${source.filepath}]", e)
}
}
buffer.bytes #:: restOfStream
}
}
restOfStream
}
}
}
class ZipBuffer(capacity: Int) {
private val buf = new ByteArrayOutputStream(capacity)
private var closed = false
val zipStream = new ZipOutputStream(buf)
def close(): Unit = {
if (!closed) {
closed = true
reset
zipStream.finish()
zipStream.close // writes central directory
}
}
def flush() = {
zipStream.flush()
}
def isClosed = closed
def reset: Unit = buf.reset
def full: Boolean = buf.size >= capacity
def bytes: Array[Byte] = {
val result = buf.toByteArray
reset
result
}
}

Is there a way to get the item being mapped when an exception is thrown akka streams?

I want to be able to log certain attributes of the item being mapped if there is an exception thrown, so I was wondering is there a way to get the item being mapped when an exception is thrown akka streams?
If I have:
val decider: Supervision.Decider = { e =>
//val item = getItemThatCausedException
logger.error("Exception in stream with itemId:"+item.id, e)
Supervision.Resume
}
implicit val actorSystem = ActorSystem()
val materializerSettings = ActorMaterializerSettings(actorSystem).withSupervisionStrategy(decider)
implicit val materializer = ActorMaterializer(materializerSettings)(actorSystem)
Source(List(item1,item2,item3)).map { item =>
if (item.property < 0) {
throw new RuntimeException("Error")
} else {
i
}
}
Is there a way of getting the failed item in Supervision.Decider or after the map is done?
Not with a Supervision.Decide but you could achieve it in a different way.
Check out this program:
object Streams extends App{
implicit val system = ActorSystem("test")
implicit val mat = ActorMaterializer()
val source = Source(List("1", "2", "3")).map { item =>
Try {
if (item == "2") {
throw new RuntimeException("Error")
} else {
item
}
}
}
source
.alsoTo(
Flow[Try[String]]
.filter(_.isFailure)
.to(Sink.foreach(t => println("failure: " + t))))
.to(
Flow[Try[String]]
.filter(_.isSuccess)
.to(Sink.foreach(t => println("success " + t)))).run()
}
Outputs:
success Success(1)
failure: Failure(java.lang.RuntimeException: Error)
success Success(3)
This is somewhat convoluted but you can do this by wrapping your mapping function in a stream and using flatMapConcat like so:
Source(List(item1, item2, item3)).flatMapConcat { item =>
Source(List(item))
.map(mapF)
.withAttributes(ActorAttributes.supervisionStrategy { e: Throwable =>
logger.error("Exception in stream with itemId:" + item.id, e)
Supervision.Resume
})
}
def mapF(item: Item) =
if (item.property < 0) {
throw new RuntimeException("Error")
} else {
i
}
This is possible because each stream stage can have its own supervision strategy.
You can use Supervision.Decider to log those attributes.
object Test extends App {
implicit val system = ActorSystem("test")
implicit val mat = ActorMaterializer()
val testSupervisionDecider: Supervision.Decider = {
case ex: RuntimeException =>
println(s"some run time exception ${ex.getMessage}")
Supervision.Resume
case ex: Exception =>
//if you want to stop the stream
Supervision.Stop
}
val source = Source(List("1", "2", "3")).map { item =>
if (item == "2") {
throw new RuntimeException(s"$item")
} else {
item
}
}
source
.to(Sink.foreach(println(_)))
.withAttributes(ActorAttributes.supervisionStrategy(testSupervisionDecider))
.run
}
The Output is:
1
some run time exception 2
3

mongodb with scala maven project

I am stuck on an issue. I have created a scala maven project where I am trying to communicate with mongo. My project is getting build successfuly and I do not get any error when I execute my project, but still no data is getting inserted in my mongodb.
If I insert data manually through mongo-cli then my code can read it but it is not even inserting a single record.
Could you please help me in finding where I am making mistakes.
My mongo version is 3.2.10
App.scala
package com.assignment.scala
import org.mongodb.scala._
import org.mongodb.scala.model.Aggregates._
import org.mongodb.scala.model.Filters._
import org.mongodb.scala.model.Projections._
import org.mongodb.scala.model.Sorts._
import org.mongodb.scala.model.Updates._
import org.mongodb.scala.model._
/**
* #author ${user.name}
*/
object App {
def main(args: Array[String]) {
println("Calling insertDoc==")
insertDoc()
}
def insertDoc() = {
val mongoClient = MongoClient()
val database = mongoClient.getDatabase("assignment")
val collection = database.getCollection("links")
println("collection find : " + collection.find())
collection.find().subscribe(
(user: Document) => println("document------------"+user.toJson()),
(error: Throwable) => println(s"Query failed: ${error.getMessage}"),
() => println("Done"))
collection.drop()
val bufferedSource = io.Source.fromFile("/home/impadmin/ServiceSource/ml-latest-small/links.csv")
var firstLine = List[String]();
var docList = List[Document]();
for ((line,count) <- bufferedSource.getLines.zipWithIndex) {
val cols = line.split(",").map(_.trim)
if (firstLine.size <= 0 && cols.length > 0) {
firstLine = cols.toList;
} else {
var doc: Document = Document("_id" -> count)
for ((a, b) <- firstLine zip cols) {
doc ++= Document(a -> b)
}
docList = docList :+ doc
collection.insertOne(doc)
}
}
val doc3 = Document("name" -> "MongoDB", "type" -> "database",
"count" -> 1, "info" -> Document("x" -> 203, "y" -> 102))
collection.insertOne(doc)
collection.find().subscribe((doc1: Document) => println(doc1.toJson()),
(e: Throwable) => println(s"There was an error: $e"))
}
}
It worked using subscribe:
val insertObservable1: Observable[Completed] = collection.insertMany(docList);
val latch1 = new CountDownLatch(1)
insertObservable1.subscribe(new Observer[Completed] {
override def onError(e: Throwable): Unit = {
println("Error1")
e.printStackTrace()
}
override def onSubscribe(subscription: Subscription): Unit = {
subscription.request(java.lang.Long.MAX_VALUE)
println("inserting1-------------")
}
override def onComplete(): Unit = {
println("Completed1")
latch1.countDown()
}
override def onNext(result: Completed): Unit = {
println("Next1-----------")
}
})
latch1.await()
}
})

Testing Actor preStart()

I moved from Casbah to Reactive Mongo and from that moment I couldn't make work the test of my actor.
I have a dao for the persistence layer and tests for that tier. All the tests passed. So, the only thing that comes to my mind its a problem of synchronization.
" UserActor " should {
val socketActorProbe = new TestProbe(system)
val peyiProbe = new TestProbe(system)
val identifyId = 1
val emailCsr = "csr#gmail.com"
val emailPeyi = "peyi#gmail.com"
val point = new Point[LatLng](new LatLng(-31.4314041, -64.1670626))
" test preStart() " in new WithApplication {
db.createDB(id1, id2, id3)
val userActorRefCsr = TestActorRef[UserActor](Props(classOf[UserActor], emailCsr, socketActorProbe.ref))
val csr = userActorRefCsr.underlyingActor
val userActorRef = TestActorRef[UserActor](Props(classOf[UserActor], emailPeyi, socketActorProbe.ref))
val peyi = userActorRef.underlyingActor
peyi.receive(ActorIdentity(identifyId, Option(userActorRefCsr)))
db.clearDB()
}
Actor class.
class UserActor(email: String, upstream: ActorRef) extends Actor {
import UserActor._
val identifyId = 1
val usersFromDB = ReactiveMongoFactory.db.collection[BSONCollection]("users")
val userDao = new UserDao(usersFromDB)
val meFuture = userDao.findMeByEmail(email)
var friends: Map[String, ActorRef] = Map()
override def preStart() = {
meFuture onComplete { result =>
val emailsFriends: List[String] = userDao.getMyFriendsEmail(result.get.get)
println(emailsFriends)
for (email <- emailsFriends) {
println("sending msg to " + email)
context.actorSelection("/user/" + email) ! Identify(identifyId)
}
}
}
private def giveMyFriend(email: String): Option[ActorRef] = {
for(friend <- friends){
if (friend._1 == email) new Some(friend._2)
}
None
}
def active(another: ActorRef): Actor.Receive = {
case Terminated(`another`) => context.stop(self)
}
def receive = {
case ActorIdentity(`identifyId`, Some(actorRef)) =>
meFuture onComplete { result =>
println(" ... subscribing ... " + result.get.get.basicProfile.email)
actorRef ! Subscribe(result.get.get.basicProfile.email.get)
context.watch(actorRef)
context.become(active(actorRef))
}
case Subscribe(email) =>
friends += (email -> sender)
context watch sender
case Terminated(user) => {
for(friend <- friends){
if (friend._2 == user ) friends -= friend._1 //removing by key
}
}
case UserMoved(email, point) =>
upstream ! UserPosition(email, System.currentTimeMillis(), point.coordinates)
}
}
Im receiving the following output.
The exception is thrown in the following lines of code.
def findMeByEmail(email: String): Future[Option[User]] = {
val query = BSONDocument("email" -> email)
println( " .... finding user ..... email: " + email )
val cursor = users.find(query).cursor[BSONDocument]
val userFuture = cursor.headOption.map(
doc => Some(userReader.read(doc.get))
)
userFuture
}
If I run the test for that method, it's all ok.
describe("get my friends emails") {
it("returns a list of emails") {
val futureUser = userDao.findMeByEmail("csr#gmail.com")
ScalaFutures.whenReady(futureUser) { result =>
val friends = userDao.getMyFriendsEmail(result.get)
assert(friends.length == 2)
}
}
}
Basically, Im trying to look my friends (Other actor) and then register them in a map to have a reference. I couldn't find any good example which shows tests using Reactive Mongo with Actors.
I hope somebody can help me to understand whats going on here. Thanks in advance.