Slick update operation returns before object is flushed in the database - scala

I am experiencing a scenario where when I fetch an object immediately after updating it, sometimes the result I get from the DB does not contain the most recent changes.
This has led me to think that the update thread returns before the object is actually committed in the DB. Is this expected behavior?
I would think that the update method would only return after the changes have been successfully flushed to the DB however it looks like this not guaranteed.
Below is pseudo code demonstrating what I am talking about.
def processObject = {
for {
objectId: Option[Long] <- saveObjectInDb
_ <- {
//perform other synchronous business logic and then update created object details
dao.findById(objectId.get).map { objectOption: Option[MyObject] =>
dao.update(objectOption.get.copy(processingStep = "third-step"))
}
}
mostRecentMyObject <- dao.findById(objectId.get)
} yield mostRecentMyObject
}
Below is how my update logic looks like
def update(myObject: MyObject): Future[Int] = {
db.run(table.filter(_.id === myObject.id).update(myObject))
}

The problem is that you are not considering the inner Future returned by the update method.
Given the signature of findById:
def findById(id: Long): Future[Option[MyObject]]
the snippet:
dao.findById(objectId.get).map { objectOption: Option[MyObject] =>
dao.update(objectOption.get.copy(processingStep = "third-step"))
}
will gave an object of type Future[Future[Int]].
You should instead flatMap instead of map over the findById future, like so:
dao.findById(objectId.get).flatMap { objectOption: Option[MyObject] =>
dao.update(objectOption.get.copy(processingStep = "third-step"))
}
this will simplify to a single future (Future[Int]), and so you can be sure retrieve the object only once inserted.
Moreover you can rewrite this as:
def processObject = {
for {
objectId: Option[Long] <- saveObjectInDb
objectOption <- dao.findById(objectId.get)
_ <- dao.update(objectOption.get.copy(processingStep = "third-step"))
mostRecentMyObject <- dao.findById(objectId.get)
} yield mostRecentMyObject
}
because, into for-comprehension, the <- is a syntactic sugar for the flatMap

Related

MVar tryPut returns true and isEmpty also returns true

I wrote simple callback(handler) function which i pass to async api and i want to wait for result:
object Handlers {
val logger: Logger = Logger("Handlers")
implicit val cs: ContextShift[IO] =
IO.contextShift(ExecutionContext.Implicits.global)
class DefaultHandler[A] {
val response: IO[MVar[IO, A]] = MVar.empty[IO, A]
def onResult(obj: Any): Unit = {
obj match {
case obj: A =>
println(response.flatMap(_.tryPut(obj)).unsafeRunSync())
println(response.flatMap(_.isEmpty).unsafeRunSync())
case _ => logger.error("Wrong expected type")
}
}
def getResponse: A = {
response.flatMap(_.take).unsafeRunSync()
}
}
But for some reason both tryPut and isEmpty(when i'd manually call onResult method) returns true, therefore when i calling getResponse it sleeps forever.
This is the my test:
class HandlersTest extends FunSuite {
test("DefaultHandler.test") {
val handler = new DefaultHandler[Int]
handler.onResult(3)
val response = handler.getResponse
assert(response != 0)
}
}
Can somebody explain why tryPut returns true, but nothing puts. And what is the right way to use Mvar/channels in scala?
IO[X] means that you have the recipe to create some X. So on your example, yuo are putting in one MVar and then asking in another.
Here is how I would do it.
object Handlers {
trait DefaultHandler[A] {
def onResult(obj: Any): IO[Unit]
def getResponse: IO[A]
}
object DefaultHandler {
def apply[A : ClassTag]: IO[DefaultHandler[A]] =
MVar.empty[IO, A].map { response =>
new DefaultHandler[A] {
override def onResult(obj: Any): IO[Unit] = obj match {
case obj: A =>
for {
r1 <- response.tryPut(obj)
_ <- IO(println(r1))
r2 <- response.isEmpty
_ <- IO(println(r2))
} yield ()
case _ =>
IO(logger.error("Wrong expected type"))
}
override def getResponse: IO[A] =
response.take
}
}
}
}
The "unsafe" is sort of a hint, but every time you call unsafeRunSync, you should basically think of it as an entire new universe. Before you make the call, you can only describe instructions for what will happen, you can't actually change anything. During the call is when all the changes occur. Once the call completes, that universe is destroyed, and you can read the result but no longer change anything. What happens in one unsafeRunSync universe doesn't affect another.
You need to call it exactly once in your test code. That means your test code needs to look something like:
val test = for {
handler <- TestHandler.DefaultHandler[Int]
_ <- handler.onResult(3)
response <- handler.getResponse
} yield response
assert test.unsafeRunSync() == 3
Note this doesn't really buy you much over just using the MVar directly. I think you're trying to mix side effects inside IO and outside it, but that doesn't work. All the side effects need to be inside.

Error value flatMap is not a member of Product with Serializable in for with Future[Option]

The following block of code fails to build with error :
value flatMap is not a member of Product with Serializable
[error] if (matchingUser.isDefined) {
Here's the code:
for {
matchingUser <- userDao.findOneByEmail(email)
user <- {
if (matchingUser.isDefined) {
matchingUser.map(u => {
// update u with new values...
userDao.save(u)
u
})
} else {
val newUser = new User(email)
userDao.create(newUser)
newUser
}
}
} yield user
Method userDao.findOneByEmail(email) returns anFuture[Option[User]]object.
My Google searches are only aboutEitherwithRightandLeft` types.
Maybe I'm not doing this the proper way, please teach me how to properly do this.
The first branch of the if statement returns Option[User], the other one returns User. So, the result of the entire statement is inferred to have type Product with Serializable because it is the only common supertype of the two.
You could wrap the last statement inside the if into an Option (just do Option(newUser) instead of newUser) or, better yet, use fold instead of the whole if(matchingUser.isDefined) {...} thingy:
matchingUser.fold {
val u = new User(email)
userDao.create(u)
u
} { u =>
userDao.save(u)
u
}
This will make the result of that statement to be Option[User] as you probably intended ... but it still will not compile.
The problem is that you cannot mix different types of monads in the for-comprehension: since the first one was Future, all the others have to be as well. You can't have an Option there.
How to get around that? Well, one possibility is to make userDao.create and userDao.save return a future of the object they just saved. That is, probably a better thing to do in general, then what you have, because now you are returning the user before it was actually stored ... What if the create operation fails afterwards? Then you can just rewrite your for-comprehension like this:
for {
matchingUser <- userDao.findOneByEmail(email)
user <- matchingUser.fold(userDao.create(new User(email)))(userDao.save)
} yield user
Or just get rid of it entirely (for-comprehension is an overkill for simple cases like this):
userDao
.findOneByEmail(email)
.flatMap(_.fold(usrDao.create(new User(email)))(userDao.save))
Or, it may look a little nicer with pattern matching instead of fold in this case:
userDao
.findOneByEmail(email)
.flatMap {
case Some(u) => userDao.save(u)
case None => userDao.create(new User(email))
}
Here is my test example to reproduce your issue with solution.
Basically your user returns wrapper around Future (Option of Future), but it expected to be Future, as the first statement in for-comprehension.
That is why I've applied some addition unwrapping. See sample below.
Note: it does not looks so nice, I'd prefer to rewrite it with flatMap map.
object T {
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def future1: Future[Option[Int]] = ???
def future2(i: Int): Future[Double] = ???
for {
matchingUser <- future1
user <- {
if (matchingUser.isDefined) {
matchingUser.map { i =>
future2(i)
}
} else {
Some(future2(42))
}
} match {
case None => Future.successful(-42.0)
case Some(x) => x
}
} yield user
}
And the same implemented with flatMap:
val userFuture = future1.flatMap {
case Some(i) => future2(i)
case None => future2(42)
}

Why a Thread.sleep or closing the connection is required after waiting for a remove call to complete?

I'm again seeking you to share your wisdom with me, the scala padawan!
I'm playing with reactive mongo in scala and while I was writting a test using scalatest, I faced the following issue.
First the code:
"delete" when {
"passing an existent id" should {
"succeed" in {
val testRecord = TestRecord(someString)
Await.result(persistenceService.persist(testRecord), Duration.Inf)
Await.result(persistenceService.delete(testRecord.id), Duration.Inf)
Thread.sleep(1000) // Why do I need that to make the test succeeds?
val thrownException = intercept[RecordNotFoundException] {
Await.result(persistenceService.read(testRecord.id), Duration.Inf)
}
thrownException.getMessage should include(testRecord._id.toString)
}
}
}
And the read and delete methods with the code initializing connection to db (part of the constructor):
class MongoPersistenceService[R](url: String, port: String, databaseName: String, collectionName: String) {
val driver = MongoDriver()
val parsedUri: Try[MongoConnection.ParsedURI] = MongoConnection.parseURI("%s:%s".format(url, port))
val connection: Try[MongoConnection] = parsedUri.map(driver.connection)
val mongoConnection = Future.fromTry(connection)
def db: Future[DefaultDB] = mongoConnection.flatMap(_.database(databaseName))
def collection: Future[BSONCollection] = db.map(_.collection(collectionName))
def read(id: BSONObjectID): Future[R] = {
val query = BSONDocument("_id" -> id)
val readResult: Future[R] = for {
coll <- collection
record <- coll.find(query).requireOne[R]
} yield record
readResult.recover {
case NoSuchResultException => throw RecordNotFoundException(id)
}
}
def delete(id: BSONObjectID): Future[Unit] = {
val query = BSONDocument("_id" -> id)
// first read then call remove. Read will throw if not present
read(id).flatMap { (_) => collection.map(coll => coll.remove(query)) }
}
}
So to make my test pass, I had to had a Thread.sleep right after waiting for the delete to complete. Knowing this is evil usually punished by many whiplash, I want learn and find the proper fix here.
While trying other stuff, I found instead of waiting, entirely closing the connection to the db was also doing the trick...
What am I misunderstanding here? Should a connection to the db be opened and close for each call to it? And not do many actions like adding, removing, updating records with one connection?
Note that everything works fine when I remove the read call in my delete function. Also by closing the connection, I mean call close on the MongoDriver from my test and also stop and start again embed Mongo which I'm using in background.
Thanks for helping guys.
Warning: this is a blind guess, I've no experience with MongoDB on Scala.
You may have forgotten to flatMap
Take a look at this bit:
collection.map(coll => coll.remove(query))
Since collection is Future[BSONCollection] per your code and remove returns Future[WriteResult] per doc, so actual type of this expression is Future[Future[WriteResult]].
Now, you have annotated your function as returning Future[Unit]. Scala often makes Unit as a return value by throwing away possibly meaningful values, which it does in your case:
read(id).flatMap { (_) =>
collection.map(coll => {
coll.remove(query) // we didn't wait for removal
() // before returning unit
})
}
So your code should probably be
read(id).flatMap(_ => collection.flatMap(_.remove(query).map(_ => ())))
Or a for-comprehension:
for {
_ <- read(id)
coll <- collection
_ <- coll.remove(query)
} yield ()
You can make Scala warn you about discarded values by adding a compiler flag (assuming SBT):
scalacOptions += "-Ywarn-value-discard"

Logging the value of a future before returning it in Scala

def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.flatMap(x => {
println(x) // logging the value of x
xFuture
})
}
This is the way I'm currently doing it. To provide more context:
This function is being called inside an API when a request is made and I'd like the log message to be printed just before the value computed in the request is returned. Which is why, the following is not a good solution for me:
def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.map(x => {
println(x) // logging the value of x
})
xFuture
}
Logging is a side-effect, meaning that you don't want the operation to fail if the logging fails for any reason (e.g. a call to toString throwing NPE).
Future#andThen is perfect for this use case. From the docs:
Applies the side-effecting function to the result of this future, and returns a new future with the result of this future.
This method allows one to enforce that the callbacks are executed in a specified order.
Note that if one of the chained andThen callbacks throws an exception, that exception is not propagated to the subsequent andThen callbacks. Instead, the subsequent andThen callbacks are given the original value of this future.
Your example becomes:
def returnFuture[A](x: A): Future[A] = {
Future { x } // suppose an API call that returns a future
.andThen { case Success(v) => println(v) }
}
You can use onComplete callback:
def returnFuture[A](x: A): Future[A] = {
val f = Future { x }
f.onComplete(println)
f
}
A map will work too:
def returnFuture[A](x: A): Future[A] = {
Future { x }.map { v =>
println(v)
v
}
}
Keep in mind that the whole point of using Futures is that you are trying to avoid blocking and that you don't control exactly when the Future will be executed. So, if you want more detailed logs while keeping the asynchronous nature of a Future, do something like this:
def doSomething(param: String): String = {
// log something here
val result = param.toUpperCase
// log something else here
result
}
def asFuture(param: String) = Future {
doSomething(param)
}
In other words, if this is an option, add logs to the x operation instead.

Scala Synchronising Asynchronous calls with Future

I have a method that does a couple of database look up and performs some logic.
The MyType object that I return from the method is as follows:
case class MyResultType(typeId: Long, type1: Seq[Type1], type2: Seq[Type2])
The method definition is like this:
def myMethod(typeId: Long, timeInterval: Interval) = async {
// 1. check if I can find an entity in the database for typeId
val myTypeOption = await(db.run(findMyTypeById(typeId))) // I'm getting the headOption on this result
if (myTypeOption.isDefined) {
val anotherDbLookUp = await(doSomeDBStuff) // Line A
// the interval gets split and assume that I get a List of thse intervals
val intervalList = splitInterval(interval)
// for each of the interval in the intervalList, I do database look up
val results: Seq[(Future[Seq[Type1], Future[Seq[Type2])] = for {
interval <- intervalList
} yield {
(getType1Entries(interval), getType2Entries(interval))
}
// best way to work with the results so that I can return MyResultType
}
else {
None
}
}
Now the getType1Entries(interval) and getType2Entries(interval) each returns a Future of Seq(Type1) and Seq(Type2) entries!
My problem now is to get the Seq(Type1) and Seq(Type2) out of the Future and stuff that into the MyResultType case class?
You could refer to this question you asked
Scala transforming a Seq with Future
so you get the
val results2: Future[Seq([Iterable[Type1], [Iterable[Type2])] = ???
and then call await on it and you have no Futures at all, you can do what you want.
I hope I understood the question correctly.
Oh and by the way you should map myTypeOption instead of checking if it's defined and returning None if it's not
if (myTypeOption.isDefined) {
Some(x)
} else {
None
}
can be simply replaced with
myTypeOption.map { _ => // ignoring what actually was inside option
x // return whatever you want, without wrapping it in Some
}
If I understood your question correctly, then this should do the trick.
def myMethod(typeId: Long, timeInterval: Interval): Option[Seq[MyResultType]] = async {
// 1. check if I can find an entity in the database for typeId
val myTypeOption = await(db.run(findMyTypeById(typeId))) // I'm getting the headOption on this result
if (myTypeOption.isDefined) {
// the interval gets split and assume that I get a List of thse intervals
val intervalList = splitInterval(interval)
// for each of the interval in the intervalList, I do database look up
val results: Seq[(Future[Seq[Type1]], Future[Seq[Type2]])] = for {
interval <- intervalList
} yield {
(getType1Entries(interval), getType2Entries(interval))
}
// best way to work with the results so that I can return MyResultType
Some(
await(
Future.sequence(
results.map{
case (l, r) =>
l.zip(r).map{
case (vl, vr) => MyResultType(typeId, vl, vr)
}
})))
}
else {
None
}
}
There are two parts to your problem, 1) how to deal with two dependent futures, and 2) how to extract the resulting values.
When dealing with dependent futures, I normally compose them together:
val future1 = Future { 10 }
val future2 = Future { 20 }
// results in a new future with type (Int, Int)
val combined = for {
a <- future1
b <- future2
} yield (a, b)
// then you can use foreach/map, Await, or onComplete to do
// something when your results are ready..
combined.foreach { ((a, b)) =>
// do something with the result here
}
To extract the results I generally use Await if I need to make a synchronous response, use _.onComplete() if I need to deal with potential failure, and use _.foreach()/_.map() for most other circumstances.