Run transactionally and retrieve result in Future - scala

How to run a transactionally statement in Slick 3.1.x, and capture the result in a Future (without the use of Await)?
This works (but uses Await)
val action = db.run((for {
_ <- table1.filter(_.id1 === id).delete
_ <- table2.filter(_.id2=== id).delete
} yield ()).transactionally)
val result = Await.result(action, Duration.Inf)
However this does not print anything:
val future = db.run((for {
_ <- table1.filter(_.id1 === id).delete
_ <- table2.filter(_.id2=== id).delete
} yield ()).transactionally)
future.map { result => println("result:"+result) }
UPDATE
This is the real code taken from the program that doesn't work. It prints "1" but it never prints "2"
case class UserRole (sk: Int, name: String)
class UserRoleDB(tag: Tag) extends Table[UserRole](tag, "user_roles") {
def sk = column[Int]("sk", O.PrimaryKey)
def name = column[String]("name")
def * = (sk, name) <> ((UserRole.apply _).tupled, UserRole.unapply)
}
class Test extends Controller {
def index = Action.async { request =>
val db = Database.forConfig("db1")
val userRoles = TableQuery[UserRoleDB]
val ur = UserRole(1002,"aaa")
try {
val action = (for {
userRole2 <- userRoles += ur
} yield (userRole2)).transactionally
val future = db.run(action)
println(1)
// val result = Await.result(future, Duration.Inf)
future.map { result => {
println(2)
Ok("Finished OK")
}
}
}
finally db.close
}
}

Coming from the other question you asked: You are opening and then immediately closing the db connection in the finally clause. Therefore your async db operation runs against a closed db connection. That's also why it works by using Await since that blocks the execution of db.close until you received the result set.
So how to fix this?
Either you move db.close into future.map or better you let play-slick handle db connections for you.
Side note
You should close your other question and update this thread accordingly instead.

Your second example is fine. My guess is that you are either running it in standalone program or in test - and it simply finishes before future has a chance to be executed.
Try to add some sleep after your code in your second sample and you'll see it is getting printed. This is definitely not something (this sleep) you would do in your actual code but it will show you it works as it should.

Related

'A test is using time, but is not advancing the test clock' in ZIO Test

After migrating my Test to RC18, I get the following warning and the test is hanging:
Warning: A test is using time, but is not advancing the test clock, which may result in the test hanging. Use TestClock.adjust to manually advance the time.
I have the following test:
val testLayer: ZLayer[Live, Nothing, Loggings with Blocking with Clock] = (Console.live >>> loggings.consoleLogger) ++ Blocking.live ++ TestClock.default
testM("curl on invalid URL") {
for {
fork <- composer.curl("https://bad.xx", 1).flip.fork
_ <- TestClock.adjust(3.second * 2)
r <- fork.join
} yield
assert(r)(isSubtype[DockerComposerException](hasField("msg", _.msg.trim, equalTo("https://bad.xx could not be reached!"))))
}.provideCustomLayer(testLayer)
I suspect, that I created the Layers wrong, as this is the only change I made for the migration.
Also the standard test from the documentation failed (time was 0):
testM("One can move time very fast") {
for {
startTime <- currentTime(TimeUnit.SECONDS)
_ <- TestClock.adjust(Duration.fromScala(1 minute))
endTime <- currentTime(TimeUnit.SECONDS)
} yield assert(endTime - startTime)(isGreaterThanEqualTo(60L))
}
When I define the Layers like:
val testLayer: ZLayer[Live, Nothing, Loggings with Blocking with Clock] = (Console.live >>> loggings.consoleLogger) ++ Blocking.live ++ Clock.live
the time is not adjusted at all.
Here is the code that I want to test:
def curl(host: String, attempt: Int = 200): ZIO[Loggings with Clock, Throwable, Unit] = {
ZIO.effect(
Process(Seq("curl", "--output", "/dev/null", "--silent", "--head", "--fail", host)).!!
).flatMap(r =>
info(s"\n$host is ready to use") *> ZIO.succeed()
).catchAll(t =>
if (attempt == 0)
ZIO.fail(DockerComposerException(s"\n$host could not be reached!", Some(t)))
else
info(s"still waiting ;(") *>
ZIO.sleep(3.second) *>
curl(host, attempt - 1)
)
}
So I want to fast forward ZIO.sleep(3.seconds).
You need to call sleep(duration) after adjust(duration) to have the adjusted time reflected when you call currentTime. So the correct version of the example above is:
testM("One can move time very fast") {
for {
startTime <- currentTime(TimeUnit.SECONDS)
_ <- TestClock.adjust(1.minute)
_ <- ZIO.sleep(1.minute)
endTime <- currentTime(TimeUnit.SECONDS)
} yield assert(endTime - startTime)(isGreaterThanEqualTo(60L))
}
Note that this is reflected in the current version of the documentation here but is not yet reflected on the website because we currently only publish changes when we do a release.
However, I don't think that is your issue as it looks like your effect is just using sleep.
I'm not able to reproduce your code entirely but the following slightly simplified example works correctly:
import zio._
import zio.clock.Clock
import zio.console.Console
import zio.duration._
import zio.test._
import zio.test.environment.TestClock
object ExampleSpec extends DefaultRunnableSpec {
type Logging = Has[Logging.Service]
object Logging {
trait Service {
def logLine(line: String): UIO[Unit]
}
val live: ZLayer[Console, Nothing, Logging] =
ZLayer.fromService { console =>
new Logging.Service {
def logLine(line: String): UIO[Unit] =
console.putStrLn(line)
}
}
def logLine(line: String): ZIO[Logging, Nothing, Unit] =
ZIO.accessM(_.get.logLine(line))
}
def effect(n: Int): ZIO[Clock with Logging, String, Unit] =
if (n == 0) ZIO.fail("fail")
else Logging.logLine("retrying") *> ZIO.sleep(3.seconds) *> effect(n -1)
def spec = suite("ExampleSpec") {
testM("test") {
for {
fiber <- effect(1).flip.fork
_ <- TestClock.adjust(6.seconds)
_ <- fiber.join
} yield assertCompletes
}.provideCustomLayer(Logging.live)
}
}
Could there be something else going on in your test?

Why a Thread.sleep or closing the connection is required after waiting for a remove call to complete?

I'm again seeking you to share your wisdom with me, the scala padawan!
I'm playing with reactive mongo in scala and while I was writting a test using scalatest, I faced the following issue.
First the code:
"delete" when {
"passing an existent id" should {
"succeed" in {
val testRecord = TestRecord(someString)
Await.result(persistenceService.persist(testRecord), Duration.Inf)
Await.result(persistenceService.delete(testRecord.id), Duration.Inf)
Thread.sleep(1000) // Why do I need that to make the test succeeds?
val thrownException = intercept[RecordNotFoundException] {
Await.result(persistenceService.read(testRecord.id), Duration.Inf)
}
thrownException.getMessage should include(testRecord._id.toString)
}
}
}
And the read and delete methods with the code initializing connection to db (part of the constructor):
class MongoPersistenceService[R](url: String, port: String, databaseName: String, collectionName: String) {
val driver = MongoDriver()
val parsedUri: Try[MongoConnection.ParsedURI] = MongoConnection.parseURI("%s:%s".format(url, port))
val connection: Try[MongoConnection] = parsedUri.map(driver.connection)
val mongoConnection = Future.fromTry(connection)
def db: Future[DefaultDB] = mongoConnection.flatMap(_.database(databaseName))
def collection: Future[BSONCollection] = db.map(_.collection(collectionName))
def read(id: BSONObjectID): Future[R] = {
val query = BSONDocument("_id" -> id)
val readResult: Future[R] = for {
coll <- collection
record <- coll.find(query).requireOne[R]
} yield record
readResult.recover {
case NoSuchResultException => throw RecordNotFoundException(id)
}
}
def delete(id: BSONObjectID): Future[Unit] = {
val query = BSONDocument("_id" -> id)
// first read then call remove. Read will throw if not present
read(id).flatMap { (_) => collection.map(coll => coll.remove(query)) }
}
}
So to make my test pass, I had to had a Thread.sleep right after waiting for the delete to complete. Knowing this is evil usually punished by many whiplash, I want learn and find the proper fix here.
While trying other stuff, I found instead of waiting, entirely closing the connection to the db was also doing the trick...
What am I misunderstanding here? Should a connection to the db be opened and close for each call to it? And not do many actions like adding, removing, updating records with one connection?
Note that everything works fine when I remove the read call in my delete function. Also by closing the connection, I mean call close on the MongoDriver from my test and also stop and start again embed Mongo which I'm using in background.
Thanks for helping guys.
Warning: this is a blind guess, I've no experience with MongoDB on Scala.
You may have forgotten to flatMap
Take a look at this bit:
collection.map(coll => coll.remove(query))
Since collection is Future[BSONCollection] per your code and remove returns Future[WriteResult] per doc, so actual type of this expression is Future[Future[WriteResult]].
Now, you have annotated your function as returning Future[Unit]. Scala often makes Unit as a return value by throwing away possibly meaningful values, which it does in your case:
read(id).flatMap { (_) =>
collection.map(coll => {
coll.remove(query) // we didn't wait for removal
() // before returning unit
})
}
So your code should probably be
read(id).flatMap(_ => collection.flatMap(_.remove(query).map(_ => ())))
Or a for-comprehension:
for {
_ <- read(id)
coll <- collection
_ <- coll.remove(query)
} yield ()
You can make Scala warn you about discarded values by adding a compiler flag (assuming SBT):
scalacOptions += "-Ywarn-value-discard"

Submitting operations in created future

I have a Future lazy val that obtains some object and a function which submits operations in the Future.
class C {
def printLn(s: String) = println(s)
}
lazy val futureC: Future[C] = Future{Thread.sleep(3000); new C()}
def func(s: String): Unit = {
futureC.foreach{c => c.printLn(s)}
}
The problem is when Future is completed it executes operations in reverse order than they have been submited. So for example if I execute sequentialy
func("A")
func("B")
func("C")
I get after Future completion
scala> C
B
A
This order is important for me. Is there a way to preserve this order?
Of course I can use an actor who asks for future and stashing strings while future is not ready, but it seems redundant for me.
lazy val futureC: Future[C]
lazy vals in scala will be compiled in to the code which uses a synchronized block for thread safety.
Here when the func(A) is called, it will obtain the lock for the lazy val and that thread will go to sleep.
Therefore func(B) & func(C) will blocked by the lock.
When those blocked threads are run, the order cannot be guaranteed.
If you do it like below, you'll have the order as you expect. This is because the for comprehension creates a flatMap, & map based chain that gets executed sequentially.
lazy val futureC: Future[C] = Future {
Thread.sleep(1000)
new C()
}
def func(s: String) : Future[Unit] = {
futureC.map { c => c.printLn(s) }
}
val x = for {
_ <- func("A")
_ <- func("B")
_ <- func("C")
} yield ()
The order preserves even without the lazy keyword. You can remove the lazy keyword unless it is really necessary.
Hope this helps.
You can use Future.traverse to ensure the order of execution.
Something like this.. Im not sure how your func has a reference to the correct futureC, so I moved it inside.
def func(s: String): Future[Unit] = {
lazy val futureC = Future{Thread.sleep(3000); new C()}
futureC.map{c => c.printLn(s)}
}
def traverse[A,B](xs: Seq[A])(fn: A => Future[B]): Future[Seq[B]] =
xs.foldLeft(Future(Seq[B]())) { (acc, item) =>
acc.flatMap { accValue =>
fn(item).map { itemValue =>
accValue :+ itemValue
}
}
}
traverse(Seq("A","B","C"))(func)

Akka: send back response to Actor

I have the following code that sends back a response to the sender Actor (the responding actor loads a list from a table using Slick):
class ManageUsersData extends Actor {
def receive = {
case _ => {
sender ! loadUsers
}
}
def loadUsers = {
var list = new ListBuffer[String]()
val db = Database.forConfig("dbconfig")
try {
val users: TableQuery[Users] = TableQuery[Users]
val future = db.run(users.result)
future onComplete {
case Success(u) => u.foreach {
user => {
list += user.firstName
}
list
}
case Failure(t) => println("An error has occured: " + t.getMessage)
}
} finally db.close
list
}
}
The problem here is that loadUsers returns before waiting for the Future to complete. How can this be approached?
You should use the pipe pattern:
import akka.pattern.pipe
// ...
def receive = {
val originalSender = sender
loadUsers pipeTo originalSender
}
The conceptual solution is already mentioned by Jean Logeart. Regarding loadUsers, I think this is a shorter version?
def loadUsers = {
val db = Database.forConfig("dbconfig")
try {
val users: TableQuery[Users] = TableQuery[Users]
db.run(users.result).map(_.firstName)
} catch {
case e: Exception => println("An error has occured: " + e.getMessage)
} finally db.close
}
As I see it, the easiest approach would be to simply send the future back to the sender, instead of the async-filled list.
As in
def loadUsers = {
val db = Database.forConfig("dbconfig")
try {
val users: TableQuery[Users] = TableQuery[Users]
val future = db.run(users.result)
future.map { //the future
_.map { //the returning Seq
_.firstName
}
}
} finally db.close
}
Now the caller actor has the burden to handle the future or the failure.
This also has the drawback that, if the sender used the ask/? operation, the async result will be a Future wrapping a further Future.
You can overcome this issue by using the pipeTo method, which sends a future message to the caller without needing to bother about unwrapping it.
The downside of piping the result is that the sender should have a way to identify which reply belongs to which request. A possible solution is to send a request identifier that will be sent back with the answer, so the requesting actor can easily link the twos.
sidenote
Why would you map on the firstName attribute in the future result instead of using a projection within the slick query? I assume it's for the sake of keeping the example simple.

Put future inside OnFailure

I am pretty new to scala so be gentle!
I am trying to create a "logical transaction", here is a code sample:
val f1:Future[Int] = dao.insertIntoDB
f1.flatmap{
x => {
val f2 = sendHttpRequestFuture
f.onFailure{
case t => dao.revertDbChangeFuture
}
}
f1.onFailure{
logger.error("error)
}
So what I think will happened is that the outer onFailure will be executed first, and then the inner onFailure.
The thing is that onFailure return Unit, so it feels weird to put a future inside onFailure, I am not sure what is the right way to handle such use case.
I basically need to revert my changes, and the revert action is asynchronous.
Can you advice?
Thanks!
Play arround with this sample.
It shows how all works.
Actually as I see you've implemented all right.
There are two variables that you can modify job1Fail and job2Fail.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
object FutFut extends App {
val f1: Future[Int] = Future {
doJob
}
val job1Fail = false
val job2Fail = true
def doJob() = {
if (job1Fail) sys.error("Failed to do job") else {
println("job 1 done")
1
}
}
def doOtherJob() {
if (job2Fail) sys.error("Failed to do other job") else {
println("other job done")
}
}
def revertingAll() {
println("reverting all")
}
f1.flatMap {
x => {
val f2 = Future {
doOtherJob()
}
f2.onFailure {
case t => revertingAll()
}
f2
}
}
f1.onFailure {
case t => println("f1 failed")
}
Thread.sleep(1000)
}
So the results are following:
db job fail -> output message f1 failed. no revert.
db job ok -> flat map invoked -> http fail -> revert invoked -> f1 does not fail
db job ok -> flat map invoked -> http ok -> all ok -> no revert -> no fail :)
That's are pretty all use cases.
You should not use a revertDbChangeFuture, because once you have that future, it most often means that you've already triggered the underlying computation, i.e. you are already reverting the DB change, regardless fo the outcome of the insert.
Instead, you should add the method for reverting the db change in your onFailure callback:
insertIntoDBFuture.onFailure {
case t => revertDbChange()
}
val f1 = insertIntoDBFuture.flatmap(sendHttpRequestFuture(_))
f1.onFailure{
case t => logger.error("error", t)
}
f1
the flatMap in val f1 = insertIntoDBFuture.flatmap(sendHttpRequestFuture(_)) will not be executed if insertIntoDBFuture fails, so you don't have to worry about that. Instead, the result of f1 will be the same failed future, so you will log the error of either insertIntoDBFuture or sendHttpRequestFuture - or none, if no error occurs.
You should probably use the recoverWith API in this case which allows you to transform a failed future with another future:
f2.recoverWith({
case NonFatal(_) => dao.revertDbChangeFuture
})
This will return you a Future against the undo operation.