I have a service that makes use of the Scala Async library. I'm using this library primarily to time my database calls. The method that I want to test contains multiple calls to the database using the async await mechanism. A pseudo code of what I have is as below:
def myDbMethod() = async {
val firstCall = await(call the db and get the result)
val secondCall = await(all the db and get the result)
val thirdCall = await(all the db and get the result)
...
}
In my Scala test unit test, I have
Await.result(myDbMethod(), 10.seconds)
I was just trying to debug myMethod by running my unit test which would return with a test success even before getting into the secondCall. I mean I had breakpoints in all the 3 calls to the database, but the IntelliJ debugger would just exit out as soon as it finishes the first call to the database. Why is this? How can I test this behaviour using IntelliJ debugger?
I'm not sure that my answer would suffice your expectations, but it's a known issue. The problem is that async/await is quite a complicated macro, which does heavy transformations on the trees (you can check the output by enabling -Xprint:<phase_name_after_typer> flag). Unfortunately neither of existing IDEs (I'm working with Intellij and Ensime) can debug it, but I'm not familiar with their internals to explain why they can't in details.
From my experience I couldn't find any neat pros over the native for-comprehension, so you can stick with th native syntax or explicit flatmap calls, which is nicely debuggable.
This construct could be used for depdendant asynchronous calls.
async / await adds some sugar to make that easier, but to formalate that by hand you can do it like this:
def dbMethodDependant : Future[T] = for {
res1 <- firstCall
res2 <- secondCall(res2)
res3 <- thirdCall(res3)
} yield res3
Await.result(dbMethodDependant, forever)
Related
trying to upsert with the new Scala Async Driver using this code, but the DB never gets created even though this is called many times:
override def insertOrUpdateOne(user: UserNew): Future[Unit] = {
users.replaceOne(equal("_id", user._id), user)
.toFuture
.map(_ => ())
}
mongo client created fine, collection created fine, but the above causes no write to be performed in fact the db is never created (which should happen on the first write). I have other code that uses casbah and that is working synchronously.
users is the Mongo collection with no syntactic sugar from me. (edited)
toFuture seems to do the subscribe (which should trigger the write?) but it still doesn't work. I'm mixing some collections with casbah and one with the async client, the casbah ones work but not the one above using async.
Any ideas?
it should be
override def insertOrUpdateOne(user: UserNew): Future[Unit] = {
users.replaceOne(equal("_id", user._id), user, UpdateOptions().upsert(true))
.toFuture
.map(_ => ())
}
Nothing in the docs but when I read the code I see it defaults upsert to false.
BTW this allows us to ignore all the Observable and subscribe stuff and setup of callbacks. Just process the results in a Scala idiomatic someFuture.map() This makes the UsersDaonon-Mongo specific code as an interface. This in turn allows all code in the UsersDaoImplto handle Mongo specific stuff, which enables the Impl to be injected so a runtime choice can be made about which Impl to use. Mongo can be replaced without dependent code caring or knowing about the Impl.
I tried to use the Slick(3.0.2) to operate database in my project with scala.
Here is my part of code
val query = table.filter(_.username === "LeoAshin").map { user => (user.username, user.password, user.role, user.delFlg) }
val f = db.run(query.result)
How can I read the data from "f"
I have tried google the solution many times, but no answer can solved my confuse
Thanks a lot
f is a Future and there are several things you can do to get at the value, depending on just how urgently you need it. If there's nothing else you can do while you're waiting then you will just have to wait for it to complete and then get the value. Perhaps the easiest is along the following lines (from the Slick documentation):
val q = for (c <- coffees) yield c.name
val a = q.result
val f: Future[Seq[String]] = db.run(a)
f.onSuccess { case s => println(s"Result: $s") }
You could then go on to do other things that don't depend on the result of f and the result of the query will be printed on the console asynchronously.
However, most of the time you'll want to use the value for some other operation, perhaps another database query. In that case, the easiest thing to do is to use a for comprehension. Something like the following:
for (r <- f) yield db.run(q(r))
where q is a function/method that will take the result of your first query and build another one. The result of this for comprehension will also be a Future, of course.
One thing to be aware of is that if you are running this in a program that will do an exit after all the code has been run, you will need to use Await (see the Scala API) to prevent the program exiting while one of your db queries is still working.
Type of query.result is DBIO. When you call db.run, it turns into Future.
If you want to print the data, use
import scala.concurrent.ExecutionContext.Implicits.global
f.foreach(println)
To continue working with data, use f.map { case (username, password, role, delFlg) ⇒ ... }
If you want to block on the Future and get the result (if you're playing around in REPL for example), use something like
import scala.concurrent.Await
import scala.concurrent.duration._
Await.result(f, 1.second)
Bear in mind, this is not what you want to do in production code — blocking on Futures is a bad practice.
I generally recommend learning about Scala core types and Futures specifically. Slick "responsibility" ends when you call db.run.
I'm using Slick 3.0 and (of course) almost all the examples out there cover Slick 2.x. Things have changed and frankly seem more complicated, not less.
Here's an example: I want to get an object (a GPPerson) by id. This is what I have right now, and it seems very verbose... more so than Slick 2.x:
def get(id: GPID): Option[GPPerson] = Await.result(
db.run(
people.filter(_.id === id).result), Duration.Inf
).headOption
In Slick 2.x things were easier because of the implicits, among other things. But the above seems to be the most concise expression I've come up with.
It also doesn't really address exception handling, which I would need to add.
I started to use Slick 3.0 in a new project a few months ago and I had the same questions. This is what I understood:
Slick 3.0 was designed for non-blocking asynchronous (reactive) applications. Obviously it means Akka + Play / Spray nowadays. In this world you mostly interact with Futures, that's why Slick's db.run returns Future. There is no point in using Await.result - if you need blocking calls it's better to return to 2.x.
But if you use reactive stack you'll get benefits immediately. For example, Spray is completely non-blocking library that works with Futures nicely using onComplete directive. You can call a method that returns Future with a result from Slick in a Spray route and then use that result together with onComplete. In this case the whole response-reply pipeline is non-blocking.
You also mentioned exception handling, so this is exactly how you do it - using Futures.
So based on my experience I would write your method in a following way:
def get(id: GPID): Future[Option[GPPerson]] = db.run(
people.filter(_.id === id).result.map(_.headOption)
)
and then work with a Future.
you can do this.
def syncResult[R](action:slick.dbio.DBIOAction[R, slick.dbio.NoStream, scala.Nothing]):R = {
import scala.concurrent.duration.Duration
val db = Database.forConfig("db")
try {
Await.result(db.run(action), Duration.Inf)
} finally db.close
}
def get(id: GPID): Option[GPPerson] = syncResult { people.filter(_.id === id).result.headOption }
I am having a issue with the below piece of code. I want 'combine' method to get triggered after all groundCoffee,heaterWater,frothedMilk method completes. They would be triggered concurrently.All the 4 methods grind,heatWater,frothMilk,brew are concurrently executed using a future.
def prepareCappuccino(): Future[Cappuccino] = {
val groundCoffee = grind("arabica beans")
val heatedWater = heatWater(Water(20))
val frothedMilk = frothMilk("milk")
for {
ground <- groundCoffee
water <- heatedWater
foam <- frothedMilk
espresso <- brew(ground, water)
} yield combine(espresso, foam)
}
When I execute the above method the output I am getting is below
start grinding...
heating the water now
milk frothing system engaged!
And the program exits after this. I got this example from a site while I was trying to learn futures. How can the program be made to wait so that combine method get triggered after all the futures return?
The solution already posted to Await for a future is a solution when you want to deliberately block execution on that thread. Two common reasons to do this are for testing, when you want to wait for the outcome before making an assertion, and when otherwise all threads would exit (as is the case when doing toy examples).
However in a proper long lived application Await is generally to be avoided.
Your question already contains one of the correct ways to do future composition - using a for comprehension. Bear in mind here, that for-comprehensions are converted to flatMaps, maps and withFilter operations, so any futures you invoke in the for-comprehension will only be created after the others complete, ie serially.
If you want a bunch of futures to operate in concurrently, then you would create them before entering the for-comprehension as you have done.
You can use the Await here:
val f = Future.sequence(futures.toList)
Await.ready(f, Duration.Inf)
I assume, you have all the futures packed in a list. The Await.ready makes all the waiting work.
I went to a very interesting lecture on Async (https://github.com/scala/async) a new library for Scala, what I am not sure about is how Akka and Async differ.
I am new to Scala so apologies if the answer is obvious.
Thanks.
Async simplifies asynchronous and concurrent programming. Async enables programming with non-blocking APIs in a familiar direct style. Direct-style code is as simple to write as blocking code, but it has all the advantages of efficient non-blocking code.
Out-of-the-box, Async makes programming with Scala's futures more convenient. In fact, by using Async with futures, your easy-to-read direct-style code is transformed to highly-efficient non-blocking code under the hood. Async can also be connected to other APIs (see below).
Akka provides a programming model and runtime to simplify concurrency, distribution, and fault tolerance. Async does not provide a runtime--it makes existing abstractions and their runtimes easier to use. However, Async and Akka can work together in several important ways:
Using Akka's "ask" pattern, sending a message using "?" returns a future. Async makes it easy to work with those futures.
Async can be connected to APIs other than Scala's Futures API. This opens up interesting ways to leverage Async to simplify programming with Akka actors. This is one area that we are going to explore in the near future at Typesafe.
Async is a replacement for Akka's dataflow API which is simpler and more robust. Async is simpler, because it does not introduce complex types stemming from the use of Scala's CPS plugin. Async is more robust, since it works well together with features like pattern matching and try-catch which are not fully supported by CPS/Akka's dataflow API. Moreover, it is clearly specified where Async cannot be used (await cannot occur inside closures, nested classes/traits/objects, or by-name arguments).
For more Async examples and documentation see my recent talk on Async and the documentation on the Async project website.
Async just adds helpfull API (and some other good internal things) for working with Future and has nothing to do with Actor model while Akka is a framework for creating massively-distributed with Actors. They have different use cases and have nothing in common. It more correctly to compare Async with Scala/Akka Futures API. I don't have much experience with Async, but the main point, is that you have two constructs async and await. Async marks a block of asynchronous code wich contains one or more await calls, which marks a point at which the computation will be suspended until the awaited Future is complete. Such API can be compared with a standart way of using map and flatmap .
Using standart API (for construct translates to combination of map and flatmap):
def slowCalcFuture: Future[Int] = ...
val future1 = slowCalcFuture
val future2 = slowCalcFuture
def combined: Future[Int] = for {
r1 <- future1
r2 <- future2
} yield r1 + r2
And Async:
def slowCalcFuture: Future[Int] = ...
def combined: Future[Int] = async {
val future1 = slowCalcFuture
val future2 = slowCalcFuture
await(future1) + await(future2)
}