I am migrating from Cats Effect 2 to 3 and since Blocker is not available anymore I wonder how is it possible to run F in the blocking execution context?
Previously, the following code was used
val blocker: Blocker = Blocker.liftExecutionContext(blockingEc)
blocker.blockOn(F)
According to the CE3 docs, blocking contexts is inside a Runtime and this field is private. There is a method Sync[F].blocking(f: => A), but I need an alternative one with the possibility to pass F[A] to the blocking context like Sync[F].blockingF(f: F[A]).
Related
I am trying to migrate project from cats-effect 2 to cats-effect 3, i am using doobie for interacting with database. Previously i could lift ConnectionIO to IO as it was described, but with the upgrade i didn't find any implementation of LiftIO[ConnectionIO], how can be same achieved with CE3?
There's two approaches here. Broadly, there's the "don't do that" option (recommended), and the other way would be to use WeakAsync
"Don't do that"
Generally, it's not recommended to interleave arbitrary IO into a ConnectionIO program, because ConnectionIO is transactional, and you cannot roll back an IO. A ConnectionIO might run multiple times, for example to retry a recoverable failure.
When possible, it's a good idea to refactor the code to not do the non-transactional operation inside a transaction, but instead to perform it afterwards
However, when that's not practical, ConnectionIO provides a Sync instance, and so there's a number of things you can do with it easily that don't require lifting from IO in the first place:
printing to the console can be done by summoning Console[ConnectionIO]
Getting the current time can be done with Clock[ConnectionIO]
You can create a log4cats Logger[ConnectionIO] as needed using the appropriate factory for your backend
Generating a UUID value can be done with UUIDGen[ConnectionIO]
Using WeakAsync
Doobie's WeakAsync provides a way to get a Resource[F, F ~> ConnectionIO]
Note that because this is a resource, you must take care to complete the transaction inside of use - the lifecycle of the FunctionK from the resource will be shut down once use completes.
Usually that means something like this:
def doStuff(rows: List[Int]): F[Unit] = ???
WeakAsync.liftK[F, ConnectionIO].use { fk =>
val transaction = for {
rows <- sql"select 1".query[Int].to[List]
_ <- fk(doStuff(rows))
} yield ()
transaction.transact(xa)
}
I found the way to achieve it by
def f()(implicit liftToConnectionIO: FunctionK[IO, ConnectionIO]): IO[Unit] = IO.unit
implicit val liftToConnectionIO = WeakAsync.liftK[IO, ConnectionIO]
liftToConnectionIO.use(implicit lift => f())
I have some cats IO operations, and Future among then. Simplified:
IO(getValue())
.flatMap(v => IO.fromFuture(IO(blockingProcessValue(v)))(myBlockingPoolContextShift))
.map(moreProcessing)
So I have some value in IO, then I need to do some blocking operation using a library that returns Future, and then I need to do some more processing on the value returned from Future
Future runs on a dedicated thread pool - so far so good. The problem is after Future is completed. moreProcessing runs on the same thread the Future was running on.
Is there a way to get back to the thread getValue() was running on?
After the discussion in the chat, the conclusion is that the only thing OP needs is to create a ContextShift in the application entry point using the appropriate (compute) EC and then pass it down to the class containing this method.
// Entry point
val computeEC = ???
val cs = IO.contextShift(computeEC)
val myClass = new MyClass(cs, ...)
// Inside the method on MyClass
IO(getValue())
.flatMap(v => IO.fromFuture(IO(blockingProcessValue(v)))(myBlockingPoolContextShift))
.flatTap(_ => cs.shift)
.map(moreProcessing)
This Scastie showed an approach using Blocker and other techniques common in the Typelevel ecosystem but were not really suitable for OP's use case; anyways I find it useful for future readers who may have a similar problem.
Given:
AtoB: GraphStageWithMaterializedValue[A, B, Future[AtoBRuntimeApi]];
BtoC: GraphStageWithMaterializedValue[B, C, Future[BtoCRuntimeApi]].
Wanted: AtoC: GraphStageWithMaterializedValue[A, C, Future[AtoCRuntimeApi]].
In my particular case it's really convenient to implement AtoCRuntimeApi in terms of both AtoBRuntimeApi and BtoARuntimeApi.
So I would like to define AtoCRuntimeApi as case class AtoCRuntimeApi(a2b: AtoBRuntimeApi, b2c: BtoCRuntimeApi);
And to define the new compound stage as stageAtoB.viaMat(stageBtoC)(combineIntoAtoC) where
combineIntoAtoC: (Future[AtoBRuntimeApi], Future[B2CRuntimeApi]) => Future[AtoCRuntimeApi].
Obviously the implementation of combineIntoAtoC requires some instance of ExecutionContext in order to map the futures.
The question: what execution context should I use in the described case?
Options, I would rather avoid:
bake in an instance that is currently available (while composition of stages) — the "blueprint" will not be safe to materialise if that execution-context becomes unavailable;
ExecutionContext.global — well, it's global. It seems terribly wrong to use it (probably, once per materialisation — is not that big deal).
The most wanted execution-context is the one that is available as the property of materializer (mat.executionContext). But there is no way I can access it within that combine-function.
I usually use the actor system context in such cases, which is indeed by default can be referred to from the materializer. As a general solution, though, you can pass the execution context to whatever function you construct the stream graph in with an implicit parameter:
def makeGraph(...)(implicit ec: ExecutionContext): RunnableGraph = {
def combineIntoAtoC(...) = ... // uses ec implicitly
}
This allows you to push the decision about which context to use up the call stack. At the appropriate level there most certainly will be some kind of access to the actor system's dispatcher.
The reason why I prefer to use the actor system's dispatcher instead of the global one is because it reduces the surface of dependencies - all execution contexts in this case come from one source, and you know how to configure them if the need arises.
The use case is actually fairly typical. A lot of web services use authorization tokens that you retrieve at the start of a session and you need to send those back on subsequent requests.
I know I can do it like this:
lazy val myData = {
val request = ws.url("/some/url").withAuth(user, password, WSAuthScheme.BASIC).withHeaders("Accept" -> "application/json")
Await.result(request.get().map{x => x.json }, 120.seconds)
}
That just feels wrong as all the docs say never us Await.
Is there a Future/Promise Scala style way of handling this?
I've found .onComplete which allows me to run code upon the completion of a Promise however without using a (mutable) var I see no way of getting a value in that scope into a lazy val in a different scope. Even with a var there is a possible timing issue -- hence the evils of mutable variables :)
Any other way of doing this?
Unfortunately, there is no way to make this non-blocking - lazy vals are designed to be synchronous and to block any thread accessing them until they are completed with a value (internally a lazy val is represented as a simple synchronized block).
A Future/Promise Scala way would be to use a Future[T] or a Promise[T] instead of a val x: T, but that way implies a great deal of overhead with executionContexts and maps upon each use of the val, and more optimal resource utilization may not be worth the decreased readability in all cases, so it may be OK to leave the Await there if you extensively use the value in many parts of your application.
Came across a problem I did not find an answer yet.
Running on playframework 2 with Scala.
Was required to write an Action method that performs multiple Future calls.
My question:
1) Is the attached code non-blocking and hence looking the way it should be ?
2) Is there a guarantee that both DAO results are caught at any given time ?
def index = Action.async {
val t2:Future[Tuple2[List[PlayerCol],List[CreatureCol]]] = for {
p <- PlayerDAO.findAll()
c <- CreatureDAO.findAlive()
}yield(p,c)
t2.map(t => Ok(views.html.index(t._1, t._2)))
}
Thanks for your feedback.
Is the attached code non-blocking and hence looking the way it should be ?
That depends on a few things. First, I'm going to assume that PlayerDAO.findAll() and CreatureDAO.findAlive() return Future[List[PlayerCol]] and Future[List[CreatureCol]] respectively. What matters most is what these functions are actually calling themselves. Are they making JDBC calls, or using an asynchronous DB driver?
If the answer is JDBC (or some other synchronous db driver), then you're still blocking, and there's no way to make it fully "non-blocking". Why? Because JDBC calls block their current thread, and wrapping them in a Future won't fix that. In this situation, the most you can do is have them block a different ExecutionContext than the one Play is using to handle requests. This is generally a good idea, because if you have several db requests running concurrently, they can block Play's internal thread pool used for handling HTTP requests, and suddenly your server will have to wait to handle other requests (even if they don't require database calls).
For more on different ExecutionContexts see the thread pools documentation and this answer.
If you're answer is an asynchronous database driver like reactive mongo (there's also scalike-jdbc, and maybe some others), then you're in good shape, and I probably made you read a little more than you had to. In that scenario your index controller function would be fully non-blocking.
Is there a guarantee that both DAO results are caught at any given time ?
I'm not quite sure what you mean by this. In your current code, you're actually making these calls in sequence. CreatureDAO.findAlive() isn't executed until PlayerDAO.findAll() has returned. Since they are not dependent on each other, it seems like this isn't intentional. To make them run in parallel, you should instantiate the Futures before mapping them in a for-comprehension:
def index = Action.async {
val players: Future[List[PlayerCol]] = PlayerDAO.findAll()
val creatures: Future[List[CreatureCol]] = CreatureDAO.findAlive()
val t2: Future[(List[PlayerCol], List[CreatureCol])] = for {
p <- players
c <- creatures
} yield (p, c)
t2.map(t => Ok(views.html.index(t._1, t._2)))
}
The only thing you can guarantee about having both results being completed is that yield isn't executed until the Futures have completed (or never, if they failed), and likewise the body of t2.map(...) isn't executed until t2 has been completed.
Further reading:
Are there any benefits in using non-async actions in Play Framework 2.2?
Understanding the Difference Between Non-Blocking Web Service Calls vs Non-Blocking JDBC