I'm just trying to learn ZIO and everything was going great guns until I ended up with a Seq[URIO[Any, Fiber.Runtime[Throwable, Unit]]] - I'm not really sure what to do with that - None of my Fibres are running, probably because I need to evaluate each URIO but if that is the case, how do I do that in the zio.App? I have tried mapping over the sequence but then I seem to keep ending up with Seq[Nothing]...
I have tried wrapping the Seq in an effect but that doesn't seem to help me. I have looked at the examples on the ZIO website but they don't deal with list of fibers so now I'm lost...
EDIT
OK - I guess my question is: How do I run a list of effects?
package testmesomemore
import zio._
import zio.console._
import zio.internal.Platform
import java.io.IOException
case class testme(
s: String,
n: Int,
l: String) {
def runme() = UIO.effectTotal {
println(s"${s}${l}")
}
}
object mytest extends zio.App {
override val platform: Platform = zio.Runtime.default.platform.withReportFailure(_ => ())
def run(args: List[String]) = {
val testlist: Seq[testme] = List[testme](testme("Hi there", 1, "."), testme("This is a test", 3, "..."))
val result = for {
// r <- testlist.map(d => d.runme().forkDaemon) // How to run a list of effects?
a <- testme("Hi there", 1, ".").runme().forkDaemon
b <- testme("This is a test", 3, "...").runme().forkDaemon
} yield()
result.exitCode
}
}
I can fork the effects individually but if I have a list of them how would I fork them all?
I think I have been looking at this too long and now nothing makes sense... Sorry If I'm being dim and missing the obvious?
unless you have some specific need to manage fibers, just use Zio.foreach or ZIO.foreachPar for parallel, like so
ZIO.foreachPar(testlist)(_.runme()).exitCode
Related
I am trying to write a custom Akka SnapshotStore plugin.
I am at the point where I want to implement this method:
def loadAsync(persistenceId: String, criteria: SnapshotSelectionCriteria): Future[Option[SelectedSnapshot]]
This is what I have so far:
import cats.data.OptionT
import cats.implicits._
...
override def loadAsync(
persistenceId: String,
criteria: SnapshotSelectionCriteria
): Future[Option[SelectedSnapshot]] = {
// same as in the original plugin
val metadata = snapshotMetadatas(persistenceId, criteria).sorted.takeRight(maxLoadAttempts)
// need to get rid of this one!
import scala.concurrent.ExecutionContext.Implicits.global
val getSnapshotAndReportMetric = for {
snapshot <- OptionT.fromOption[Future](getMaybeSnapshotFromMetadata(metadata))
_ <- OptionT.liftF(Future {
observabilityService
.recordMetric(
LongCounterMetric(readVehicleSnapshotFromDiskCounter, SnapShotDirectoryScannerCommandOptions)
)
}(directorySnapshotScanningDispatcher))
} yield snapshot
getSnapshotAndReportMetric.value.recoverWith {
// retry if we listed an older snapshot that was deleted before loading
case _: NoSuchFileException => loadAsync(persistenceId, criteria)
}(streamDispatcher)
}
For reference here is the signature of getMaybeSnapshotFromMetadata. It's signature can be modified to add a Future, but ultimately the wrapped response must be of type Option[SelectedSnapshot].
private def getMaybeSnapshotFromMetadata(metadata: Seq[SnapshotMetadata]): Option[SelectedSnapshot]
The code as is compiles, but only because I have imported the implicit global ExecutionContext. My goal is to use different explicit execution contexts (different configurable dispatchers), but I don't see how to do it for the first line (the one that calls getMaybeSnapshotFromMetadata) within for-comprehensions. If I used OptionT.liftF then I could do it, e.g.
...
snapshot <- OptionT.liftF( Future { getMaybeSnapshotFromMetadata(metadata)}(streamDispatcher))
...but then I get a OptionT[Future, Option[SelectedSnapshot] as a result.
Is there a solution to what I want to achieve? If not I can work around with mere Futures and it's andThen chain method:
Future {
getMaybeSnapshotFromMetadata(metadata)
}(streamDispatcher)
.andThen(selectedSnapshot => {
observabilityService
.recordMetric(
LongCounterMetric(readVehicleSnapshotFromDiskCounter, SnapShotDirectoryScannerCommandOptions)
)
selectedSnapshot
})(opentelemetryDispatcher)
.recoverWith {
// retry if we listed an older snapshot that was deleted before loading
case _: NoSuchFileException => loadAsync(persistenceId, criteria)
}(streamDispatcher)
Update For the second line within for-comprehension liftF is actually a viable solution - I updated the code block.
Do not overuse cats. It's a nice tool for some things, but when used improperly, it just adds complexity and hurts readability without any benefit.
Future(getMaybeSnapshotFromMetadata(metadata))(streamDispatcher)
.andThen { case _ =>
observabilityService.recordWiseMetric(...)
}(directorySnapshotScanningDispatcher)
.recoverWith(...)(streamDispatcher)
This is equivalent to your code without all the complication ...
It is not a "workaround", but an actual proper way to write this.
I have a small test of fs2 streams, process elements, waiting and then write them to a file.
I'm getting a type error saying and I could not figure out what it does mean:
Error : required: fs2.Stream[[x]cats.effect.IO[x],Unit] => fs2.Stream[[+A]cats.effect.IO[A],Unit],
found : [F[_]]fs2.Pipe[F,Byte,Unit]
import java.nio.file.Paths
import cats.effect.{Blocker, ExitCode, IO, IOApp, Timer}
import fs2.Stream
import fs2.io
import fs2.concurrent.Queue
import scala.concurrent.duration._
import scala.util.Random
class StreamTypeIntToDouble(q: Queue[IO, Int])(implicit timer: Timer[IO]) {
import core.Processing._
val blocker: Blocker =
Blocker.liftExecutionContext(
scala.concurrent.ExecutionContext.Implicits.global
)
def storeInQueue: Stream[IO, Unit] = {
Stream(1, 2, 3)
.covary[IO]
.evalTap(n => IO.delay(println(s"Pushing $n to Queue")))
.metered(Random.between(1, 20).seconds)
.through(q.enqueue)
}
def getFromQueue: Stream[IO, Unit] = {
q.dequeue
.evalMap(n => IO.delay(println(s"Pulling from queue $n")))
.through(
io.file
.writeAll(Paths.get("file.txt"), blocker)
)
}
}
object Five extends IOApp {
override def run(args: List[String]): IO[ExitCode] = {
val program = for {
q <- Queue.bounded[IO, Int](10)
b = new StreamTypeIntToDouble(q)
_ <- b.storeInQueue.compile.drain.start
_ <- b.getFromQueue.compile.drain
} yield ()
program.as(ExitCode.Success)
}
}
There are a couple of issues here, and the first is the most confusing. writeAll is polymorphic in its context F[_], but it requires a ContextShift instance for F (as well as Sync). You don't currently have a ContextShift[IO] in scope, so the compiler won't infer that the F for writeAll should be IO. If you add something like this:
implicit val ioContextShift: ContextShift[IO] =
IO.contextShift(scala.concurrent.ExecutionContext.Implicits.global)
…then the compiler will infer IO as you'd expect.
My suggestion for cases like this is to skip the type inference. Writing it out with the type parameter is only marginally more verbose:
.through(
io.file
.writeAll[IO](Paths.get("file.txt"), blocker)
)
…and it means that you'll get useful error messages for things like missing type class instances.
Once you've fixed that issue there are a couple of others. The next is that using evalMap in this context means that you'll have a stream of () values. If you change it to evalTap, the logging side effects will still happen appropriately, but you won't lose the actual values of the stream you call it on.
The last issue is that writeAll requires a stream of bytes, while you've given it a stream of Ints. How you want to deal with that discrepancy depends on the intended semantics, but for the sake of example something like .map(_.toByte) would make it compile.
I'm a newbie trying to grasp the intuition behind fs2 Queues.
I'm trying to do a basic example for pulling data from a Stream[IO, Int]. But the documentation for me is not enough as it directly dives into advanced stuff directly.
Here what I've done so far:
import cats.effect.{ ExitCode, IO, IOApp}
import fs2._
import fs2.concurrent.Queue
class QueueInt(q: Queue[IO, Int]) {
def startPushingtoQueue: Stream[IO, Unit] = {
Stream(1, 2, 3).covary[IO].through(q.enqueue)
q.dequeue.evalMap(n => IO.delay(println(s"Pulling element $n from Queue")))
}
}
object testingQueues extends IOApp {
override def run(args: List[String]): IO[ExitCode] = {
val stream = for {
q <- Queue.bounded(10)
b = new QueueInt(q)
_ <- b.startPushingtoQueue.drain
} yield ()
}
}
Question 1: I'm getting No implicit argument of type Concurrent[F_],
Knowing that I'm not using any concurrent effect I can not seem to figure out what a am I missing?
Question 2: What can I do to print the results.
Question 3: Can someone direct me to some resources to learn fs2
I found several issues in your code:
If you're getting an error about missing implicits you can very often fix them with explicitly stating type arguments:
q <- Queue.bounded[IO, Unit](10) // it will fix your error with implicits
The resulting type of your for comprehension is IO[Unit], but in order to make it run you'd have to return it from run method. You'd also need to change the type from unit to ExitCode:
stream.as(ExitCode.Success)
In your method startPushingToQueue you're creating Steam but you're not assigning it anywhere. It will just create a description of the stream, but it won't be run.
What I think you wanted to achieve is to create on the method which will push elements to queue and another which would get elements from the queue and print them. Please check my solution:
import cats.effect.{ ExitCode, IO, IOApp}
import fs2._
import fs2.concurrent.Queue
import scala.concurrent.duration._
class QueueInt(q: Queue[IO, Int])(implicit timer: Timer[IO]) { //I need implicit timer for metered
def startPushingToQueue: Stream[IO, Unit] = Stream(1, 2, 3)
.covary[IO]
.evalTap(n => IO.delay(println(s"Pushing element $n to Queue"))) //eval tap evaluates effect on an element but doesn't change stream
.metered(500.millis) //it will create 0.5 delay between enqueueing elements of stream,
// I added it to make visible that elements can be pushed and pulled from queue concurrently
.through(q.enqueue)
def pullAndPrintElements: Stream[IO, Unit] = q.dequeue.evalMap(n => IO.delay(println(s"Pulling element $n from Queue")))
}
object testingQueues extends IOApp {
override def run(args: List[String]): IO[ExitCode] = {
val program = for {
q <- Queue.bounded[IO, Int](10)
b = new QueueInt(q)
_ <- b.startPushingToQueue.compile.drain.start //start at the end will start running stream in another Fiber
_ <- b.pullAndPrintElements.compile.drain //compile.draing compiles stream into io byt pulling all elements.
} yield ()
program.as(ExitCode.Success)
}
}
On console, you will see lines telling about pushing and pulling from queue interleaved.
If you remove start you will see that firstly stream from startPushingToQueue finishes after pushing all elements, and only then pullAndPrintElements starts.
If you're looking for good sources to learn fs2, I would suggest that you should start with checking out fs2-related talks. Prefer newer talks, than the old one, because they could reference the older API.
You should also check guide on fs2 documentation.
I am working on a todo list app using
scalajs,
cats (free monads), and
scalajs-react
When I am using a simple model like the code below, everything works like expected.
class TodoModel() {
private object State {
var todos = Seq.empty[Todo]
def mod(f: Seq[Todo] => Seq[Todo]): Callback = {
val newTodos = f(todos)
Callback(todos = newTodos)
}
}
def add(t: Todo): Callback = State.mod(_ :+ t)
def todos: Seq[Todo] = State.todos
}
Once I use the free monads from cats, I have an odd behaviour. The first click always inserts two todo entries. Every click afterwards works like expected. See the pictures below.
What is wrong here?
import cats.free.Free
import cats.free.Free.liftF
import japgolly.scalajs.react._
import japgolly.scalajs.react.vdom.html_<^._
import org.scalajs.dom
case class Todo(text: String)
sealed trait TodoModelOp[A]
case class Add(todo: Todo) extends TodoModelOp[Unit]
case class Todos() extends TodoModelOp[Seq[Todo]]
object FreeTodoModelOps {
// type alias for lifted TodoModelOp
type TodoModelOpF[A] = Free[TodoModelOp, A]
def add(Todo: Todo): TodoModelOpF[Unit] = liftF[TodoModelOp, Unit](Add(Todo))
def todos: TodoModelOpF[Seq[Todo]] = liftF[TodoModelOp, Seq[Todo]](Todos())
}
object StateInterpreter {
import cats.arrow.FunctionK
import cats.{ Id, ~> }
val interpet: TodoModelOp ~> Id = new (TodoModelOp ~> Id) {
val todos = scala.collection.mutable.ArrayBuffer.empty[Todo]
def apply[A](fa: TodoModelOp[A]): Id[A] = fa match {
case Add(todo) => todos += todo; ()
case Todos() => todos.toSeq
}
}
}
class TodoModel() {
import cats.instances.list._
import cats.syntax.traverse._
import FreeTodoModelOps._
def add(t: Todo): Callback = {
def program: TodoModelOpF[Unit] = for {
_ <- FreeTodoModelOps.add(t)
} yield ()
Callback(program.foldMap(StateInterpreter.interpet))
}
def todos: Seq[Todo] = {
def program: TodoModelOpF[Seq[Todo]] = for {
n <- FreeTodoModelOps.todos
} yield n
program.foldMap(StateInterpreter.interpet)
}
}
object TodoPage {
case class Props(model: TodoModel)
case class State(todos: Seq[Todo])
class Backend($: BackendScope[Props, State]) {
val t = Todo("a new todo")
def onSubmit(e: ReactEventFromInput) =
e.preventDefaultCB >>
$.modState(s => State(s.todos :+ t)) >>
$.props.flatMap(P => P.model.add(t))
def render(S: State) =
<.div(
<.form(
^.onSubmit ==> onSubmit,
<.button("Add #", S.todos.length + 1)),
<.ul(S.todos.map(t => <.li(t.text)): _*))
}
val component = ScalaComponent.builder[Props]("Todo")
.initialStateFromProps(p => State(p.model.todos))
.renderBackend[Backend]
.build
def apply(model: TodoModel) = component(Props(model))
}
object Test {
val model = new TodoModel()
def main(args: Array[String]): Unit = {
TodoPage.apply(model).renderIntoDOM(dom.document.getElementById("mount-node"))
}
}
empty, no click on button
first click on button
second click on button
In your first snippet there's a bug:
Here you've got a variable todos (inpure) which you're accessing in a pure context:
def mod(f: Seq[Todo] => Seq[Todo]): Callback = {
val newTodos = f(todos)
Callback(todos = newTodos)
Impurity should be in Callback. Even reading a variable outside of a Callback is unsafe, so it should be:
def mod(f: Seq[Todo] => Seq[Todo]): Callback =
Callback(todos = f(todos))
(See scalajs-react's Ref.scala an example of safely working with a variable.)
Secondly, with regards to your larger snippet, scalajs-react is very FP friendly but that's very unconventional way of trying to use it, and has some significant problems:
StateInterpreter.interpet isn't referentially-transparent; there's shared global state underlying that. Fails the FP test. Stops being a lawful natural transformation.
You're tracking two sets of identical state separately: the component state and the state in TodoModel (impure, fails the FP test). Not only is this approach redundant and runs the risk of the two states getting out-of-sync, but it also makes the component less reusable; imagine you decided to draw it twice on the same screen for the same data - they're going to go out of sync. Best to keep the component stateless and pure.
If you're going to transform a free structure into a component effect, it's best to transform it into a state monad, see here for an example.
It's really, really awesome that you're learning about free monads and scalajs-react. FP will make your entire program really, really easy to reason about and prevent confusing surprises in behaviour, but you've got to not cut any corners and make sure that you keep all of your code pure. Any impurity will render the entire stack all the way up to the entrypoint impure and remove those nice dependable FP properties from those layers. I'd suggest making everything as pure as possible using the points above as starting points, and then I think you'll find that the bug just disappears, or at least is very easy to then detect. Cheers
I am a newbie to scala futures and I have a doubt regarding the return value of scala futures.
So, generally syntax for a scala future is
def downloadPage(url: URL) = Future[List[Int]] {
}
I want to know how to access the List[Int] from some other method which calls this method.
In other words,
val result = downloadPage("localhost")
then what should be the approach to get List[Int] out of the future ?
I have tried using map method but not able to do this successfully.`
The case of Success(listInt) => I want to return the listInt and I am not able to figure out how to do that.
The best practice is that you don't return the value. Instead you just pass the future (or a version transformed with map, flatMap, etc.) to everyone who needs this value and they can add their own onComplete.
If you really need to return it (e.g. when implementing a legacy method), then the only thing you can do is to block (e.g. with Await.result) and you need to decide how long to await.
You need to wait for the future to complete to get the result given some timespan, here's something that would work:
import scala.concurrent.duration._
def downloadPage(url: URL) = Future[List[Int]] {
List(1,2,3)
}
val result = downloadPage("localhost")
val myListInt = result.result(10 seconds)
Ideally, if you're using a Future, you don't want to block the executing thread, so you would move your logic that deals with the result of your Future into the onComplete method, something like this:
result.onComplete({
case Success(listInt) => {
//Do something with my list
}
case Failure(exception) => {
//Do something with my error
}
})
I hope you already solved this since it was asked in 2013 but maybe my answer can help someone else:
If you are using Play Framework, it support async Actions (actually all Actions are async inside). An easy way to create an async Action is using Action.async(). You need to provide a Future[Result]to this function.
Now you can just make transformations from your Future[List[Int]] to Future[Result] using Scala's map, flatMap, for-comprehension or async/await. Here an example from Play Framework documentation.
import play.api.libs.concurrent.Execution.Implicits.defaultContext
def index = Action.async {
val futureInt = scala.concurrent.Future { intensiveComputation() }
futureInt.map(i => Ok("Got result: " + i))
}
You can do something like that. If The wait time that is given in Await.result method is less than it takes the awaitable to execute, you will have a TimeoutException, and you need to handle the error (or any other error).
import scala.concurrent._
import ExecutionContext.Implicits.global
import scala.util.{Try, Success, Failure}
import scala.concurrent.duration._
object MyObject {
def main(args: Array[String]) {
val myVal: Future[String] = Future { silly() }
// values less than 5 seconds will go to
// Failure case, because silly() will not be done yet
Try(Await.result(myVal, 10 seconds)) match {
case Success(extractedVal) => { println("Success Happened: " + extractedVal) }
case Failure(_) => { println("Failure Happened") }
case _ => { println("Very Strange") }
}
}
def silly(): String = {
Thread.sleep(5000)
"Hello from silly"
}
}
The best way I’ve found to think of a Future is a box that will, at some point, contain the thing that you want. The key thing with a Future is that you never open the box. Trying to force open the box will lead you to blocking and grief. Instead, you put the Future in another, larger box, typically using the map method.
Here’s an example of a Future that contains a String. When the Future completes, then Console.println is called:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
object Main {
def main(args:Array[String]) : Unit = {
val stringFuture: Future[String] = Future.successful("hello world!")
stringFuture.map {
someString =>
// if you use .foreach you avoid creating an extra Future, but we are proving
// the concept here...
Console.println(someString)
}
}
}
Note that in this case, we’re calling the main method and then… finishing. The string’s Future, provided by the global ExecutionContext, does the work of calling Console.println. This is great, because when we give up control over when someString is going to be there and when Console.println is going to be called, we let the system manage itself. In constrast, look what happens when we try to force the box open:
val stringFuture: Future[String] = Future.successful("hello world!")
val someString = Future.await(stringFuture)
In this case, we have to wait — keep a thread twiddling its thumbs — until we get someString back. We’ve opened the box, but we’ve had to commandeer the system’s resources to get at it.
It wasn't yet mentioned, so I want to emphasize the point of using Future with for-comprehension and the difference of sequential and parallel execution.
For example, for sequential execution:
object FuturesSequential extends App {
def job(n: Int) = Future {
Thread.sleep(1000)
println(s"Job $n")
}
val f = for {
f1 <- job(1)
f2 <- job(2)
f3 <- job(3)
f4 <- job(4)
f5 <- job(5)
} yield List(f1, f2, f3, f4, f5)
f.map(res => println(s"Done. ${res.size} jobs run"))
Thread.sleep(6000) // We need to prevent main thread from quitting too early
}
And for parallel execution (note that the Future are before the for-comprehension):
object FuturesParallel extends App {
def job(n: Int) = Future {
Thread.sleep(1000)
println(s"Job $n")
}
val j1 = job(1)
val j2 = job(2)
val j3 = job(3)
val j4 = job(4)
val j5 = job(5)
val f = for {
f1 <- j1
f2 <- j2
f3 <- j3
f4 <- j4
f5 <- j5
} yield List(f1, f2, f3, f4, f5)
f.map(res => println(s"Done. ${res.size} jobs run"))
Thread.sleep(6000) // We need to prevent main thread from quitting too early
}