I am new to ZHub and ZStream and wanted to familiarize myself with their APIs.
Unfortnuately, I could not make this simple example work:
for
hub <- Hub.bounded[String](4)
stream = ZStream.fromHub(hub)
_ <- hub.publish("Hello")
_ <- hub.publish("World")
collected <- stream.runCollect
_ <- ZIO.foreach(collected) { msg => console.putStrLn(msg) }
yield
()
This program does not terminate, I suspect, because I am trying to collect an infinite stream. I have also tried to print the messages using stream.tap(...) or to shut down the hub. Nothing has helped.
What am I missing here? Any help is appreciated, thanks.
#adamgfraser kindly provided a working example on GitHub:
import zio._
import zio.stream._
object Example extends App {
def run(args: List[String]): URIO[ZEnv, ExitCode] =
for {
promise <- Promise.make[Nothing, Unit]
hub <- Hub.bounded[String](2)
stream = ZStream.managed(hub.subscribe).flatMap { queue =>
ZStream.fromEffect(promise.succeed(())) *>
ZStream.fromQueue(queue)
}
fiber <- stream.take(2).runCollect.fork
_ <- promise.await
_ <- hub.publish("Hello")
_ <- hub.publish("World")
collected <- fiber.join
_ <- ZIO.foreach(collected)(console.putStrLn(_)).orDie
} yield ExitCode.success
}
My mistake was to publish values to the hub before waiting for the subscription to complete.
Related
I have a stream that just repeats every x seconds.
In my unit tests I want to test certain business logic, so I need my clock to start at 1:30pm and run until 1:45pm.
How can I mock this type of behaviour?
import java.time.{ZoneId, ZonedDateTime}
val zoneId = ZoneId.of("America/New_York")
Stream
.repeatEval {
for {
realTime <- Clock[F].realTimeInstant
zdt = ZonedDateTime.ofInstant(realTime, zoneId)
_ <- std.Console[F].println(s"zdt=$zdt")
} yield ()
}
You can use the TestControl structure from cats-effect-testkit to have fine-grained control over time in tests. It provides a full IORuntime, so your code just operates as normal
See the example of advancing the clock manually:
test("backoff appropriately between attempts") {
case object TestException extends RuntimeException
val action = IO.raiseError(TestException)
val program = Random.scalaUtilRandom[IO] flatMap { random =>
retry(action, 1.minute, 5, random)
}
TestControl.execute(program) flatMap { control =>
for {
_ <- control.results.assertEquals(None)
_ <- control.tick
_ <- 0.until(4) traverse { i =>
for {
_ <- control.results.assertEquals(None)
interval <- control.nextInterval
_ <- IO(assert(interval >= 0.nanos))
_ <- IO(assert(interval < (1 << i).minute))
_ <- control.advanceAndTick(interval)
} yield ()
}
_ <- control.results.assertEquals(Some(Outcome.failed(TestException)))
} yield ()
}
}
Im trying to alter ZIO's example code to fit what I want, but ran into a problem. I want to implement functionalities between different rpc calls, but I can't seem to get it to work, since in below example, only the while loop, rcpMethod3() and rcpMethod4() gets executed, whereas rcpMethod1() and rcpMethod2() doesn't.
I want to execute all of the rcpMethods and the while loop.
object Client extends App {
val clientLayer: Layer[Throwable, ClientZLayer] =
ClientZLayer.live(
ZManagedChannel(
ManagedChannelBuilder.forAddress("localhost", 8980).usePlaintext()
)
)
// rcp methods
def rcpMethod1(): ZIO[ClientZLayer with Console, Status, Unit] = {
for {
response <- ClientZLayer.rcpMethod1(Greeting("Hi"))
_ <- putStrLn(s"""Greeted: "${response.name}".""")
} yield ()
}
// Run the code
final def run(args: List[String]) = {
(for {
_ <- rcpMethod1()
_ <- rcpMethod2()
} yield ()).provideCustomLayer(clientLayer).exitCode
while(condition) {
// bla bla
}
(for {
_ <- rcpMethod3()
_ <- rcpMethod4()
} yield ()).provideCustomLayer(clientLayer).exitCode
}
}
The ZIO data type is a functional effect. A functional effect is a description of a workflow. This is why we have the run method at the end of the world. This run method executes the provided ZIO effect.
All the rcpMethodN methods are ZIO effect, so they are just a description of running workflow. If you want to run these effects sequentially you should compose them using for-comprehension or flatMap like this:
for {
_ <- rcpMethod1()
_ <- rcpMethod2()
_ <- rcpMethod3()
_ <- rcpMethod4()
} yield ()
The while(condition){ ... } is another mistake in your code. You should introduce these loop structs with ZIO data type. For example:
for {
_ <- rcpMethod1()
_ <- rcpMethod2()
_ <- ZIO.effect(???).repeatWhile(condition)
_ <- rcpMethod3()
_ <- rcpMethod4()
} yield ()
help me how to organize a directory scan on ZIO. This is my version, but it doesn't track all file creation events (miss some events).
object Main extends App {
val program = for {
stream <- ZIO.succeed(waitEvents)
_ <- stream.run(ZSink.foreach(k => putStrLn(k.map(e => (e.kind(), e.context())).mkString("\n"))))
} yield ()
val managedWatchService = ZManaged.make {
for {
watchService <- FileSystem.default.newWatchService
path = Path("c:/temp")
_ <- path.register(watchService,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_DELETE
)
} yield watchService
}(_.close.orDie)
val lookKey = ZManaged.make {
managedWatchService.use(watchService => watchService.take)
}(_.reset)
val waitEvents = ZStream.fromEffect {
lookKey.use(key => key.pollEvents)
}.repeat(Schedule.forever)
override def run(args: List[String]): ZIO[zio.ZEnv, Nothing, ExitCode] =
program
.provideLayer(Console.live ++ Blocking.live ++ Clock.live)
.exitCode
}
Thank you for your advice.
You are forcing your WatchService to shutdown and recreate every time you poll for events. Since that probably involves some system handles it is likely fairly slow so you would probably missing file events that occur in between. More likely you want to produce the WatchService once and then poll it repeatedly. I would suggest something like this instead:
object Main extends App {
val managedWatchService = ZManaged.make {
for {
watchService <- FileSystem.default.newWatchService
path = Path("c:/temp")
_ <- path.register(watchService,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_DELETE
)
} yield watchService
}(_.close.orDie)
// Convert ZManaged[R, E, ZStream[R, E, A]] into ZStream[R, E, A]
val waitEvents = ZStream.unwrapManaged(
managedWatchService.mapM(_.take).map { key =>
// Use simple effect composition instead of a managed for readability.
ZStream.repeatEffect(key.pollEvents <* key.reset)
// Optional: Flatten the `List` of values that is returned
.flattenIterables
}
)
val program = waitEvents
.map(e => (e.kind(), e.context()).toString)
.foreach(putStrLn).unit
override def run(args: List[String]): ZIO[zio.ZEnv, Nothing, ExitCode] =
program
.provideLayer(Console.live ++ Blocking.live ++ Clock.live)
.exitCode
}
Also as a side note, when using ZManaged, you probably don't want to do
ZManaged.make(otherManaged.use(doSomething))(tearDown)
because you will cause the finalizers to execute out of order. ZManaged can already handle the ordering of teardown just through normal flatMap composition.
otherManaged.flatMap { other => ZManaged.make(doSomething(other))(tearDown) }
I expected that canceling Fiber makes it joinable. Here is an example of what I mean:
object TestFiber extends App {
implicit val contextShift: ContextShift[IO] =
IO.contextShift(ExecutionContext.global)
implicit val timer: Timer[IO] = IO.timer(ExecutionContext.global)
val test = for {
fiber <- IO.never.attempt.start
_ <- fiber.cancel
_ <- fiber.join
_ <- IO(println("Finished"))
} yield ()
test.unsafeRunSync() //blocks instead of printing "Finished" and exit
}
I expected that after cancel, join would immediately return and Finished would be printed.
But the actual behavior is that the program hangs up. How to make a Fiber finished so it can be joinable?
This seems to be the intended, documented behaviour of cats-effect. The ZIO equivalent to cancel is interrupt, and it returns an Exit type that will allow you to handle all possible cases (success, error, interruption) very easily, so if possible, I'd encourage you to just switch to ZIO.
But if you need to use cats-effect, the best that I could come up with (and it's not very good) is to use a Deferred:
def catchCancel[F[_]: Concurrent, A](io: F[A]): F[(F[Unit], F[Either[Option[Throwable], A]])] =
for {
deferred <- Deferred[F, Either[Option[Throwable], A]]
fiber <- (for {
a <- Concurrent[F].unit.bracketCase(_ => io) {
case (_, ExitCase.Completed) => Concurrent[F].unit
case (_, ExitCase.Error(e)) => deferred.complete(Left(Some(e))).as(())
case (_, ExitCase.Canceled) => deferred.complete(Left(None))
}
_ <- deferred.complete(Right(a))
} yield ()).start
} yield (fiber.cancel, deferred.get)
And you can use it like so:
val test = for {
x <- catchCancel[IO, Nothing](IO.never)
(cancel, getResult) = x
_ <- cancel
result <- getResult
_ <- IO(println(s"Finished $result"))
} yield ()
But seriously, why would you inflict this onto yourself? Just switch to ZIO, it fixes this problem and a boatload of others.
After migrating my Test to RC18, I get the following warning and the test is hanging:
Warning: A test is using time, but is not advancing the test clock, which may result in the test hanging. Use TestClock.adjust to manually advance the time.
I have the following test:
val testLayer: ZLayer[Live, Nothing, Loggings with Blocking with Clock] = (Console.live >>> loggings.consoleLogger) ++ Blocking.live ++ TestClock.default
testM("curl on invalid URL") {
for {
fork <- composer.curl("https://bad.xx", 1).flip.fork
_ <- TestClock.adjust(3.second * 2)
r <- fork.join
} yield
assert(r)(isSubtype[DockerComposerException](hasField("msg", _.msg.trim, equalTo("https://bad.xx could not be reached!"))))
}.provideCustomLayer(testLayer)
I suspect, that I created the Layers wrong, as this is the only change I made for the migration.
Also the standard test from the documentation failed (time was 0):
testM("One can move time very fast") {
for {
startTime <- currentTime(TimeUnit.SECONDS)
_ <- TestClock.adjust(Duration.fromScala(1 minute))
endTime <- currentTime(TimeUnit.SECONDS)
} yield assert(endTime - startTime)(isGreaterThanEqualTo(60L))
}
When I define the Layers like:
val testLayer: ZLayer[Live, Nothing, Loggings with Blocking with Clock] = (Console.live >>> loggings.consoleLogger) ++ Blocking.live ++ Clock.live
the time is not adjusted at all.
Here is the code that I want to test:
def curl(host: String, attempt: Int = 200): ZIO[Loggings with Clock, Throwable, Unit] = {
ZIO.effect(
Process(Seq("curl", "--output", "/dev/null", "--silent", "--head", "--fail", host)).!!
).flatMap(r =>
info(s"\n$host is ready to use") *> ZIO.succeed()
).catchAll(t =>
if (attempt == 0)
ZIO.fail(DockerComposerException(s"\n$host could not be reached!", Some(t)))
else
info(s"still waiting ;(") *>
ZIO.sleep(3.second) *>
curl(host, attempt - 1)
)
}
So I want to fast forward ZIO.sleep(3.seconds).
You need to call sleep(duration) after adjust(duration) to have the adjusted time reflected when you call currentTime. So the correct version of the example above is:
testM("One can move time very fast") {
for {
startTime <- currentTime(TimeUnit.SECONDS)
_ <- TestClock.adjust(1.minute)
_ <- ZIO.sleep(1.minute)
endTime <- currentTime(TimeUnit.SECONDS)
} yield assert(endTime - startTime)(isGreaterThanEqualTo(60L))
}
Note that this is reflected in the current version of the documentation here but is not yet reflected on the website because we currently only publish changes when we do a release.
However, I don't think that is your issue as it looks like your effect is just using sleep.
I'm not able to reproduce your code entirely but the following slightly simplified example works correctly:
import zio._
import zio.clock.Clock
import zio.console.Console
import zio.duration._
import zio.test._
import zio.test.environment.TestClock
object ExampleSpec extends DefaultRunnableSpec {
type Logging = Has[Logging.Service]
object Logging {
trait Service {
def logLine(line: String): UIO[Unit]
}
val live: ZLayer[Console, Nothing, Logging] =
ZLayer.fromService { console =>
new Logging.Service {
def logLine(line: String): UIO[Unit] =
console.putStrLn(line)
}
}
def logLine(line: String): ZIO[Logging, Nothing, Unit] =
ZIO.accessM(_.get.logLine(line))
}
def effect(n: Int): ZIO[Clock with Logging, String, Unit] =
if (n == 0) ZIO.fail("fail")
else Logging.logLine("retrying") *> ZIO.sleep(3.seconds) *> effect(n -1)
def spec = suite("ExampleSpec") {
testM("test") {
for {
fiber <- effect(1).flip.fork
_ <- TestClock.adjust(6.seconds)
_ <- fiber.join
} yield assertCompletes
}.provideCustomLayer(Logging.live)
}
}
Could there be something else going on in your test?