Calling slow future returning function in scala for each item in a Seq - scala

I have this sample code currently but the issue is that slowFunction() below isn't running at all.
Is it possible to trigger the next function call after the function call on the previous item returns? It doesn't really matter as long as the function gets executed at least once for each item but thought it would help with the load on the system.
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.{Failure, Success}
val inputsFromDb = Future { Seq(1, 2, 3) }
def slowFunction(input: Int): Future[Boolean] = {
Future {
Thread.sleep(1000)
println(s"Returning from slow function for input $input")
true
}
}
val results: Future[Boolean] = inputsFromDb.flatMap { inputs =>
val updateResults = inputs.map { input =>
println(s"Calling slow function for input $input")
slowFunction(input)
}
Future.sequence(updateResults).map(_ => true)
}
results // Future(<not completed>)
Here's the output if I run the code as is.
Calling slow function for input 1
Calling slow function for input 2
Calling slow function for input 3
However it runs fine when I comment out the sleep part Thread.sleep(1000)
Calling slow function for input 1
Calling slow function for input 2
Calling slow function for input 3
Returning from slow function for input 3
Returning from slow function for input 2
Returning from slow function for input 1
If I can't get this working, I might just add a delay between each call to slowFuction by using java.util.Timer

It looks like your program (or worksheet) is terminating before all the Futures reach completion.
Your code produces the expected output if, at some point before the program exits, you Await until results completes.
import scala.concurrent.duration._
scala.concurrent.Await.result(results, 999.millis)
As demonstrated in this Scastie session.

Related

fs2.Stream[IO, Something] does not return on take(1)

I have a function that returns a fs2.Stream of Measurements.
import cats.effect._
import fs2._
def apply(sds: SerialPort, interval: Int)(implicit cs: ContextShift[IO]): Stream[IO, SdsMeasurement] =
for {
blocker <- Stream.resource(Blocker[IO])
stream <- io.readInputStream(IO(sds.getInputStream), 1, blocker)
.through(SdsStateMachine.collectMeasurements())
} yield stream
Normally it is an infinite Stream, unless I pass it a test flag, in which case it should output one value and halt.
val infiniteSource: Stream[IO, SdsMeasurement] = ...
val source = if (isTest) infiniteSource.take(1) else infiniteSource
source.compile.drain
The infinite Stream works fine. It gives me all Measurements infinitely. The test Stream indeed gives me only the first measurement, nothing more. The problem I have is that the Stream does not return after this last measurements. It blocks forever. What am I doing wrong?
Note: I think I abstracted the essential code, but for more context, please take a look at my project: https://github.com/jkransen/fijnstof/blob/ZIO/src/main/scala/nl/kransen/fijnstof/Main.scala
The code you've presented here looks fine, I don't think the issue lies within that code. If it blocks, then presumably one of the underlying APIs blocks, for instance it might be the close method of the InputStream. What I usually do in such situations is to add log statements before and after every function call that might block.

Is map of Future lazy or not?

Basically I mean:
for(v <- Future(long time operation)) yield v*someOtherValue
This expression returns another Future, but the question is, is the v*someOhterValue operation lazy or not? Will this expression block on getting the value of Future(long time operation)?
Or it is like a chain of callbacks?
A short experiment can test this question.
import concurrent._;
import concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
object TheFuture {
def main(args: Array[String]): Unit = {
val fut = for (v <- Future { Thread.sleep(2000) ; 10 }) yield v * 10;
println("For loop is finished...")
println(Await.ready(fut, Duration.Inf).value.get);
}
}
If we run this, we see For loop is finished... almost immediately, and then two seconds later, we see the result. So the act of performing map or similar operations on a future is not blocking.
A map (or, equivalently, your for comprehension) on a Future is not lazy: it will be executed as soon as possible on another thread. However, since it runs on another thread, it isn't blocking, either.
If you want to do the definition and execution of the Future separately, then you have to use something like a Monix Task.
https://monix.io/api/3.0/monix/eval/Task.html

Unpacking Future[Option[MyType]] in Scala

I am new to Scala and Play Framework so I am not quite sure what is wrong. I am trying to unpack a Future[Option[MyType]] given by a Slick DB controler (Play Framework). MyType is called BoundingBox in the code:
def getBoundingBoxByFileName(name: String) = {
val selectByName = boundingBoxTableQuery.filter{ boundingBoxTable =>
boundingBoxTable.name === name
}
db.run(selectByName.result.headOption)
}
BoundingBox type has a field called product_name. To retrieve this field I do the following:
val boundingBoxFutOpt = BoundingBoxQueryActions.getBoundingBoxByFileName("some_file")
val res = for {
optBb : Option[db.BoundingBox] <- boundingBoxFutOpt
} yield{
for(bb : db.BoundingBox <- optBb) yield {
println(s"${bb.product_name}")
}
}
This code does not yield anything on the output, though I have no compilation errors. If I change the println statement for some random text (not using the bb reference), it is also not printed on the console. To me it seems that the println statement is never executed.
I'll appreciate some directions on this problem.
It's likely that your program is terminating before the future has a chance to run the println. I think this will get you what you want:
import scala.concurrent.Await
import scala.concurrent.duration.Duration
// your code here
Await.result(res, Duration.Inf)
In your above example you're running a thread but then not giving it a chance to finish execution. The above will block until the future is complete.
It's worth nothing that you shouldn't use Await in production code as the blocking done negates the value of having code run in a separate thread.

Should a Scala process be marked scala.concurrent.blocking within a Scala Future?

I have created a Scala process, I read that this is blocking, should it be wrapped in a scala.concurrent.blocking? I'm confused, since the Process is going to call back to the log parameter here which would require this thread to do something I guess, so its not completely blocking?
import scala.sys.process.Process
import scala.concurrent.Future
import scala.sys.process.ProcessLogger
def something = Future {
val log = ProcessLogger(normal => {
lines = lines :+ normal
Logger.info("SFTPline added: " + normal)
},
error => {
Logger.info("SFTPERROR: " + error)
})
val success = scala.concurrent.blocking {
val result = Process(command).!(log)
lines.mkString("").contains("\"success\":true") && (result == 0)
}
}
Scala Process is actually get execute by the run function which return Process and exteciute it without blocking. the '!' is function that call the 'run' function and then call the exitValue() function of the Process. the 'exitValue()' Blocks until proccess exits)
calling Process(command).!(log) is actually same as run(log).exitValue() (well, not exactly but you can get the idea... see ProcessBuilderImpl.scala)
If you don't want to be blocked just call the run function instead.
Given that what you're returning is not a Future, your code must be blocking somewhere (I don't see how your example is "within a scala Future") - if you call a function that calls a callback, but then block waiting for that callback to be called, you're still blocking. So yes, declare it blocking.

future take a long time to run, why not return immediately?

I have the following code,and I expected a.success(burncpu(14969)) to return instantly since it's run in a future, but why it took a long time to run.
import scala.concurrent._
val a=Promise[Unit]()
// why the following took long time here, shouldn't it be in async mode, and return very quick
a.success(burncpu(14969))
a.future
def burncpu(a:Int):Int = {
val point=new Date().getTime()
while ((new Date()).getTime()-point< a) {
a
}
a
}
You are using the Promise wrong.
a.success method completes the promise with given argument, it doesn't run the expression you pass to it asynchronously.
What you probably want to do is something like this:
val f = Future(burncpu(6000))
Assuming you have an ExecutionContext available (if you don't, you can do import ExecutionContext.Implicits.global), this will construct a Future which will run your function asynchronously.
You can see how it works in Scala REPL (f.value returns None until the method has returned)
scala> val f = Future(burncpu(6000))
f: scala.concurrent.Future[Int] = scala.concurrent.impl.Promise$DefaultPromise#4d4d8fcf
scala> f.value
res27: Option[scala.util.Try[Int]] = None
scala> f.value
res28: Option[scala.util.Try[Int]] = None
scala> f.value
res29: Option[scala.util.Try[Int]] = Some(Success(6000))
Promise.success is not executed asynchronously, basically your code is the equivalent of:
Future.successful(burncpu(14969))
You could try this:
Future {
burncpu(14969)
}
This will call Future.apply and execute your function asynchronously.
The call to future is indeed called asynchronously. However, nothing in the API suggests that completing the Promise, i.e. calling success in your case, will be executed asynchronously. That is your responsibility to ensure.
It makes sense when you consider what a Promise is - a "marker" to another execution path, notifying it that a calculation has been completed and a result (or failure) is available.