private class FutMemorizer[T](valid: T => Boolean)(f: () => Future[T]) {
private val ref = new AtomicReference[Promise[T]]
#scala.annotation.tailrec
final def get(): Future[T] = {
val nullableRef = ref.get()
val valid = checkPromise(ref.get())
if(valid) {
nullableRef.future
} else {
val p = Promise[T]
val success = ref.compareAndSet(nullableRef, p)
if(success) {
p.completeWith(f())
p.future
} else {
get()
}
}
}
private def checkPromise(nullable: Promise[T]) = {
nullable != null && {
nullable.future.value match {
case None => true // future is not complete all caller should wait
case Some(Success(v)) => valid(v)
case _ => false
}
}
}
}
I am implementing an Future memorizer that only cache a valid future value.
It must meet following requirements
Futures created by f never executed paralleled
get never return a invalid value (once invalid recall f() to reload)
Is my implementation correct ?
Is there a more functional or simpler way to do this (because I hardly prove correntness of mime)?
As far as I understand this is wrong:
p.completeWith(f())
The caller gets a future the value of which is (or will be sometimes) that of the future returned by f(), but it's not checked anywhere that this value satisfies or will satisfy valid(...); same for other callers that came while the resulting future returned by f() is in progress if it takes time. It's only when result of f() completes will the next caller probably start "fixing" it.
I would probably go about fixing this problem the following way (see the fixed method), with some stylistic changes:
class FutMemorizer[T](valid: T => Boolean)(f: () => Future[T]) {
private val ref = new AtomicReference[Future[T]]
#tailrec
final def get: Future[T] = {
val current = ref.get
if (current != null && isValid(current)) current
else {
val p = Promise[T]
val pf = p.future
if (ref.compareAndSet(current, pf)) {
p.completeWith(fixed(f))
pf
} else get
}
}
private def fixed(f: () => Future[T]): Future[T] =
f() flatMap { t =>
if (valid(t)) Future.successful(t) else fixed(f)
}
private def isValid(future: Future[T]) =
future.value match {
case None => true // future is not complete all caller should wait
case Some(Success(v)) => valid(v)
case _ => false
}
}
As for your question about a more functional way of doing it I guess f and valid having effects on the external state and basing their computations on it (which I guess is the point of having a memorizer with invalidation) would seriously hinder it.
Just find spray-cache already have this feature
Related
I need to execute a Future method on some elements I have in a list simultaneously. My current implementation works sequentially, which is not optimal for saving time. I did this by mapping my list and calling the method on each element and processing the data this way.
My manager shared a link with me showing how to execute Futures simultaneously using for-comprehension but I cannot see/understand how I can implement this with my List.
The link he shared with me is https://alvinalexander.com/scala/how-use-multiple-scala-futures-in-for-comprehension-loop/
Here is my current code:
private def method1(id: String): Tuple2[Boolean, List[MyObject]] = {
val workers = List.concat(idleWorkers, activeWorkers.keys.toList)
var ready = true;
val workerStatus = workers.map{ worker =>
val option = Await.result(method2(worker), 1 seconds)
var status = if (option.isDefined) {
if (option.get._2 == id) {
option.get._1.toString
} else {
"INVALID"
}
} else "FAILED"
val status = s"$worker: $status"
if (option.get._1) {
ready = false
}
MyObject(worker.toString, status)
}.toList.filterNot(s => s. status.contains("INVALID"))
(ready, workerStatus)
}
private def method2(worker: ActorRef): Future[Option[(Boolean, String)]] = Future{
implicit val timeout: Timeout = 1 seconds;
Try(Await.result(worker ? GetStatus, 1 seconds)) match {
case Success(extractedVal) => extractedVal match {
case res: (Boolean, String) => Some(res)
case _ => None
}
case Failure(_) => { None }
case _ => { None }
}
}
If someone could suggest how to implement for-comprehension in this scenario, I would be grateful. Thanks
For method2 there is no need for the Future/Await mix. Just map the Future:
def method2(worker: ActorRef): Future[Option[(Boolean, String)]] =
(worker ? GetStatus).map{
case res: (Boolean, String) => Some(res)
case _ => None
}
For method1 you likewise need to map the result of method2 and do the processing inside the map. This will make workerStatus a List[Future[MyObject]] and means that everything runs in parallel.
Then use Future.sequence(workerStatus) to turn the List[Future[MyObject]] into a Future[List[MyObject]]. You can then use map again to do the filtering/ checking on that List[MyObject]. This will happen when all the individual Futures have completed.
Ideally you would then return a Future from method1 to keep everything asynchronous. You could, if absolutely necessary, use Await.result at this point which would wait for all the asynchronous operations to complete (or fail).
I wrote simple callback(handler) function which i pass to async api and i want to wait for result:
object Handlers {
val logger: Logger = Logger("Handlers")
implicit val cs: ContextShift[IO] =
IO.contextShift(ExecutionContext.Implicits.global)
class DefaultHandler[A] {
val response: IO[MVar[IO, A]] = MVar.empty[IO, A]
def onResult(obj: Any): Unit = {
obj match {
case obj: A =>
println(response.flatMap(_.tryPut(obj)).unsafeRunSync())
println(response.flatMap(_.isEmpty).unsafeRunSync())
case _ => logger.error("Wrong expected type")
}
}
def getResponse: A = {
response.flatMap(_.take).unsafeRunSync()
}
}
But for some reason both tryPut and isEmpty(when i'd manually call onResult method) returns true, therefore when i calling getResponse it sleeps forever.
This is the my test:
class HandlersTest extends FunSuite {
test("DefaultHandler.test") {
val handler = new DefaultHandler[Int]
handler.onResult(3)
val response = handler.getResponse
assert(response != 0)
}
}
Can somebody explain why tryPut returns true, but nothing puts. And what is the right way to use Mvar/channels in scala?
IO[X] means that you have the recipe to create some X. So on your example, yuo are putting in one MVar and then asking in another.
Here is how I would do it.
object Handlers {
trait DefaultHandler[A] {
def onResult(obj: Any): IO[Unit]
def getResponse: IO[A]
}
object DefaultHandler {
def apply[A : ClassTag]: IO[DefaultHandler[A]] =
MVar.empty[IO, A].map { response =>
new DefaultHandler[A] {
override def onResult(obj: Any): IO[Unit] = obj match {
case obj: A =>
for {
r1 <- response.tryPut(obj)
_ <- IO(println(r1))
r2 <- response.isEmpty
_ <- IO(println(r2))
} yield ()
case _ =>
IO(logger.error("Wrong expected type"))
}
override def getResponse: IO[A] =
response.take
}
}
}
}
The "unsafe" is sort of a hint, but every time you call unsafeRunSync, you should basically think of it as an entire new universe. Before you make the call, you can only describe instructions for what will happen, you can't actually change anything. During the call is when all the changes occur. Once the call completes, that universe is destroyed, and you can read the result but no longer change anything. What happens in one unsafeRunSync universe doesn't affect another.
You need to call it exactly once in your test code. That means your test code needs to look something like:
val test = for {
handler <- TestHandler.DefaultHandler[Int]
_ <- handler.onResult(3)
response <- handler.getResponse
} yield response
assert test.unsafeRunSync() == 3
Note this doesn't really buy you much over just using the MVar directly. I think you're trying to mix side effects inside IO and outside it, but that doesn't work. All the side effects need to be inside.
No need to pay attention to the purpose of the function here, it's only for demonstration:
def readAllByPersonOrFail(person: Person, otherPersonId: Long): Future[List[Person]] = {
val personSiblingsFuture: Future[List[Person]] = personSiblingsDomain.readAllByPersonId(person.id)
personSiblingsFuture.map { persons =>
persons.find(_.id == otherPersonId) match {
case Some(person) =>
person.isActive match {
case true => person
case false => throw new IllegalArgumentException("something inactive")
}
case None => throw new IllegalArgumentException("something wrong ehre")
}
}
personSiblingsFuture
}
I would like to return personSiblingsFuture above iff it validates (makes sure correct person is in the list and is active), otherwise throw the exception. I don't think the above code is doing the right thing as it is not existing upon failure.
Take a look at scala.concurrent.Future.map. This creates a new future, whose value is resolved by applying a function to the successful result of this future.
Note that here you're throwing away the resulting future you just created with .map() too.
There are a few areas to solve your problem, though you should question more deeply the use of exceptions with Futures. Scala provides concepts like Future, Option, and Try specifically to avoid throwing exceptions and have a clearer control flow.
Option 1, return the mapped future
In your funciton,
def func(...): Future[List[Person]] {
val personSiblingsFuture = ...;
personSiblingsFuture.map { persons =>
...
}
}
// note we're not returning personSiblingsFuture,
// but the mapped result
When someone actually tries to get the value of the future, e.g. by using .value, they might see an exception intead:
def main() {
val future = func(...); // this is fine
val my_list = future.value; // awaits Future, might throw here
}
Option 2, actually await the list and throw in the function
Returning a future that might throw is strange, it might be a bit easier if the you actually explicitly a had a function that might throw, e.g.
/** jsdoc describing function **/
def funcMightThrow(...): List[Person] {
val personSiblingsFuture = ...;
val personSiblings = personSiblingsFuture.value;
personSiblings.find(_.id == otherPersonId) match {
case Some(person) =>
person.isActive match {
case true => personSiblings
case false => throw new IllegalArgumentException("something inactive")
}
case None => throw new IllegalArgumentException("something wrong ehre")
}
}
Option 3, consider making return types more explicit
def func(...): Future[Try[List[Person]]] {
val personSiblingsFuture = ...;
personSiblingsFuture.map { persons =>
...
// successful case returns 'persons' (List[Person])
// fail cases return Failure(...) instead
}
} // return the mapped future
You can also return Try[List[Person]] rather than a Future[] of that, by using .value, which makes func a blocking function.
val value = authenticateUser
private def authenticateUser = {
val holder = WS.url(platformUrl + "/userApi/auth/login?username=testuser&password=testPass")
val res = holder.post(Results.EmptyContent()).onComplete {
case Success(response) => response.cookies.map{cookie => println(cookie.value.get)}
case Failure(errors) => println("")
// The `Future` failed.
}
}
How to return cookie.value.get from authenticateUser method?
First of all, you seem to have many cookies, so it's not clear, the value of which one you want to return.
More importantly, you should not actually return the value itself. The idea is to return the Future, that can be further transformed downstream:
def authenticateUser = WS.url(..)
.post(Results.EmptyContent)
.map { _.cookies }
.map { _.find(_.name == "Auth").flatMap(_.value).getOrElse("") }
Now, somewhere downstream, you can have something like:
def serve(request: Request): Future[Response] = authenticateUser
.map { user =>
serveRequest(user, request)
}.map { result => makeResponse(result) }
I don't know the specifics of Play, so, consider the Request/Response stuff "pseudocode", but the basic idea is that your entire processing should be a chain of Future transformations. The general rule is to never block the service thread.
def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.flatMap(x => {
println(x) // logging the value of x
xFuture
})
}
This is the way I'm currently doing it. To provide more context:
This function is being called inside an API when a request is made and I'd like the log message to be printed just before the value computed in the request is returned. Which is why, the following is not a good solution for me:
def returnFuture[A](x: A): Future[A] = {
val xFuture = Future { x } // suppose an API call that returns a future
xFuture.map(x => {
println(x) // logging the value of x
})
xFuture
}
Logging is a side-effect, meaning that you don't want the operation to fail if the logging fails for any reason (e.g. a call to toString throwing NPE).
Future#andThen is perfect for this use case. From the docs:
Applies the side-effecting function to the result of this future, and returns a new future with the result of this future.
This method allows one to enforce that the callbacks are executed in a specified order.
Note that if one of the chained andThen callbacks throws an exception, that exception is not propagated to the subsequent andThen callbacks. Instead, the subsequent andThen callbacks are given the original value of this future.
Your example becomes:
def returnFuture[A](x: A): Future[A] = {
Future { x } // suppose an API call that returns a future
.andThen { case Success(v) => println(v) }
}
You can use onComplete callback:
def returnFuture[A](x: A): Future[A] = {
val f = Future { x }
f.onComplete(println)
f
}
A map will work too:
def returnFuture[A](x: A): Future[A] = {
Future { x }.map { v =>
println(v)
v
}
}
Keep in mind that the whole point of using Futures is that you are trying to avoid blocking and that you don't control exactly when the Future will be executed. So, if you want more detailed logs while keeping the asynchronous nature of a Future, do something like this:
def doSomething(param: String): String = {
// log something here
val result = param.toUpperCase
// log something else here
result
}
def asFuture(param: String) = Future {
doSomething(param)
}
In other words, if this is an option, add logs to the x operation instead.