I am attempting to pass an ActorRef to a calling client. Here is some code:
object Sub {
implicit val timeout = Timeout(5 seconds)
lazy val default = {
val subActor = Akka.system.actorOf(Props[Sub], "sub")
subActor
}
def apply(pChannel: Concurrent.Channel[JsValue]):ActorRef = {
(default ? Register(callback)).map {
case ref:ActorRef => ref
}
}
}
The client invoking this is simply calling val sub:ActorRef = Sub(channel)
The problem I get here however is:
[error] found : scala.concurrent.Future[akka.actor.ActorRef]
[error] required: akka.actor.ActorRef
How can I modify the code above to get an ActorRef for the calling code to get the ref it needs?
Future is the promise of a certain value at a later time. In this case Future[ActorRef] is a value that represents an ActorRef now or at some point in the future.
You don't really want to get the ActorRef directly, you probably want to compose your calling code with the future that is returned.
For instance, if your code does:
val sub = Sub(channel)
doSomething(sub)
you'd want to rewrite it as:
Sub(channel).map { sub =>
doSomething(sub)
}
as that will create a new future that automatically calls doSomething(sub) when the sub value is available. You can also rewrite the example as:
for(sub <- Sub(channel)) yield doSomething(sub)
If you're looking to block in the calling code and return the value when available (which goes against the design principles of Akka, Play and reactive programming in general), you can always use Await, such as:
// Await.result() takes a Future[T] and returns a T
val sub = Await.result(Sub(channel), 10 seconds)
but it is poor design to do this in library code and isn't recommended. You should only wait on futures at the very end of your processing, and even then, the framework will usually handle that for you.
Related
I'm new to parallel programming and ZIO, i'm trying to get data from an API, by parallel requests.
import sttp.client._
import zio.{Task, ZIO}
ZIO.foreach(files) { file =>
getData(file)
Task(file.getName)
}
def getData(file: File) = {
val data: String = readData(file)
val request = basicRequest.body(data).post(uri"$url")
.headers(content -> "text", char -> "utf-8")
.response(asString)
implicit val backend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()
request.send().body
resquest.Response match {
case Success(value) => {
val src = new PrintWriter(new File(filename))
src.write(value.toString)
src.close()
}
case Failure(exception) => log error
}
when i execute the program sequentially, it work as expected,
if i tried to run parallel, by changing ZIO.foreach to ZIO.foreachPar.
The program is terminating prematurely, i get that, i'm missing something basic here,
any help is appreciated to help me figure out the issue.
Generally speaking I wouldn't recommend mixing synchronous blocking code as you have with asynchronous non-blocking code which is the primary role of ZIO. There are some great talks out there on how to effectively use ZIO with the "world" so to speak.
There are two key points I would make, one ZIO lets you manage resources effectively by attaching allocation and finalization steps and two, "effects" we could say are "things which actually interact with the world" should be wrapped in the tightest scope possible*.
So lets go through this example a bit, first of all, I would not suggest using the default Identity backed backend with ZIO, I would recommend using the AsyncHttpClientZioBackend instead.
import sttp.client._
import zio.{Task, ZIO}
import zio.blocking.effectBlocking
import sttp.client.asynchttpclient.zio.AsyncHttpClientZioBackend
// Extract the common elements of the request
val baseRequest = basicRequest.post(uri"$url")
.headers(content -> "text", char -> "utf-8")
.response(asString)
// Produces a writer which is wrapped in a `Managed` allowing it to be properly
// closed after being used
def managedWriter(filename: String): Managed[IOException, PrintWriter] =
ZManaged.fromAutoCloseable(UIO(new PrintWriter(new File(filename))))
// This returns an effect which produces an `SttpBackend`, thus we flatMap over it
// to extract the backend.
val program = AsyncHttpClientZioBackend().flatMap { implicit backend =>
ZIO.foreachPar(files) { file =>
for {
// Wrap the synchronous reading of data in a `Task`, but which allows runs this effect on a "blocking" threadpool instead of blocking the main one.
data <- effectBlocking(readData(file))
// `send` will return a `Task` because it is using the implicit backend in scope
resp <- baseRequest.body(data).send()
// Build the managed writer, then "use" it to produce an effect, at the end of `use` it will automatically close the writer.
_ <- managedWriter("").use(w => Task(w.write(resp.body.toString)))
} yield ()
}
}
At this point you will just have the program which you will need to run using one of the unsafe methods or if you are using a zio.App through the main method.
* Not always possible or convenient, but it is useful because it prevents resource hogging by yielding tasks back to the runtime for scheduling.
When you use a purely functional IO library like ZIO, you must not call any side-effecting functions (like getData) except when calling factory methods like Task.effect or Task.apply.
ZIO.foreach(files) { file =>
Task {
getData(file)
file.getName
}
}
I've been facing an issue the past few days regarding saving & handling data from Futures in Scala. I'm new to the language and the concept of both. Lagom's documentation on Cassandra says to implement roughly 9 files of code and I want to ensure my database code works before spreading it out over that much code.
Specifically, I'm currently trying to implement a proof of concept to send data to/from the cassandra database that lagom implements for you. So far I'm able to send and retrieve data to/from the database, but I'm having trouble returning that data since this all runs asynchronously, and also returning that the data returned successfully.
I've been playing around for a while; The retrieval code looks like this:
override def getBucket(logicalBucket: String) = ServiceCall[NotUsed, String] {
request => {
val returnList = ListBuffer[String]()
println("Retrieving Bucket " + logicalBucket)
val readingFromTable = "SELECT * FROM data_access_service_impl.s3buckets;"
//DB query
var rowsFuture: Future[Seq[Row]] = cassandraSession.selectAll(readingFromTable)
println(rowsFuture)
Await.result(rowsFuture, 10 seconds)
rowsFuture onSuccess {
case rows => {
println(rows)
for (row <- rows) println(row.getString("name"))
for (row <- rows) returnList += row.getString("name")
println("ReturnList: " + returnList.mkString)
}
}
rowsFuture onFailure {
case e => println("An error has occured: " + e.getMessage)
Future {"An error has occured: " + e.getMessage}
}
Future.successful("ReturnList: " + returnList.mkString)
}
}
When this runs, I get the expected database values to 'println' in the onSuccess callback. However, that same variable, which I use in the return statement, outside of the callback prints as empty (and returns empty data as well). This also happens in the 'insertion' function I use, where it doesn't always return variables I set within callback functions.
If I try to put the statement within the callback function, I'm given an error of 'returns Unit, expects Future[String]'. So I'm stuck where I can't return from within the callback functions, so I can't guarantee I'm returning data).
The goal for me is to return a string to the API so that it shows a list of all the s3 bucket names within the DB. That would mean iterating through the Future[Seq[Row]] datatype, and saving the data into a concatenated string. If somebody could help with that, they'll solve 2 weeks of problems I've had reading through Lagom, Akka, Datastax, and Cassandra documentation. I'm flabbergasted at this point (information overload) and there's no clearcut guide I've found on this.
For reference, here's the cassandraSession documentation:
LagomTutorial/Documentation Style Information with their only cassandra-query example
CassandraSession.scala code
The key thing to understand about Future, (and Option, and Either, and Try) is that you do not (in general) get values out of them, you bring computations into them. The most common way to do that is with the map and flatMap methods.
In your case, you want to take a Seq[Row] and transform it into a String. However, your Seq[Row] is wrapped in this opaque data structure called Future, so you can't just rows.mkString as you would if you actually had a Seq[Row]. So, instead of getting the value and performing computation on it, bring your computation rows.mkString to the data:
//DB query
val rowsFuture: Future[Seq[Row]] = cassandraSession.selectAll(readingFromTable)
val rowToString = (row: Row) => row.getString("name")
val computation = (rows: Seq[Row]) => rows.map(rowToString).mkString
// Computation to the data, rather than the other way around
val resultFuture = rowsFuture.map(computation)
Now, when rowsFuture is completed, the new future that you created by calling rowsFuture.map will be fulfilled with the result of calling computation on the Seq[Row] that you actually care about.
At that point you can just return resultFuture and everything will work as anticipated, because the code that calls getBucket is expecting a Future and will handle it as is appropriate.
Why is Future opaque?
The simple reason is because it represents a value that may not currently exist. You can only get the value when the value is there, but when you start your call it isn't there. Rather than have you poll some isComplete field yourself, the code lets you register computations (callbacks, like onSuccess and onFailure) or create new derived future values using map and flatMap.
The deeper reason is because Future is a Monad and monads encompass computation, but do not have an operation to extract that computation out of them
Replace the select for your specific select and the field that you want to obtain for your specific field.The example is only for test, is not a architecture propose.
package ldg.com.dbmodule.model
/**
* Created by ldipotet on 05/11/17.
*/
import com.datastax.driver.core.{Cluster, ResultSet, ResultSetFuture}
import scala.util.{Failure, Success, Try}
import java.util.concurrent.TimeUnit
import scala.collection.JavaConversions._
//Use Implicit Global Context is strongly discouraged! we must create
//our OWN execution CONTEXT !
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{Future, _}
import scala.concurrent.duration._
object CassandraDataStaxClient {
//We create here a CallBack in Scala with the DataStax api
implicit def resultSetFutureToScala(rf: ResultSetFuture):
Future[ResultSet] = {
val promiseResult = Promise[ResultSet]()
val producer = Future {
getResultSet(rf) match {
//we write a promise with an specific value
case Success(resultset) => promiseResult success resultset
case Failure(e) => promiseResult failure (new
IllegalStateException)
}
}
promiseResult.future
}
def getResultSet: ResultSetFuture => Try[ResultSet] = rsetFuture => {
Try(
// Other choice can be:
// getUninterruptibly(long timeout, TimeUnit unit) throws
TimeoutException
// for an specific time
//can deal an IOException
rsetFuture.getUninterruptibly
)
}
def main(args: Array[String]) {
def defaultFutureUnit() = TimeUnit.SECONDS
val time = 20 seconds
//Better use addContactPoints and adds more tha one contact point
val cluster = Cluster.builder().addContactPoint("127.0.0.1").build()
val session = cluster.connect("myOwnKeySpace")
//session.executeAsync es asyncronous so we'll have here a
//ResultSetFuture
//converted to a resulset due to Implicitconversion
val future: Future[ResultSet] = session.executeAsync("SELECT * FROM
myOwnTable")
//blocking on a future is strongly discouraged!! next is an specific
//case
//to make sure that all of the futures have been completed
val results = Await.result(future,time).all()
results.foreach(row=>println(row.getString("any_String_Field"))
session.close()
cluster.close()
}
}
Im using Play and have an action in which I want to do two things:-
firstly check my cache for a value
secondly, call a web service with the value
Since WS API returns a Future, I'm using Action.async.
My Redis cache module also returns a Future.
Assume I'm using another ExecutionContext appropriately for the potentially long running tasks.
Q. Can someone confirm if I'm on the right track by doing the following. I know I have not catered for the Exceptional cases in the below - just keeping it simple for brevity.
def token = Action.async { implicit request =>
// 1. Get Future for read on cache
val cacheFuture = scala.concurrent.Future {
cache.get[String](id)
}
// 2. Map inside cache Future to call web service
cacheFuture.map { result =>
WS.url(url).withQueryString("id" -> result).get().map { response =>
// process response
Ok(responseData)
}
}
}
My concern is that this may not be the most efficient way of doing things because I assume different threads may handle the task of completing each of the Futures.
Any recommendations for a better approach are greatly appreciated.
That's not specific to Play. I suggest you have a look at documentations explaining how Futures work.
val x: Future[FutureOp2ResType] = futureOp1(???).flatMap { res1 => futureOp2(res1, ???) }
Or with for-comprehension
val x: Future[TypeOfRes] = for {
res1 <- futureOp1(???)
res2 <- futureOp2(res1, ???)
// ...
} yield res
As for how the Futures are executed (using threads), it depends on which ExecutionContext you use (e.g. the global one, the Play one, ...).
WS.get returning a Future, it should not be called within cacheFuture.map, or it will returns a Future[Future[...]].
Given that we must avoid...
1) Modifying state
2) Blocking
...what is a correct end-to-end usage for a Future?
The general practice in using Futures seems to be transforming them into other Futures by using map, flatMap etc. but it's no good creating Futures forever.
Will there always be a call to onComplete somewhere, with methods writing the result of the Future to somewhere external to the application (e.g. web socket; the console; a message broker) or is there a non-blocking way of accessing the result?
All of the information on Futures in the Scaladocs - http://docs.scala-lang.org/overviews/core/futures.html seem to end up writing to the console. onComplete doesn't return anything, so presumably we have to end up doing some "fire-and-forget" IO.
e.g. a call to println
f onComplete {
case Success(number) => println(number)
case Failure(err) => println("An error has occured: " + err.getMessage)
}
But what about in more complex cases where we want to do more with the result of the Future?
As an example, in the Play framework Action.async can return a Future[Result] and the framework handles the rest. Will it eventually have to expect never to get a result from the Future?
We know the user needs to be returned a Result, so how can a framework do this using only a Unit method?
Is there a non-blocking way to retrieve the value of a future and use it elsewhere within the application, or is a call to Await inevitable?
Best practice is to use callbacks such as onComplete, onSuccess, onFailure for side effecting operations, e.g. logging, monitoring, I/O.
If you need the continue with the result of of your Future computation as opposed to do a side-effecting operation, you should use map to get access to the result of your computation and compose over it.
Future returns a unit, yes. That's because it's an asynchronous trigger. You need to register a callback in order to gather the result.
From your referenced scaladoc (with my comments):
// first assign the future with expected return type to a variable.
val f: Future[List[String]] = Future {
session.getRecentPosts
}
// immediately register the callbacks
f onFailure {
case t => println("An error has occurred: " + t.getMessage)
}
f onSuccess {
case posts => for (post <- posts) println(post)
}
Or instead of println-ing you could do something with the result:
f onSuccess {
case posts: List[String] => someFunction(posts)
}
Try this out:
import scala.concurrent.duration._
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
val f: Future[Int] = Future { 43 }
val result: Int = Await.result(f, 0 nanos)
So what is going on here?
You're defining a computation to be executed on a different thread.
So you Future { 43 } returns immediately.
Then you can wait for it and gather the result (via Await.result) or define computation on it without waiting for it to be completed (via map etc...)
Actually, the kind of Future you are talking about are used for side-effects. The result returned by a Future depends its type :
val f = Future[Int] { 42 }
For example, I could send the result of Future[Int] to another Future :
val f2 = f.flatMap(integer => Future{ println(integer) }) // may print 42
As you know, a future is a process that happens concurrently. So you can get its result in the future (that is, using methods such as onComplete) OR by explicitly blocking the current thread until it gets a value :
import scala.concurrent.Await
import akka.util.Timeout
import scala.concurrent.duration._
implicit val timeout = Timeout(5 seconds)
val integer = Await.result(Future { 42 }, timeout.duration)
Usually when you start dealing with asynchronous processes, you have to think in terms of reactions which may never occur. Using chained Futures is like declaring a possible chain of events which could be broken at any moment. Therefore, waiting for a Future's value is definitely not a good practice as you may never get it :
val integer = Await.result(Future { throw new RuntimeException() }, timeout.duration) // will throw an uncaught exception
Try to think more in terms of events, than in procedures.
Is it appropriate to use Futures and Promises for delayed initialization, rather than using an Option var or some mutable variable?
You could create a factory class that encapsulates the promise:
class IntFactory{
val intPromise = Promise[Int]
def create () : Future[Int] = intPromise.future
def init (data : String) : Unit = intPromise success data.length
}
An actor or some other class could then use it like this:
class MyActor(factory : IntFactory) extends Actor{
val future_int = factory.create()
def receive = {
case (msg : String) => factory.init(msg) // Now the promise is fulfilled
}
}
Is there anything wrong with doing something like this? It may not have been ideal to use an actor as an example, as I think there are better alternatives for actors (become or FSM). I am currently considering using this with a non-actor class. Some of the instance variables are nothing until certain events occur. I was considering doing this instead of using a var Option and setting it to None. If this is bad, what are some other alternatives?
EDIT:
I thought of situations where this might be more useful. If I had multiple things that needed to be initialized, and I had some async action that I wanted to perform when it was all done:
class MyActor(factory1 : IntFactory, factory2 : IntFactory) extends Actor{
val future_int1 = factory1.create()
val future_int2 = factory2.create()
for{
x <- future_int1
y <- future_int2
} // Do some stuff when both are complete
def receive = {
case "first" => factory1.init("first")
case "second" => factory2.init("second")
}
}
Then I would not have to check which ones are None every time I get another piece.
MORE EDITS:
Some additional information that I failed to specify in my original question:
The data needed to initialize the objects will come in asynchronously.
The data passed to the init function is required for initialization. I edited my example code so that this is now the case.
I am not using Akka. I thought Akka would be helpful for throwing together a quick example and thought that experienced Akka people could provide useful feedback.
Yes, this is certainly a better approach than using mutable variables (whether Option or not). Using lazy val, as #PatrykĆwiek suggested, is even better if you can initialize the state at any time instead of waiting for external events and don't need to do it asynchronously.
Judging from your IntFactory, you don't really need the data string (it's not used anywhere), so I think the base case could be rewritten like this:
class Foo {
lazy val first = {
Thread.sleep(2000) // Some computation, initialization etc.
25
}
lazy val second = {
Thread.sleep(1000) // Some computation, initialization etc.
11
}
def receive(s : String) = s match {
case "first" => first
case "second" => second
case _ => -1
}
}
now, let's say you do this:
val foo = new Foo()
println(foo.receive("first")) // waiting for 2 seconds, initializing
println(foo.receive("first")) // returns immediately
println(foo.receive("second")) // waiting for 1 second, initializing
Now both first and second can be initialized at most once.
You can't pass parameters to lazy vals, so if the data string is somehow important to the initialization, then you'd probably be better off using factory method with memoization (IMO).