i have a problem. I'm trying to do some scala tests. My problem is that I don't know how to take the return of a method to test whether it returns a sequence to me or not. How can i do it?
My Test class is: (It works)
"The game actor" should {
"accept a specific numbers of players and notify that the game is started with an initial state" in {
val gameActor = TestActorRef[GameMatchActor](GameMatchActor.props(NUMBER_OF_PLAYERS))
val player1 = TestProbe()
val player2 = TestProbe()
gameActor ! GamePlayers(Seq(GamePlayer("id1", "player1", player1.ref), GamePlayer("id2", "player2", player2.ref)))
player1.expectMsgType[MatchFound]
player1.send(gameActor, PlayerReadyServer("id1", player1.ref))
player2.expectMsgType[MatchFound]
player2.send(gameActor, PlayerReadyServer("id2", player2.ref))
player1.expectMsgType[GamePlayersClient]
player2.expectMsgType[GamePlayersClient]
}
}
And the method that return me GamePlayersClient is:
private def defineRoles(players: Seq[GamePlayer]): Seq[Player] = {
var playersRole: Seq[Player] = Seq()
val rand1 = Random.nextInt(players.length)
val rand2 = Random.nextInt(players.length)
for (n <- 0 until players.length) {
n match {
case n if n == rand1 || n == rand2 =>
playersRole = playersRole :+ Impostor(players(n).id, players(n).username, Point2D(0,0))
case _ => playersRole = playersRole :+ Crewmate(players(n).id, players(n).username, Point2D(0,0))
}
}
playersRole
}
And:
// watch the players with the new actor ref
val playersRole = defineRoles(players)
this.players.foreach(p => {
p.actorRef ! GamePlayersClient(playersRole)
context.watch(p.actorRef)
})
So, how can i take GamePlayersClient(playersRole) and search if inside there's a Sequence of Players and one of they is a "Crewmate". Thanks
You can use one of the receive... methods on TestProbe to collect the GamePlayersClient messages sent to the probe and then use ScalaTest's matchers to assert things about those messages.
For example, you could replace
player1.expectMsgType[GamePlayersClient]
with
val gpcMessageOpt1: Option[GamePlayersClient] = Option(player1.receiveN(1)).flatMap {
case g: GamePlayersClient => Some(g)
case _ => None
}
gpcMessageOpt1 shouldNot be(empty) // effectively the same as the expectMessage
If the field of interest is playersRole, then you might have
// To my mind, it's generally OK to do unsafe things like get in a test...
gpcMessageOpt1.get.playersRole shouldNot be(empty)
Related
I need to execute a Future method on some elements I have in a list simultaneously. My current implementation works sequentially, which is not optimal for saving time. I did this by mapping my list and calling the method on each element and processing the data this way.
My manager shared a link with me showing how to execute Futures simultaneously using for-comprehension but I cannot see/understand how I can implement this with my List.
The link he shared with me is https://alvinalexander.com/scala/how-use-multiple-scala-futures-in-for-comprehension-loop/
Here is my current code:
private def method1(id: String): Tuple2[Boolean, List[MyObject]] = {
val workers = List.concat(idleWorkers, activeWorkers.keys.toList)
var ready = true;
val workerStatus = workers.map{ worker =>
val option = Await.result(method2(worker), 1 seconds)
var status = if (option.isDefined) {
if (option.get._2 == id) {
option.get._1.toString
} else {
"INVALID"
}
} else "FAILED"
val status = s"$worker: $status"
if (option.get._1) {
ready = false
}
MyObject(worker.toString, status)
}.toList.filterNot(s => s. status.contains("INVALID"))
(ready, workerStatus)
}
private def method2(worker: ActorRef): Future[Option[(Boolean, String)]] = Future{
implicit val timeout: Timeout = 1 seconds;
Try(Await.result(worker ? GetStatus, 1 seconds)) match {
case Success(extractedVal) => extractedVal match {
case res: (Boolean, String) => Some(res)
case _ => None
}
case Failure(_) => { None }
case _ => { None }
}
}
If someone could suggest how to implement for-comprehension in this scenario, I would be grateful. Thanks
For method2 there is no need for the Future/Await mix. Just map the Future:
def method2(worker: ActorRef): Future[Option[(Boolean, String)]] =
(worker ? GetStatus).map{
case res: (Boolean, String) => Some(res)
case _ => None
}
For method1 you likewise need to map the result of method2 and do the processing inside the map. This will make workerStatus a List[Future[MyObject]] and means that everything runs in parallel.
Then use Future.sequence(workerStatus) to turn the List[Future[MyObject]] into a Future[List[MyObject]]. You can then use map again to do the filtering/ checking on that List[MyObject]. This will happen when all the individual Futures have completed.
Ideally you would then return a Future from method1 to keep everything asynchronous. You could, if absolutely necessary, use Await.result at this point which would wait for all the asynchronous operations to complete (or fail).
I am looking to expand the following code to work for an unknown amount of actor ask requests.
implicit val timeout = Timeout(100 millis)
val sendRequestActor = context.actorOf(Props(new SendRequest(request)), "Send_Request_".concat(getActorNumber))
val sendRequestActor2 = context.actorOf(Props(new SendRequest(request)), "Send_Request_".concat(getActorNumber))
val a1 = ask(sendRequestActor, Request).fallbackTo(Future.successful(RequestTimeout))
val a2 = ask(sendRequestActor2, Request).fallbackTo(Future.successful(RequestTimeout))
val result = for {
r1 <- a1
r2 <- a2
} yield(r1, r2)
val r = Await.result(result, 100 millis)
r match {
case (b: SuccessResponse, b2: SuccessResponse) => {
//Process Results
}
case (b: SuccessResponse, b2: RequestTimeout) => {
//Process Results
}
case (b: RequestTimeout, b2: SuccessResponse) => {
//Process Results
}
case (b: RequestTimeout, b2: RequestTimeout) => {
//Process Results
}
case _ => {}
}
I am trying to send out requests to a List of recipients(gotten from a previous database call). The number of recipients will vary each time this function is called. Recipients have a maximum of 100 milliseconds to respond before I want to time out their requests and record a RequestTimeout. The SendRequest actor will reply with SuccessResponse if the recipients respond. I am assuming I will have to change the val result for-loop to process a list, but I am unsure of how to structure everything so that I will wait the minimum amount of time(either when all actors return or when the timeout hits, whichever is lower). I do not need everything in a single return value like the example, I am fine with a list of results and matching type on each iteration.
Any help would be appreciated, please let me know if I can provide any other information.
Thank you
Edit:
Calling Class:
case object GetResponses
def main(args: Array[String]) {
val route = {
get {
complete {
//stuff
val req_list = List(req1,req2,req3)
val createRequestActor = system.actorOf(Props(new SendAll(req_list)), "Get_Response_Actor_" + getActorNumber)
val request_future = ask(createRequestActor, GetResponses).mapTo[List[Any]]
Thread.sleep(1000)
println(request_future)
//more stuff
}
}
}
Http().bindAndHandle(route, "localhost", 8080)
}
Updated Sending Class:
class SendAll(requests: List[Request]) extends Actor {
import context.{become,dispatcher}
var numProcessed = 0
var results: List[Any] = List()
requests.foreach(self ! _)
implicit val timeout = Timeout(100 millis)
def receive = {
case r: RequestMsg =>
val sendRequestActor = context.actorOf(Props(new SendRequest(r)), "Send_Request_".concat(getActorNumber))
(sendRequestActor ? Request).pipeTo(self)
case s: SuccessResponse =>
println("Got Success")
results = results :+ s
println(results.size + " == " + requests.size)
if(results.size == requests.size) {
println("Before done")
become(done)
}
case akka.actor.Status.Failure(f) =>
println("Got Failed")
results = results :+ RequestTimeout
if(results.size == requests.size) {
become(done)
}
case m =>
println("Got Other")
}
def done: Receive = {
case GetResponses =>
println("Done")
sender ! results
case _ => {
println("Done as well")
}
}
}
Output
Got Success
1 == 3
Got Success
2 == 3
Got Success
3 == 3
Before done
Future(<not completed>)
I would pass the list of requests to the actor, then pipe the responses from the child actors to self instead of using Await.result. For example:
class Handler(requests: List[RequestMsg]) extends Actor {
import context.{become, dispatcher}
var numProcessed = 0
var results: List[Any] = List()
requests.foreach(self ! _)
implicit val timeout = Timeout(100.millis)
def receive = {
case r: RequestMsg =>
val sendRequestActor = context.actorOf(Props(new SendRequest(r)), "Send_Request".concat(getActorNumber))
(sendRequestActor ? Request).pipeTo(self)
case s: SuccessResponse =>
println(s"response: $s")
results = results :+ s
if (results.size == requests.size)
become(done)
case akka.actor.Status.Failure(f) =>
println("a request failed or timed out")
results = results :+ RequestTimeout
if (results.size == requests.size)
become(done)
case m =>
println(s"Unhandled message received while processing requests: $m")
sender ! NotDone
}
def done: Receive = {
case GetResponses =>
println("sending responses")
sender ! results
}
}
You would instantiate an actor for every list of requests:
val requests1 = List(RequestMsg("one"), RequestMsg("two"), RequestMsg("three"))
val handler1 = system.actorOf(Props(new Handler(requests1)))
In this example--following the principle that an actor should have a distinct, limited sphere of responsibility--the actor simply coordinates requests and responses; it doesn't perform any processing on the collected responses. The idea is that another actor would send this actor a GetResponses messages in order to get the responses and process them (or this actor would proactively send the results to a processing actor).
The simplest solution is put all your actor refs into the List map it to List[Future] and use Future.sequence to obtain Future[List].
val route = {
get {
val listActorRefs = List(actorRef1, actorRef2, ...)
val futureListResponses = Future.sequence(listActorRefs.map(_ ? Request))
onComplete(futureListResponses) {
case Success(listResponse) => ...
complete(...)
case Failure(exception) => ...
}
}
}
A better solution is avoid a lot of actor' asks, prepare some ResponseCollector actor which will send all your message (I suggest to look at BroadcastPool) and schedule one message for itself to stop waiting and return result.
Let's say we have a fake data source which will return data it holds in batch
class DataSource(size: Int) {
private var s = 0
implicit val g = scala.concurrent.ExecutionContext.global
def getData(): Future[List[Int]] = {
s = s + 1
Future {
Thread.sleep(Random.nextInt(s * 100))
if (s <= size) {
List.fill(100)(s)
} else {
List()
}
}
}
object Test extends App {
val source = new DataSource(100)
implicit val g = scala.concurrent.ExecutionContext.global
def process(v: List[Int]): Unit = {
println(v)
}
def next(f: (List[Int]) => Unit): Unit = {
val fut = source.getData()
fut.onComplete {
case Success(v) => {
f(v)
v match {
case h :: t => next(f)
}
}
}
}
next(process)
Thread.sleep(1000000000)
}
I have mine, the problem here is some portion is more not pure. Ideally, I would like to wrap the Future for each batch into a big future, and the wrapper future success when last batch returned 0 size list? My situation is a little from this post, the next() there is synchronous call while my is also async.
Or is it ever possible to do what I want? Next batch will only be fetched when the previous one is resolved in the end whether to fetch the next batch depends on the size returned?
What's the best way to walk through this type of data sources? Are there any existing Scala frameworks that provide the feature I am looking for? Is play's Iteratee, Enumerator, Enumeratee the right tool? If so, can anyone provide an example on how to use those facilities to implement what I am looking for?
Edit----
With help from chunjef, I had just tried out. And it actually did work out for me. However, there was some small change I made based on his answer.
Source.fromIterator(()=>Iterator.continually(source.getData())).mapAsync(1) (f=>f.filter(_.size > 0))
.via(Flow[List[Int]].takeWhile(_.nonEmpty))
.runForeach(println)
However, can someone give comparison between Akka Stream and Play Iteratee? Does it worth me also try out Iteratee?
Code snip 1:
Source.fromIterator(() => Iterator.continually(ds.getData)) // line 1
.mapAsync(1)(identity) // line 2
.takeWhile(_.nonEmpty) // line 3
.runForeach(println) // line 4
Code snip 2: Assuming the getData depends on some other output of another flow, and I would like to concat it with the below flow. However, it yield too many files open error. Not sure what would cause this error, the mapAsync has been limited to 1 as its throughput if I understood correctly.
Flow[Int].mapConcat[Future[List[Int]]](c => {
Iterator.continually(ds.getData(c)).to[collection.immutable.Iterable]
}).mapAsync(1)(identity).takeWhile(_.nonEmpty).runForeach(println)
The following is one way to achieve the same behavior with Akka Streams, using your DataSource class:
import scala.concurrent.Future
import scala.util.Random
import akka.actor.ActorSystem
import akka.stream._
import akka.stream.scaladsl._
object StreamsExample extends App {
implicit val system = ActorSystem("Sandbox")
implicit val materializer = ActorMaterializer()
val ds = new DataSource(100)
Source.fromIterator(() => Iterator.continually(ds.getData)) // line 1
.mapAsync(1)(identity) // line 2
.takeWhile(_.nonEmpty) // line 3
.runForeach(println) // line 4
}
class DataSource(size: Int) {
...
}
A simplified line-by-line overview:
line 1: Creates a stream source that continually calls ds.getData if there is downstream demand.
line 2: mapAsync is a way to deal with stream elements that are Futures. In this case, the stream elements are of type Future[List[Int]]. The argument 1 is the level of parallelism: we specify 1 here because DataSource internally uses a mutable variable, and a parallelism level greater than one could produce unexpected results. identity is shorthand for x => x, which basically means that for each Future, we pass its result downstream without transforming it.
line 3: Essentially, ds.getData is called as long as the result of the Future is a non-empty List[Int]. If an empty List is encountered, processing is terminated.
line 4: runForeach here takes a function List[Int] => Unit and invokes that function for each stream element.
Ideally, I would like to wrap the Future for each batch into a big future, and the wrapper future success when last batch returned 0 size list?
I think you are looking for a Promise.
You would set up a Promise before you start the first iteration.
This gives you promise.future, a Future that you can then use to follow the completion of everything.
In your onComplete, you add a case _ => promise.success().
Something like
def loopUntilDone(f: (List[Int]) => Unit): Future[Unit] = {
val promise = Promise[Unit]
def next(): Unit = source.getData().onComplete {
case Success(v) =>
f(v)
v match {
case h :: t => next()
case _ => promise.success()
}
case Failure(e) => promise.failure(e)
}
// get going
next(f)
// return the Future for everything
promise.future
}
// future for everything, this is a `Future[Unit]`
// its `onComplete` will be triggered when there is no more data
val everything = loopUntilDone(process)
You are probably looking for a reactive streams library. My personal favorite (and one I'm most familiar with) is Monix. This is how it will work with DataSource unchanged
import scala.concurrent.duration.Duration
import scala.concurrent.Await
import monix.reactive.Observable
import monix.execution.Scheduler.Implicits.global
object Test extends App {
val source = new DataSource(100)
val completed = // <- this is Future[Unit], completes when foreach is done
Observable.repeat(Observable.fromFuture(source.getData()))
.flatten // <- Here it's Observable[List[Int]], it has collection-like methods
.takeWhile(_.nonEmpty)
.foreach(println)
Await.result(completed, Duration.Inf)
}
I just figured out that by using flatMapConcat can achieve what I wanted to achieve. There is no point to start another question as I have had the answer already. Put my sample code here just in case someone is looking for similar answer.
This type of API is very common for some integration between traditional Enterprise applications. The DataSource is to mock the API while the object App is to demonstrate how the client code can utilize Akka Stream to consume the APIs.
In my small project the API was provided in SOAP, and I used scalaxb to transform the SOAP to Scala async style. And with the client calls demonstrated in the object App, we can consume the API with AKKA Stream. Thanks for all for the help.
class DataSource(size: Int) {
private var transactionId: Long = 0
private val transactionCursorMap: mutable.HashMap[TransactionId, Set[ReadCursorId]] = mutable.HashMap.empty
private val cursorIteratorMap: mutable.HashMap[ReadCursorId, Iterator[List[Int]]] = mutable.HashMap.empty
implicit val g = scala.concurrent.ExecutionContext.global
case class TransactionId(id: Long)
case class ReadCursorId(id: Long)
def startTransaction(): Future[TransactionId] = {
Future {
synchronized {
transactionId += transactionId
}
val t = TransactionId(transactionId)
transactionCursorMap.update(t, Set(ReadCursorId(0)))
t
}
}
def createCursorId(t: TransactionId): ReadCursorId = {
synchronized {
val c = transactionCursorMap.getOrElseUpdate(t, Set(ReadCursorId(0)))
val currentId = c.foldLeft(0l) { (acc, a) => acc.max(a.id) }
val cId = ReadCursorId(currentId + 1)
transactionCursorMap.update(t, c + cId)
cursorIteratorMap.put(cId, createIterator)
cId
}
}
def createIterator(): Iterator[List[Int]] = {
(for {i <- 1 to 100} yield List.fill(100)(i)).toIterator
}
def startRead(t: TransactionId): Future[ReadCursorId] = {
Future {
createCursorId(t)
}
}
def getData(cursorId: ReadCursorId): Future[List[Int]] = {
synchronized {
Future {
Thread.sleep(Random.nextInt(100))
cursorIteratorMap.get(cursorId) match {
case Some(i) => i.next()
case _ => List()
}
}
}
}
}
object Test extends App {
val source = new DataSource(10)
implicit val system = ActorSystem("Sandbox")
implicit val materializer = ActorMaterializer()
implicit val g = scala.concurrent.ExecutionContext.global
//
// def process(v: List[Int]): Unit = {
// println(v)
// }
//
// def next(f: (List[Int]) => Unit): Unit = {
// val fut = source.getData()
// fut.onComplete {
// case Success(v) => {
// f(v)
// v match {
//
// case h :: t => next(f)
//
// }
// }
//
// }
//
// }
//
// next(process)
//
// Thread.sleep(1000000000)
val s = Source.fromFuture(source.startTransaction())
.map { e =>
source.startRead(e)
}
.mapAsync(1)(identity)
.flatMapConcat(
e => {
Source.fromIterator(() => Iterator.continually(source.getData(e)))
})
.mapAsync(5)(identity)
.via(Flow[List[Int]].takeWhile(_.nonEmpty))
.runForeach(println)
/*
val done = Source.fromIterator(() => Iterator.continually(source.getData())).mapAsync(1)(identity)
.via(Flow[List[Int]].takeWhile(_.nonEmpty))
.runFold(List[List[Int]]()) { (acc, r) =>
// println("=======" + acc + r)
r :: acc
}
done.onSuccess {
case e => {
e.foreach(println)
}
}
done.onComplete(_ => system.terminate())
*/
}
I want to simplify my for comprehension code to make it as simple as possible.
Here is the code
case object Message
class SimpleActor extends Actor {
def receive = {
case Message => sender ! Future { "Hello" }
}
}
object SimpleActor extends App {
val test = ActorSystem("Test")
val sa = test.actorOf(Props[SimpleActor])
implicit val timeout = Timeout(2.seconds)
val fRes = for {
f <- (sa ? Message).asInstanceOf[Future[Future[String]]]
r <- f
} yield r
println {
Await.result(fRes, 5.seconds)
}
}
Is it possible to get rid of this part
.asInstanceOf[Future[Future[String]]]
?
Look at the pipeTo function which is all about passing a Future between Actors. See here on ask patterns.
myFuture pipeTo sender
The return type of your ask will be Future[String] in your case, which as your comment below asks will need the mapTo[String] to actually get the result type to be a String. Thus your for-comp could be ditched and directly called:
val fRes = (sa ? Message).mapTo[String]
Given two scala play enumerators A and B that each provide sorted integers, is there a way to derive an enumerator of integers that exist in B that don't exist in A?
For example:
val A: Enumerator[Int] = Enumerator(1,3,5,9,11,13)
and
val B: Enumerator[Int] = Enumerator(1,3,5,7,9,11,13)
I would somehow get:
val C: Enumerator[Int] // This enumerator will output 7
Doing it in a reactive way with enumerators/iteratees/enumeratees is preferred.
One solution I've thought of is to interleave the enumerators and somehow use Iteratee.fold to maintain a buffer to compare the two streams but that seems like it should be unnecessary.
I had somewhat similar question
How to merge 2 Enumerators in one, based on merge rule
I modified given answer, to fit your needs
object Disjunction {
def disjunction[E: Ordering](enumA: Enumerator[E], enumB: Enumerator[E])(implicit ec: ExecutionContext) = new Enumerator[E] {
def apply[A](iter: Iteratee[E, A]) = {
case class IterateeReturn(o: Option[(Promise[Promise[IterateeReturn]], E)])
val failP: Promise[Nothing] = Promise() // Fail promise
val failPF: Future[Nothing] = failP.future // Fail promise future
val initState1: Future[Seq[IterateeReturn]] = Future.traverse(Seq(enumA, enumB)) {
enum =>
val p: Promise[IterateeReturn] = Promise[IterateeReturn]()
// The flow to transform Enumerator in IterateeReturn form
enum.run(Iteratee.foldM(p)({
(oldP: Promise[IterateeReturn], elem: E) =>
val p = Promise[Promise[IterateeReturn]]()
// Return IterateeReturn pointing to the next foldM Future reference, and current element
oldP success IterateeReturn(Some(p, elem))
// Return new Future as a Result of foldM
p.future
}) map ({
promise => promise success IterateeReturn(None) // Finish last promise with empty IterateeReturn
})
) onFailure {
// In case of failure main flow needs to be informed
case t => failP failure t
}
p.future
}
val initState: Future[List[(Promise[Promise[IterateeReturn]], E)]] = initState1 map (_.map(_.o).flatten.toList)
val newEnum: Enumerator[Option[E]] = Enumerator.unfoldM(initState) { fstate =>
// Whatever happens first, fstate returned of failure happened during iteration
Future.firstCompletedOf(Seq(fstate, failPF)) map { state =>
// state is List[(Promise[Promise[IterateeReturn]], E)
// sort elements by E
if (state.isEmpty) {
None
} else if (state.length == 1) {
val (oldP, elem) = state.head
val p = Promise[IterateeReturn]()
oldP success p
// Return newState, with this iterator moved
val newState: Future[List[(Promise[Promise[IterateeReturn]], E)]] = p.future.map(ir => ir.o.map(List(_)).getOrElse(Nil))
Some(newState, Some(elem))
} else {
val sorted = state.sortBy(_._2)
val (firstP, fe) = sorted.head
val (secondP, se) = sorted.tail.head
if (fe != se) {
// Move first and combine with the second
val p = Promise[IterateeReturn]()
firstP success p
val newState: Future[List[(Promise[Promise[IterateeReturn]], E)]] = p.future.map(ir => ir.o.map(List(_, (secondP, se))).getOrElse(List((secondP, se))))
// Return new state
Some(newState, Some(fe))
} else {
// Move future 1
val p1 = Promise[IterateeReturn]()
firstP success p1
val fState: Future[Option[(Promise[Promise[IterateeReturn]], E)]] = p1.future.map(ir => ir.o)
// Move future 2
val p2 = Promise[IterateeReturn]()
secondP success p2
val sState: Future[Option[(Promise[Promise[IterateeReturn]], E)]] = p2.future.map(ir => ir.o)
// Combine in new state
val newState = Future.sequence(List(fState, sState)).map(_.flatten)
// Return
Some(newState , None)
}
}
}
}
newEnum &>
Enumeratee.filter(_.isDefined) &>
Enumeratee.map(_.get) apply iter
}
}
}
I checked, it works.