I have a Future[Future[Set[String]]. I flatmap it to return a Future[Set[String]]. I wait for infinity (I know this is bad) and then check if the Future is complete. It returns true. However, when I get the value returned by Future and try to add it to a Set, it doesn't add it. Statements at the end of the callback get printed first. When I return the Set from the method, it is always empty though the Future does return values. I have added comments as to explain the sequence of events happening. Please point me if I am doing something wrong! Thanks!
val googleXfpSegments = Set[String]()
val customCriteriaFuture: Future[List[Long]] = getCustomCriteriaValueIds(lineItemPage.getResults())
// getCustomCriteriaValues returns a Future[Set[String]]
val criteriaValueList = customCriteriaFuture.flatMap(criteriaIdList => getCustomCriteriaValues(criteriaIdList.toSet))
// I wait for infinite time
Await.result(criteriaValueList, scala.concurrent.duration.Duration.Inf)
log.info("Future Completed = " + criteriaValueList.isCompleted) // This returns true
criteriaValueList onComplete {
case Success(segmentNames) => {
// This statement gets printed after the "TEST" value gets printed
log.info("SegmentNameList = " + segmentNames.mkString(","))
googleXfpSegments ++= segmentNames
// This prints nothing which means the segmentNames is not getting added
log.info("Google Segments = " + googleXfpSegments.mkString(","))
}
case Failure(error) => log.error("Error occurred ", error)
}
googleXfpSegments += "TEST"
// This prints TEST
log.info("Google Segments = " + googleXfpSegments.mkString(","))
Log output is shown below:
[info] [INFO] [2014-12-09 17:25:01,147] [service.GoogleXFPService] Future Completed = true
[info] [INFO] [2014-12-09 17:25:01,148] [service.GoogleXFPService] Google Segments = TEST
[info] [INFO] [2014-12-09 17:25:01,148] [service.GoogleXFPService] SegmentNameList = vt.cm,vt.cp,ex.qy,ex.ahe,cm.gannett_trav
You need to assign the output of Await.result(criteriaValueList, scala.concurrent.duration.Duration.Inf) to a value. Currently you are waiting for it to complete, doing nothing with the output, then attaching an onComplete callback to an already completed future, then your program terminates before the callback can run.
val segmentNames = Await.result(criteriaValueList, scala.concurrent.duration.Duration.Inf)
log.info("Google Segments = " + segmentNames.mkString(","))
Related
I've set the timeout duration to "2 minutes" as follows:
def updateAcrossEvents (tuple3: Tuple3[String, String, String], inputs: Iterator[R00tJsonObject],
oldState: GroupState[MyState]): OutputRow = {
println("$$$$ Inside updateAcrossEvents with : " + tuple3._1 + ", " + tuple3._2 + ", " + tuple3._3)
var state: MyState = if (oldState.exists) oldState.get else MyState(tuple3._1, tuple3._2, tuple3._3)
if (oldState.hasTimedOut) {
println("##### oldState has timed out ####")
// Logic to Write OutputRow
OutputRow("some values here...")
} else {
for (input <- inputs) {
state = updateWithEvent(state, input)
oldState.update(state)
oldState.setTimeoutDuration("2 minutes")
}
OutputRow(null, null, null)
}
}
I have also specified ProcessingTimeTimeout in 'mapGroupsWithState' as follows...
.mapGroupsWithState(GroupStateTimeout.ProcessingTimeTimeout())(updateAcrossEvents)
But 'hasTimedOut' is never true so I don't get any output! What am I doing wrong?
It seems it only works if input data is continuously flowing. I had stopped the input job because I had enough data but it seems timeouts work only if the data is continuously fed. Not sure why it's designed that way. Makes it a bit harder to write unit/integration tests BUT I am sure there's a reason why it's designed this way. Thanks.
I want to look for an entire list of items to be found before I complete and if that entire list isn't found, then an exception (a Timeout or custom one) is to be thrown. Like the built in Observable.timer() but instead of the test passing once the first item is emitted, I want it to require all of the items in a list to be found.
Here is an example. Let's say I have some test function that emits Observable<FoundNumber>. It looks like this:
var emittedList: List<String?> = listOf(null, "202", "302", "400")
data class FoundNumber(val numberId: String?)
fun scanNumbers(): Observable<FoundNumber> = Observable
.intervalRange(0,
emittedList.size.toLong(),
0,
1,
TimeUnit.SECONDS).map { index ->
FoundNumber(emittedList[index.toInt()]) }
That function will then be called to get numbers that will be compared to a list of expected numbers. It doesn't matter if there are additional numbers coming from scanForNumbers that aren't in the "target" list. They will just be ignored. Something like this:
val expectedNumbers = listOf("202", "302","999")
scanForNumbers(expectedNumbers)
.observeOn(AndroidSchedulers.mainThread())
.subscribeOn(Schedulers.io())
.subscribe { value -> Log.d(TAG, "Was returned a $value") }
So, the expected numbers (202, 302, and 999) don't exactly match with the numbers that will be emitted (202, 302, and 400). So, a timeout SHOULD occur, but with the built in version of Observable.timer(), it will not time out since at least one item was observed.
Here is kind of what I'd like to have. Anyone know how to code this up in RxJava/RxKotlin?
fun scanForNumbers(targets: List<String>): Observable<FoundNumber> {
val accumulator: Pair<Set<Any>, FoundNumber?> = targets.toSet() to null
return scanNumbers()
.SPECIAL_TIMEOUT_FOR_LIST(5, TimeUnit.SECONDS, List)
.scan(accumulator) { acc, next ->
val (set, previous) = acc
val stringSet:MutableSet<String> = hashSetOf()
set.forEach { stringSet.add(it.toString()) }
val item = if (next.numberId in stringSet) {
next
} else null
(set - next) to item // return set and nullable item
}
.filter { Log.d(TAG, "Filtering on ${it.second}")
it.second != null } // item not null
.take(targets.size.toLong()) // limit to the number of items
.map { it.second } // unwrap the item from the pair
.map { FoundController(it.numberId) } // wrap in your class
}
How do you code, hopefully using RxJava/Kotlin, a means to timeout on a list as mentioned?
I think I get it now, you want the timeout to begin counting from the moment you subscribe, not after you observe items.
If this is what you need, then the takeUntil operator could help you:
return scanNumbers()
.takeUntil(Observable.timer(5, TimeUnit.SECONDS))
.scan(accumulator) { acc, next -> ...
In this case, the timer will begin counting as soon as you subscribe. If the main observable completes before then great, if not, then the timer will complete the main observable anyways.
But takeUntil by itself will not throw an error, it will just complete. If you need it to end with an error, then you could use the following combination:
return scanNumbers()
.takeUntil(
Observable
.error<Void>(new TimeoutError("timeout!"))
.delay(5, TimeUnit.SECONDS, true))
.scan(accumulator) { acc, next -> ...
Trying to process a list of long running jobs in a vertx way
One would hope one could do something like:
use the executeBlocking to process the long running job in an async manner
use the composite future to wait for the futures to complete
I'm aware this approach does not work .. the list of Futures is not complete before the code drops into the CompositeFuture.
Is there a executeBlocking approach or does one have to use either the eventbus, vertx utils that support lists?
java.util.ArrayList futureList = new ArrayList()
for (i = 0; i < 100; i ++){
vertx.executeBlocking({ future ->
int id = i
println "Running " + id
java.lang.Thread.sleep(1000)
println "Thread done " + id
future.complete()
}, true , { res ->
if (res.succeeded()) {
print "."
} else {
print "x"
}
})
}
CompositeFuture.join(futureList).setHandler({ ar ->
if (ar.succeeded()) {
System.err.println "all threads should be done.."
}
})
Results in .. "all threads should be done" printing early
Running 84
Running 87
Running 87
Running 95
all threads should be done..
done.
Thread done 3
Thread done 36
Thread done 3
Thread done 0
In your example, futureList is empty so CompositeFuture.join(futureList) is completed immediately.
Change your example like this:
java.util.ArrayList futureList = new ArrayList()
for (i = 0; i < 100; i ++){
Future jobFuture = Future.future()
futureList.add(jobFuture)
vertx.executeBlocking({ future ->
int id = i
println "Running " + id
java.lang.Thread.sleep(1000)
println "Thread done " + id
future.complete()
}, true , { res ->
if (res.succeeded()) {
print "."
} else {
print "x"
}
jobFuture.complete()
})
}
Notice the jobFuture creation:
Future jobFuture = Future.future()
futureList.add(jobFuture)
As well as completion:
jobFuture.complete()
Now the CompositeFuture.join(futureList) handler will be executed only after all jobs complete.
In an implementation of an FSM (old story: modeling an Elevator/Lift), I am trying to see if I can test whether the FSM follows a transition from and to the same State. It seems, it is not happening, but that surprises me because according to this discussion on StackOverFlow, the fix is available in Akka 2.4.x.
Here's the relevant portion of the code:
class LiftCarriageWithMovingState extends Actor
with FSM[LiftState,LiftData]
with ActorLogging
{
import context._
val mxTimeToWaitStopping = 500 milliseconds
private var pendingPassengerRequests: Vector[NextStop] = Vector.empty
private var currentFloorID = 0 // Always start at Ground Floor
val timeToReachNextFloor = 1000 millis // up or down, same time taken
private val dontCare: StateFunction = { /* ... */ }
private val powerYourselfOff: StateFunction = { /* ... */ }
private val powerYourselfOn: StateFunction = { /* ... */ }
private val beReady: StateFunction = { /* ... */ }
private val moveToWaitingPassenger: StateFunction = { /* ... */ }
private val transportPassengerToDest: StateFunction = {/* ... */ }
private val actWhenStoppedForLongEnough: StateFunction = {
case Event(StateTimeout,_) =>
if (this.pendingPassengerRequests isEmpty) {
println("timeout in Stopped, no passenger requests pending")
goto (Stopped)
}
else {
println("timeout in Stopped, moving to floor(" + this.pendingPassengerRequests.head + ")")
simulateMovementTo(this.pendingPassengerRequests.head)
goto(Moving)
}
}
startWith(PoweredOff,InitialData)
when (PoweredOff) (powerYourselfOn orElse
dontCare)
when (PoweredOn) (powerYourselfOff orElse
beReady orElse
dontCare)
when (Ready) (moveToWaitingPassenger orElse
transportPassengerToDest )
when (Stopped) (actWhenStoppedForLongEnough orElse
moveToWaitingPassenger orElse
transportPassengerToDest )
when (Moving) {
case Event(ReachedFloor(floorID, PurposeOfMovement.ToWelcomeInAnWaitingPassenger),_) =>
currentFloorID = pendingPassengerRequests.head.floorID
pendingPassengerRequests = pendingPassengerRequests.tail
goto (Stopped) forMax(this.mxTimeToWaitStopping)
case Event(ReachedFloor(floorID,PurposeOfMovement.ToAllowATransportedPassengerAlight),_) =>
currentFloorID = pendingPassengerRequests.head.floorID
pendingPassengerRequests = pendingPassengerRequests.tail
goto (Stopped) forMax(this.mxTimeToWaitStopping)
// .... Other events
}
whenUnhandled {
// Event handlers...
}
onTransition {
case Stopped -> Stopped => {
println("Remaining in Stopped ...")
}
}
// Rest of the Actor
I have pruned unnecessary parts, because the main point is this:
When the actor receives a
Event(ReachedFloor(floorID, PurposeOfMovement.ToWelcomeInAnWaitingPassenger),_)
when it is in the Moving state, it switches to Stopping state, with a specified timeout. When that timer fires while the FSM is Stopped, the handler that is supposed to execute, is named (shown above too):
actWhenStoppedForLongEnough
Here, I am checking a certain exit condition, and if I am not happy, I am switching back to same Stopped state again. I am using goto() now, but the original code had stay().
I have set up a onTransition block too (shown above again), to prove that the transition indeed does happen.
onTransition {
case Stopped -> Stopped => {
println("Remaining in Stopped ...")
}
}
When I run the code, the println statement is not executed, at all.
I also have a testcode; the relevant testcase is this:
var carriage: ActorRef = _
override def beforeAll(): Unit = {
super.beforeAll()
carriage = TestActorRef(Props(new LiftCarriageWithMovingState))
carriage ! SubscribeTransitionCallBack(testActor)
}
"A LiftCarriage" must {
"be ready, when it settles down after being PoweredOn" in {
expectMsg(Duration(100, TimeUnit.MILLISECONDS), new CurrentState(carriage,PoweredOff))
carriage ! InstructedToPowerOn
expectMsg (new Transition(carriage,PoweredOff,PoweredOn))
carriage ! BeReady
expectMsg((new Transition(carriage,PoweredOn,Ready)))
}
"move to where a passenger is waiting, if ready" in {
carriage ! ReportCurrentFloor
expectMsg(StoppedAt(0))
carriage ! PassengerIsWaitingAt(3)
expectMsg(new Transition(carriage,Ready,Moving))
expectMsg(Duration(5000, TimeUnit.MILLISECONDS), new Transition(carriage,Moving, Stopped))
carriage ! ReportCurrentFloor
expectMsg(StoppedAt(3))
}
"let a passenger in and transport her to her destination" in {
carriage ! ReportCurrentFloor
expectMsg(StoppedAt(3))
carriage ! PassengerRequestsATransportTo(Vector(7))
expectMsg(new Transition(carriage,Stopped,Moving))
expectMsg(Duration(5000, TimeUnit.MILLISECONDS), new Transition(carriage,Moving,Stopped))
carriage ! ReportCurrentFloor
expectMsg(StoppedAt(7))
// This always times out and fails.
expectMsg(Duration(5000, TimeUnit.MILLISECONDS), new Transition(carriage,Stopped,Stopped))
}
I am trying to think hard as to if and where the gap in my understanding lies, but failing. Any hints?
Here's the build.sbt:
name := """akka-sample-fsm-scala"""
version := "2.4"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
// https://mvnrepository.com/artifact/com.typesafe.akka/akka-actor_2.11
"com.typesafe.akka" % "akka-actor_2.11" % "2.4.12",
// https://mvnrepository.com/artifact/org.scalatest/scalatest_2.11
"org.scalatest" % "scalatest_2.11" % "2.1.3",
// https://mvnrepository.com/artifact/com.typesafe.akka/akka-testkit_2.11,
"com.typesafe.akka" % "akka-testkit_2.11" % "2.3.15"
)
Another question - which is quite similar to mine - has been asked earlier, but I don't see any responses. May be, the answer has been found already but not yet shared.
Right now I'm trying to build some code to handle lists over 100 items, as returned by the Amazon API endpoints. This requires building page-support into our data-gathering routines. This is my first time doing much with coffeescript, so I'm running into some conceptual walls here.
In a less async language, what I'm trying to do would be handleable using an until loop:
puts "Fetching launch configs"
next_token = ''
do
if next_token.length > 0
page_list = get_autoscale_configs(next_token)
else
page_list = get_autoscale_configs
if page_list.NextToken is undefined
next_token = ''
else
next_token = page_list.NextToken
until(next_token.length == 0)
The method of doing this in coffeescript is eluding me. What I have now...
populate_configs = ( ) ->
process_results = ( err data ) ->
if err
return err
# do some logic
if data.NextToken
saved.next_token = data.NextToken
else
saved.next_token = ''
return console.log "Finished parsing #{data.LaunchConfigurations.length} items."
if saved.next_token = ''
autoscaling.describeLaunchConfigurations {
MaxRecords: 100, StartToken: next_token
}, ( err, data ) -> process_results( err, data )
else
autoscaling.describeLaunchConfigurations {
MaxRecords: 100
}, ( err, data ) -> process_results( err, data )
And then in the body of the code, this function is invoked:
saved = {}
async.series [
( series_cb ) ->
saved.next_token = ''
async.doWhilst populate_configs,
saved.next_token.length > 4,
( err ) ->
if err
# complain about it.
# else, log success
return series_cb()
# more callbacks
]
The idea here being that populate_configs is called by doWhilst, which then fetches a list of launch_configs out of amazon. The data is then passed into another function called process_results, which persists things that should be persisted and sets variables for next_token. It returns, and doWhilst tests to see if the test is passing (the string-length of saved.next_token is long enough to be data); if it passes, it runs through populate_configs again, if it fails, it runs the third callback.
What I'm getting right now is that the first iteration of that populate_configs block is executed, but then the entire execution stops dead at that point. None of the calls in the error-handler of doWhilst are being executed.
Clearly, I'm misunderstanding how callbacks work and how to get myself out of this hole. This part needs to be synchronous. Once I have this list built, I can do all sorts of async fun with the list I'm building. But I need the list first.
I think the issue is here: if saved.next_token = ''. You set next_token to '' so populate_configs runs only once. The comparaison is done with == or is in CoffeeScript.
Also, ( err, data ) -> process_results( err, data ) can be replaced by process_results.