Need list outside map in play ws api call - scala

In a play-scala application, Im making a WS call:
def getaddresses : ListBuffer[String] = {
var lb = new ListBuffer[String]()
var url = "xyz#xyx.com/someapi"
WS.url(url).get.map {
response =>
var emailsSeq = (response.json \\ "email")
emailsSeq.foreach(lb+=_.toString())
lb.foreach(println) //This is filled with values
}
lb.foreach(println) //This is empty
lb
}
Inside map a sequence is returned whose entries I put in a ListBuffer lb to be returned by this method.
The problem is that the the ListBuffer shows values inside map{} but does not show outside it. As a result empty ListBuffer is passed from the method.
I thought there might be a delay in ws response so tried the following which had no advantage:
WS.url(url).withRequestTimeout(10.seconds).get.map {
Please help me in getting filled list buffer to be returned

I think that ws.url(url).get method is async, so when you call lb.foreach(println), there is nothing to print. Try to add Thread.sleep just after map block of code. If it is so, you should make getaddresses method async, or use future/promise.
PS: you should use val instead of var, just for cleaner code.
EDIT: Sample to try:
def getaddresses : ListBuffer[String] = {
val url = "xyz#xyx.com/someapi"
val promiseOfAddresses = Promise[ListBuffer[String]]()
WS.url(url).get.map {
response =>
var emailsSeq = (response.json \\ "email")
promiseOfAddresses.success(emailsSeq) // here you will complete promise with actual value
}
val lb = promiseOfAddresses.future.get // here you will obtain future from promise and also get value of future. Method get is blocking and it will wait till promise is fullfiled or failed
lb.foreach(println) //This is empty
lb
}
PS2: Probably best help with future/promises can be found here: the-neophytes-guide-to-scala-part-9-promises-and-futures-in-practice (I have not enough reputation, so google this blog) It is something like CompletableFuture in java world. But remember, that best way is to stay all time in red side (red = asynchronous functions/methods. see nice, but chatty, blog what-color-is-your-function)

Related

Scala IO wait during map external call

I will start mentioning I am very new to Scala but I have now to maintain a legacy code where some new feature are being tried to be include.
I have the following code:
Where a list is coming as a parameter where a new output needs to be processed. However it seems like code is not waiting for the response to the external service when processing.
def historyBet(jackpotListUser : List[JackpotBetHistory])(implicit MC: AppMarkerContext) : List[LegacyJackpotHistoryResponse] =
for {
bet <- jackpotListUser
prize = jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId) match {
case Some(jackpot : JackpotResponse) =>
...
extra code extracting price from jackpot : JackpotResponse
...
extra code generating result with prize
} yield result
How can I do a call to jackpotIntegratorService.findJackpotByJackpotHumanId to execute at that time. instead of returning something that F[Option....?
def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit MC: AppMarkerContext): F[Option[JackpotResponse]] =
jackpotIntegratorRepo.findJackpotByJackpotHumanId(jackpotHumanId)
where it is finally implemented as:
override def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit mc: AppMarkerContext): IO[Option[JackpotResponse]] =
... code calling an API which return the IO.
Thanks!
I thought I could do IO.await somewhere... but not sure where or how...
because in the "historyBet" function I got a F[] when it was an IO... so what is the syntax to be able to wait for the response and the continue?
Extra Comment:
The real issue we notice is that the method call is starting (the logs shows part of it) but the caller with in the maps continues too.
prize = jackpotIntegratorService.findJackpotByJackpotHumanId
this part of the code continues even when prize, which we want the final object JackpotResponse, not the IO or F.
So, if your method needs to call an IO then it must return an IO unless you unsafeRunSync them... but, as the name suggest, you should not do that.
So the return type is now: IO[List[LegacyJackpotHistoryResponse]
And can be implemented like this:
def historyBet(jackpotListUser: List[JackpotBetHistory])(implicit MC: AppMarkerContext): IO[List[LegacyJackpotHistoryResponse]] =
jackpotListUser.traverse { bet =>
jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId).map {
case Some(jackpot) =>
// ...
case None =>
// ...
}
}

Gatling. What would be the best way to pass feeder data to other functions?

I'm trying to write a load test for service.
I want to build DeliveryObjects and publish them, each delivery must have a unique id.
The problem I encounter is that I cant pass variables from the session to a function that I wrote (I know the documentation says I can't), also I can't "catch" the value on run time as I saw in several examples. So this is one thing I have tried:
object AdminClient extends FireClient {
def getDeliveryStateByDeliveryId(name: String = "Get delivery state by ID",
#Nullable deliveryId: Expression[String] = "${delivery_uuid}")
: HttpClientRequest = {
// The deliveryId resolve to something like this: io.gatling.core.session.el.ElCompiler$$$Lambda$372/1144897090#473b3b7a
println("delivery id in adminclient is: " + deliveryId)
get(name)
.uri(s"/url/${deliveryId}")
.requestModeAdmin
.allowOkStatus
}
}
and the scenario looks like this (to make things simpler):
object LoadTest extends FireScenarios {
val csvFeeder = csv("deliveries.csv")
fireScenario("Load test starts")(_
.feed(csvFeeder)
.exec { session =>
// Here delivery_uuid get a real value something like "b6070d6b-ce10-5fd3-b81d-ed356665f0e1"
println("delivery id id:" + session.get("delivery_uuid").as[String])
session
}
.exec(AdminClient.getDeliveryStateByDeliveryId())
)
}
So I guess my question is how can I pass a value to the var "${delivery_uuid}" in the "getDeliveryStateByDeliveryId" method?
Note that I also can't just call the getDeliveryStateByDeliveryId method from withing the
exec{ session =>
AdminClient.getDeliveryStateByDeliveryId(deliveryId = session.get("delivery_uuid"))
session
}
Although the method gets the variable as I want, the Gatling throws an error that no request was sent and no report will be produced.
I'm very confused after reading the docs too many times, any help will be much appreciated.
Let's sum up what you can find in the official documentation:
Expression[String] is a Scala type alias for (making it simple) scala.Function[Session, String]. Similarly, in Gatling's Java DSL, you directly pass java.util.Function<Session, String>.
Gatling's Scala DSL has some implicit conversion that transform String values passed to methods expecting a Function parameter into a proper function.
So if we make explicit what you've written, you actually have (doesn't compile, but you'll get the idea):
def getDeliveryStateByDeliveryId(name: String = "Get delivery state by ID",
#Nullable deliveryId: Expression[String] = session => session("delivery_uuid").as[String])
: HttpClientRequest = {
println("delivery id in adminclient is: " + deliveryId)
get(name)
.uri("/url/" + deliveryId)
.requestModeAdmin
.allowOkStatus
}
This cannot possibly work. You're concatenating a String and a Function which, like in Java uses the toString inherited from Object.
Now, as you're a beginner, why do you need deliveryId to be a function? Can't it just be a String with the name of the desired attribute?
def getDeliveryStateByDeliveryId(name: String = "Get delivery state by ID",
deliveryId: String = "delivery_uuid")
: HttpClientRequest =
get(name)
.uri(s"/url/#{$deliveryId}")
.requestModeAdmin
.allowOkStatus
object LoadTest extends FireScenarios {
val csvFeeder = csv("deliveries.csv")
fireScenario("Load test starts")(_
.feed(csvFeeder)
.exec(AdminClient.getDeliveryStateByDeliveryId())
)
}

How to test `Var`s of `scala.rx` with scalatest?

I have a method which connects to a websocket and gets stream messages from some really outside system.
The simplified version is:
def watchOrders(): Var[Option[Order]] = {
val value = Var[Option[Order]](None)
// onMessage( order => value.update(Some(order))
value
}
When I test it (with scalatest), I want to make it connect to the real outside system, and only check the first 4 orders:
test("watchOrders") {
var result = List.empty[Order]
val stream = client.watchOrders()
stream.foreach {
case Some(order) =>
result = depth :: result
if (result.size == 4) { // 1.
assert(orders should ...) // 2.
stream.kill() // 3.
}
case _ =>
}
Thread.sleep(10000) // 4.
}
I have 4 questions:
Is it the right way to check the first 4 orders? there is no take(4) method found in scala.rx
If the assert fails, the test still passes, how to fix it?
Is it the right way to stop the stream?
If the thread doesn't sleep here, the test will pass the code in case Some(order) never runs. Is there a better way to wait?
One approach you might consider to get a List out of a Var is to use the .fold combinator.
The other issue you have is dealing with the asynchronous nature of the data - assuming you really want to talk to this outside real world system in your test code (ie, this is closer to the integration test side of things), you are going to want to look at scalatest's support for async tests and will probably do something like construct a future out of a promise that you can complete when you accumulate the 4 elements in your list.
See: http://www.scalatest.org/user_guide/async_testing

Returning value from Scala future completion

Coming from a Java background, I have been trying to teach myself Scala for some time now. As part of that, I am doing a small pet project that exposes a HTTP endpoint that saves the registration numberof a vehicle against the owner and returns the status.
To give more context, I am using Slick as FRM which performs DB operations asynchronously and returns a Future.
Based on the output of this Future, I want to set the status variable to return back to the client.
Here, is the code
def addVehicleOwner(vehicle: Vehicle): String = {
var status = ""
val addFuture = db.run((vehicles returning vehicles.map(_.id)) += vehicle)
addFuture onComplete {
case Success(id) => {
BotLogger.info(LOGTAG, s"Vehicle registered at $id ")
status = String.format("Registration number - '%s' mapped to owner '%s' successfully", vehicle.registration,
vehicle.owner)
println(s"status inside success $status") //--------- (1)
}
case Failure(e: SQLException) if e.getMessage.contains("SQLITE_CONSTRAINT") => {
status = updateVehicleOwner(vehicle)
BotLogger.info(LOGTAG, s"Updated owner='${vehicle.owner}' for '${vehicle.registration}'")
}
case Failure(e) => {
BotLogger.error(LOGTAG, e)
status = "Sorry, unable to add now!"
}
}
exec(addFuture)
println(s"Status=$status") //--------- (2)
status
}
// Helper method for running a query in this example file:
def exec[T](sqlFuture: Future[T]):T = Await.result(sqlFuture, 1 seconds)
This was fairly simple in Java. With Scala, I am facing the following problems:
The expected value gets printed at (1), but (2) always prints empty string and same is what method returns. Can someone explain why?
I even tried marking the var status as #volatile var status, it still evaluates to empty string.
I know, that the above is not the functional way of doing things as I am muting state. What is the clean way of writing code for such cases.
Almost all the examples I could find described how to map the result of Success or handle Failure by doing a println. I want to do more than that.
What are some good references of small projects that I can refer to? Specially, that follow TDD.
Instead of relying on status to complete inside the closure, you can recover over the Future[T] which handle the exception if they occur, and always returns the result you want. This is taking advantage of the nature of expressions in Scala:
val addFuture =
db.run((vehicles returning vehicles.map(_.id)) += vehicle)
.recover {
case e: SQLException if e.getMessage.contains("SQLITE_CONSTRAINT") => {
val status = updateVehicleOwner(vehicle)
BotLogger.info(
LOGTAG,
s"Updated owner='${vehicle.owner}' for '${vehicle.registration}'"
)
status
}
case e => {
BotLogger.error(LOGTAG, e)
val status = "Sorry, unable to add now!"
status
}
}
val result: String = exec(addFuture)
println(s"Status = $result")
result
Note that Await.result should not be used in any production environment as it synchronously blocks on the Future, which is exactly the opposite of what you actually want. If you're already using a Future to delegate work, you want it to complete asynchronously. I'm assuming your exec method was simply for testing purposes.

How to properly use spray.io LruCache

I am quite an unexperienced spray/scala developer, I am trying to properly use spray.io LruCache. I am trying to achieve something very simple. I have a kafka consumer, when it reads something from its topic I want it to put the value it reads to cache.
Then in one of the routings I want to read this value, the value is of type string, what I have at the moment looks as follows:
object MyCache {
val cache: Cache[String] = LruCache(
maxCapacity = 10000,
initialCapacity = 100,
timeToLive = Duration.Inf,
timeToIdle = Duration(24, TimeUnit.HOURS)
)
}
to put something into cache i use following code:
def message() = Future { new String(singleMessage.message()) }
MyCache.cache(key, message)
Then in one of the routings I am trying to get something from the cache:
val res = MyCache.cache.get(keyHash)
The problem is the type of res is Option[Future[String]], it is quite hard and ugly to access the real value in this case. Could someone please tell me how I can simplify my code to make it better and more readable ?
Thanks in advance.
Don't try to get the value out of the Future. Instead call map on the Future to arrange for work to be done on the value when the Future is completed, and then complete the request with that result (which is itself a Future). It should look something like this:
path("foo") {
complete(MyCache.cache.get(keyHash) map (optMsg => ...))
}
Also, if singleMessage.message does not do I/O or otherwise block, then rather than creating the Future like you are
Future { new String(singleMessage.message) }
it would be more efficient to do it like so:
Future.successful(new String(singleMessage.message))
The latter just creates an already completed Future, bypassing the use of an ExecutionContext to evaluate the function.
If singleMessage.message does do I/O, then ideally you would do that I/O with some library (like Spray client, if it's an HTTP request) that returns a Future (rather than using Future { ... } to create another thread which will block).