I want to show future response in UI in a liftweb framework.
Following is a Loggable snippet method(processRequest) I'm currently using to respond back future object to UI textarea using SetHtml but blocking with Thread.sleep
27 def render = {
45 def processRequest() : JsCmd = {
46
47 val future_ : LAFuture[String] = new LAFuture()
48
49 val urlRequest = url(request)
51 val future = Http(urlRequest OK as.String)
57
58 future onSuccess {
59 case json =>
60 resp = json
61 logger.info(s"[onSuccess] response => ${json}")
62 SetHtml("response", Text(resp+"")) //FIXME
63 logger.info(s"[onSuccess] response => ${json}") //works fine
64
65 }
66
67 future onFailure {
68 case exception =>
69 logger.info(s"${exception.getMessage()}")
70 }
71 Thread.sleep(2000) //TEMPORARY FIX
72 logger.info("[outside] response => ${resp}")
73 SetHtml("response", Text(resp)) //it works in blocking mode
74 }
77 "#requestid" #> text(requestid, s => requestid = s) &
78 "#request" #> text(request , s => request = s) &
79 "#response" #> textarea(responseString, s=>responseString = s) &
81 "#processreq" #> ajaxSubmit("Send Request", processRequest)
75 }
SetHtml doesn't work inside onSuccess, though I get json there. SetHtml has following doc,
Replaces the content of the node with the provided id with the markup
given by content
This is analogous to assigning a new value to a DOM object's innerHtml
property in Javascript.
I have all this app available at github.
What I want is solution to SetHtml in handling future object where the request itself is through ajax.
SetHtml returns a JsCmd, but in order for that to be sent to the browser it would need to be part of a server response. However, as you are calling it asynchronously there is no response to send it with and therefore the browser never receives it. To fix that you have two options:
Wait for the future to finish by blocking, something like having
your function return future.get
Use an actor which allows for server push communication with the browser. In that way, your future could send a message to the actor upon completion and then the actor would send the setHtml command.
There is a quick example with Lift actors here (unfortunately uses the old bind syntax in render) which illustrates how to send a message and have that execute setHtml: https://www.assembla.com/wiki/show/liftweb/Comet_Support
Related
I will start mentioning I am very new to Scala but I have now to maintain a legacy code where some new feature are being tried to be include.
I have the following code:
Where a list is coming as a parameter where a new output needs to be processed. However it seems like code is not waiting for the response to the external service when processing.
def historyBet(jackpotListUser : List[JackpotBetHistory])(implicit MC: AppMarkerContext) : List[LegacyJackpotHistoryResponse] =
for {
bet <- jackpotListUser
prize = jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId) match {
case Some(jackpot : JackpotResponse) =>
...
extra code extracting price from jackpot : JackpotResponse
...
extra code generating result with prize
} yield result
How can I do a call to jackpotIntegratorService.findJackpotByJackpotHumanId to execute at that time. instead of returning something that F[Option....?
def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit MC: AppMarkerContext): F[Option[JackpotResponse]] =
jackpotIntegratorRepo.findJackpotByJackpotHumanId(jackpotHumanId)
where it is finally implemented as:
override def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit mc: AppMarkerContext): IO[Option[JackpotResponse]] =
... code calling an API which return the IO.
Thanks!
I thought I could do IO.await somewhere... but not sure where or how...
because in the "historyBet" function I got a F[] when it was an IO... so what is the syntax to be able to wait for the response and the continue?
Extra Comment:
The real issue we notice is that the method call is starting (the logs shows part of it) but the caller with in the maps continues too.
prize = jackpotIntegratorService.findJackpotByJackpotHumanId
this part of the code continues even when prize, which we want the final object JackpotResponse, not the IO or F.
So, if your method needs to call an IO then it must return an IO unless you unsafeRunSync them... but, as the name suggest, you should not do that.
So the return type is now: IO[List[LegacyJackpotHistoryResponse]
And can be implemented like this:
def historyBet(jackpotListUser: List[JackpotBetHistory])(implicit MC: AppMarkerContext): IO[List[LegacyJackpotHistoryResponse]] =
jackpotListUser.traverse { bet =>
jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId).map {
case Some(jackpot) =>
// ...
case None =>
// ...
}
}
I am working with an API that limits me to 40 requests per second, and 200 every 120 seconds.
I am currently designing an async pattern in Rust using reqwest and tokio. I want to incorporate the rate limiting constraints. I have done similar things in Python using Semaphores and have looked into semaphores in Rust, but am not quite sure how to structure my code.
Ideally, I'd like to:
Send in batches of 40 requests (never more than 40 per second)
Once I hit 200 requests and the timer hasn't hit 120 seconds. Stop and wait 120 seconds. Hitting 429 will incur a 120 second wait so the goal is to fill the bucket until that limit, then wait until I can begin sending requests again.
After all requests are finished, collect the responses in a Vec
Curious other thoughts and ideas on how best to handle this. I've read several other questions about this type of situation but haven't found something that works yet. Also am completely new to async-await in Rust so any refactoring advice helps.
The current async pattern is like the below:
use tokio::time::{ sleep, Duration };
use reqwest::header::HeaderMap;
async fn _make_requests(
headers: &HeaderMap,
requests: &Vec<String>
) -> Result<Vec<String>, Box<dyn std::error::Error>>
{
let client = reqwest::Client::new();
// Each req is a string URL which will pull back the response text from the API
for req in requests
{
let client = client.clone();
tokio::spawn(
match async move {
let resp = client.get(req)
.headers(headers.to_owned())
.send()
.await?
.text()
.await?;
Ok(resp)
}
.await
// Handle resp status in match
{
Ok(resp) => println!("{:?}", resp),
Err(e) => eprintln!("{}", e),
}
);
}
}
fn main()
{
// Create sample headers
let mut headers = HeaderMap::new();
headers.insert("Accept", "application/json".parse().unwrap());
let rt = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
// Run event loop until complete
rt.block_on(_make_requests(&headers, &requests));
Ok(())
}
I would like to chain http request using akka-http-client as Stream. Each http request in a chain depends on a success/response of a previous requests and uses it to construct a new request. If a request is not successful, the Stream should return the response of the unsuccessful request.
How can I construct such a stream in akka-http?
which akka-http client level API should I use?
If you're making a web crawler, have a look at this post. This answer tackles a more simple case, such as downloading paginated resources, where the link to the next page is in a header of the current page response.
You can create a chained source - where one item leads to the next - using the Source.unfoldAsync method. This takes a function which takes an element S and returns Future[Option[(S, E)]] to determine if the stream should continue emitting elements of type E, passing the state to the next invocation.
In your case, this is kind of like:
taking an initial HttpRequest
producing a Future[HttpResponse]
if the response points to another URL, returning Some(request -> response), otherwise None
However, there's a wrinkle, which is that this will not emit a response from the stream if it doesn't contain a pointer to the next request.
To get around this, you can make the function passed to unfoldAsync return Future[Option[(Option[HttpRequest], HttpResponse)]]. This allows you to handle the following situations:
the current response is an error
the current response points to another request
the current response doesn't point to another request
What follows is some annotated code which outlines this approach, but first a preliminary:
When streaming HTTP requests to responses in Akka streams, you need to ensure that the response body is consumed otherwise bad things will happen (deadlocks and the like.) If you don't need the body you can ignore it, but here we use a function to convert the HttpEntity from a (potential) stream into a strict entity:
import scala.concurrent.duration._
def convertToStrict(r: HttpResponse): Future[HttpResponse] =
r.entity.toStrict(10.minutes).map(e => r.withEntity(e))
Next, a couple of functions to create an Option[HttpRequest] from an HttpResponse. This example uses a scheme like Github's pagination links, where the Links header contains, e.g: <https://api.github.com/...> rel="next":
def nextUri(r: HttpResponse): Seq[Uri] = for {
linkHeader <- r.header[Link].toSeq
value <- linkHeader.values
params <- value.params if params.key == "rel" && params.value() == "next"
} yield value.uri
def getNextRequest(r: HttpResponse): Option[HttpRequest] =
nextUri(r).headOption.map(next => HttpRequest(HttpMethods.GET, next))
Next, the real function we'll pass to unfoldAsync. It uses the Akka HTTP Http().singleRequest() API to take an HttpRequest and produce a Future[HttpResponse]:
def chainRequests(reqOption: Option[HttpRequest]): Future[Option[(Option[HttpRequest], HttpResponse)]] =
reqOption match {
case Some(req) => Http().singleRequest(req).flatMap { response =>
// handle the error case. Here we just return the errored response
// with no next item.
if (response.status.isFailure()) Future.successful(Some(None -> response))
// Otherwise, convert the response to a strict response by
// taking up the body and looking for a next request.
else convertToStrict(response).map { strictResponse =>
getNextRequest(strictResponse) match {
// If we have no next request, return Some containing an
// empty state, but the current value
case None => Some(None -> strictResponse)
// Otherwise, pass on the request...
case next => Some(next -> strictResponse)
}
}
}
// Finally, there's no next request, end the stream by
// returning none as the state.
case None => Future.successful(None)
}
Note that if we get an errored response, the stream will not continue since we return None in the next state.
You can invoke this to get a stream of HttpResponse objects like so:
val initialRequest = HttpRequest(HttpMethods.GET, "http://www.my-url.com")
Source.unfoldAsync[Option[HttpRequest], HttpResponse](
Some(initialRequest)(chainRequests)
As for returning the value of the last (or errored) response, you simply need to use Sink.last, since the stream will end either when it completes successfully or on the first errored response. For example:
def getStatus: Future[StatusCode] = Source.unfoldAsync[Option[HttpRequest], HttpResponse](
Some(initialRequest))(chainRequests)
.map(_.status)
.runWith(Sink.last)
I was hoping someone can briefly go over the various ways of consuming a service (this one just returns a string, normally it would be JSON but I just want to understand the concepts here).
My service:
def ping = Action {
Ok("pong")
}
Now in my Play (2.3.x) application, I want to call my client and display the response.
When working with Futures, I want to display the value.
I am a bit confused, what are all the ways I could call this method i.e. there are some ways I see that use Success/Failure,
val futureResponse: Future[String] = WS.url(url + "/ping").get().map { response =>
response.body
}
var resp = ""
futureResponse.onComplete {
case Success(str) => {
Logger.trace(s"future success $str")
resp = str
}
case Failure(ex) => {
Logger.trace(s"future failed")
resp = ex.toString
}
}
Ok(resp)
I can see the trace in STDOUT for success/failure, but my controller action just returns "" to my browser.
I understand that this is because it returns a FUTURE and my action finishes before the future returns.
How can I force it to wait?
What options do I have with error handling?
If you really want to block until feature is completed look at the Future.ready() and Future.result() methods. But you shouldn't.
The point about Future is that you can tell it how to use the result once it arrived, and then go on, no blocks required.
Future can be the result of an Action, in this case framework takes care of it:
def index = Action.async {
WS.url(url + "/ping").get()
.map(response => Ok("Got result: " + response.body))
}
Look at the documentation, it describes the topic very well.
As for the error-handling, you can use Future.recover() method. You should tell it what to return in case of error and it gives you new Future that you should return from your action.
def index = Action.async {
WS.url(url + "/ping").get()
.map(response => Ok("Got result: " + response.body))
.recover{ case e: Exception => InternalServerError(e.getMessage) }
}
So the basic way you consume service is to get result Future, transform it in the way you want by using monadic methods(the methods that return new transformed Future, like map, recover, etc..) and return it as a result of an Action.
You may want to look at Play 2.2 -Scala - How to chain Futures in Controller Action and Dealing with failed futures questions.
I am trying to build a service that grab some pages from another web service and process the content and return results to users. I am using Play 2.2.3 Scala.
val aas = WS.url("http://localhost/").withRequestTimeout(1000).withQueryString(("mid", mid), ("t", txt)).get
val result = aas.map {
response =>
(response.json \ "status").asOpt[Int].map {
st => status = st
}
(response.json \ "msg").asOpt[String].map {
txt => msg = txt
}
}
val rs1 = Await.result(result, 5 seconds)
if (rs1.isDefined) {
Ok("good")
}
The problem is that the service will wait 5 seconds to return "good" even the WS request takes 100 ms. I also cannot set Await time to 100ms because the other web service I am requesting may take between 100ms to 1 second to respond.
My question is: is there a way to process and serve the results as soon as they are ready instead of wait a fixed amount of time?
#wingedsubmariner already provided the answer. Since there is no code example, I will just post what it should be:
def wb = Action.async{ request =>
val aas = WS.url("http://localhost/").withRequestTimeout(1000).get
aas.map(response =>{
Ok("responded")
})
}
Now you don't need to wait until the WS to respond and then decide what to do. You can just tell play to do something when it responds.