I am trying to run my webapp that is based on http4s on jetty. The http4s library provides AsyncHttp4sServlet[IO] class to extend and I use as follows:
class UserSvcServlet
extends AsyncHttp4sServlet[IO](
service = UserSvcServer.start,
servletIo = BlockingServletIo(4096, Blocker.liftExecutorService(Executors.newCachedThreadPool())),
serviceErrorHandler = DefaultServiceErrorHandler
)
As you can see on the service property, I provide my http service that has the following implementation:
def start[F[_]: ConcurrentEffect: ContextShift: Sync: Timer]: HttpApp[F] =
for {
a <- EnvironmentLoader.db.load[F].map(create_transactor)
b <- EnvironmentLoader.cors.load[F].map(origin)
http = Logger.httpApp(true, true)(CORS(UserRoutes(UserProgram(LiveUserQuery.make(a))).routes, b).orNotFound)
} yield http
The start method should return HttpApp[F] but unfortunately the for block returns F[Http[F]]. However at the end the F will be an IO type.
Here is the definition of HttpApp[F]:
type Http[F[_], G[_]] = Kleisli[F, Request[G], Response[G]]
The both EnvironmentLoader.db.load[F].map(create_transactor) and EnvironmentLoader.cors.load[F].map(origin) is in the context of F and they load environment variables. For loading environment variables, I use the library https://cir.is/.
I know, it is not possible to get Http[F] out of the context F. Do I have here to restructure the code to make it work?
Update
One possible workaround would be:
class UserSvcServlet
extends AsyncHttp4sServlet[IO](
service = UserSvcServer.start[IO].unsafeRunSync(),
servletIo = BlockingServletIo(4096, Blocker.liftExecutorService(Executors.newCachedThreadPool())),
serviceErrorHandler = DefaultServiceErrorHandler
)
It is ugly but it works.
Yes, you have to.
When you have F[Http[F]] you are supposed to compose it into your IO program:
for {
dep1 <- createDependency1
dep1 <- createDependency2
http <- createHttp // : F[HttpApp[F]] = F[Request[F] => F[Response[F]]]
// or "use side effects to create a function that
// will take side-effecting request to create
// recipe for side-effecting response" (e.g.
// reading from stream to server and returning
// a stream from server)
dep4 = needHttp(http)
...
} yield ...
In your code a change from F[HttpApp[F]] to HttpApp[F] seems to literally require just changing:
http = ...
into
http <- ...
which kind of suggest, that you are not paying enough attention to what your code does.
Related
I will start mentioning I am very new to Scala but I have now to maintain a legacy code where some new feature are being tried to be include.
I have the following code:
Where a list is coming as a parameter where a new output needs to be processed. However it seems like code is not waiting for the response to the external service when processing.
def historyBet(jackpotListUser : List[JackpotBetHistory])(implicit MC: AppMarkerContext) : List[LegacyJackpotHistoryResponse] =
for {
bet <- jackpotListUser
prize = jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId) match {
case Some(jackpot : JackpotResponse) =>
...
extra code extracting price from jackpot : JackpotResponse
...
extra code generating result with prize
} yield result
How can I do a call to jackpotIntegratorService.findJackpotByJackpotHumanId to execute at that time. instead of returning something that F[Option....?
def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit MC: AppMarkerContext): F[Option[JackpotResponse]] =
jackpotIntegratorRepo.findJackpotByJackpotHumanId(jackpotHumanId)
where it is finally implemented as:
override def findJackpotByJackpotHumanId(
jackpotHumanId: JackpotHumanId
)(implicit mc: AppMarkerContext): IO[Option[JackpotResponse]] =
... code calling an API which return the IO.
Thanks!
I thought I could do IO.await somewhere... but not sure where or how...
because in the "historyBet" function I got a F[] when it was an IO... so what is the syntax to be able to wait for the response and the continue?
Extra Comment:
The real issue we notice is that the method call is starting (the logs shows part of it) but the caller with in the maps continues too.
prize = jackpotIntegratorService.findJackpotByJackpotHumanId
this part of the code continues even when prize, which we want the final object JackpotResponse, not the IO or F.
So, if your method needs to call an IO then it must return an IO unless you unsafeRunSync them... but, as the name suggest, you should not do that.
So the return type is now: IO[List[LegacyJackpotHistoryResponse]
And can be implemented like this:
def historyBet(jackpotListUser: List[JackpotBetHistory])(implicit MC: AppMarkerContext): IO[List[LegacyJackpotHistoryResponse]] =
jackpotListUser.traverse { bet =>
jackpotIntegratorService.findJackpotByJackpotHumanId(bet.jackpotHumanId).map {
case Some(jackpot) =>
// ...
case None =>
// ...
}
}
I am playing with ZIO and built a simple application that get content via HTTP :
for {
options <- Options.parse(args)
http = HttpClient(args)
content <- Download.execute(args.resource).provide(http)
} yield ()
It does the job but the client is backed by Play StandaloneWsClient and I would like to close it and terminate the actor system as described in the documentation: https://github.com/playframework/play-ws#scala-1
So I created a finaliser method but it seems that is has no effect:
// ...
content <- Download.execute(args.resource).ensuring(http.disconnect()).provide(http)
// ...
class HttpClient {
// ...
def disconnect():UIO[Unit] = ZIO.effectTotal {
client.close()
system.terminate()
}
How can I instruct ZIO to call a finaliser method to free up my resources ?
In a play-scala application, Im making a WS call:
def getaddresses : ListBuffer[String] = {
var lb = new ListBuffer[String]()
var url = "xyz#xyx.com/someapi"
WS.url(url).get.map {
response =>
var emailsSeq = (response.json \\ "email")
emailsSeq.foreach(lb+=_.toString())
lb.foreach(println) //This is filled with values
}
lb.foreach(println) //This is empty
lb
}
Inside map a sequence is returned whose entries I put in a ListBuffer lb to be returned by this method.
The problem is that the the ListBuffer shows values inside map{} but does not show outside it. As a result empty ListBuffer is passed from the method.
I thought there might be a delay in ws response so tried the following which had no advantage:
WS.url(url).withRequestTimeout(10.seconds).get.map {
Please help me in getting filled list buffer to be returned
I think that ws.url(url).get method is async, so when you call lb.foreach(println), there is nothing to print. Try to add Thread.sleep just after map block of code. If it is so, you should make getaddresses method async, or use future/promise.
PS: you should use val instead of var, just for cleaner code.
EDIT: Sample to try:
def getaddresses : ListBuffer[String] = {
val url = "xyz#xyx.com/someapi"
val promiseOfAddresses = Promise[ListBuffer[String]]()
WS.url(url).get.map {
response =>
var emailsSeq = (response.json \\ "email")
promiseOfAddresses.success(emailsSeq) // here you will complete promise with actual value
}
val lb = promiseOfAddresses.future.get // here you will obtain future from promise and also get value of future. Method get is blocking and it will wait till promise is fullfiled or failed
lb.foreach(println) //This is empty
lb
}
PS2: Probably best help with future/promises can be found here: the-neophytes-guide-to-scala-part-9-promises-and-futures-in-practice (I have not enough reputation, so google this blog) It is something like CompletableFuture in java world. But remember, that best way is to stay all time in red side (red = asynchronous functions/methods. see nice, but chatty, blog what-color-is-your-function)
I create a thrift server using Finagle like this
val server = Thrift.serveIface(bindAddr(), new MyService[Future] {
def myRPCFuction() {}
})
But, I found that the maximum number of concurrent requests is five( why 5? when more than 5, the server just ignore the excessed ones.) I look through the doc of Finagle really hard (http://twitter.github.io/finagle/guide/Protocols.html#thrift-and-scrooge), but find nothing hint to configure the max-request-limit.
How to config the maximum concurrent request num of Finagle? Thanks
I've solved this problem by myself and I share it here to help others who may run into the same case. Because I m a thrift user before and in Thrift when you return from the RPC function you return the values back to calling client. While in Finagle only when you use Future.value() you return the value to client. And when use Finagle, you should totally use the asynchronous way, that's to say you had better not sleep or do some other RPC synchronously in the RPC function.
/* THIS is BAD */
val server = Thrift.serveIface(bindAddr(), new MyService[Future] {
def myRPCFuction() {
val rpcFuture = rpcClient.callOtherRpc() // call other rpc which return a future
val result = Await.result(rpcFuture, TwitterDuration(rpcTimeoutSec()*1000, MILLISECONDS))
Future.value(result)
}
})
/* This is GOOD */
val server = Thrift.serveIface(bindAddr(), new MyService[Future] {
def myRPCFuction() {
val rpcFuture = rpcClient.callOtherRpc() // call other rpc which return a future
rpcFuture onSuccess { // do you job when success (you can return to client using Future.value) }
rpcFuture onFailure { // do your job when fail }
}
})
Then, can get a satisfactory concurrency. Hope it helps others who have the same issue.
I want to forbid the full access to the Solr core from outside, and let it be used only for querying. Thus I am launching secondary server w/ connector instance inside Jetty servlet container (besides, the main webapp) on the port, that is not accessible from the WWW.
When there is incoming HTTP request to the liftweb application, I hook with RestHelper:
object Dispatcher extends RestHelper {
serve {
case List("api", a # _*) JsonGet _ => JString("API is not implemented yet. rest: " + a)
}
}
Targeting my browser to http://localhost/api/solr/select?q=region I get a response "API is not implemented yet. rest: List(solr, select)", so it seems to work. Now I want to do a connection on internal port (where Solr resides) in order to pass the query using the post-api part of the URL (i.e. http://localhost:8080/solr/select?q=region). I am catching the trailing REST-part of the URL (by means of a # _*), but how can I access URL parameters? It would be ideal to pass a raw string (after api path element) to the Solr instance, just to prevent redundant parse/build steps. So applies to the Solr's response: I would like to avoid parsing building JsonResponse.
This seems to be a good example on doing some HTTP-redirection, but then I would have to open the hidden Solr's port, as far as I can understand.
What is the most effective way to cope with this task?
EDIT:
Well, I missed that after JsonGet comes Req value, which has all the needed info. But is there still a way to avoid unwanted parsing/composing URL to hidden port and JSON-response?
SOLUTION:
This is what I've got consdering Dave's suggestion:
import net.liftweb.common.Full
import net.liftweb.http.{JsonResponse, InMemoryResponse}
import net.liftweb.http.provider.HTTPRequest
import net.liftweb.http.rest.RestHelper
import dispatch.{Http, url}
object ApiDispatcher extends RestHelper {
private val SOLR_PORT = 8080
serve { "api" :: Nil prefix {
case JsonGet(path # List("solr", "select"), r) =>
val u = localApiUrl(SOLR_PORT, path, r.request)
Http(url(u) >> { is =>
val bytes = Stream.continually(is.read).takeWhile(-1 !=).map(_.toByte).toArray
val headers = ("Content-Length", bytes.length.toString) ::
("Content-Type", "application/json; charset=utf-8") :: JsonResponse.headers
Full(InMemoryResponse(bytes, headers, JsonResponse.cookies, 200))
})
}}
private def localApiUrl(port: Int, path: List[String], r: HTTPRequest) =
"%s://localhost:%d/%s%s".format(r.scheme, port, path mkString "/", r.queryString.map("?" + _).openOr(""))
}
I'm not sure that I understand your question, but if you want to return the JSON you receive from solr without parsing it you could use a net.liftweb.http.InMemoryResponse that contains a byte[] representation of the JSON.