I'm using the scala sttp client and in particular, using the Monix backend and I have the following code:
def httpTask(href: String): Task[HttpBinaryResponse] = {
val request = basicRequest
.get(uri"$href")
AsyncHttpClientMonixBackend
.resource()
.use { backend =>
request.send(backend).map {
response: Response[Either[String, String]] =>
println(s"For URL: $href >> Got response code: ${response.code}")
HttpBinaryResponse(
href,
response.code.isSuccess,
response.headers.map(elem => (elem.name, elem.value)))
}
}
}
I have a set of URL's that I run through this method:
hrefs.toSet.flatten.filter(_.startsWith("http")).map(httpTask) // Set[Task[HttpBinaryResponse]]
In my caller function, I then execute the task:
val taskSeq = appBindings.httpService.parse(parserFilter)
val chunks = taskSeq.sliding(30, 30).toSeq
val batchedTasks = chunks.map(chunk => Task.parSequence(chunk))
val allBatches = Task.sequence(batchedTasks).map(_.flatten)
I then pattern match on the materialized type and return, bit what I want here is not just a Success and a Failure, but I would like to have both the values at once:
allBatches.runToFuture.materialize.map {
case Success(elem) =>
Ok(
Json
.obj("status" -> "Ok", "URL size" -> s"${elem.size}")
).enableCors
case Failure(err) =>
Ok(
Json
.obj("status" -> "error", "message" -> s"${err.getMessage}")
).enableCors
}
I need to get a List of URL's with a corresponding Success or a Failure message. With this approach of what I have, I get only either one of those, which is obvious as I pattern match. How can I collect both Success and Failure?
Related
I am trying to redirect to an external URL with some authorization code attached to it. I am using the "Redirect" function available in play framework using Scala. But instead of redirecting to the provided URL, the message in "Ok" gets printed and is not redirected
I am a beginner in Play and Scala. It is my understanding that an "Action" should send a "Result". This "Result" can either be "Ok" or a "Redirect". In the below code:
1) If I don't mention "Ok", there is a compile error
2) If I execute the code below, it doesn't get redirected, it just prints the message in "Ok"
// This is inside a controller
// it is defined as a GET in the routes file
def test = Action { implicit request =>
async {
await(userAuth.getAuth()) match{
case Some(userAuth) if <>
val url = <FunctionReturnsUrlwithcode>
Redirect(url)
case _ if flag
val url = <FunctionReturnsUrlwithcode>
Redirect(url)
}
Ok("Some message") // Without this , there is an error
}
I assume this is pseudocode, since that pattern matching isn't syntactically correct...
Your pattern matching isn't exhaustive. For example:
def test(): String = {
val x = Some(1)
x match {
case Some(y) if y == 2 => return "matched 1"
case _ if false == true => return "matched 2"
}
"matched nothing"
}
The above code will return "matched nothing", since 1 != 2 and false != true. There needs to be a default case, otherwise if any of the specified conditions aren't met it'll ignore the pattern matching altogether.
In your case, I'm guessing that neither of the conditions in your pattern matching are being met. Therefore, the pattern matching is skipped over and Ok(whatever) is returned - if you take out the Ok, your app will blow up since an Action must return a Result. Your code will also return the Ok if you write it like this:
def test = Action { implicit request =>
async {
await(userAuth.getAuth()) match{
case Some(userAuth) if <> =>
val url = <FunctionReturnsUrlwithcode>
Redirect(url)
case _ if flag =>
val url = <FunctionReturnsUrlwithcode>
Redirect(url)
case _ =>
Ok("Some message")
}
}
}
Also,
This "Result" can either be "Ok" or a "Redirect"
This is not true; it isn't limited to only two values. It can be a number of other things, such as NoContent, Created, BadRequest, InternalServerError, etc. (basically any valid HTTP status).
You can try the below:
def test: Action[AnyContent] = Action.async { implicit request =>
//your logic
async {
await(userAuth.getAuth()) match{
case Some(userAuth) if <> =>
val url = <FunctionReturnsUrlwithcode>
(Redirect(url, Map("traceid"-> Set("trace")))
case _ if flag =>
val url = <FunctionReturnsUrlwithcode>
(Redirect(url, Map("traceid"-> Set("trace")))
case _ =>
Ok("Some message")
}
}
I am using Akka HTTP cache to cache my result. But i am facing issue to test it.
class GoogleAnalyticsController #Inject()(cache: Cache[String, HttpResponse],
googleAnalyticsApi: GoogleAnalyticsTrait,
googleAnalyticsHelper: GoogleAnalyticsHelper)
(implicit system: ActorSystem, materializer: ActorMaterializer) {
def routes: Route =
post {
pathPrefix("pageviews") {
path("clients" / Segment) { accountsClientId =>
entity(as[GoogleAnalyticsMetricsRequest]) { googleAnalyticsMetricsRequest =>
val googleAnalyticsMetricsKey = "key"
complete(
cache.getOrLoad(googleAnalyticsMetricsKey, _ => getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest))
)
}
}
}
}
private def getGoogleAnalyticsMetricsData(accountsClientId: String,
request: GoogleAnalyticsMetricsRequest) = {
val payload = generate(request)
val response = googleAnalyticsApi.googleAnalyticsMetricResponseHandler(accountsClientId, payload) // response from another microservice
googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(
googleAnalyticsMetricsRequest.metricName, response)
}
}
class GoogleAnalyticsHelper extends LoggingHelper {
def googleAnalyticsMetricResponseHandler(metricName: String, response: Either[Throwable, Long]): Future[HttpResponse] =
response.fold({ error =>
logger.error(s"An exception has occurred while getting $metricName from behavior service and error is ${error.getMessage}")
Marshal(FailureResponse(error.getMessage)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.InternalServerError))
}, value =>
Marshal(MetricResponse(metricName, value)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
)
}
Test case: Sharing only the relevant part
"get success metric response for " + pageviews + " metric of given accounts client id" in { fixture =>
import fixture._
val metricResponse = MetricResponse(pageviews, 1)
val eventualHttpResponse = Marshal(metricResponse).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
when(googleAnalyticsApi.getDataFromGoogleAnalytics(accountsClientId, generate(GoogleAnalyticsRequest(startDate, endDate, pageviews))))
.thenReturn(ApiResult[Long](Some("1"), None))
when(googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(pageviews, Right(1))).thenReturn(eventualHttpResponse)
Post(s"/pageviews/clients/$accountsClientId").withEntity(requestEntity) ~>
googleAnalyticsController.routes ~> check {
status shouldEqual StatusCodes.OK
responseAs[String] shouldEqual generate(metricResponse)
}
}
By doing this, I am best to test if the cache has the key but not able to test if cache misses the hit. In code coverage, it misses following highlighted part
cache.getOrLoad(googleAnalyticsMetricsKey, _ =>
getGoogleAnalyticsMetricsData(accountsClientId,
googleAnalyticsMetricsRequest))
If there is a design issue, please feel free to guide me on how can I make my design testable.
Thanks in advance.
I think you don't need to mock the cache. You should create an actual object for cache instead of mocked one.
What you have done is, you have mocked the cache, in this case, the highlighted part will be not called as you are providing the mocked value. In the following stubbing, whenever cache.getOrLoad is found, eventualHttpResponse is returned:
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
and hence the function getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest) is never called.
Q: Where is the correct place to be calling Ok() to send http response from an async database call?
I have taken the very basic Scala Play framework tutorial play-scala-starter-example as a starting point and adding some additional basic Controller / Service classes that make use of ReactiveCouchbase for database access.
The application successfully:
Connects to Couchbase
Stores a JSON document in Couchbase
Retrieves the stored JSON document from Couchbase
Logs the contents of the JSON to the console
I am new to Scala/Play and can't work out the correct way to succesfully write the JSON back to the http response using Ok(), when the async db call completes.
Inside the Controller class is the following function:
def storeAndRead() = Action {
testBucket
.insert[JsValue]("key1", Json.obj("message" -> "Hello World", "type" -> "doc"))
val res = testBucket
.get("key1")
.map(i => Json.toJson(i.get))
.map(j => Ok(j)) // PROBLEM LINE
}
Looking at the "//PROBLEM LINE", having the Ok() call inside the map leads to a compile error:
CouchbaseController.scala:30:19: Cannot write an instance of Unit to HTTP response. Try to define a Writeable[Unit]
Placing the call to Ok() later on, fails with a different compile error:
def storeAndRead() = Action {
testBucket
.insert[JsValue]("key1", Json.obj("message" -> "Hello World", "type" -> "doc"))
val res = testBucket
.get("key1")
.map(i => Json.toJson(i.get))
Ok(res)
}
Compile error:
CouchbaseController.scala:35:7: Cannot write an instance of scala.concurrent.Future[play.api.libs.json.JsValue] to HTTP response. Try to define a Writeable[scala.concurrent.Future[play.api.libs.json.JsValue]]
In this second case, I believe the issue is that the Future may not yet have completed when Ok() is called?
Lastly, I have tried placing the call to Ok() inside an onSuccess() function, in an effort to ensure it is called after the async function has completed:
def storeAndRead() = Action {
testBucket
.insert[JsValue]("key1", Json.obj("message" -> "Hello World", "type" -> "doc"))
val res = testBucket
.get("key1")
.map(i => Json.toJson(i.get))
.onSuccess {
//case doc => Console.println("completed: " + doc)
case doc => Ok(doc)
}
}
Again...compilation error:
CouchbaseController.scala:22:24: overloaded method value apply with alternatives:
[error] (block: => play.api.mvc.Result)play.api.mvc.Action[play.api.mvc.AnyContent] <and>
[error] (block: play.api.mvc.Request[play.api.mvc.AnyContent] => play.api.mvc.Result)play.api.mvc.Action[play.api.mvc.AnyContent] <and>
[error] [A](bodyParser: play.api.mvc.BodyParser[A])play.api.mvc.ActionBuilder[play.api.mvc.Request,A]
[error] cannot be applied to (Unit)
[error] def storeAndRead() = Action {
Question:
I am clearly missing something fairly fundamental:
Where should Ok() be called in this kind of basic scenario? I assume it needs to be called as a result of a callback when the async db request has completed?
What's the correct and appropriate way to structure this in an async way for Scala/Play?
Handle future results using Action.async
Play knows how to handle Future (async call). You have to use Action.async.
For instance :
def myAction = Action.async {
// ...
myFuture.map(resp => Ok(Json.toJson(resp)))
}
In your case :
def storeAndRead() = Action.async {
// by the way, the expression behind probably returns a future, you should handle it
testBucket
.insert[JsValue]("key1", Json.obj("message" -> "Hello World", "type" -> "doc"))
testBucket
.get("key1")
.map(i => Json.toJson(i.get))
.map(j => Ok(j))
}
You have to return a Result (or a Future[Result])
You get the error CouchbaseController.scala:30:19: Cannot write an instance of Unit to HTTP response. Try to define a Writeable[Unit] because you don't return anything. A Result is expected here.
Handle futures chain
Futhermore, you should handle the call of several Futures. If you don't, you will get silent errors even if the client received the http response.
For instance :
def storeAndRead() = Action.async {
for {
_ <- testBucket.insert[JsValue]("key1", Json.obj("message" -> "Hello World", "type" -> "doc"))
value <- testBucket.get("key1")
} yield Ok(Json.toJson(value))
}
I have a function which will take a Token via ajax request. It will return a response a list of directories while the token is valid.
def all = Action.async(parse.json) {
implicit request => tokenForm.bind(request.body).fold(
formWithErrors => Future.successful(BadRequest(formWithErrors.toString)),
form => checkToken(form.token).map(token => {
val directories = Directories.all.map(directory => {
Json.toJson(directory)
})
Ok(Json.obj("status" -> {if (token.get.id.getOrElse(0) >= 1) true else false}, "message" -> {if (token.get.id.getOrElse(0) >= 1) Json.toJson(directories) else "Invalid token"}))
})
)
}
While I run the above code It says [As ^ sign indicate the position the error found]
No Json serializer found for type scala.concurrent.Future[play.api.libs.json.JsValue]. Try to implement an implicit Writes or Format for this type.
Ok(Json.obj("status" -> {if (token.get.id.getOrElse(0) >= 1) true else false}, "message" -> {if (token.get.id.getOrElse(0) >= 1) Json.toJson(directories) else "Invalid token"}))
^
Here is what I come up with
def all = Action.async(parse.json) {
implicit request => tokenForm.bind(request.body).fold(
formWithErrors => Future.successful(BadRequest(formWithErrors.toString)),
form => for {
(token, directories) <- checkToken(form.token) zip Directories.all
} yield {
val isTokenValid = token.isDefined
val responseBody = Json.obj(
"status" -> isTokenValid,
"message" -> {
if (isTokenValid)
Json.toJson(directories)
else
"Invalid token"
}
)
Ok(responseBody)
}
)
}
asuming checkToken and Directories.all returns Future.
Your function needs to return Future[Result]. When you are folding, the first branch is correct
formWithErrors => Future.successful(BadRequest(formWithErrors.toString))
However in the second branch it seems like you started a new Future by calling Directories.all, and you want to serialize it and return as json. There is no serializer for Future[JsValue] as this makes no sense, it would have to block to get result. You need to somehow get to the values of both checkToken(form.token) and Directories.all.
You can do this as I did above, or for example like this:
form => {
val futureToken = checkToken(form.token)
val futureDirectories = Directories.all
for {
token <- futureToken
directories <- futureDirectories
} yield {
// same code as above
}
}
Notice that if you would inline futureToken and futureDirectories they would be executed serially.
Also note that you are converting your directory list to json twice.
Once here
val directories = Directories.all.map(directory => {
Json.toJson(directory)
})
Asuming Directories.all returns Future[List[Directory]], then when you use map, the function you pass operates on List[Directory] so the variable should be named directories not directory. It will work, play easly converts list of anything convertible to json, no need to do it manually.
Second time you did it here
Json.toJson(directories)
I am trying to make a basic http client posting some data to a REST API with Akka HTTP but I cannot make the following code work :
def sendData(e : GenericEvent): Future[Either[(String,StatusCode),(GenericEvent,StatusCode)]] = {
val request = Marshal(e).to[RequestEntity]
val responseFuture: Future[HttpResponse] = request map { req =>
Source.single(HttpRequest(method = HttpMethods.POST, uri = s"/data-ingest", headers = List(auth), entity = req))
.via(dataIngestFlow)
.runWith(Sink.head)
}
responseFuture.flatMap { response =>
response.status match {
case OK => Unmarshal(response.entity).to[GenericEvent].map(Right(_, response.status))
case BadRequest => Future.successful(Left(s"$e.data: incorrect data", response.status))
case _ => Unmarshal(response.entity).to[String].flatMap { entity =>
val error = s"generic event ingest failed with status code ${response.status} and entity $entity"
logger.error(error)
Future.failed(new IOException(error))
}
}
}
I got the following error
polymorphic expression cannot be instantiated to expected type;
[error] found :
[T]akka.stream.scaladsl.Sink[T,scala.concurrent.Future[T]]
[error] required:
akka.stream.Graph[akka.stream.SinkShape[akka.http.scaladsl.model.HttpResponse],akka.http.scaladsl.model.HttpResponse]
[error] .runWith(Sink.head)
Here is the code for the dataIngestFlow
val dataIngestFlow = Http().outgoingConnection(config.endpointUrl,config.endpointPort)
Here is the code on server side :
val routes = {
logRequestResult("akka-http-microservice") {
path("event-ingest") {
post {
entity(as[GenericEvent]) { eventIngest =>
log.info(s"Ingesting {} and publishing event to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
complete {
eventIngest
}
}~
entity(as[List[GenericEvent]]) { eventIngestList =>
eventIngestList.foreach{ eventIngest=>
log.info(s"Ingesting {} and publishing event List to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
}
complete {
eventIngestList
}
}
}
}
}
}
I tried another simple client, it builds well but the ingestion stop after 160 events, the server doesn't receive anymore events.
The first problem I see with your example is that RequestEntity does not have a map function. Therefore, the following line
val responseFuture: Future[HttpResponse] = request map { ...
should not compile.
Further, if request is actually a Future (which I infer from the map) then responseFuture is actually of type Future[Future[HttpResponse]] because the stream materializes into its own Future. To solve this problem you can use Future.flatMap instead of map. Namely:
val responseFuture: Future[HttpResponse] = request flatMap { req =>
This is the monadic bind operation within Futures.