How to retrieve error message in HTTP4S client? - scala

I am trying to use Blaze client for http4s to make http calls. It works fine when 200 response is returned but in case of HTTP 500 or 400 I am not able to figure out how to retrieve the detailed error message returned from server.
I am can only see folllwing in the logs for this client.
2022-11-10 23:37:40 INFO - Shutting down connection pool:
curAllocated=1 idleQueues.size=1 waitQueue.size=0
maxWaitQueueLimit=256 closed=false org.http4s.client.UnexpectedStatus:
unexpected HTTP status: 500 Internal Server Error
def callEffect(client: Client[IO]): IO[String] = {
val uri = serverUrl
.withPath("/abc")
val request = GET(uri, Accept(MediaType.application.json))
client.expect[String](request).map { res =>
println(res)
res
}
}
def deploy(implicit cs: ContextShift[IO]): IO[ExitCode] = {
BlazeClientBuilder[IO](scala.concurrent.ExecutionContext.global).resource
.map(x => println(callEffect(x).unsafeRunSync()))
.use(_ => IO.unit.as(ExitCode.Success))
}

Since the result is provided inside an IO, you can use any of IO's error handling features to handle the issues, for example with redeem
def run(args: List[String]): IO[ExitCode] = {
BlazeClientBuilder[IO].resource
.use(client =>
for {
result <- callEffect(client).redeem(
error => "could not get a result",
something => s"this is what I got: $something"
)
_ <- IO.println(result)
} yield ExitCode.Success
)
}
There's a great article on this topic here:
https://softwaremill.com/practical-guide-to-error-handling-in-scala-cats-and-cats-effect/
For even more control, you can handle errors from the client itself, for example using expectOr() instead of expect(), which gives you access to a convenient onError: Response[F] => F[Throwable].

Related

Facing issue testing akka http cache

I am using Akka HTTP cache to cache my result. But i am facing issue to test it.
class GoogleAnalyticsController #Inject()(cache: Cache[String, HttpResponse],
googleAnalyticsApi: GoogleAnalyticsTrait,
googleAnalyticsHelper: GoogleAnalyticsHelper)
(implicit system: ActorSystem, materializer: ActorMaterializer) {
def routes: Route =
post {
pathPrefix("pageviews") {
path("clients" / Segment) { accountsClientId =>
entity(as[GoogleAnalyticsMetricsRequest]) { googleAnalyticsMetricsRequest =>
val googleAnalyticsMetricsKey = "key"
complete(
cache.getOrLoad(googleAnalyticsMetricsKey, _ => getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest))
)
}
}
}
}
private def getGoogleAnalyticsMetricsData(accountsClientId: String,
request: GoogleAnalyticsMetricsRequest) = {
val payload = generate(request)
val response = googleAnalyticsApi.googleAnalyticsMetricResponseHandler(accountsClientId, payload) // response from another microservice
googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(
googleAnalyticsMetricsRequest.metricName, response)
}
}
class GoogleAnalyticsHelper extends LoggingHelper {
def googleAnalyticsMetricResponseHandler(metricName: String, response: Either[Throwable, Long]): Future[HttpResponse] =
response.fold({ error =>
logger.error(s"An exception has occurred while getting $metricName from behavior service and error is ${error.getMessage}")
Marshal(FailureResponse(error.getMessage)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.InternalServerError))
}, value =>
Marshal(MetricResponse(metricName, value)).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
)
}
Test case: Sharing only the relevant part
"get success metric response for " + pageviews + " metric of given accounts client id" in { fixture =>
import fixture._
val metricResponse = MetricResponse(pageviews, 1)
val eventualHttpResponse = Marshal(metricResponse).to[HttpResponse].map(httpResponse => httpResponse.copy(status = StatusCodes.OK))
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
when(googleAnalyticsApi.getDataFromGoogleAnalytics(accountsClientId, generate(GoogleAnalyticsRequest(startDate, endDate, pageviews))))
.thenReturn(ApiResult[Long](Some("1"), None))
when(googleAnalyticsHelper.googleAnalyticsMetricResponseHandler(pageviews, Right(1))).thenReturn(eventualHttpResponse)
Post(s"/pageviews/clients/$accountsClientId").withEntity(requestEntity) ~>
googleAnalyticsController.routes ~> check {
status shouldEqual StatusCodes.OK
responseAs[String] shouldEqual generate(metricResponse)
}
}
By doing this, I am best to test if the cache has the key but not able to test if cache misses the hit. In code coverage, it misses following highlighted part
cache.getOrLoad(googleAnalyticsMetricsKey, _ =>
getGoogleAnalyticsMetricsData(accountsClientId,
googleAnalyticsMetricsRequest))
If there is a design issue, please feel free to guide me on how can I make my design testable.
Thanks in advance.
I think you don't need to mock the cache. You should create an actual object for cache instead of mocked one.
What you have done is, you have mocked the cache, in this case, the highlighted part will be not called as you are providing the mocked value. In the following stubbing, whenever cache.getOrLoad is found, eventualHttpResponse is returned:
when(cache.getOrLoad(anyString, any[String => Future[HttpResponse]].apply)).thenReturn(eventualHttpResponse)
and hence the function getGoogleAnalyticsMetricsData(accountsClientId, googleAnalyticsMetricsRequest) is never called.

How to get error from akka-stream propagate to akka-http to both be logged and notify the client properly?

Right now I am using akka-stream and akka-HTTP to build a file streaming API. As such I am injecting a streaming source into an entity to have data streamed directly to the HTTP client like so:
complete(HttpEntity(ContentTypes.`application/octet-stream`, source))
However, if for some reason the stream fails, the connection gets closed by akka-http without further explanation or logging.
I would need 2 things:
How can I get the exception logs?
How can I notify my client with a message before closing the connection?
Thank you
As mentioned in comment HTTP protocol does not allow to signal error to the client side.
As to logging:
For me it boils down to missing proper access log directive in akka http.
In my current project we have decorator which register onComplete handler for http entity before giving it to akka http for rendering.
private def onResponseStreamEnd(response: HttpResponse)(action: StatusCode => Unit): HttpResponse =
if (!response.status.allowsEntity() || response.entity.isKnownEmpty()) {
action(response.status)
response
} else {
val dataBytes =
onStreamEnd(response.entity) { result =>
val overallStatusCode =
result match {
case Success(_) =>
response.status
case Failure(e) =>
logger.error(e, s"error streaming response [${e.getMessage}]")
StatusCodes.InternalServerError
}
action(overallStatusCode)
}
response.withEntity(response.entity.contentLengthOption match {
case Some(length) => HttpEntity(response.entity.contentType, length, dataBytes)
case None => HttpEntity(response.entity.contentType, dataBytes)
})
}
private def onStreamEnd(entity: HttpEntity)(onComplete: Try[Done] ⇒ Unit): Source[ByteString, _] =
entity.dataBytes.alsoTo { Sink.onComplete(onComplete) }
Usage:
complete(onResponseStreamEnd(HttpResponse(StatusCodes.OK, HttpEntity(ContentTypes.`application/octet-stream`, source))){ statusCode => .... })
Similar approach but using custom graph stage you can find here

Akka-http how to log server error responses?

I have a http server built with scala and akka-http. Is there a way I can log the error responses sent to clients?
You can take a look at the documentation LogResult Directive
Here you can find the same snipped of code shown in the documentation:
// different possibilities of using logResponse
// The first alternatives use an implicitly available LoggingContext for logging
// marks with "get-user", log with debug level, HttpResponse.toString
DebuggingDirectives.logResult("get-user")
// marks with "get-user", log with info level, HttpResponse.toString
DebuggingDirectives.logResult(("get-user", Logging.InfoLevel))
// logs just the response status at debug level
def responseStatus(res: RouteResult): String = res match {
case RouteResult.Complete(x) => x.status.toString
case RouteResult.Rejected(rejections) => "Rejected: " + rejections.mkString(", ")
}
DebuggingDirectives.logResult(responseStatus _)
// logs just the response status at info level
def responseStatusAsInfo(res: RouteResult): LogEntry = LogEntry(responseStatus(res), Logging.InfoLevel)
DebuggingDirectives.logResult(responseStatusAsInfo _)
// This one doesn't use the implicit LoggingContext but uses `println` for logging
def printResponseStatus(res: RouteResult): Unit = println(responseStatus(res))
val logResultPrintln = DebuggingDirectives.logResult(LoggingMagnet(_ => printResponseStatus))
// tests:
Get("/") ~> logResultPrintln(complete("logged")) ~> check {
responseAs[String] shouldEqual "logged"
}

How to keep connection open for all the time in websockets

Server code :
object EchoService {
def route: Route = path("ws-echo") {
get {
handleWebSocketMessages(flow)
}
} ~ path("send-client") {
get {
sourceQueue.map(q => {
println(s"Offering message from server")
q.offer(BinaryMessage(ByteString("ta ta")))
} )
complete("Sent from server successfully")
}
}
val (source, sourceQueue) = {
val p = Promise[SourceQueue[Message]]
val s = Source.queue[Message](100, OverflowStrategy.backpressure).mapMaterializedValue(m => {
p.trySuccess(m)
m
})
(s, p.future)
}
val flow =
Flow.fromSinkAndSourceMat(Sink.ignore, source)(Keep.right)
}
Client Code :
object Client extends App {
implicit val actorSystem = ActorSystem("akka-system")
implicit val flowMaterializer = ActorMaterializer()
val config = actorSystem.settings.config
val interface = config.getString("app.interface")
val port = config.getInt("app.port")
// print each incoming strict text message
val printSink: Sink[Message, Future[Done]] =
Sink.foreach {
case message: TextMessage.Strict =>
println(message.text)
case _ => println(s"received unknown message format")
}
val (source, sourceQueue) = {
val p = Promise[SourceQueue[Message]]
val s = Source.queue[Message](100, OverflowStrategy.backpressure).mapMaterializedValue(m => {
p.trySuccess(m)
m
})
(s, p.future)
}
val flow =
Flow.fromSinkAndSourceMat(printSink, source)(Keep.right)
val (upgradeResponse, sourceClosed) =
Http().singleWebSocketRequest(WebSocketRequest("ws://localhost:8080/ws-echo"), flow)
val connected = upgradeResponse.map { upgrade =>
// just like a regular http request we can get 404 NotFound,
// with a response body, that will be available from upgrade.response
if (upgrade.response.status == StatusCodes.SwitchingProtocols || upgrade.response.status == StatusCodes.OK ) {
Done
} else {
throw new RuntimeException(s"Connection failed: ${upgrade.response.status}")
}
}
connected.onComplete(println)
}
when i hit http://localhost:8080/send-client i see messages coming to client but after a while if try to send to client again i don't see any messages on client side :s . I also tried source.concatMat(Source.maybe)(Keep.right) but no luck :(
Edit : I tested with js client, somehow connection/flow closed on server end , is there anyway to prevent this ? and how can i listen to this event while using akka-http websocket client :s
Hi,
The reason why it does not keep connected is because by default all
HTTP connections have idle-timeout on by default to keep the system
from leaking connections if clients disappear without any signal.
One way to overcome this limitation (and actually my recommended
approach) is to inject keep-alive messages on the client side
(messages that the server otherwise ignore, but informs the underlying
HTTP server that the connection is still live).
You can override the idle-timeouts in the HTTP server configuration to
a larger value but I don't recommend that.
If you are using stream based clients, injecting heartbeats when
necessary is as simple as calling keepAlive and providing it a time
interval and a factory for the message you want to inject:
http://doc.akka.io/api/akka/2.4.7/index.html#akka.stream.scaladsl.Flow#keepAliveU>:Out:FlowOps.this.Repr[U]
That combinator will make sure that no periods more than T will be
silent as it will inject elements to keep this contract if necessary
(and will not inject anything if there is enough background traffic)
-Endre
thank you Endre :) , working snippet ..
// on client side
val (source, sourceQueue) = {
val p = Promise[SourceQueue[Message]]
val s = Source.queue[Message](Int.MaxValue, OverflowStrategy.backpressure).mapMaterializedValue(m => {
p.trySuccess(m)
m
}).keepAlive(FiniteDuration(1, TimeUnit.SECONDS), () => TextMessage.Strict("Heart Beat"))
(s, p.future)
}

Basic akka http client post not working

I am trying to make a basic http client posting some data to a REST API with Akka HTTP but I cannot make the following code work :
def sendData(e : GenericEvent): Future[Either[(String,StatusCode),(GenericEvent,StatusCode)]] = {
val request = Marshal(e).to[RequestEntity]
val responseFuture: Future[HttpResponse] = request map { req =>
Source.single(HttpRequest(method = HttpMethods.POST, uri = s"/data-ingest", headers = List(auth), entity = req))
.via(dataIngestFlow)
.runWith(Sink.head)
}
responseFuture.flatMap { response =>
response.status match {
case OK => Unmarshal(response.entity).to[GenericEvent].map(Right(_, response.status))
case BadRequest => Future.successful(Left(s"$e.data: incorrect data", response.status))
case _ => Unmarshal(response.entity).to[String].flatMap { entity =>
val error = s"generic event ingest failed with status code ${response.status} and entity $entity"
logger.error(error)
Future.failed(new IOException(error))
}
}
}
I got the following error
polymorphic expression cannot be instantiated to expected type;
[error] found :
[T]akka.stream.scaladsl.Sink[T,scala.concurrent.Future[T]]
[error] required:
akka.stream.Graph[akka.stream.SinkShape[akka.http.scaladsl.model.HttpResponse],akka.http.scaladsl.model.HttpResponse]
[error] .runWith(Sink.head)
Here is the code for the dataIngestFlow
val dataIngestFlow = Http().outgoingConnection(config.endpointUrl,config.endpointPort)
Here is the code on server side :
val routes = {
logRequestResult("akka-http-microservice") {
path("event-ingest") {
post {
entity(as[GenericEvent]) { eventIngest =>
log.info(s"Ingesting {} and publishing event to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
complete {
eventIngest
}
}~
entity(as[List[GenericEvent]]) { eventIngestList =>
eventIngestList.foreach{ eventIngest=>
log.info(s"Ingesting {} and publishing event List to Kafka topic {}.", eventIngest.eventType,config.kafkaTopic)
kafka ! eventIngest
}
complete {
eventIngestList
}
}
}
}
}
}
I tried another simple client, it builds well but the ingestion stop after 160 events, the server doesn't receive anymore events.
The first problem I see with your example is that RequestEntity does not have a map function. Therefore, the following line
val responseFuture: Future[HttpResponse] = request map { ...
should not compile.
Further, if request is actually a Future (which I infer from the map) then responseFuture is actually of type Future[Future[HttpResponse]] because the stream materializes into its own Future. To solve this problem you can use Future.flatMap instead of map. Namely:
val responseFuture: Future[HttpResponse] = request flatMap { req =>
This is the monadic bind operation within Futures.