I've got an akka-http application that uses akka-streams for data processing. So, it makes some sense to complete the request with Source[Result, _] to get backpressure across HTTP boundary for free.
Versions:
akka-http 10.0.7
akka-streams 2.5.2
akka 2.5.2
This is the simplified version of the code, and it works just fine.
pathEnd { post { entity(asSourceOf[Request]) { _ =>
complete {
Source.single("ok")
}
}}}
Since this enpoint is supposed to create and entity, instead of returning 200 OK to the requester I'd like to return 204 CREATED status code. However, I wasn't able to find a way to do that:
complete { Created -> source.single("ok") } fails compilation with Type mismatch, expected: ToResponseMarshallable, actual: (StatusCodes.Success, Source[String, NotUsed])
complete { source.single((Created, "ok")) } fails with Type mismatch, expected: ToResponseMarshallable, actual: Source[(StatusCodes.Success, String), NotUsed]
complete(Created) { Source.single("ok") } fails with Type mismatch, expected: RequestContext, actual: Source[String,NotUsed]
complete(Created, Source.signle("ok") fails with too many arguments for method complete(m: => ToResponseMarshallable)
It looks like custom marshaller might be a way to achieve that, but it'll basically mean I'll need one unmarshaller per endpoint, which isn't quite convenient or clear.
So, the question is, are there a (more convenient than custom unmarshaller) way to complete the request with Source[_,_] while also providing status code.
From the documentation:
complete(Created -> "bar")
If you want to provide some Source of data then construct the HttpResponse and pass it to complete:
import akka.http.scaladsl.model.HttpEntity.Chunked
import akka.http.scaladsl.model.ContentTypes
import akka.http.scaladsl.model.HttpEntity.ChunkStreamPart
complete {
val entity =
Chunked(ContentTypes.`text/plain(UTF-8)`,
Source.single("ok").map(ChunkStreamPart.apply))
HttpResponse(status = Created, entity=entity)
}
I hit this problem and took the approach of using mapResponse to override the status code. This is the simplest approach I've found.
mapResponse(_.copy(status = StatusCodes.Accepted)) {
complete {
Source.single("ok")
}
}
The drawback of Ramon's answer is that you become responsible for both marshalling the stream (to a ByteString), and for the content negotiation.
Related
I'm trying to mock Http calls for unit test.
To do that I have done the following, I have created a RequestMock case class:
case class RequestMock() {
def sendRequest(httpRequest: HttpRequest)(implicit actorSystem: ActorSystem): Future[HttpResponse] = {
Http().singleRequest(httpRequest)
}
}
and in my service, I have written the following piece of code :
case class Service(requestHandler: RequestMock) {
....
for {
response <- {
requestHandler.sendRequest(
HttpRequest(
method = HttpMethods.GET,
uri = "http://database:9000"
)
)
} yield {
response
}
}
For the unit test, I'm trying to mock HttpCalls, to do that, I have done the following :
def test_2 = mock[RequestMock]
And for defining the mock behaviour I have done the following
when(test_2.sendRequest(
HttpRequest(
method = HttpMethods.GET,
uri = "http://database:9000")
)).thenReturn{
Future(
HttpResponse(
StatusCodes.OK,
entity = HttpEntity(ContentTypes.`text/plain(UTF-8)`,"connection established"))
But, when I execute unit tests, I always get the following error:
java.lang.NullPointerException
Does anyone know how I can solve this issue ?
A couple of problems.
First of all, test_2 should be a val, not a def.
With def like you have it, you get a different instance every time you access it. So, you define the stub on one instance, but then create your Service with a different one, that does not have sendRequest defined, so returns null by default, and that causes your NPE.
The next problem, that you will probably encounter after you fix this one is that you are not defining all of the behavior.
when(test_2.sendRequest(
HttpRequest(
method = HttpMethods.GET,
uri = "http://database:9000")
))
Only creates a stub for a method call with this specific parameter value. So, if your tests try to make a POST for example or hit a different endpoint, you'll get an NPE again.
Even if you only ever use one request, it is better to define the stub for any argument, to avoid weird NPE failures if the code happens to send a different one (you are writing a test, so should not just assume automatically, that the code will always do what you expect - you would not need the test in the first place if that was the case):
when(test2.sendRequest(any)),thenReturn(Future.successful(...))
(Note Future.successful above – that's the correct way to create Future that is immediately satisfied, what you are doing makes it run on a thread ... not a big deal in your case, but still icky).
Then, after the test code is run, you can check that the parameter value passed to the sendRequest was actually what you expect:
verify(test2)
.sendRequest(HttpRequest(method = HttpMethods.GET, uri = "http://database:9000"))
i'm on my journey to deepen my knowledge in fs2, and want to try fs2-kafka for a use case where i would replace akka stream. The idea is simple, read from kafka and post data via http request to a sink, then commit back to kafka on success. So far i can't really figure out the http part. In akka stream / akka http you have out of the box a flow for that https://doc.akka.io/docs/akka-http/current/client-side/host-level.html#using-a-host-connection-pool
Flow[(HttpRequest, T), (Try[HttpResponse], T), HostConnectionPool]
Which integrate flawlessly with akka stream.
I was trying to see if i could do something similar with http4s and fs2 .
Does anyone has any reference, code sample, blog and what not that shows how to do that kind of integration. So far the only thing i could think of was, wrapping the the stream into the use method of the client resource i.e
BlazeClientBuilder[IO](IORuntime.global.compute).resource.use { ..... run stream here ..... }
Even then i am not sure of the entire thing
The thing with the typelevel ecosystem is that everything is just a library, you don't need examples on how many of them interact together, you just need to understand how each library works and the basic rules of composition.
def createClient(/** whatever arguments you need */): Resource[IO, Client[IO]] = {
// Fill this based on the documentation of the client of your choice:
// I would recommend the ember client from http4s:
// https://http4s.org/v0.23/api/org/http4s/ember/client/emberclientbuilder
}
def sendHttpRequest(client: Client[IO])(data: Data): IO[Result] = {
// Fill this based on the documentation of your client:
// https://http4s.org/v0.23/client/
// https://http4s.org/v0.23/api/org/http4s/client/client
}
def getStreamOfRecords(/** whatever arguments you need */): Stream[IO, CommittableConsumerRecord[IO, Key, Data]] = {
// Fill this based on the documentation of fs2-kafka:
// https://fd4s.github.io/fs2-kafka/docs/consumers
}
def program(/** whatever arguments you need */): Stream[IO, Unit] = {
// Based on the documentation of fs2 and fs2-kafka I would guess something like this:
Stream.fromResource(createClient(...)).flatMap { client =>
getStreamOfRecords(...).evalMapFilter { committable =>
sendHttpRequest(client)(data = committable.record).map { result =>
if (result.isSuccess) Some(committable.offset)
else None
}
}.through(commitBatchWithin(...))
}
}
object Main extends IOApp.Simple {
override final val run: IO[Unit] =
program(...).compile.drain
}
Note that I wrote all this on top of my head and with just a quick glimpse of the documentation, you need to change many things (especially types, like Data & Result). As well as tunning things like error handling and when to commit back to Kafka.
However, I expect this helps you to get an idea of how to structure your code.
I am trying to handle a future I got from slick in order to generate a response to a request, but I'm stuck at the "async" part.
Here is a snippet:
get("/tmp") {
new AsyncResult() {
override val is: Future[_] = db.run(Users.findUserWithLogin("user"))
}
}
Now, the db.run call returns a Future[Option[User]]. How do I returns a response depending on the content of the option?
In this case, you need to map the future returned by Slick to the result that you want rather than setting is directly to the Slick result. So, the following would be an example of how you might handle it:
get("/tmp") {
new AsyncResult() {
val is = db.run(Users.findUserWithLogin("user")) map {
case Some(u) => //return some stuff about the user
case None => //return some stuff about user not being found
}
}
}
As noted in the comments below, the AsyncResult is not strictly necessary. You can find more details in the Scalatra documentation.
I'm new rather new to Scala so I think this might be a very small problem.
I'm currently trying to change the method chat from using the deprecated WebSocket.async to WebSocket.tryAccept. The application uses the sample chat found at PlayFramework websocket-chat
I'm having trouble creating the complex Future type that the method requires.
This is the old method:
def chat() = WebSocket.async[JsValue] {
request =>
ChatRoom.join("User: 1")
}
New method:
def chat2() = WebSocket.tryAccept[JsValue] {
request =>
try {
// ChatRoom.join returns (iteratee,enumerator)
ChatRoom.join("User: 1").map(e => Right(e))
} catch {
case e: Exception =>
Left(Ok("Failed")) // Error here
}
}
My error message:
found : Left[Result,Nothing]
required: Future[Either[Result,(Iteratee[JsValue, _], Enumerator[JsValue])]]
I have no idea how I am supposed to create such a complex result for such a simple message.
Although ChatRoom.join("User: 1").map(e => Right(e)) doesn't show any errors now, I'm unsure if this is the correct implementation.
I'm not in front of an IDE at the moment, so I can't answer fully, but the return type it's asking for isn't as complex as it seems. An "Either" is a "Left" or a "Right" in the same way that an "Option" is a "Some" or a "None". So what it's asking for is a Future (which Websocket.async should also have required) that contains either a Left[Result] -- the fail-to-connect case, or a Right[(Iteratee, Enumerator)] -- the success case. Assuming that Chatroom.join returns a Future[(Iteratee, Enumerator)], the map operation is simply wrapping that in a "Right". The first thing I'd try is wrapping Left(Ok("Failed")) in a Future and see what happens.
In spray I would like to respond with different content-types, depending on the given Accept header. I've seen a couple of suggestions in the question by rompetroll, but I would like to hear if there are any canonical way of doing it (i. e. simple or already implemented).
In essence what I imagine should happen is something like:
path("somepath") {
get {
// Find whatever we would like to return (lazily)
...
// Marshall resource and complete depending on the `Accept` header
...
}
}
Thanks in advance.
See the tests in this commit.
I copied it here for reference:
case class Data(name: String, age: Int)
object Data {
import spray.json.DefaultJsonProtocol._
import spray.httpx.SprayJsonSupport._
// don't make those `implicit` or you will "ambiguous implicit" errors when compiling
val jsonMarshaller: Marshaller[Data] = jsonFormat2(Data.apply)
val xmlMarshaller: Marshaller[Data] =
Marshaller.delegate[Data, xml.NodeSeq](MediaTypes.`text/xml`) { (data: Data) ⇒
<data><name>{ data.name }</name><age>{ data.age }</age></data>
}
implicit val dataMarshaller: ToResponseMarshaller[Data] =
ToResponseMarshaller.oneOf(MediaTypes.`application/json`, MediaTypes.`text/xml`) (jsonMarshaller, xmlMarshaller)
}
You then using complete should suffice in your route, content-type negotiation is automatically taken care of:
get {
complete(Data("Ida", 83))
}
Spray is actually looking into the Accept header value and validates against it. So if route is returning application/json or text/plain and client accepts image/jpeg than spray will return 406 Not Acceptable. If client will request application/json ortext/plain from this route than he will receive repsonse with matching Content-Type.
The main trick here is to use correct marshallers for return objects.
You can read more about marshalling here.
Also you can override MediaType with respondWithMediaType directive, but I think it is better to use correct marshallers.