I left the Akka world for a few months, and apparently I've lost my mojo. I am trying to write a web service that returns either an XML or a JSON document, based on the Accept header.
However, I can't get the Marshallers to work (returns 406, only accepts text/plain). This is what I have:
trait MyMarshallers extends DefaultJsonProtocol with SprayJsonSupport with ScalaXmlSupport {
implicit def ec: ExecutionContext
implicit val itemJsonFormat = jsonFormat3(MyPerson)
def marshalCatalogItem(obj: MyPerson): NodeSeq =
<MyPerson>
<id>
{obj.ID}
</id>
<name>
{obj.Name}
</name>
<age>
{obj.Age}
</age>
</MyPerson>
def marshalCatalogItems(items: Iterable[MyPerson]): NodeSeq =
<Team>
{items.map(marshalCatalogItem)}
</Team>
implicit def catalogXmlFormat = Marshaller.opaque[Iterable[MyPerson], NodeSeq](marshalCatalogItems)
implicit def catalogItemXmlFormat = Marshaller.opaque[MyPerson, NodeSeq](marshalCatalogItem)
implicit val catalogMarshaller: ToResponseMarshaller[Iterable[MyPerson]] = Marshaller.oneOf(
Marshaller.withFixedContentType(MediaTypes.`application/json`) { catalog ⇒
HttpResponse(entity = HttpEntity(ContentType(MediaTypes.`application/json`),
catalog.map(i ⇒ MyPerson(i.ID, i.Name, i.Age))
.toJson.compactPrint))
}
,
Marshaller.withOpenCharset(MediaTypes.`application/xml`) { (catalog, charset) ⇒
HttpResponse(entity = HttpEntity.CloseDelimited(ContentType(MediaTypes.`application/xml`, HttpCharsets.`UTF-8`),
Source.fromFuture(Marshal(catalog.map(i => MyPerson(i.ID, i.Name, i.Age)))
.to[NodeSeq])
.map(ns ⇒ ByteString(ns.toString()))
)
)
}
)
}
and my route looks like this:
class MyService extends MyMarshallers {
implicit val system = ActorSystem("myService")
implicit val materializer = ActorMaterializer()
implicit val ec: ExecutionContext = system.dispatcher
...
def route = ...
(get & path("teams")) {
parameters('name.as[String]) { id =>
complete {
getTeam(id)
}
}
...
}
And if I request /, I get plain text, but if I request application/xml or application/json, I get a 406.
How does AKKA-HTTP decide what content types it will accept? I just updated everything to Akka 2.4.6.
Sorry, I figured it out. The above code had two issues, one of them fixed the problem:
Issue 1 - changed to use fromIterator instead of fromFuture in the
XML marshaller.
Issue 2 - my getTeam() method was returning an Iterable. I changed that to a Seq.
Now everything works.
Related
I'm building a toy project to learn Scala 3 and i'm stuck in one problem, first of all i'm following the tagless-final approach using cats-effect, the approach is working as expected except for the entity serialization, when i try to create a route using akka-http i have the following problem:
def routes: Route = pathPrefix("security") {
(path("auth") & post) {
entity(as[LoginUserByCredentialsCommand]) {
(command: LoginUserByCredentialsCommand) =>
complete {
login(command)
}
}
}}
F[
Either[com.moralyzr.magickr.security.core.errors.AuthError,
com.moralyzr.magickr.security.core.types.TokenType.Token
]
]
Required: akka.http.scaladsl.marshalling.ToResponseMarshallable
where: F is a type in class SecurityApi with bounds <: [_] =>> Any
For what i understood, akka-http does not know how to serialize the F highly-kinded type, by searching a little bit i found the following solution, it consists of creating an implicit called marshallable to show the akka-http how to serialize the type, however when i implement it i get a StackOverflow error :(
import akka.http.scaladsl.marshalling.ToResponseMarshaller
import cats.effect.IO
trait Marshallable[F[_]]:
def marshaller[A: ToResponseMarshaller]: ToResponseMarshaller[F[A]]
object Marshallable:
implicit def marshaller[F[_], A : ToResponseMarshaller](implicit M: Marshallable[F]): ToResponseMarshaller[F[A]] =
M.marshaller
given ioMarshaller: Marshallable[IO] with
def marshaller[A: ToResponseMarshaller] = implicitly
I'm really stuck right now, does anyone have an idea on how can i fix this problem? The complete code can be found here
Edit: This is the login code
For clarity, here are the class that instantiate the security api and the security api itself
object Magickr extends IOApp:
override def run(args: List[String]): IO[ExitCode] =
val server = for {
// Actors
actorsSystem <- ActorsSystemResource[IO]()
streamMaterializer <- AkkaMaterializerResource[IO](actorsSystem)
// Configs
configs <- Resource.eval(MagickrConfigs.makeConfigs[IO]())
httpConfigs = AkkaHttpConfig[IO](configs)
databaseConfigs = DatabaseConfig[IO](configs)
flywayConfigs = FlywayConfig[IO](configs)
jwtConfig = JwtConfig[IO](configs)
// Interpreters
jwtManager = JwtBuilder[IO](jwtConfig)
authentication = InternalAuthentication[IO](
passwordValidationAlgebra = new SecurityValidationsInterpreter(),
jwtManager = jwtManager
)
// Database
_ <- Resource.eval(
DbMigrations.migrate[IO](flywayConfigs, databaseConfigs)
)
transactor <- DatabaseConnection.makeTransactor[IO](databaseConfigs)
userRepository = UserRepository[IO](transactor)
// Services
securityManagement = SecurityManagement[IO](
findUser = userRepository,
authentication = authentication
)
// Api
secApi = new SecurityApi[IO](securityManagement)
routes = pathPrefix("api") {
secApi.routes()
}
akkaHttp <- AkkaHttpResource.makeHttpServer[IO](
akkaHttpConfig = httpConfigs,
routes = routes,
actorSystem = actorsSystem,
materializer = streamMaterializer
)
} yield (actorsSystem)
return server.useForever
And
class SecurityApi[F[_]: Async](
private val securityManagement: SecurityManagement[F]
) extends LoginUserByCredentials[F]
with SecurityProtocols:
def routes()(using marshaller: Marshallable[F]): Route = pathPrefix("security") {
(path("auth") & post) {
entity(as[LoginUserByCredentialsCommand]) {
(command: LoginUserByCredentialsCommand) =>
complete {
login(command)
}
}
}
}
override def login(
command: LoginUserByCredentialsCommand
): F[Either[AuthError, Token]] =
securityManagement.loginWithCredentials(command = command).value
================= EDIT 2 =========================================
With the insight provided by Luis Miguel, it makes a clearer sense that i need to unwrap the IO into a Future at the Marshaller level, something like this:
def ioToResponseMarshaller[A: ToResponseMarshaller](
M: Marshallable[IO]
): ToResponseMarshaller[IO[A]] =
Marshaller.futureMarshaller.compose(M.entity.unsafeToFuture())
However, i have this problem:
Found: cats.effect.unsafe.IORuntime => scala.concurrent.Future[A]
Required: cats.effect.IO[A] => scala.concurrent.Future[A]
I think i'm close! Is there a way to unwrap the IO keeping the IO type?
I managed to make it work! Thanks to #luismiguel insight, the problem was that the Akka HTTP Marshaller was not able to deal with Cats-Effect IO monad, so the solution was an implementation who unwraps the IO monad using the unsafeToFuture inside the marshaller, that way i was able to keep the Tagless-Final style from point to point, here's the solution:
This implicit fetches the internal marshaller for the type
import akka.http.scaladsl.marshalling.ToResponseMarshaller
import cats.effect.IO
trait Marshallable[F[_]]:
def marshaller[A: ToResponseMarshaller]: ToResponseMarshaller[F[A]]
object Marshallable:
implicit def marshaller[F[_], A: ToResponseMarshaller](implicit
M: Marshallable[F]
): ToResponseMarshaller[F[A]] = M.marshaller
given ioMarshallable: Marshallable[IO] with
def marshaller[A: ToResponseMarshaller] = CatsEffectsMarshallers.ioMarshaller
This one unwraps the IO monad and flatMaps the marshaller using a future, which akka-http knows how to deal with.
import akka.http.scaladsl.marshalling.{
LowPriorityToResponseMarshallerImplicits,
Marshaller,
ToResponseMarshaller
}
import cats.effect.IO
import cats.effect.unsafe.implicits.global
trait CatsEffectsMarshallers extends LowPriorityToResponseMarshallerImplicits:
implicit def ioMarshaller[A](implicit
m: ToResponseMarshaller[A]
): ToResponseMarshaller[IO[A]] =
Marshaller(implicit ec => _.unsafeToFuture().flatMap(m(_)))
object CatsEffectsMarshallers extends CatsEffectsMarshallers
I try to write a unit test for a small function in a controller using Play/ScalaTest+ Play. The function I want to test looks like this:
def functionToTest(id: String) = Action.async {
implicit request =>
lang
deeperFunction{ implicit context =>
...
}
}
The deeperFunction
def deeperFunction(block: Context => Future[Result])(implicit request: RequestHeader): Future[Result] = {
// returns Future.successful(Found("DummyUrltoRedirect"))
}
}
The deeperFunction is inherited from a trait and I don't want use the real one here because it's a unit test and so I want to use a matcher instead
val deeperMock = mock[Rainmaker]
val contextMock = mock[Context]
val controller = new Controller()(.....) // list of implicit arguments
"Controller" must {
"return something" in {
val request = FakeRequest("GET", "/something")
when(deeperMock.deeperFunction(anyObject)(anyObject)) thenReturn Future.successful(Found("DummyUrlToRedirect"))
val id = "id"
val result = controller.functionToTest(id).apply(request)
status(result) mustBe Ok
}
}
But when I run this, the line "val result = controller.functionToTest(id).apply(request)" still seems to call the real deeperFunction, not the fake one and therefore throws a null matcher at some point.
I also tried to use
when(controller.deeperFunction(anyObject)(anyObject)) thenReturn Future.successful(Found("DummyUrlToRedirect"))
instead, because the deeperFunction is inherited, but with the same result.
I tried to stick to theses instructions
ScalaTest+Play
dzone
but it seems I am still missing some basics/understanding. Thanks in advance.
I am migrating from code spray (version: 1.3.4) to akka-http (version: 10.0.13).
We have an existing clients they send Content-Type like application/vnd.awesome.value; mykey="custom/custom"
Following code works fine in spray but in akka-http parameter (mykey) is stripped off
request.header[`Content-Type`].map { header =>
// HERE PARAMS IS EMPTY MAP
val myKeyValue = header.contentType.mediaType.params.get("mykey").map(_.replace("\"", ""))
myKeyValue.flatMap(_.toMediaType).getOrElse(header.contentType.mediaType)
}.getOrElse(defaultMediaType)
StringToMediaTypeConversion.scala
val customMediaTypes: List[MediaType] = ???
implicit class StringToMediaType(private val str: String) extends AnyVal {
private def parseMediaType(input: String): Option[MediaType] = MediaType.parse(input).fold(_ => None, Some(_))
private def isSupportedType(mediaType: MediaType): Boolean = customMediaTypes.contains(mediaType)
private def extractMediaTypeParamIfExists(mediaType: MediaType): Option[MediaType] = mediaType.params.get("mykey") match {
case Some(value) => parseMediaType(value)
case None => Some(mediaType)
}
def toMediaType: Option[MediaType] = parseMediaType(str).flatMap(extractMediaTypeParamIfExists).filter(isSupportedType)
}
Following unit test is working is passing
val customMediaType: Option[MediaType] = """application/vnd.awesome.value; mykey="custom/custom""""".toMediaType
customMediaType should not be None
customMediaType.get.mainType shouldBe "custom"
customMediaType.get.subType shouldBe "custom"
Any help is appreciated.
How to extract raw value? or How to extract param value of media type?
I was registering custom media types as specified in the documentation, if I remove it I am seeing expected behaviour.
Can someone help me get a hold on the spray code to better understand how to spray ?
I'm the context of sending a file as multipart data.
The code that was suggested to me is :
import akka.actor.ActorSystem
import spray.client.pipelining._
import spray.http.{MediaTypes, BodyPart, MultipartFormData}
object UploadFileExample extends App {
implicit val system = ActorSystem("simple-spray-client")
import system.dispatcher // execution context for futures below
val pipeline = sendReceive
val payload = MultipartFormData(Seq(BodyPart(new File("/tmp/test.pdf"), "datafile", MediaTypes.`application/pdf`)))
val request =
Post("http://localhost:8080/file-upload", payload)
pipeline(request).onComplete { res =>
println(res)
system.shutdown()
}
}
Which is fine and works of course.
However i want to understand what is under the hood so i can do things by myself:
Here are the confusion coming from this code:
BodyPart(new File("/tmp/test.pdf"), "datafile", MediaTypes.`application/pdf`)
is the first issue, indeed, BodyPart only has one apply method that is closely match:
def apply(file: File, fieldName: String, contentType: ContentType): BodyPart =
apply(HttpEntity(contentType, HttpData(file)), fieldName, Map.empty.updated("filename", file.getName))
it takes a contentType and not a MediaType.
However i found in contentType
object ContentType {
private[http] case object `; charset=` extends SingletonValueRenderable
def apply(mediaType: MediaType, charset: HttpCharset): ContentType = apply(mediaType, Some(chars et))
implicit def apply(mediaType: MediaType): ContentType = apply(mediaType, None) }
But ContentType is not in the scope of the Main (see first code at the top.) Hence i don't know where that implicit conversion comes from?
Then the last thing that i do not understand here is
val payload = MultipartFormData(Seq(BodyPart(new File("/tmp/test.pdf"),
"datafile", MediaTypes.`application/pdf`)))
Post("http://localhost:8080/file-upload", payload)
The problem here is that, it is based on the RequestBuilding (as can be found in the RequestBuilding Source)
val Post = new RequestBuilder(POST)
with an object that contain the apply method:
def apply[T: Marshaller](uri: String, content: Option[T]): HttpRequest = apply(Uri(uri), content)
....
....
def apply[T: Marshaller](uri: Uri, content: Option[T]): HttpRequest = {
val ctx = new CollectingMarshallingContext {
override def startChunkedMessage(entity: HttpEntity, ack: Option[Any],
headers: Seq[HttpHeader])(implicit sender: ActorRef) =
sys.error("RequestBuilding with marshallers producing chunked requests is not supported")
}
content match {
case None ⇒ HttpRequest(method, uri)
case Some(value) ⇒ marshalToEntityAndHeaders(value, ctx) match {
case Right((entity, headers)) ⇒ HttpRequest(method, uri, headers.toList, entity)
case Left(error) ⇒ throw error
}
}
}
MultiPartFormData is not a Marshaler , hence i do not understand how does it works.
Can someone help me figure that out ?
Many thanks,
M
In one part of my application I have to send a CSV file back to the browser. I have an actor that replies with a Stream[String], each element of the Stream is one line in the file that should be sent to the user.
This is what I currently have. Note that I'm currently returning MediaType text/plain for debugging purposes, it will be text/csv in the final version:
trait TripService extends HttpService
with Matchers
with CSVTripDefinitions {
implicit def actorRefFactory: ActorRefFactory
implicit val executionContext = actorRefFactory.dispatcher
lazy val csvActor = Ridespark.system.actorSelection(s"/user/listeners/csv").resolveOne()(1000)
implicit val stringStreamMarshaller = Marshaller.of[Stream[String]](MediaTypes.`text/plain`) { (v, ct, ctx) =>
ctx.marshalTo(HttpEntity(ct, v.mkString("\n")))
}
val route = {
get {
path("trips" / Segment / DateSegment) { (network, date) =>
respondWithMediaType(MediaTypes.`text/plain`) {
complete(askTripActor(date, network))
}
}
}
}
def askTripActor(date: LocalDate, network: String)
(implicit timeout: Timeout = Timeout(1000, TimeUnit.MILLISECONDS)): Future[Stream[String]] = {
csvActor flatMap { actor => (actor ? GetTripsForDate(date, Some(network))).mapTo[Stream[String]] }
}
}
class TripApi(info: AuthInfo)(implicit val actorRefFactory: ActorRefFactory) extends TripService
My problem with this is that it will load the whole CSV into memory (due to the mkString in stringStreamMarshaller). Is there a way of not doing this, streaming the lines of the CSV as they are generated?
You don't need the stringStreamMarshaller. Using the built-in stream marshaller should already do what you want without adding the extra line-breaks. However, adding the linebreaks shouldn't be more difficult than using streamOfStrings.map(_ + "\n") or just adding the linebreaks when rendering the lines in the csv-actor in the first place.