I’m just heading an issue when I’m trying to create an endpoint with multiple bodies shape.
My model looks like this:
sealed trait FileExampleTrait {
def kind: String
}
case class FileExampleOne(name: String, length: Int) extends FileExampleTrait {
override def kind: String = “one”
}
case class FileExampleTwo(name: String) extends FileExampleTrait {
override def kind: String = “two”
}
case class FileExampleResponse(message: String)
And I’m trying to create this endpoint:
val fileExample = baseEndpoint.post
.in(“example”)
.in(jsonBody[FileExampleTrait])
.out(jsonBody[FileExampleResponse])
.summary(“something”)
.description(“something”)
The implementation of the endpoint looks like this:
private val fileExample = toAkkaRoute(jwtConsumer, errorHandler)(
FileApi.fileExample, { (scope: RequestScope, input: (FileExampleTrait)) =>
print(scope)
input match {
case FileExampleOne(name, _) => Future.successful(FileExampleResponse(name).asRight)
case FileExampleTwo(name) => Future.successful(FileExampleResponse(name).asRight)
}
}
)
This is just an example on what I’m trying to create. I added the schema derivation based on this:
val sOne = Schema.derived[FileExampleOne]
val sTwo = Schema.derived[FileExampleTwo]
implicit val sExampleTrait: Schema[FileExampleTrait] =
Schema.oneOfUsingField[FileExampleTrait, String](_.kind, _.toString)(“one” -> sOne, “two” -> sTwo)
I created a test for trying the endpoint based on Akka HTTP:
test(“Example test”) {
new Fixture() {
val request = FileExampleOne(“name”, 1)
Post(s”/api/v1/files/example”, jsonEntity(request)).withHeaders(requestHeaders) ~> wrappedRoute ~> check {
response should be(successful)
contentType shouldEqual ContentTypes.`application/json`
}
}
}
The error I’m getting is the following:
Response error: {“code”:400,“message”:“Invalid value for: body (No constructor for type FileExampleTrait, JObject(List((name,JString(name)), (length,JInt(1)))))“}
I was following this documentation.
Well that's because a trait doesn't have a constructor as indicated in the error itself. I think I see where you're going, you want to try parsing the body as one of the traits subclasses. So imagine you have this type/class hierarchy:
T // parent trait
/ \
C1 C2 // class 1, etc...
/
C3
Now you want to deserialize some JSON into trait T, you need to define your custom behavior, like "First try converting into C3, if failed, try converting to C2, if failed again, try converting to C1", and you'll get your T value. Now depending on the JSON library you use, the implementation might differ, see the documentation by softwaremill to get more information about how to deal with JSONs in tapir, and if you use Play Json, I can recommend:
object FileExampleOne {
implicit val reader: Reads[FileExampleOne] = Json.reads
}
object FileExampleTwo {
implicit val reader: Reads[FileExampleTwo] = Json.reads
}
object FileExampleTrait {
implicit val reads: Reads[FileExampleTrait] = json => {
json.validate[FileExampleOne] orElse json.validate[FileExampleTwo]
}
}
You can see it running on scastie. And based on the tapir documentations, you need a Codec for your types, and one of the approaches to create your coded for JSON, is using one of tapir's supported libraries (circe, Play Json, ...).
Related
I have a very generic message object that I get back from a queue like:
case class Message(key: String, properties: Map[String, String])
I then have a bunch of very specific classes that represent a message, and I use properties.get("type") to determine which particular message it is:
sealed trait BaseMessage
case class LoginMessage(userId: Int, ....) extends BaseMessage
case class RegisterMessage(email: String, firstName: String, ....) extends BaseMessage
Now in my code I have to convert from a generic Message to a particular message in many places, and I want to create this in a single place like:
Currently I am doing something like:
val m = Message(....)
val myMessage = m.properties.get("type") match {
case Some("login") => LoginMessage(m.properties("userID"), ...)
case ...
}
What options do I have in making this less cumbersome in scala?
I don't know all your context here, but I can suggest using implicit conversions if you don't want to bring another library in your project. Anyway, implicit conversions can help you separate a lot the implementation or override it "on-the-fly" as needed.
We can start by defining a MessageConverter trait that is actually a function:
/**
* Try[T] here is useful to track deserialization errors. If you don't need it you can use Option[T] instead.
*/
trait MessageConverter[T <: BaseMessage] extends (Message => Try[T])
Now define an object that holds both the implementations and also enables a nice #as[T] method on Message instances:
object MessageConverters {
/**
* Useful to perform conversions such as:
* {{{
* import MessageConverters._
*
* message.as[LoginMessage]
* message.as[RegisterMessage]
* }}}
*/
implicit class MessageConv(val message: Message) extends AnyVal {
def as[T <: BaseMessage : MessageConverter]: Try[T] =
implicitly[MessageConverter[T]].apply(message)
}
// Define below message converters for each particular type
implicit val loginMessageConverter = new MessageConverter[LoginMessage] {
override def apply(message: Message): Try[LoginMessage] = {
// Parse the properties and build the instance here or fail if you can't.
}
}
}
That's it! It may not be the best solution as implicits bring complexity and they make code harder to follow. However, if you follow a well-defined structure for storing these implicit values and be careful how you pass them around, then you shouldn't have any issues.
You can convert the properties map to Json and read it as a case class. Assuming that the keys to the map have the same name as your case class fields you can write a formatter using playjson:
object LoginMessage {
implicit val fmtLoginMessage = Json.format[LoginMessage]
}
If the fields don't have the same name you will have to specify the reads object manually. Your code to convert it into a case class would be something like:
object BaseMessageFactory {
def getMessage(msg: Message): Option[BaseMessage] = {
val propertiesJson = Json.toJson(msg.properties)
msg.properties.get("type").map {
case "login" => propertiesJson.as[LoginMessage]
...
case _ => //Some error
}
}
}
The signature may differ depending on how you want to deal with error handling.
I'm trying to parse different types of events in json format using json4s. I have written some case classes to represent the different events all inheriting from a base class Event:
abstract class Event{ def EventType : String }
case class StartSession(val EventType: String, val Platform: String) extends Event
case class AdView(val EventType: String, val EventSubtype: String) extends Event
This is the function I use to parse the StartSession event:
def parser(json: String): Event = {
val parsedJson = parse(json)
val s = parsedJson.extract[StartSession]
return s
}
This function will correctly parse a json like {"EventType":"StartSession","Platform":"Portal"}
I am looking for a way to generalize the parser function so I can use it to parse all types of events that inherits from Event and then do pattern matching over the return value of the function.
Type hints provide the solution to the problem. If you have a polymorphic type, such as Event, the type hint (here EventType) tells json4s into which actual type a json object should be deserialized.
For reference, have a look at the github page of json4s. There is a section called "Serializing polymorphic Lists".
Since the type hint is only necessary for deserialization, we can get rid of the field EventType in Event.
abstract class Event
case class StartSession(Platform: String) extends Event
case class AdView(EventSubtype: String) extends Event
We need to bring a Formats instance into the scope of extract. The instance tells json4s how to perform deserialization. In our case, we need to specialize typeHintFieldName and typeHints. The first is the key of our type hint, in the example it is "EventType". The latter is a mapping from string values to classes. If we are using the class name as value, then ShortTypeHints will do the trick. Othwerise, we could implement our own, specialized TypeHints.
A concrete solution could look as follows:
def main(args: Array[String]): Unit ={
val json =
s"""
|[
| {
| "EventType": "StartSession",
| "Platform": "Portal"
| },
| {
| "EventType": "AdView",
| "EventSubtype": "SpecializedView"
| }
|]
""".stripMargin
implicit val formats = new DefaultFormats {
override val typeHintFieldName: String = "EventType"
override val typeHints: TypeHints =
ShortTypeHints(
List(classOf[StartSession], classOf[AdView])
)
}
val parsedJson = parse(json)
val s = parsedJson.extract[List[Event]]
println(s)
}
I am trying to figure out how to get a custom JsonFormat write method to be invoked when using the routing directive complete. JsonFormat created with the jsonFormat set of helper functions work fine, but defining a complete JsonFormat will not get called.
sealed trait Error
sealed trait ErrorWithReason extends Error {
def reason: String
}
case class ValidationError(reason: String) extends ErrorWithReason
case object EntityNotFound extends Error
case class DatabaseError(reason: String) extends ErrorWithReason
case class Record(a: String, b: String, error: Error)
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object ErrorJsonFormat extends JsonFormat[Error] {
def write(err: Error) = failure match {
case e: ErrorWithReason => JsString(e.reason)
case x => JsString(x.toString())
}
def read(value: JsValue) = {
value match {
//Really only intended to serialize to JSON for API responses, not implementing read
case _ => throw new DeserializationException("Can't reliably deserialize Error")
}
}
}
implicit val record2Json = jsonFormat3(Record)
}
And then a route like:
import MyJsonProtocol._
trait TestRoute extends HttpService with Json4sSupport {
path("testRoute") {
val response: Record = getErrorRecord()
complete(response)
}
}
If I add logging, I can see that the ErrorJsonFormat.write method never gets called.
The ramifications are as follows showing what output I'm trying to get and what I actually get. Let's say the Record instance was Record("something", "somethingelse", EntityNotFound)
actual
{
"a": "something",
"b": "somethingelse",
"error": {}
}
intended
{
"a": "something",
"b": "somethingelse",
"error": "EntityNotFound"
}
I was expecting that the complete(record) uses the implicit JsonFormat for Record which in turn relies on the implicit object ErrorJsonFormat that specifies the write method that creates the appropriate JsString field. Instead it seems to both recognize the provided ErrorJsonFormat while ignoring its instructions for serializing.
I feel like there should be a solution that does not involve needing to replace implicit val record2Json = jsonFormat3(Record) with an explicit implicit object RecordJsonFormat extends JsonFormat[Record] { ... }
So to summarize what I am asking
Why does the serialization of Record fail to call the ErrorJsonFormat write method (what does it even do instead?) answered below
Is there a way to match my expectation while still using complete(record)?
Edit
Digging through the spray-json source code, there is an sbt-boilerplate template that seems to define the jsonFormat series of methods: https://github.com/spray/spray-json/blob/master/src/main/boilerplate/spray/json/ProductFormatsInstances.scala.template
and the relevant product for jsonFormat3 from that seems to be :
def jsonFormat3[P1 :JF, P2 :JF, P3 :JF, T <: Product :ClassManifest](construct: (P1, P2, P3) => T): RootJsonFormat[T] = {
val Array(p1,p2,p3) = extractFieldNames(classManifest[T])
jsonFormat(construct, p1, p2, p3)
}
def jsonFormat[P1 :JF, P2 :JF, P3 :JF, T <: Product](construct: (P1, P2, P3) => T, fieldName1: String, fieldName2: String, fieldName3: String): RootJsonFormat[T] = new RootJsonFormat[T]{
def write(p: T) = {
val fields = new collection.mutable.ListBuffer[(String, JsValue)]
fields.sizeHint(3 * 4)
fields ++= productElement2Field[P1](fieldName1, p, 0)
fields ++= productElement2Field[P2](fieldName2, p, 0)
fields ++= productElement2Field[P3](fieldName3, p, 0)
JsObject(fields: _*)
}
def read(value: JsValue) = {
val p1V = fromField[P1](value, fieldName1)
val p2V = fromField[P2](value, fieldName2)
val p3V = fromField[P3](value, fieldName3)
construct(p1v, p2v, p3v)
}
}
From this it would seem that jsonFormat3 itself is perfectly fine (if you trace into the productElement2Field it grabs the writer and directly calls write). The problem must then be that the complete(record) doesn't involve JsonFormat at all and somehow alternately marshals the object.
So this seems to answer part 1: Why does the serialization of Record fail to call the ErrorJsonFormat write method (what does it even do instead?). No JsonFormat is called because complete marshals via some other means.
It seems the remaining question is if it is possible to provide a marshaller for the complete directive that will use the JsonFormat if it exists otherwise default to its normal behavior. I realize that I can generally rely on the default marshaller for basic case class serialization. But when I get a complicated trait/case class setup like in this example I need to use JsonFormat to get the proper response. Ideally, this distinction shouldn't have to be explicit for someone writing routes to need to know the situations where its the default marshaller as opposed to needing to invoke JsonFormat. Or in other words, needing to distinguish if the given type needs to be written as complete(someType) or complete(someType.toJson) feels wrong.
After digging further, it seems the root of the problem has been a confusion of the Json4s and Spray-Json libraries in the code. In trying to track down examples of various elements of JSON handling, I didn't recognize the separation between the two libraries readily and ended up with code that mixed some of each, explaining the unexpected behavior.
In this question, the offending piece is pulling in the Json4sSupport in the router. The proper definition should be using SprayJsonSupport:
import MyJsonProtocol._
trait TestRoute extends HttpService with SprayJsonSupport {
path("testRoute") {
val response: Record = getErrorRecord()
complete(response)
}
}
With this all considered, the answers are more apparent.
1: Why does the serialization of Record fail to call the ErrorJsonFormat write method (what does it even do instead)?.
No JsonFormat is called because complete marshals via some other means. That other means is the marshaling provided implicitly by Json4s with Json4sSupport. You can use record.toJson to force spray-json serialization of the object, but the output will not be clean (it will include nested JS objects and "fields" keys).
Is there a way to match my expectation while still using complete(record)?
Yes, using SprayJsonSupport will use implicit RootJsonReader and/or RootJsonWriter where needed to automatically create a relevant Unmarshaller and/or Marshaller. Documentation reference
So with SprayJsonSupport it will see the RootJsonWriter defined by the jsonFormat3(Record) and complete(record) will serialize as expected.
void whatever() {
// ...
val parser = new MyParser
val parse = parser.parse(input)
if (parse successful) {
semanticAnalysis(parse)
}
}
void semanticAnalysis(parse: DontKnowTheCorrectType) {
// ...
}
What type do I have to give to the formal parameter parse? Hovering over parse inside whatever says val parse: parser.ParseResult[parsing.Program], but of course that doesn't work as a parameter type of semanticAnalysis, because the local variable parse is not in scope there.
Parse results are path-dependent types because they are results of this specific parser and there's no guarantee they are compatible.
This is why parsers are usually not instantiated the way you use them (new MyParser), but as object.
object MyParser extends RegexParsers {
def statements : Parser[List[Statement]] = // ...
}
def handleResult(res: MyParser.ParseResult[List[Statement]]) = { // ... }
val res = MyParser.parseAll(MyParser.statements, "/* */")
If you need more dynamic behaviour (or want concurrent parsing, parser combinators aren't thread-safe, ouch), you'll just have to keep the parser object accessible (and stable) wherever you want to use its results.
Sadly, passing a parser and its result around together is not trivial because you run into the prohibition of dependent method types, e.g.
def fun(p: scala.util.parsing.combinator.Parsers, res: p.ParseResult[_]) = {}
won't compile ("illegal dependent method type"), but there are ways to get around that if you must, like the answer to this question.
You should be able to define semanticAnalysis as:
def semanticAnalysis(parse: MyParser#ParseResult[parsing.Program]) = {
...
}
Note the use of # instead of . for the type. There are more details about type projections here, but basically, parser.ParseResult is different for each parser, while MyParser#ParseResult is the same for all parser instances.
But, as themel says, you should probably be using object instead of class.
You can rewrite it like this:
def whatever() {
// ...
val parser = new MyParser
def semanticAnalysis(parse: parser.ParseResult) {
// ...
}
val parse = parser.parse(input)
if (parse successful) {
semanticAnalysis(parse)
}
}
If you have this being called from multiple places, then maybe this:
class SemanticAnalysis(parser: Parser) {
def apply(parse: parser.ParseResult) {
// ...
}
}
And then
if (parse successful) {
new SemanticAnalysis(parser)(parse)
}
When I need to use the result of parser combinators in other parts of a program, I extract the success result or error message from the path-dependent ParseResult and put the data into an independent type. It's more verbose than I like, but I want to keep the combinator instance an implementation detail of the parser.
sealed abstract class Result[+T]
case class Success[T](result: T) extends Result[T]
case class Failure(msg: String) extends Result[Nothing]
case class Error(msg: String) extends Result[Nothing]
/** Parse the package declarations in the file. */
def parse(file: String): Result[List[Package]] = {
val stream = ... // open the file...
val parser = new MyParser
val result = parser.parseAll(parser.packages, stream)
stream.close()
result match {
case parser.Success(packages, _) => Success(packages)
case parser.Failure(msg, _) => Failure(msg)
case parser.Error(msg, _) => Error(msg)
}
}
(Essentially I need some kind of a synthesis of these two questions (1, 2), but I'm not smart enough to combine them myself.)
I have a set of JAXB representations in Scala like this:
abstract class Representation {
def marshalToXml(): String = {
val context = JAXBContext.newInstance(this.getClass())
val writer = new StringWriter
context.createMarshaller.marshal(this, writer)
writer.toString()
}
}
class Order extends Representation {
#BeanProperty
var name: String = _
...
}
class Invoice extends Representation { ... }
The problem I have is with my unmarshalling "constructor" methods:
def unmarshalFromJson(marshalledData: String): {{My Representation Subclass}} = {
val mapper = new ObjectMapper()
mapper.getDeserializationConfig().withAnnotationIntrospector(new JaxbAnnotationIntrospector())
mapper.readValue(marshalledData, this.getClass())
}
def unmarshalFromXml(marshalledData: String): {{My Representation Subclass}} = {
val context = JAXBContext.newInstance(this.getClass())
val representation = context.createUnmarshaller().unmarshal(
new StringReader(marshalledData)
).asInstanceOf[{{Type of My Representation Subclass}}]
representation // Return the representation
}
Specifically, I can't figure out how to attach these unmarshalling methods in a typesafe and DRY way to each of my classes, and then to call them from Scala (and hopefully sometimes by using only abstract type information). In other words, I would like to do this:
val newOrder = Order.unmarshalFromJson(someJson)
And more ambitiously:
class Resource[R <: Representation] {
getRepresentation(marshalledData: String): R =
{{R's Singleton}}.unmarshalFromXml(marshalledData)
}
In terms of my particular stumbling blocks:
I can't figure out whether I should define my unmarshalFrom*() constructors once in the Representation class, or in a singleton Representation object - if the latter, I don't see how I can automatically inherit that down through the class hierarchy of Order, Invoice etc.
I can't get this.type (as per this answer) to work as a way of self-typing unmarshalFromJson() - I get a compile error type mismatch; found: ?0 where type ?0 required: Representation.this.type on the readValue() call
I can't figure out how to use the implicit Default[A] pattern (as per this answer) to work down my Representation class hierarchy to call the singleton unmarshalling constructors using type information only
I know this is a bit of a mammoth question touching on various different (but related) issues - any help gratefully received!
Alex
The key is to not try and attach the method to the class but rather pass it in as a parameter. To indicate the type you are expecting and let the type system handle passing it in. I tried to make the unmarshal invocation something that reads a little DSL like.
val order = UnMarshalXml( xml ).toRepresentation[Order]
The following is a fully testable code snippet
abstract class Representation {
def marshalToXml(): String = {
val context = JAXBContext.newInstance(this.getClass)
val writer = new StringWriter
context.createMarshaller.marshal(this, writer)
writer.toString
}
}
#XmlRootElement
class Order extends Representation {
#BeanProperty
var name: String = _
}
case class UnMarshalXml( xml: String ) {
def toRepresentation[T <: Representation](implicit m:Manifest[T]): T = {
JAXBContext.newInstance(m.erasure).createUnmarshaller().unmarshal(
new StringReader(xml)
).asInstanceOf[T]
}
}
object test {
def main( args: Array[String] ) {
val order = new Order
order.name = "my order"
val xml = order.marshalToXml()
println("marshalled: " + xml )
val received = UnMarshalXml( xml ).toRepresentation[Order]
println("received order named: " + received.getName )
}
}
You should see the following output if you run test.main
marshalled: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><order><name>my order</name></order>
received name: my order
Here's the updated version of Neil's code which I used to support the second use case as well as the first:
case class UnmarshalXml(xml: String) {
def toRepresentation[T <: Representation](implicit m: Manifest[T]): T =
toRepresentation[T](m.erasure.asInstanceOf[Class[T]])
def toRepresentation[T <: Representation](typeT: Class[T]): T =
JAXBContext.newInstance(typeT).createUnmarshaller().unmarshal(
new StringReader(xml)
).asInstanceOf[T]
}
This supports simple examples like so:
val order = UnmarshalXml(xml).toRepresentation[Order]
But also for abstract type based usage, you can use like this:
val order = UnmarshalXml(xml).toRepresentation[T](typeOfT)
(Where you have grabbed and stored typeOfT using another implicit Manifest at the point of declaring T.)