How to make only few datatype which is not related to each other acceptable by generics - scala

There is a trait which works perfectly. However, I would like to refactor the part related to generic [T] in order to limit the data type which could be accepted by generic [T] (I need only Option[JsValue] , JsValue , StringEnumEntry , String ). Is it possible to solve this problem through shapeless coproduct? Maybe there are other solutions?
trait ParameterBinders extends Log {
def jsonBinder[T](json: T, jsonType: java.lang.String = "json"): ParameterBinderWithValue = {
val jsonObject = new PGobject()
jsonObject.setType(jsonType)
json match {
case json: Option[JsValue] =>
jsonObject.setValue(json.map(Json.stringify).orNull)
case json: JsValue =>
jsonObject.setValue(Json.stringify(json))
case json: StringEnumEntry =>
jsonObject.setValue(json.value)
case json: String =>
jsonObject.setValue(json)
case _ =>
logger.error("unexpected data type ")
}
if (jsonType == "JSONSCHEMATYPE" || jsonType == "SYSPROPERTYTYPE") {
ParameterBinder(this, (ps, i) => {
ps.setObject(i, jsonObject)
})
} else {
ParameterBinder(json, (ps, i) => {
ps.setObject(i, jsonObject)
})
}
}
}

The easiest way is to use an ADT as described in the link of the first comment.
If you don't want to change the types that are accepted in jsonBinder then you can solve the problem by using a typeclass.
e.g.
trait JsonBindValue[T] {
def value(t: T): String
}
you would then have to provide instances for your accepted datatypes
object JsonBindValue {
implicit val OptJsBinder = new JsonBindValue[Option[JsValue]] {
def value(t: Option[JsValue]): String = {
t.map(Json.stringify).orNull
}
}
... more instances here
}
finally your function would look like this:
def jsonBinder[T : JsonBindValue](json: T, jsonType: java.lang.String = "json"): ParameterBinderWithValue = {
val binder = implicitly[JsonBindValue[T]]
jsonObject.setType(jsonType)
jsonObject.setValue(binder.value(json))
...
}
if you call the function without a implicit instance in scope you will get a compile time error.

Related

(Scala 2.12.8) pattern type is incompatible with expected type for parameterized type inside of parameterized class

(Scala 2.12.8)
Full Example
So lets say you have some "TypeEvidence" for some specific concrete types:
sealed trait TypeEvidence[+T]
object TypeEvidence{
case object DoubleType extends TypeEvidence[Double]
case object LongType extends TypeEvidence[Long]
case object StringType extends TypeEvidence[String]
}
I can match on those evidence objects like this:
object ThisIsOk{
def apply[T](ev: TypeEvidence[T]): Option[T] = {
ev match {
case TypeEvidence.DoubleType => Some(123.456)
case TypeEvidence.LongType => Some(1234L)
case _ => None
}
}
}
But not like this:
class ThisFails[T]{
def apply(ev: TypeEvidence[T]): Option[T] = {
ev match {
case TypeEvidence.DoubleType => Some(123.456)
case TypeEvidence.LongType => Some(1234L)
case _ => None
}
}
}
This fails to compile with:
pattern type is incompatible with expected type;
found : TypeEvidence.DoubleType.type
required: TypeEvidence[T]
pattern type is incompatible with expected type;
found : TypeEvidence.LongType.type
required: TypeEvidence[T]
Why is this? And how can this be worked around?
What if you do this with a type class?
class ThisFails[T]{
def apply(ev: TypeEvidence[T])(implicit
teto: TypeEvidenceToOption[T]
): Option[T] = teto(ev)
}
trait TypeEvidenceToOption[T] {
def apply(ev: TypeEvidence[T]): Option[T]
}
object TypeEvidenceToOption {
implicit val double: TypeEvidenceToOption[Double] =
{ case TypeEvidence.DoubleType => Some(123.456) }
implicit val long: TypeEvidenceToOption[Long] =
{ case TypeEvidence.LongType => Some(1234L) }
implicit def default[T]: TypeEvidenceToOption[T] = _ => None
}
new ThisFails[Double].apply(TypeEvidence.DoubleType) // Some(123.456)
new ThisFails[Long].apply(TypeEvidence.LongType) // Some(1234)
new ThisFails[String].apply(TypeEvidence.StringType) // None
I'm not entirely sure why it didn't work the way you expected. The only reasonable explanation I tell to myself is that the compiler has no issue using T as a placeholder for any type while T belongs to the parameterized method, whereas when it belongs to a parameterized class, it's more restrictive in terms of what you can do with it.
As a workaround, you can use .type and .asIstanceOf:
class ThisFails[T] {
def apply(ev: TypeEvidence[T]): Option[T] = {
ev match {
case _: TypeEvidence.DoubleType.type => Some(123.456.asInstanceOf[T])
case _: TypeEvidence.LongType.type => Some(1234L.asInstanceOf[T])
case _ => None
}
}
}
val x = new ThisFails[Long]
val x2 = new ThisFails[Double]
val x3 = new ThisFails[String]
println(x.apply(TypeEvidence.LongType)) // Some(1234)
println(x2.apply(TypeEvidence.DoubleType)) // Some(123.456)
println(x3.apply(TypeEvidence.StringType)) // None
I'm aware it's not nice, but you asked for a workaround. This pattern match now matches an object of the given type. Of course, with this workaround, having the objects "caseable" is redundant now.
I had a peek at the disassembled code of object ThisIsOk to figure out this workaround:
public static class ThisIsOk$
{
public static final ThisIsOk$ MODULE$;
static {
ThisIsOk$.MODULE$ = new ThisIsOk$();
}
public <T> Option<T> apply(final Main.TypeEvidence<T> ev) {
Object module$;
if (Main.TypeEvidence$.DoubleType$.MODULE$.equals(ev)) {
module$ = new Some((Object)BoxesRunTime.boxToDouble(123.456));
}
else if (Main.TypeEvidence$.LongType$.MODULE$.equals(ev)) {
module$ = new Some((Object)BoxesRunTime.boxToLong(1234L));
}
else {
module$ = None$.MODULE$;
}
return (Option<T>)module$;
}
}
I saw here the way pattern match works is by calling equals on their types, but since equals is not overridden, what this actually does is a reference check. Recall:
// in Object.java file:
public boolean equals(Object obj) {
return (this == obj);
}
Also I saw a cast to Option<T> in the end.
So with these observations, the class ThisFails[T] workaround I just showed earlier gets disassembled into this:
public static class ThisFails<T>
{
public Option<T> apply(final Main.TypeEvidence<T> ev) {
Object module$;
if (Main.TypeEvidence$.DoubleType$.MODULE$ == ev) {
module$ = new Some((Object)BoxesRunTime.boxToDouble(123.456));
}
else if (Main.TypeEvidence$.LongType$.MODULE$ == ev) {
module$ = new Some((Object)BoxesRunTime.boxToLong(1234L));
}
else {
module$ = None$.MODULE$;
}
return (Option<T>)module$;
}
}
Which is almost identical to the object ThisIsOk's disassembled code, except for using == instead of equals, which does not matter in this case, as I explained earlier.
I tested it with both Scala 2.13.8 and your version here on Scastie, and both worked.

Comparing the json data types at runtime using Jackson and Scala

I have an incoming JSON data that looks like below:
{"id":"1000","premium":29999,"eventTime":"2021-12-22 00:00:00"}
Now, I have created a class that will accept this record and will check whether the data type of the incoming record is according to the data types defined in the case class. However, when I am calling the method it is always calling the Failure part of the match case.
case class Premium(id: String, premium: Long, eventTime: String)
class Splitter extends ProcessFunction[String, Premium] {
val outputTag = new OutputTag[String]("failed")
def fromJson[T](json: String)(implicit m: Manifest[T]): Either[String, T] = {
Try {
println("inside")
lazy val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
mapper.readValue[T](json)
} match {
case Success(x) => {
Right(x)
}
case Failure(err) => {
Left(json)
}
}
}
override def processElement(i: String, context: ProcessFunction[String, Premium]#Context, collector: Collector[Premium]): Unit = {
fromJson(i) match {
case Right(data) => {
collector.collect(data)
println("Good Records: " + data)
}
case Left(json) => {
context.output(outputTag, json)
println("Bad Records: " + json)
}
}
}
}
Based on the sample record above, it should pass the Success value but no matter what I pass, it always enters the Failure part. What else is missing?
I am using Scala 2.11.12 and I tried examples from this link and this link but no luck.

How to write a custom decoder for [Option[Option[A]] in Circe?

I had written a Reads converter in play-json for Option[Option[A]] that had the following behavior:
//given this case class
case class MyModel(field: Option[Option[String]])
//this JSON -- maps to --> this MyModel:
//"{ \"field\": \"value\" }" --> MyModel(field = Some(Some("value")))
//"{ \"field\": null, ... }" --> MyModel(field = Some(None))
//"{ }" --> MyModel(field = None)
So, providing the value mapped to Some[Some[A]], providing null mapped to Some[None] (i.e. Some[Option.empty[A]]), and not providing the value mapped to just None (i.e. Option.empty[Option[A]]). Here's the play-json converter:
def readOptOpt[A](implicit r: Reads[A]): Reads[Option[Option[A]]] = {
Reads[Option[Option[A]]] { json =>
path.applyTillLast(json).fold(
identity,
_.fold(_ => JsSuccess(None), {
case JsNull => JsSuccess(Some(None))
case js => r.reads(js).repath(path).map(a => Some(Some(a)))
})
)
}
}
Now I am converting my play-json code to Circe, but I can't figure out how to write a Decoder[Option[Option[A]] that has the same behavior. That is, I need
def optOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]] = ??? //help!
Any ideas on how I can make this work? Thanks
I figured this out:
There were two problems:
1) How to deal with the case where the field was completely missing from the JSON. Turns out you have to use Decoder.reattempt in your custom decoder, following Circe's decodeOption code, which works.
2) How to have the compiler recognize cases of Option[Option[A]] when your decoder code is sitting in a helper object (or wherever). Turns out if you're using semi-auto derivation, you can create an implicit in the companion object and that will override the defaults:
//companion object
object MyModel {
implicit def myModelOptOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]]] =
MyHelperObject.optOptDecoder
implicit val myModelDecoder: Decoder[MyModel] = deriveDecoder
}
Anyway, I don't think this will be much help to anybody in the future, so unless I get any upvotes in the next few hours I think I'll just delete this.
Edit2: Okay it was answered so I won't delete it. Stay strong, esoteric circe question, stay strong...
An Option[Option[A]] is a bit odd. I understand and mostly agree with the reasoning, but I think it's weird enough that it may warrant just replacing it with your own class (and writing a decoder for that). Something like:
sealed trait OptionalNull[+A] {
def toOption: Option[Option[A]]
}
object NotPresent extends OptionalNull[Nothing] {
override def toOption = None
}
object PresentButNull extends OptionalNull[Nothing] {
override def toOption = Some(None)
}
case class PresentNotNull[A](value: A) extends OptionalNull[A] {
override def toOption = Some(Some(value))
}
This has the additional benefit of not having to worry about implicit precedence and stuff like that. Might simplify your decoder.
Here is another solution I found (This is not my gist):
sealed trait UpdateOrDelete[+A]
case object Delete extends UpdateOrDelete[Nothing]
final case class UpdateOptionalFieldWith[A](value: A) extends UpdateOrDelete[A]
object UpdateOrDelete {
implicit def optionalDecoder[A](implicit decodeA: Decoder[A]): Decoder[UpdateOptionalField[A]] =
Decoder.withReattempt {
// We're trying to decode a field but it's missing.
case c: FailedCursor if !c.incorrectFocus => Right(None)
case c =>
Decoder.decodeOption[A].tryDecode(c).map {
case Some(a) => Some(UpdateOptionalFieldWith(a))
case None => Some(Delete)
}
}
// Random UUID to _definitely_ avoid collisions
private[this] val marker: String = s"$$marker-${UUID.randomUUID()}-marker$$"
private[this] val markerJson: Json = Json.fromString(marker)
implicit def optionalEncoder[A](implicit encodeA: Encoder[A]): Encoder[UpdateOptionalField[A]] =
Encoder.instance {
case Some(Delete) => Json.Null
case Some(UpdateOptionalFieldWith(a)) => encodeA(a)
case None => markerJson
}
def filterMarkers[A](encoder: Encoder.AsObject[A]): Encoder.AsObject[A] =
encoder.mapJsonObject { obj =>
obj.filter {
case (_, value) => value =!= markerJson
}
}
}

missing FromRequestUnmarshaller[Entity] on akka post route

Let me start by saying that i am very new to akka-http, none of the books i have covered the marsheling topic well. So it is bit of a blackbox for me. I was able to obtain the following (Un)Marsheller which is capable of returning both json and protobuf based on a request header.
This part of the code works fine and i have a get route defined in akka-http and it works fine.
trait PBMarshaller {
private val protobufContentType = ContentType(MediaType.applicationBinary("octet-stream", Compressible, "proto"))
private val applicationJsonContentType = ContentTypes.`application/json`
implicit def PBFromRequestUnmarshaller[T <: GeneratedMessage with Message[T]](companion: GeneratedMessageCompanion[T]): FromEntityUnmarshaller[T] = {
Unmarshaller.withMaterializer[HttpEntity, T](_ => implicit mat => {
case entity#HttpEntity.Strict(`applicationJsonContentType`, data) =>
val charBuffer = Unmarshaller.bestUnmarshallingCharsetFor(entity)
FastFuture.successful(JsonFormat.fromJsonString(data.decodeString(charBuffer.nioCharset().name()))(companion))
case entity#HttpEntity.Strict(`protobufContentType`, data) =>
FastFuture.successful(companion.parseFrom(CodedInputStream.newInstance(data.asByteBuffer)))
case entity =>
Future.failed(UnsupportedContentTypeException(applicationJsonContentType, protobufContentType))
})
}
implicit def PBToEntityMarshaller[T <: GeneratedMessage]: ToEntityMarshaller[T] = {
def jsonMarshaller(): ToEntityMarshaller[T] = {
val contentType = applicationJsonContentType
Marshaller.withFixedContentType(contentType) { value =>
HttpEntity(contentType, JsonFormat.toJsonString(value))
}
}
def protobufMarshaller(): ToEntityMarshaller[T] = {
Marshaller.withFixedContentType(protobufContentType) { value =>
HttpEntity(protobufContentType, value.toByteArray)
}
}
Marshaller.oneOf(protobufMarshaller(), jsonMarshaller())
}
}
the issue i am facing is on the post route.
(post & entity(as[PropertyEntity])) { propertyEntity =>
complete {
saveProperty(propertyEntity)
}
}
During compilation time, i get the following error
Error:(20, 24) could not find implicit value for parameter um: akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[PropertyEntity]
(post & entity(as[PropertyEntity])) { propertyEntity =>
I am not sure exactly what i am missing. Do i need to define an implicit FromRequestUnmarshaller ? if so what should it have?
i was able to hack something together that works for the moment, but i still don't know how to create a general Unmarshaller that can decode any ScalaPB case class
implicit val um:Unmarshaller[HttpEntity, PropertyEntity] = {
Unmarshaller.byteStringUnmarshaller.mapWithCharset { (data, charset) =>
val charBuffer = Unmarshaller.bestUnmarshallingCharsetFor(data)
JsonFormat.fromJsonString(data.decodeString(charBuffer.nioCharset().name()))(PropertyEntity)
/*PropertyEntity.parseFrom(CodedInputStream.newInstance(data.asByteBuffer))*/
}
}
even this i don't know how to have both decoders enabled at the same time. so i have commented one out.

How to dynamically select class type when parsing JSON to object in Play Framework?

The following sample code uses Play Framework to parse JSON to an object:
def createEvent(): Action[JsValue] = Action.async(parse.tolerantJson) {
request => {
request.body.validate[SomeEvent] match {
case o:JsSuccess[SomeEvent] => {
//do something
Future(Created)
}
}
}
}
Is it possible to generalise it so it can handle different event types? e.g.
def createEvent(): Action[JsValue] = Action.async(parse.tolerantJson) {
request => {
val eventType = request.contentType match {
case Some("SomeEventType") => SomeEvent
case Some("OtherEventType") => OtherEvent
}
request.body.validate[eventType] match {
case o:JsSuccess[eventType] => {
//do something
Future(Created)
}
}
}
}
Currently, the above code will fail in the line request.body.validate[eventType]
You can extract body.validate[T] into a function and call it from your patten matching construct with a proper type, i.e:
def extract[T: JsonFormat](implicit req: Request[AnyContent]) = req.body.valudate[T]
request.contentType match {
case Some("SomeEventType") => extract[SomeEvent]
case Some("OtherEventType") => extract[OtherEvent]
}
You can read and create class from contentType dynamically. But you can have problem, if will be no implicit Format for extracted type in scope:
error: No Json formatter found for type A. Try to implement an implicit Format for this type.
or
java.lang.ClassNotFoundException: models.Bazz
The available data:
model
package models
case class Bar(name: String)
request
request.contentType: Option[String] = Some("models.Bar")
incoming body
request.body: JsValue = """{"name": "bar name"}"""
format[Bar]
implicit val BarFormat = Json.format[models.Bar]
extractor:
def extract[A <: Any](ct: String, json: JsValue)(implicit format: Format[A]) = {
val manifest:Manifest[A] = Manifest.classType[A](Class.forName(ct))
json.validate[A]
}
request.contentType.map(extract(_, request.body))
res1: Option[play.api.libs.json.JsResult[models.Bar]] = Some(JsSuccess(Bar(bar name),/name))
you can see https://github.com/strongtyped/fun-cqrs/blob/demos/shop-sample/samples/shop/src/main/scala/funcqrs/json/TypedJson.scala for other way of deserializing self-contained json.