In some cases default values make more sense than optionals in case classes:
case class Car(numberOfWheels:Int = 4, color:String)
case class Car(numbeOfWheels:Option[Int], color:String) //silly
In the first case I'd expect to be able to easily convert the following json to an instance:
{"color":"red"}
But with a standard jsonFormat2(Car), spray-json complains about missing value for numberOfWheels.
How do I work around this most cleanly?
I stumbled upon the same problem. I've create a patch that solves it for me. It makes fields with a default value optional.
https://github.com/spray/spray-json/pull/56
update: PR is updated and still open https://github.com/spray/spray-json/pull/93
I have never used spray, but here's my guess about what may work:
case class Car(numberOfWheels: Int, color: String) {
def this(color: String) = this(4, color)
}
object Car {
def apply(color: String) = new Car(color)
}
Maybe now jsonFormat1(Car) will work.
The fix I found for the same issue was to implement my own jsonFormat:
implicit object carFormat extends JsonFormat[Car] {
def write(car: Car): JsObject = {
val fields = List(
("numberOfWheels" -> JsNumber(car.numberOfWheels)),
("color" -> JsString(car.color))
)
JsObject(fields: _*)
}
def read(json: JsValue): Car = {
val numberOfWheels = fromField[Option[Int]](json, "numberOfWheels")
val color = fromField[String](json, "color")
Car(numberOfWheels.getOrElse(4), color)
}
}
Related
I had written a Reads converter in play-json for Option[Option[A]] that had the following behavior:
//given this case class
case class MyModel(field: Option[Option[String]])
//this JSON -- maps to --> this MyModel:
//"{ \"field\": \"value\" }" --> MyModel(field = Some(Some("value")))
//"{ \"field\": null, ... }" --> MyModel(field = Some(None))
//"{ }" --> MyModel(field = None)
So, providing the value mapped to Some[Some[A]], providing null mapped to Some[None] (i.e. Some[Option.empty[A]]), and not providing the value mapped to just None (i.e. Option.empty[Option[A]]). Here's the play-json converter:
def readOptOpt[A](implicit r: Reads[A]): Reads[Option[Option[A]]] = {
Reads[Option[Option[A]]] { json =>
path.applyTillLast(json).fold(
identity,
_.fold(_ => JsSuccess(None), {
case JsNull => JsSuccess(Some(None))
case js => r.reads(js).repath(path).map(a => Some(Some(a)))
})
)
}
}
Now I am converting my play-json code to Circe, but I can't figure out how to write a Decoder[Option[Option[A]] that has the same behavior. That is, I need
def optOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]] = ??? //help!
Any ideas on how I can make this work? Thanks
I figured this out:
There were two problems:
1) How to deal with the case where the field was completely missing from the JSON. Turns out you have to use Decoder.reattempt in your custom decoder, following Circe's decodeOption code, which works.
2) How to have the compiler recognize cases of Option[Option[A]] when your decoder code is sitting in a helper object (or wherever). Turns out if you're using semi-auto derivation, you can create an implicit in the companion object and that will override the defaults:
//companion object
object MyModel {
implicit def myModelOptOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]]] =
MyHelperObject.optOptDecoder
implicit val myModelDecoder: Decoder[MyModel] = deriveDecoder
}
Anyway, I don't think this will be much help to anybody in the future, so unless I get any upvotes in the next few hours I think I'll just delete this.
Edit2: Okay it was answered so I won't delete it. Stay strong, esoteric circe question, stay strong...
An Option[Option[A]] is a bit odd. I understand and mostly agree with the reasoning, but I think it's weird enough that it may warrant just replacing it with your own class (and writing a decoder for that). Something like:
sealed trait OptionalNull[+A] {
def toOption: Option[Option[A]]
}
object NotPresent extends OptionalNull[Nothing] {
override def toOption = None
}
object PresentButNull extends OptionalNull[Nothing] {
override def toOption = Some(None)
}
case class PresentNotNull[A](value: A) extends OptionalNull[A] {
override def toOption = Some(Some(value))
}
This has the additional benefit of not having to worry about implicit precedence and stuff like that. Might simplify your decoder.
Here is another solution I found (This is not my gist):
sealed trait UpdateOrDelete[+A]
case object Delete extends UpdateOrDelete[Nothing]
final case class UpdateOptionalFieldWith[A](value: A) extends UpdateOrDelete[A]
object UpdateOrDelete {
implicit def optionalDecoder[A](implicit decodeA: Decoder[A]): Decoder[UpdateOptionalField[A]] =
Decoder.withReattempt {
// We're trying to decode a field but it's missing.
case c: FailedCursor if !c.incorrectFocus => Right(None)
case c =>
Decoder.decodeOption[A].tryDecode(c).map {
case Some(a) => Some(UpdateOptionalFieldWith(a))
case None => Some(Delete)
}
}
// Random UUID to _definitely_ avoid collisions
private[this] val marker: String = s"$$marker-${UUID.randomUUID()}-marker$$"
private[this] val markerJson: Json = Json.fromString(marker)
implicit def optionalEncoder[A](implicit encodeA: Encoder[A]): Encoder[UpdateOptionalField[A]] =
Encoder.instance {
case Some(Delete) => Json.Null
case Some(UpdateOptionalFieldWith(a)) => encodeA(a)
case None => markerJson
}
def filterMarkers[A](encoder: Encoder.AsObject[A]): Encoder.AsObject[A] =
encoder.mapJsonObject { obj =>
obj.filter {
case (_, value) => value =!= markerJson
}
}
}
I have a case where I wish to apply modifications to an object based on the presence of (a few, say, 5 to 10) optionals. So basically, if I were to do it imperatively, what I'm aiming for is :
var myObject = ...
if (option.isDefined) {
myObject = myObject.modify(option.get)
}
if (option2.isDefined) {
myObject = myObject.someOtherModification(option2.get)
}
(Please note : maybe my object is mutable, maybe not, that is not the point here.)
I thought it'd look nicer if I tried to implement a fluent way of writing this, such as (pseudo code...) :
myObject.optionally(option, _.modify(_))
.optionally(option2, _.someOtherModification(_))
So I started with a sample code, which intelliJ does not highlight as an error, but that actually does not build.
class MyObject(content: String) {
/** Apply a transformation if the optional is present */
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject =
optional.map(operation(_, this)).getOrElse(this)
/** Some possible transformation */
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object Test {
val my = new MyObject("test")
val option = Option(2)
my.optionally(option, (size, value) => value.resized(size))
}
Now, in my case, the MyObject type is of some external API, so I created an implicit conversion to help, so what it really does look like :
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
// What I did : create a rich type over MyObject
class MyRichObject(myObject: MyObject) {
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject = optional.map(operation(_, myObject)).getOrElse(myObject)
}
// And an implicit conversion
object MyRichObject {
implicit def apply(myObject: MyObject): MyRichObject = new MyRichObject(myObject)
}
And then, I use it this way :
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyRichObject._
my.optionally(option, (size, value) => value.resized(size))
}
And this time, it fails in IntelliJ and while compiling because the type of the Option is unknown :
Error:(8, 26) missing parameter type
my.optionally(option, (size, value) => value.resized(size))
To make it work, I can :
Actively specify a type of the size argument : my.optionally(option, (size: Int, value) => value.resized(size))
Rewrite the optionally to a curried-version
None of them is really bad, but if I may ask :
Is there a reason that a curried version works, but a multi argument version seems to fail to infer the parametrized type,
Could it be written in a way that works without specifying the actual types
and as a bonus (although this might be opinion based), how would you write it (some sort of foldLeft on a sequence of optionals come to my mind...) ?
One option for your consideration:
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object MyObjectImplicits {
implicit class OptionalUpdate[A](val optional: Option[A]) extends AnyVal {
def update(operation: (A, MyObject) => MyObject): MyObject => MyObject =
(obj: MyObject) => optional.map(a => operation(a, obj)).getOrElse(obj)
}
}
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyObjectImplicits._
Seq(
option.update((size, value) => value.resized(size)),
// more options...
).foldLeft(my)(_)
}
Might as well just use a curried-version of your optionally, like you said.
A nicer way to think about the need to add the type there is write it this way:
object Test {
val my = new MyObject("test")
val option = Some(2)
my.optionally[Int](option, (size, value) => value.resized(size))
}
Another way, if you only will manage one type since the object creation, is to move the generic to the class creation, but be careful, with this option you only can have one type per instance:
class MyObject[A](content: String) {
def optionally(optional: Option[A], operation: (A, MyObject[A]) => MyObject[A]): MyObject[A] =
optional.map(operation(_, this)).getOrElse(this)
def resized(length : Int): MyObject[A] = new MyObject[A](content.substring(0, length))
}
object Test {
val my = new MyObject[Int]("test")
val option = Some(2)
my.optionally(option, (size, value) => value.resized(size))
}
As you can see, now all the places where the generics was is taken by the Int type, because that is what you wanted in the first place, here is a pretty answer telling why:
(just the part that I think applies here:)
4)When the inferred return type would be more general than you intended, e.g., Any.
Source: In Scala, why does a type annotation must follow for the function parameters ? Why does the compiler not infer the function parameter types?
In order to be able to handle large amounts of different request types I created a .proto file like this:
message Message
{
string typeId = 1;
bytes message = 2;
}
I added the typeId so that one knows what actual protobuf bytes represents. (Self-describing)
Now my problem is handling that different "concrete types" in an elegant way. (Note: All works fine if I simple use a switch-case-like approach!)
I thought about a solution like this:
1) Have a trait the different handlers have to implement, e.g.:
trait Handler[T]
{
def handle(req: T): Any
}
object TestHandler extends Handler[Test]
{
override def handle(req: Test): String =
{
s"A success, $req has been handled by TestHandler
}
}
object OtherHandler extends Handler[Other]
{
override def handle(req: Other): String =
{
s"A success, $req has been handled by OtherHandler
}
}
2) provide some kind of registry to query the right handler for a given message:
val handlers = Map(
Test -> TestHandler,
Other -> OtherHandler
)
3) If a request comes in it identifies itself, so we need another Mapper:
val reqMapper = Map(
"Test" -> Test
"Other" -> Other
)
4) If a request comes in, handle it:
val request ...
// Determine the requestType
val requestType = reqMapper(request.type)
// Find the correct handler for the requestType
val handler = handlers(requestType)
// Parse the actual request
val actualRequest = requestType.parse(...) // type of actualRequest can only be Test or Other in our little example
Now, until here everything looks fine and dandy, but then this line breaks my whole world:
handler.handle(actualRequest)
It leads to:
type mismatch; found : com.trueaccord.scalapb.GeneratedMessage with Product with com.trueaccord.scalapb.Message[_ >: tld.test.proto.Message.Test with tld.test.proto.Message.Other <: com.trueaccord.scalapb.GeneratedMessage with Product] with com.trueaccord.lenses.Updatable[_ >: tld.test.proto.Message.Other with tld.test.proto.Message.Test <: com.trueaccord.scalapb.GeneratedMessage with Product]{def companion: Serializable} required: _1
As far as I understand - PLEASE CORRECT ME HERE IF AM WRONG - the compiler cannot be sure here, that actualRequest is "handable" by a handler. That means it lacks the knowledge that the actualRequest is definitely somewhere in that mapper AND ALSO that there is a handler for it.
It's basically implicit knowledge a human would get, but the compiler cannot infer.
So, that being said, how can I overcome that situation elegantly?
your types are lost when you use a normal Map. for eg
object Test{}
object Other{}
val reqMapper = Map("Test" -> Test,"Other" -> Other)
reqMapper("Test")
res0: Object = Test$#5bf0fe62 // the type is lost here and is set to java.lang.Object
the most idomatic way to approach this is to use pattern matching
request match {
case x: Test => TestHandler(x)
case x: Other => OtherHandler(x)
case _ => throw new IllegalArgumentException("not supported")
}
if you still want to use Maps to store your type to handler relation consider HMap provided by Shapeless here
Heterogenous maps
Shapeless provides a heterogenous map which supports an arbitrary
relation between the key type and the corresponding value type,
I settled for this solution for now (basically thesamet's, a bit adapted for my particular use-case)
trait Handler[T <: GeneratedMessage with Message[T], R]
{
implicit val cmp: GeneratedMessageCompanion[T]
def handle(bytes: ByteString): R = {
val msg: T = cmp.parseFrom(bytes.newInput())
handler(msg)
}
def apply(t: T): R
}
object Test extends Handler[Test, String]
{
override def apply(t: Test): String = s"$t received and handled"
override implicit val cmp: GeneratedMessageCompanion[Test] = Test.messageCompanion
}
One trick you can use is to capture the companion object as an implicit, and combine the parsing and handling in a single function where the type is available to the compiler:
case class Handler[T <: GeneratedMessage with Message[T]](handler: T => Unit)(implicit cmp: GeneratedMessageCompanion[T]) {
def handle(bytes: ByteString): Unit = {
val msg: T = cmp.parseFrom(bytes.newInputStream)
handler(t)
}
}
val handlers: Map[String, Handler[_]] = Map(
"X" -> Handler((x: X) => Unit),
"Y" -> Handler((x: Y) => Unit)
)
// To handle the request:
handlers(request.typeId).handle(request.message)
Also, take a look at any.proto which defines a structure very similar to your Message. It wouldn't solve your problem, but you can take advantage of it's pack and unpack methods.
I'm using scala and slick here, and I have a baserepository which is responsible for doing the basic crud of my classes.
For a design decision, we do have updatedTime and createdTime columns all handled by the application, and not by triggers in database. Both of this fields are joda DataTime instances.
Those fields are defined in two traits called HasUpdatedAt, and HasCreatedAt, for the tables
trait HasCreatedAt {
val createdAt: Option[DateTime]
}
case class User(name:String,createdAt:Option[DateTime] = None) extends HasCreatedAt
I would like to know how can I use reflection to call the user copy method, to update the createdAt value during the database insertion method.
Edit after #vptron and #kevin-wright comments
I have a repo like this
trait BaseRepo[ID, R] {
def insert(r: R)(implicit session: Session): ID
}
I want to implement the insert just once, and there I want to createdAt to be updated, that's why I'm not using the copy method, otherwise I need to implement it everywhere I use the createdAt column.
This question was answered here to help other with this kind of problem.
I end up using this code to execute the copy method of my case classes using scala reflection.
import reflect._
import scala.reflect.runtime.universe._
import scala.reflect.runtime._
class Empty
val mirror = universe.runtimeMirror(getClass.getClassLoader)
// paramName is the parameter that I want to replacte the value
// paramValue is the new parameter value
def updateParam[R : ClassTag](r: R, paramName: String, paramValue: Any): R = {
val instanceMirror = mirror.reflect(r)
val decl = instanceMirror.symbol.asType.toType
val members = decl.members.map(method => transformMethod(method, paramName, paramValue, instanceMirror)).filter {
case _: Empty => false
case _ => true
}.toArray.reverse
val copyMethod = decl.declaration(newTermName("copy")).asMethod
val copyMethodInstance = instanceMirror.reflectMethod(copyMethod)
copyMethodInstance(members: _*).asInstanceOf[R]
}
def transformMethod(method: Symbol, paramName: String, paramValue: Any, instanceMirror: InstanceMirror) = {
val term = method.asTerm
if (term.isAccessor) {
if (term.name.toString == paramName) {
paramValue
} else instanceMirror.reflectField(term).get
} else new Empty
}
With this I can execute the copy method of my case classes, replacing a determined field value.
As comments have said, don't change a val using reflection. Would you that with a java final variable? It makes your code do really unexpected things. If you need to change the value of a val, don't use a val, use a var.
trait HasCreatedAt {
var createdAt: Option[DateTime] = None
}
case class User(name:String) extends HasCreatedAt
Although having a var in a case class may bring some unexpected behavior e.g. copy would not work as expected. This may lead to preferring not using a case class for this.
Another approach would be to make the insert method return an updated copy of the case class, e.g.:
trait HasCreatedAt {
val createdAt: Option[DateTime]
def withCreatedAt(dt:DateTime):this.type
}
case class User(name:String,createdAt:Option[DateTime] = None) extends HasCreatedAt {
def withCreatedAt(dt:DateTime) = this.copy(createdAt = Some(dt))
}
trait BaseRepo[ID, R <: HasCreatedAt] {
def insert(r: R)(implicit session: Session): (ID, R) = {
val id = ???//insert into db
(id, r.withCreatedAt(??? /*now*/))
}
}
EDIT:
Since I didn't answer your original question and you may know what you are doing I am adding a way to do this.
import scala.reflect.runtime.universe._
val user = User("aaa", None)
val m = runtimeMirror(getClass.getClassLoader)
val im = m.reflect(user)
val decl = im.symbol.asType.toType.declaration("createdAt":TermName).asTerm
val fm = im.reflectField(decl)
fm.set(??? /*now*/)
But again, please don't do this. Read this stackoveflow answer to get some insight into what it can cause (vals map to final fields).
I have an optional field on my requests:
case class SearchRequest(url: String, nextAt: Option[Date])
My protocol is:
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
}
How do I mark the nextAt field optional, such that the following JSON objects will be correctly read and accepted:
{"url":"..."}
{"url":"...", "nextAt":null}
{"url":"...", "nextAt":"2012-05-30T15:23Z"}
I actually don't really care about the null case, but if you have details, it would be nice. I'm using spray-json, and was under the impression that using an Option would skip the field if it was absent on the original JSON object.
Works for me (spray-json 1.1.1 scala 2.9.1 build)
import cc.spray.json._
import cc.spray.json.DefaultJsonProtocol._
// string instead of date for simplicity
case class SearchRequest(url: String, nextAt: Option[String])
// btw, you could use jsonFormat2 method here
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
assert {
List(
"""{"url":"..."}""",
"""{"url":"...", "nextAt":null}""",
"""{"url":"...", "nextAt":"2012-05-30T15:23Z"}""")
.map(_.asJson.convertTo[SearchRequest]) == List(
SearchRequest("...", None),
SearchRequest("...", None),
SearchRequest("...", Some("2012-05-30T15:23Z")))
}
You might have to create an explicit format (warning: psuedocodish):
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit object SearchRequestJsonFormat extends JsonFormat[SearchRequest] {
def read(value: JsValue) = value match {
case JsObject(List(
JsField("url", JsString(url)),
JsField("nextAt", JsString(nextAt)))) =>
SearchRequest(url, Some(new Instant(nextAt)))
case JsObject(List(JsField("url", JsString(url)))) =>
SearchRequest(url, None)
case _ =>
throw new DeserializationException("SearchRequest expected")
}
def write(obj: SearchRequest) = obj.nextAt match {
case Some(nextAt) =>
JsObject(JsField("url", JsString(obj.url)),
JsField("nextAt", JsString(nextAt.toString)))
case None => JsObject(JsField("url", JsString(obj.url)))
}
}
}
Use NullOptions trait to disable skipping nulls:
https://github.com/spray/spray-json#nulloptions
Example:
https://github.com/spray/spray-json/blob/master/src/test/scala/spray/json/ProductFormatsSpec.scala
Don't know if this will help you but you can give that field a default value in the case class definition, so if the field is not in the json, it will assign the default value to it.
Easy.
import cc.spray.json._
trait MyJsonProtocol extends DefaultJsonProtocol {
implicit val searchFormat = new JsonWriter[SearchRequest] {
def write(r: SearchRequest): JsValue = {
JsObject(
"url" -> JsString(r.url),
"next_at" -> r.nextAt.toJson,
)
}
}
}
class JsonTest extends FunSuite with MyJsonProtocol {
test("JSON") {
val search = new SearchRequest("www.site.ru", None)
val marshalled = search.toJson
println(marshalled)
}
}
For anyone who is chancing upon this post and wants an update to François Beausoleil's answer for newer versions of Spray (circa 2015+?), JsField is deprecated as a public member of JsValue; you should simply supply a list of tuples instead of JsFields. Their answer is spot-on, though.