Left to right arguments type inference - scala

I have a case where I wish to apply modifications to an object based on the presence of (a few, say, 5 to 10) optionals. So basically, if I were to do it imperatively, what I'm aiming for is :
var myObject = ...
if (option.isDefined) {
myObject = myObject.modify(option.get)
}
if (option2.isDefined) {
myObject = myObject.someOtherModification(option2.get)
}
(Please note : maybe my object is mutable, maybe not, that is not the point here.)
I thought it'd look nicer if I tried to implement a fluent way of writing this, such as (pseudo code...) :
myObject.optionally(option, _.modify(_))
.optionally(option2, _.someOtherModification(_))
So I started with a sample code, which intelliJ does not highlight as an error, but that actually does not build.
class MyObject(content: String) {
/** Apply a transformation if the optional is present */
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject =
optional.map(operation(_, this)).getOrElse(this)
/** Some possible transformation */
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object Test {
val my = new MyObject("test")
val option = Option(2)
my.optionally(option, (size, value) => value.resized(size))
}
Now, in my case, the MyObject type is of some external API, so I created an implicit conversion to help, so what it really does look like :
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
// What I did : create a rich type over MyObject
class MyRichObject(myObject: MyObject) {
def optionally[A](optional: Option[A], operation: (A, MyObject) => MyObject): MyObject = optional.map(operation(_, myObject)).getOrElse(myObject)
}
// And an implicit conversion
object MyRichObject {
implicit def apply(myObject: MyObject): MyRichObject = new MyRichObject(myObject)
}
And then, I use it this way :
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyRichObject._
my.optionally(option, (size, value) => value.resized(size))
}
And this time, it fails in IntelliJ and while compiling because the type of the Option is unknown :
Error:(8, 26) missing parameter type
my.optionally(option, (size, value) => value.resized(size))
To make it work, I can :
Actively specify a type of the size argument : my.optionally(option, (size: Int, value) => value.resized(size))
Rewrite the optionally to a curried-version
None of them is really bad, but if I may ask :
Is there a reason that a curried version works, but a multi argument version seems to fail to infer the parametrized type,
Could it be written in a way that works without specifying the actual types
and as a bonus (although this might be opinion based), how would you write it (some sort of foldLeft on a sequence of optionals come to my mind...) ?

One option for your consideration:
// Out of my control
class MyObject(content: String) {
def resized(length : Int): MyObject = new MyObject(content.substring(0, length))
}
object MyObjectImplicits {
implicit class OptionalUpdate[A](val optional: Option[A]) extends AnyVal {
def update(operation: (A, MyObject) => MyObject): MyObject => MyObject =
(obj: MyObject) => optional.map(a => operation(a, obj)).getOrElse(obj)
}
}
object Test {
val my = new MyObject("test")
val option = Option(2)
import MyObjectImplicits._
Seq(
option.update((size, value) => value.resized(size)),
// more options...
).foldLeft(my)(_)
}
Might as well just use a curried-version of your optionally, like you said.

A nicer way to think about the need to add the type there is write it this way:
object Test {
val my = new MyObject("test")
val option = Some(2)
my.optionally[Int](option, (size, value) => value.resized(size))
}
Another way, if you only will manage one type since the object creation, is to move the generic to the class creation, but be careful, with this option you only can have one type per instance:
class MyObject[A](content: String) {
def optionally(optional: Option[A], operation: (A, MyObject[A]) => MyObject[A]): MyObject[A] =
optional.map(operation(_, this)).getOrElse(this)
def resized(length : Int): MyObject[A] = new MyObject[A](content.substring(0, length))
}
object Test {
val my = new MyObject[Int]("test")
val option = Some(2)
my.optionally(option, (size, value) => value.resized(size))
}
As you can see, now all the places where the generics was is taken by the Int type, because that is what you wanted in the first place, here is a pretty answer telling why:
(just the part that I think applies here:)
4)When the inferred return type would be more general than you intended, e.g., Any.
Source: In Scala, why does a type annotation must follow for the function parameters ? Why does the compiler not infer the function parameter types?

Related

How to write a custom decoder for [Option[Option[A]] in Circe?

I had written a Reads converter in play-json for Option[Option[A]] that had the following behavior:
//given this case class
case class MyModel(field: Option[Option[String]])
//this JSON -- maps to --> this MyModel:
//"{ \"field\": \"value\" }" --> MyModel(field = Some(Some("value")))
//"{ \"field\": null, ... }" --> MyModel(field = Some(None))
//"{ }" --> MyModel(field = None)
So, providing the value mapped to Some[Some[A]], providing null mapped to Some[None] (i.e. Some[Option.empty[A]]), and not providing the value mapped to just None (i.e. Option.empty[Option[A]]). Here's the play-json converter:
def readOptOpt[A](implicit r: Reads[A]): Reads[Option[Option[A]]] = {
Reads[Option[Option[A]]] { json =>
path.applyTillLast(json).fold(
identity,
_.fold(_ => JsSuccess(None), {
case JsNull => JsSuccess(Some(None))
case js => r.reads(js).repath(path).map(a => Some(Some(a)))
})
)
}
}
Now I am converting my play-json code to Circe, but I can't figure out how to write a Decoder[Option[Option[A]] that has the same behavior. That is, I need
def optOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]] = ??? //help!
Any ideas on how I can make this work? Thanks
I figured this out:
There were two problems:
1) How to deal with the case where the field was completely missing from the JSON. Turns out you have to use Decoder.reattempt in your custom decoder, following Circe's decodeOption code, which works.
2) How to have the compiler recognize cases of Option[Option[A]] when your decoder code is sitting in a helper object (or wherever). Turns out if you're using semi-auto derivation, you can create an implicit in the companion object and that will override the defaults:
//companion object
object MyModel {
implicit def myModelOptOptDecoder[A](implicit d: Decoder[A]): Decoder[Option[Option[A]]] =
MyHelperObject.optOptDecoder
implicit val myModelDecoder: Decoder[MyModel] = deriveDecoder
}
Anyway, I don't think this will be much help to anybody in the future, so unless I get any upvotes in the next few hours I think I'll just delete this.
Edit2: Okay it was answered so I won't delete it. Stay strong, esoteric circe question, stay strong...
An Option[Option[A]] is a bit odd. I understand and mostly agree with the reasoning, but I think it's weird enough that it may warrant just replacing it with your own class (and writing a decoder for that). Something like:
sealed trait OptionalNull[+A] {
def toOption: Option[Option[A]]
}
object NotPresent extends OptionalNull[Nothing] {
override def toOption = None
}
object PresentButNull extends OptionalNull[Nothing] {
override def toOption = Some(None)
}
case class PresentNotNull[A](value: A) extends OptionalNull[A] {
override def toOption = Some(Some(value))
}
This has the additional benefit of not having to worry about implicit precedence and stuff like that. Might simplify your decoder.
Here is another solution I found (This is not my gist):
sealed trait UpdateOrDelete[+A]
case object Delete extends UpdateOrDelete[Nothing]
final case class UpdateOptionalFieldWith[A](value: A) extends UpdateOrDelete[A]
object UpdateOrDelete {
implicit def optionalDecoder[A](implicit decodeA: Decoder[A]): Decoder[UpdateOptionalField[A]] =
Decoder.withReattempt {
// We're trying to decode a field but it's missing.
case c: FailedCursor if !c.incorrectFocus => Right(None)
case c =>
Decoder.decodeOption[A].tryDecode(c).map {
case Some(a) => Some(UpdateOptionalFieldWith(a))
case None => Some(Delete)
}
}
// Random UUID to _definitely_ avoid collisions
private[this] val marker: String = s"$$marker-${UUID.randomUUID()}-marker$$"
private[this] val markerJson: Json = Json.fromString(marker)
implicit def optionalEncoder[A](implicit encodeA: Encoder[A]): Encoder[UpdateOptionalField[A]] =
Encoder.instance {
case Some(Delete) => Json.Null
case Some(UpdateOptionalFieldWith(a)) => encodeA(a)
case None => markerJson
}
def filterMarkers[A](encoder: Encoder.AsObject[A]): Encoder.AsObject[A] =
encoder.mapJsonObject { obj =>
obj.filter {
case (_, value) => value =!= markerJson
}
}
}

Handling loads of different message-types at runtime in an elegant way

In order to be able to handle large amounts of different request types I created a .proto file like this:
message Message
{
string typeId = 1;
bytes message = 2;
}
I added the typeId so that one knows what actual protobuf bytes represents. (Self-describing)
Now my problem is handling that different "concrete types" in an elegant way. (Note: All works fine if I simple use a switch-case-like approach!)
I thought about a solution like this:
1) Have a trait the different handlers have to implement, e.g.:
trait Handler[T]
{
def handle(req: T): Any
}
object TestHandler extends Handler[Test]
{
override def handle(req: Test): String =
{
s"A success, $req has been handled by TestHandler
}
}
object OtherHandler extends Handler[Other]
{
override def handle(req: Other): String =
{
s"A success, $req has been handled by OtherHandler
}
}
2) provide some kind of registry to query the right handler for a given message:
val handlers = Map(
Test -> TestHandler,
Other -> OtherHandler
)
3) If a request comes in it identifies itself, so we need another Mapper:
val reqMapper = Map(
"Test" -> Test
"Other" -> Other
)
4) If a request comes in, handle it:
val request ...
// Determine the requestType
val requestType = reqMapper(request.type)
// Find the correct handler for the requestType
val handler = handlers(requestType)
// Parse the actual request
val actualRequest = requestType.parse(...) // type of actualRequest can only be Test or Other in our little example
Now, until here everything looks fine and dandy, but then this line breaks my whole world:
handler.handle(actualRequest)
It leads to:
type mismatch; found : com.trueaccord.scalapb.GeneratedMessage with Product with com.trueaccord.scalapb.Message[_ >: tld.test.proto.Message.Test with tld.test.proto.Message.Other <: com.trueaccord.scalapb.GeneratedMessage with Product] with com.trueaccord.lenses.Updatable[_ >: tld.test.proto.Message.Other with tld.test.proto.Message.Test <: com.trueaccord.scalapb.GeneratedMessage with Product]{def companion: Serializable} required: _1
As far as I understand - PLEASE CORRECT ME HERE IF AM WRONG - the compiler cannot be sure here, that actualRequest is "handable" by a handler. That means it lacks the knowledge that the actualRequest is definitely somewhere in that mapper AND ALSO that there is a handler for it.
It's basically implicit knowledge a human would get, but the compiler cannot infer.
So, that being said, how can I overcome that situation elegantly?
your types are lost when you use a normal Map. for eg
object Test{}
object Other{}
val reqMapper = Map("Test" -> Test,"Other" -> Other)
reqMapper("Test")
res0: Object = Test$#5bf0fe62 // the type is lost here and is set to java.lang.Object
the most idomatic way to approach this is to use pattern matching
request match {
case x: Test => TestHandler(x)
case x: Other => OtherHandler(x)
case _ => throw new IllegalArgumentException("not supported")
}
if you still want to use Maps to store your type to handler relation consider HMap provided by Shapeless here
Heterogenous maps
Shapeless provides a heterogenous map which supports an arbitrary
relation between the key type and the corresponding value type,
I settled for this solution for now (basically thesamet's, a bit adapted for my particular use-case)
trait Handler[T <: GeneratedMessage with Message[T], R]
{
implicit val cmp: GeneratedMessageCompanion[T]
def handle(bytes: ByteString): R = {
val msg: T = cmp.parseFrom(bytes.newInput())
handler(msg)
}
def apply(t: T): R
}
object Test extends Handler[Test, String]
{
override def apply(t: Test): String = s"$t received and handled"
override implicit val cmp: GeneratedMessageCompanion[Test] = Test.messageCompanion
}
One trick you can use is to capture the companion object as an implicit, and combine the parsing and handling in a single function where the type is available to the compiler:
case class Handler[T <: GeneratedMessage with Message[T]](handler: T => Unit)(implicit cmp: GeneratedMessageCompanion[T]) {
def handle(bytes: ByteString): Unit = {
val msg: T = cmp.parseFrom(bytes.newInputStream)
handler(t)
}
}
val handlers: Map[String, Handler[_]] = Map(
"X" -> Handler((x: X) => Unit),
"Y" -> Handler((x: Y) => Unit)
)
// To handle the request:
handlers(request.typeId).handle(request.message)
Also, take a look at any.proto which defines a structure very similar to your Message. It wouldn't solve your problem, but you can take advantage of it's pack and unpack methods.

Explanation on the error with for comprehension and co-variance

Question
Would like to get assistance to understand the cause of the error. The original is from Coursera Scala Design Functional Random Generators.
Task
With the factories for random int and random boolean, trying to implement a random tree factory.
trait Factory[+T] {
self => // alias of 'this'
def generate: T
def map[S](f: T => S): Factory[S] = new Factory[S] {
def generate = f(self.generate)
}
def flatMap[S](f: T => Factory[S]): Factory[S] = new Factory[S] {
def generate = f(self.generate).generate
}
}
val intFactory = new Factory[Int] {
val rand = new java.util.Random
def generate = rand.nextInt()
}
val boolFactory = intFactory.map(i => i > 0)
Problem
The implementation in the 1st block causes the error but if it changed into the 2nd block, it does not. I believe Factory[+T] meant that Factory[Inner] and Factory[Leaf] could be both treated as Factory[Tree].
I have no idea why the same if expression in for block is OK but it is not OK in yield block. I appreciate explanations.
trait Tree
case class Inner(left: Tree, right: Tree) extends Tree
case class Leaf(x: Int) extends Tree
def leafFactory: Factory[Leaf] = intFactory.map(i => new Leaf(i))
def innerFactory: Factory[Inner] = new Factory[Inner] {
def generate = new Inner(treeFactory.generate, treeFactory.generate)
}
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
} yield if (isLeaf) leafFactory else innerFactory
^^^^^^^^^^^ ^^^^^^^^^^^^
type mismatch; found : Factory[Inner] required: Tree
type mismatch; found : Factory[Leaf] required: Tree
However, below works.
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
tree <- if (isLeaf) leafFactory else innerFactory
} yield tree
I have no idea why the same if expression in for block is OK but it is
not OK in yield block
Because they are translated differently by the compiler. The former example is translated into:
boolFactory.flatMap((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactor)
Which yields the expected Factory[Tree], while the latter is being translated to:
boolFactory.map((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactory)
Which yields a Factory[Factory[Tree]], not a Factory[Tree], thus not conforming to your method signature. This isn't about covariance, but rather how for comprehension translates these statements differently.

How to define a parametric type alias

I try to define a parametric type alias :
case class A
case class B
case class C
// We need an Int to load instances of A and B, and a String to load C
object Service {
def loadA(i: Int) : A = ???
def loadB(i: Int) : B = ???
def loadC(s: String) : C = ???
}
trait Location[T] { def get : T}
class IntLocation(val i: Int)
class StringLocation(val s: String)
trait EntityLocation[E] extends Location[_]
// Aim : make the loader typesafe
// Problem : I need something like that : type EntityLocation[Composite] = IntLocation
object Family {
trait EntityLoader[EntityT] extends (EntityLocation[EntityT] => EntityT)
val ALoader = new EntityLoader[A] {def load[A](l: EntityLocation[A]) = Service.loadA(l.get)
}
I am not sure what you are trying to achieve here. Could you please explain how you want to use these types in your code?
Assuming just want to use the types IdLocation and FileLocation in your code, maybe you want to try
trait Location[T] { def get : T }
type IdLocation = Location[Id]
type FileLocation = Location[java.io.File]
Seems rather convoluted, so I'm not sure I follow exactly what your purpose here is. You seem to go into many layers of factories that create factories, that call factory methods, etc.
Seems to me that at the end of the day you need you want to have a val ALoader value that you can use to get instances of A from Location[Int] objects, so I'll go with that assumption:
// Not sure what you want this one, but let's assume that you need a wrapper class per your example.
trait Location[P] { def get: P }
class IntLocation(val i: Int) extends Location[Int]
{
override def get: Int = i
}
// P for parameter, O for output class.
def loader[O, P](creator: P => O)(param: Location[P]) = { creator(param.get) }
object Service
{
// A function somewhere, capable of taking your parameter and creating something else (in your example, an Int to an 'A')
// here Int to String to make something concrete.
// This could be any function, anywhere
def loadA(someParam: Int) = someParam.toString
}
def main(args: Array[String])
{
val myStringLoader: Location[Int] => String = loader(Service.loadA)
// Alternatively, you could have written `val myStringLoader = loader(Service.loadA)(_)`. Either the type or the underscore are needed to tell the compiler that you expect a function, not a value.
// Some definition for you wrapper class
val location3 = new Location[Int]{
override def get: Int = 3
}
// ... or just a plain old instance of it.
val otherLocation = new IntLocation(5)
// This would 'load' the kind of thing you want using the method you specified.
val myString = myStringLoader(location3)
val myOtherString = myStringLoader(otherLocation)
// This prints "3 - 5"
print(myString + " - " + myOtherString)
}
This might seem like a long answer, but in truth the line def loader[O, P](creator: P => O)(param: Location[P]) = { creator(param.get) } is the one that does it all, the rest is to make it as similar to your sample as possible and to provide a working main you can use to start from.
Of course, this would be even simpler if you don't really need the Location wrapper for your integer.

Inferring result type in continuations

Is it possible to remove some types from the following code:
import util.continuations._
object TrackingTest extends App {
implicit def trackable(x: Int) = new {
def tracked[R] = shift { cf: (Int => (R, Set[Int])) =>
cf(x) match {
case (r, ints) => (r, ints + x)
}
}
}
def track[R](body: => R #cpsParam[(R, Set[Int]), (R, Set[Int])]) = reset {
(body, Set[Int]())
}
val result = track(7.tracked[Int] + 35.tracked[Int])
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked[String].toString)
assert(differentTypes == ("9", Set(9)))
}
track function tracks calls of tracked on Int instances (e.g. 7.tracked).
Is it possible to infer type parameter on tracked implicit, so the following would compile:
track(7.tracked + 35.tracked)
Your question made me think of how continuations can track state. So I adapted that to your case and came up with this:
import util.continuations._
object TrackingTest extends App {
type State = Set[Int]
type ST = State => State
implicit class Tracked(val i: Int) extends AnyVal {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
def track[A](thunk: => A#cps[ST]): (A, State) = {
var result: A = null.asInstanceOf[A]
val finalSate = (reset {
result = thunk
(state:State) => state
}).apply(Set[Int]())
(result, finalSate)
}
val result = track(7.tracked + 35.tracked)
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked.toString)
assert(differentTypes == ("9", Set(9)))
}
This is using 2.10.1 but it works fine with 2.9.1 as well provided you replace the 2.10.x implicit value class with:
implicit def tracked(i: Int) = new {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
The key change I made is to have tracked not use any type inference, fixing to Int#cps[ST]. The CPS plugin then maps the computation to the right type (like String#cps[ST]) as appropriate. The state is threaded by the continuation returning a State=>State function that takes the current state (the set of ints) and returns the next state. The return type of reset is a function from state to state (of type ST) that will take the initial state and will return the final state.
The final trick is to use a var to capture the result while still keeping the expected type for reset.
While the exact answer to this question can be given only by the authors of the compiler, we can guess it is not possible by giving a look to the continuation plugin source code.
If you look to the source of the continuations you can see this:
val anfPhase = new SelectiveANFTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("pickler")
}
val cpsPhase = new SelectiveCPSTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("selectiveanf")
}
The anfPhase phase is executed after the pickler phase, and the cpsPhase after selectiveAnf. If you look to SelectiveANFTransform.scala
abstract class SelectiveANFTransform extends PluginComponent with Transform with
TypingTransformers with CPSUtils {
// inherits abstract value `global' and class `Phase' from Transform
import global._ // the global environment
import definitions._ // standard classes and methods
import typer.atOwner // methods to type trees
/** the following two members override abstract members in Transform */
val phaseName: String = "selectiveanf"
If we use scalac -Xshow-phases, we can see the phases during the compilation process:
parser
namer
packageobjects
typer
superaccessors
pickler
refchecks
selectiveanf
liftcode
selectivecps
uncurry
......
As you can see the typer phase is applied before the selectiveAnf and selectiveCps phases. It should be confirmed that type inference occurs in the typer phase, but if this is really the case and it would make sense, it should be now clear why you can't omit the Int type on 7.tracked and 35.tracked.
Now if you are not satisfied yet, you should know that the compiler works by performing a set of transformations on "trees", which you might look, using the following options:
-Xprint: shows your scala code after a certain phase have been executed
-Xprint: -Yshow-trees shows your scala code and the trees after the phase have been executed
-YBrowse: opens a GUI to surf both.