Get the "most specific" input type - scala

Suppose I have a function-like type e.g.
trait Parser[-Context, +Out]
and I want to be able to combine multiple parsers such that the combined Context will be the most-specific type among the combined parsers' contexts. For example:
Parser[Any, Int] + Parser[String, Long] = Parser[String, (Int, Long)]
Parser[String, Int] + Parser[Any, Long] = Parser[String, (Int, Long)]
Parser[Option[Int], Foo] + Parser[Some[Int], Bar] = Parser[Some[Int], (Foo, Bar)]
Parser[String, Foo] + Parser[Int, Bar] = <should be a compile error>
To put the example in more concrete terms, suppose I have a function combiner like
def zipFuncs[A, B1, B2](f1: A => B1, f2: A => B2): A => (B1, B2) = {
a => (f1(a), f2(a))
}
and some functions like
val f1 = { a: Any => 123 }
val f2 = { a: String => 123 }
val f3 = { a: Option[Int] => 123 }
Now I can do
> zipFuncs(f1, f2)
res1: String => (Int, Int) = <function>
> zipFuncs(f1, f3)
res2: Option[Int] => (Int, Int) = <function>
> zipFuncs(f2, f3)
res3: Option[Int] with String => (Int, Int) = <function1>
But what I want is for zipFuncs(f2, f3) to not compile at all. Since String is not a subtype of Option[Int], and Option[Int] is not a subtype of String, there's no way to construct an input value for res3.
I did create a typeclass:
// this says type `T` is the most specific type between `T1` and `T2`
sealed trait MostSpecificType[T, T1, T2] extends (T => (T1, T2))
// implementation of `object MostSpecificType` omitted
def zipFuncs[A, A1, A2, B1, B2](f1: A1 => B1, f2: A2 => B2)(
implicit mst: MostSpecificType[A, A1, A2]
): A => (B1, B2) = { a: A =>
val (a1, a2) = mst(a)
f1(a1) -> f2(a2)
}
This accomplishes the goal described above, but with a really annoying problem. IntelliJ will highlight valid combinations as errors, inferring that the "most specific type (A)" is actually Nothing when it is in fact a real value. Here's the actual issue in practice.
The highlighting issue is surely a bug in IntelliJ, and google searching seems to imply that various resets/cache wipes/etc should fix it (it didn't). Regardless of the blame, I'm hoping to find an alternate approach that both satisfies my original requirement, and doesn't confuse IntelliJ.

You can achieve that using generalized type constraints:
def zipFuncs[A1, A2, B1, B2](f1: A1 => B1, f2: A2 => B2)
(implicit ev: A2 <:< A1): A2 => (B1, B2) = {
a => (f1(a), f2(a))
}
val f1 = { a: Any => 123 }
val f2 = { a: String => 123 }
val f3 = { a: Option[Int] => 123 }
zipFuncs(f1, f2) // works
zipFuncs(f1, f3) // works
zipFuncs(f2, f3) // cannot prove that Option[Int] <:< String
However, this requires the second function to use a more specific type in the input parameter than the first one. This is OK unless you also want zipFuncs(f2, f1) to work too. If you do have that requirement, I don't see any other way than doing some implicit type gymnastics similar to the ones you already do.
EDIT: See Eduardo's answer for a neat trick on achieving this.
And yes, I also had a number of situations when IntelliJ sees something as an error when in fact it is not. I know it's tedious but I don't see a way to fix the situation other than reporting an issue and waiting.

If you want this to work only when one of the types is a subtype of the other, then you can do this:
def Zip[A,X,Y](f: A => X, g: A => Y): A => (X,Y) = a => (f(a), g(a))
implicit class ZipOps[A,X](val f: A => X) extends AnyVal {
def zip[A0, Y](g: A0 => Y)(implicit ev: A0 <:< A): A0 => (X,Y) =
Zip({a: A0 => f(a)},g)
def zip[A0 >: A, Y](g: A0 => Y): A => (X,Y) =
Zip(f,g)
}
val f1: Any => Int = { a: Any => 123 }
val f2: String => Int = { a: String => 123 }
val f3: Option[Int] => Int = { a: Option[Int] => 123 }
val x1 = f1 zip f2 // works
val x1swap = f2 zip f1 // works
val x2 = f1 zip f3 // works
val x3 = f2 zip f3 // cannot prove that Option[Int] <:< String
val x3swap = f3 zip f2 // cannot prove that String <:< Option[Int]

Related

Kleisli Inference in scala with cats

Quick question
With
trait SubEnv1
trait SubEnv2
While i understand (I think) why the following code would work:
def logic =
for {
s1 <- Kleisli{(e:SubEnv1) => Option("hello")}
s1 <- Kleisli{(e:Any) => Option("hello2")}
} yield (s1)
//cats.data.Kleisli[Option,SubEnv1,String]
I am confused as to why the following does not work:
def logic =
for {
s1 <- Kleisli{(e:SubEnv1) => Option("hello")}
s1 <- Kleisli{(e:Nothing) => Option("hello2")}
} yield (s1)
//type mismatch;
//found : Nothing => Option[String]
//required: A => Option[String]
In fact i do not understand the error.
The type of flatMap is
def flatMap[C, AA <: A](f: B => Kleisli[F, AA, C])(implicit F: FlatMap[F]): Kleisli[F, AA, C]
So we have AA <: A
The first work because of contravariance i think.
Where AA => Option[C] is expected we can pass can take Any => Option[C]
The second is rather strange given the requirement
AA <: A
and
implicitly[Nothing <:< SubEnv1]
// Nothing <:< SubEnv1 = generalized constraint
So what does //required: A => Option[String] that it requires a type but Nothing ?
I believe you are simply hitting this bug: Nothing does not conform to arbitrary type parameter #9453
Proof:
Kleisli { (e: SubEnv1) => Option("hello") }.flatMap[String, Nothing] { s1 =>
Kleisli[Option, Nothing, String] { (e: Nothing) => Option("boo") }
}
res34: Kleisli[Option, Nothing, String] = Kleisli(cats.data.Kleisli$$$Lambda$2297/0x0000000800c08840#1385e9e3)
(typechecks correctly)

Composing a sequence of functions of variable types in scala

What I'm trying to do is apply a sequence of transformations to a dataset where each function takes the output of the previous step and transforms it for the next step. E.g.
val f1: Function1[Int, Double] = _ / 2d
val f2: Function1[Double, BigDecimal] = x=>BigDecimal(x - 2.1)
val f3: Function1[BigDecimal, String] = _.toString
val chained = (f1 andThen f2 andThen f3)(_)
println(chained(10))
What I want is a function f that takes an input a Seq(f1, f2, ...) and returns the chaining of them, where f1, f2, ...fn do not all have the same input and the same output types T. But they are composable, so for example:
f1: Function1[A,B]
f2: Function1[B,C]
f3: Function1[C,D]
then the chaining function will return a function
f: [A,D].
Thanks,
Z
Two solution proposals here:
A solution that requires a special kind of list that can keep track of all the types in the chain of functions.
An asInstanceOf-heavy solution which works on ordinary lists.
Keeping track of all the types of intermediate results
An ordinary list would lose track of the types of all the intermediate results. Here is a list of functions that keeps track of all those types:
sealed trait Func1List[-In, +Res] {
def ::[I, O <: In](h: I => O): Func1List[I, Res] = ConsFunc1(h, this)
}
object Func1List {
def last[In, Res](f: In => Res): Func1List[In, Res] = LastFunc1(f)
def nil[A]: Func1List[A, A] = LastFunc1(identity)
}
case class LastFunc1[-In, +Res](f: In => Res)
extends Func1List[In, Res]
case class ConsFunc1[-In, Out, +Res](head: In => Out, tail: Func1List[Out, Res])
extends Func1List[In, Res]
Now, for a Func1List, we can define a function that concatenates all the elements:
def andThenAll[A, Z](fs: Func1List[A, Z]): A => Z = fs match {
case LastFunc1(f) => f
case c: ConsFunc1[A, t, Z] => c.head andThen andThenAll[t, Z](c.tail)
}
A little test:
val f1: Function1[Int, Double] = _ / 2d
val f2: Function1[Double, BigDecimal] = x => BigDecimal(x - 2.1)
val f3: Function1[BigDecimal, String] = _.toString
val fs = f1 :: f2 :: Func1List.last(f3)
val f = andThenAll(fs)
println(f(42)) // prints 18.9
Just asInstanceOf all the things
A somewhat less refined, but much shorter solution:
def andThenAll[X, Y](fs: List[_ => _]): X => Y = fs match {
case Nil => (identity[X] _).asInstanceOf[X => Y]
case List(f) => f.asInstanceOf[X => Y]
case hd :: tl => hd match {
case f: Function1[X #unchecked, o] => f andThen andThenAll[o, Y](tl)
}
}
This here also results in 18.9:
println(andThenAll[Int, String](List(f1, f2, f3))(42))

abstract over scala functions without losing types

I'm making a function that takes a lambda, and uses .tupled on it if possible (arity 2+). For the compiler to allow using that, it needs to know if the lambda is really a Function2 (~ Function22). However, pattern-matching for Function2[Any,Any,Any] means I'm left with a (Any, Any) => Any rather than its original types. Not even knowing the arity isn't helping either. I tried matching like case f[A <: Any, B <: Any, C <: Any]: scala.Function2[A, B, C] => f.tupled in an attempt to preserve types, but it doesn't really allow type parameters on a case.
The code:
val add = (a: Int, b: Int) => a + b
add: (Int, Int) => Int = <function2>
val tpl = (fn: Any) => {
fn match {
case f: Function0[Any] => f
case f: Function1[Any,Any] => f
case f: Function2[Any,Any,Any] => f.tupled
// case f: Function3[Any,Any,Any,Any] => f.tupled
// ...
// case _ => { throw new Exception("huh") }
}
}
// actual result:
tpl(add)
res0: Any = <function1>
// desired result is like this one:
scala> add.tupled
res3: ((Int, Int)) => Int = <function1>
Bonus points if I won't need pattern-matching cases for each possible level of arity...
The answer is ugly, as you may have expected. Using a pattern-match on a function as a val won't work. When you start out with Any you've already lost a ton of type information. And the standard library isn't really helping either, as there is no abstraction over arity of functions. That means that we can't even really use reflection to try to grab the type parameters, because we don't even know how many there are. You can figure out what FunctionN you have, but not it's contained types, as they are already lost at this point.
Another possibility is to make tpl a method and overload it for each FunctionN.
def tpl[A](f: Function0[A]): Function0[A] = f
def tpl[A, R](f: Function1[A, R]): Function1[A, R] = f
def tpl[A1, A2, R](f: Function2[A1, A2, R]): Function1[(A1, A2), R] = f.tupled
def tpl[A1, A2, A3, R](f: Function3[A1, A2, A3, R]): Function1[(A1, A2, A3), R] = f.tupled
// ... and so on
scala> val add = (a: Int, b: Int) => a + b
add: (Int, Int) => Int = <function2>
scala> tpl(add)
res0: ((Int, Int)) => Int = <function1>
It's not pretty, but it's at least type-safe. I don't think it would be too difficult to create a macro to generate the overloads 1-22.

How to compose tupled unary functions by combining their input tuples

I've been playing around with shapeless for a bit now.
But, yesterday I got stuck when trying to compose tupled functions.
What I was looking into specifically is composing two unary functions f1: T => R and f2: R => U => S into f: TU => S where T is a TupleN and TU := (t1, ... , tn, u)
import shapeless.ops.tuple._
implicit class Composable[T <: Product, R](val f1: T => R) extends AnyVal{
def compose2[U, S](f2: R => U => S)(implicit p: Prepend[T, Tuple1[U]]): (p.Out => S) = {
// how to provide the two required implicits for Last[p.Out] and Init[p.Out]?
tu => f1.andThen(f2)(tu.init)(tu.last)
}
}
val f1: ((Int, Int)) => Int = x => x._1 * x._2
val f2: ((Int, Int, Int)) => Int = f1.compose2((y: Int) => (x3: Int) => x3 + y).apply _
What I've been struggling with is providing the implicit proof for the tuple operations last and init, so the above code won't compile!
From a logical perspective it feels trivial as result of Prepend, but I couldn't figure out a way. So any idea is welcome :)
Using shapeless's facilities to abstract over arity I got somehow closer:
import shapeless.ops.function.{FnFromProduct, FnToProduct}
import shapeless.{::, HList}
implicit class Composable[F](val f: F) extends AnyVal{
// the new param U is appended upfront
def compose2[I <: HList, R, U, S](f2: R => U => S)
(implicit ftp: FnToProduct.Aux[F, I => R], ffp: FnFromProduct[U :: I => S]): ffp.Out = {
ffp(list => f2.compose(ftp(f))(list.tail)(list.head))
}
}
val f1: (Int, Int) => Int = (x1,x2) => x1 * x2
val f2: (Int, Int, Int) => Int = f1.compose2((y: Int) => (x3: Int) => x3 + y).apply _
This works, but then again I was really looking for compose2 to work on unary tupled Function1s. Also, this results in f: (U, t1, ..., tn) => S rather than f: TU => S with TU := (t1, ... , tn, u).
As Miles says, this would be more convenient with an undo for Prepend, but since the length of the second part is fixed, an approach similar to the one in my other answer isn't too bad at all:
import shapeless.ops.tuple._
implicit class Composable[T <: Product, R](val f1: T => R) extends AnyVal {
def compose2[U, S, TU](f2: R => U => S)(implicit
p: Prepend.Aux[T, Tuple1[U], TU],
i: Init.Aux[TU, T],
l: Last.Aux[TU, U]
): (p.Out => S) =
tu => f1.andThen(f2)(i(tu))(l(tu))
}
And then:
scala> val f1: ((Int, Int)) => Int = x => x._1 * x._2
f1: ((Int, Int)) => Int = <function1>
scala> val f2: ((Int, Int, Int)) => Int =
| f1.compose2((y: Int) => (x3: Int) => x3 + y).apply _
f2: ((Int, Int, Int)) => Int = <function1>
scala> f2((2, 3, 4))
res1: Int = 10
The trick is adding the output of Prepend to the type parameter list for compose2—which will generally be inferred—and then using Prepend.Aux to make sure that it's inferred appropriately. You'll often find in Shapeless that you need to refer to the output type of a type class in other type class instances in the same implicit parameter list in this way, and the Aux type members make doing so a little more convenient.

Lifting methods to function values in Scala

Does the Scala library provide any support for lifting a method of a given type to a function value?
For example, suppose I want to lift String.length. I can write
val f: String => Int = _.length
or
val f = { s: String => s.length }
However, this syntax is not always ideal (particularly in the midst of a larger expression). I think I'm looking for something that will enable expressions like
Lift[String](_.length)
Lift[Option[Int]].lift(_.filter)
and I have in mind something like this:
class Lift[T] {
def apply[R](f: T => R): T => R = f
def lift[A, R](f: (T) => (A) => R): (T, A) => R =
f(_)(_)
def lift[A1, A2, R](f: (T) => (A1, A2) => R): (T, A1, A2) => R =
f(_)(_,_)
// ... etc. ...
}
object Lift {
def apply[T] = new Lift[T]
}
Question 1: Does the standard library (or any library) provide something like this?
Question 2: If not, is it possible to write it in such a way that Option.filter can be lifted as above (rather than as Lift[Option[Int]].lift[Int => Boolean, Option[Int]](_.filter))? Without supplying the type parameters on the lift method I get the following error:
error: missing parameter type for expanded function ((x$1) => x$1.filter)
Lift[Option[Int]].lift(_.filter)
^
Update:
Apparently, the problem I'm running in to has something to do with the overloaded lift method. If I rename the overloads, I can lift Option.filter without all the extra type parameters.
What is the problem with
(_: String).length
(_: Option[Int]).filter _
?
I finally came up with a solution that I'm happy with. This version supports simple syntax and a single entry point to the API, while also providing control over the form of the lifted function (i.e. uncurried, partly curried, or fully curried).
Examples:
I'll use the following class definition in the examples below:
class Foo {
def m1: Int = 1
def m2(i: Int): Int = i
def m3(i: Int, j: Int): Int = i + j
}
The simplest form of lifting is to return the method as a partially applied function, equivalent to invoking ((_: Foo).method _):
scala> lift[Foo](_.m1) // NOTE: trailing _ not required
res0: (Foo) => Int = <function1>
scala> lift[Foo](_.m2 _) // NOTE: trailing _ required
res1: (Foo) => (Int) => Int = <function1>
scala> lift[Foo](_.m3 _)
res2: (Foo) => (Int, Int) => Int = <function1> // NOTE: the result is partly curried
By importing some implicits, one can request curried or uncurried forms:
scala> {
| import CurriedLiftables._
| lift[Foo](_.m3 _)
| }
res3: (Foo) => (Int) => (Int) => Int = <function1>
scala> {
| import UncurriedLiftables._
| lift[Foo](_.m3 _)
| }
res4: (Foo, Int, Int) => Int = <function3>
Implementation:
class Lift[T] {
def apply[R,F](f: T => R)(implicit e: (T => R) Liftable F): F = e.lift(f)
}
object lift {
def apply[T] = new Lift[T]
}
class Liftable[From, To](val lift: From => To)
class DefaultLiftables {
implicit def lift[F]: F Liftable F = new Liftable(identity)
}
object Liftable extends DefaultLiftables
class UncurriedLiftable1 extends DefaultLiftables {
implicit def lift1[T, A, R]: (T => A => R) Liftable ((T, A) => R) =
new Liftable( f => f(_)(_) )
}
class UncurriedLiftable2 extends UncurriedLiftable1 {
implicit def lift2[T, A1, A2, R]: (T => (A1, A2) => R) Liftable ((T, A1, A2) => R) =
new Liftable ( f => f(_)(_,_) )
}
// UncurriedLiftable3, UncurriedLiftable4, ...
object UncurriedLiftables extends UncurriedLiftable2
class CurriedLiftable2 extends DefaultLiftables {
implicit def lift2[T, A1, A2, R]: (T => (A1, A2) => R) Liftable (T => A1 => A2 => R) =
new Liftable( f => (x: T) => (a1: A1) => (a2: A2) => f(x)(a1, a2) )
}
// CurriedLiftable3, CurriedLiftable4, ...
object CurriedLiftables extends CurriedLiftable2
My previous solution required a separate lift method for each arity:
import Lift._
val f1 = lift0[String](_.length)
val f2 = lift1[Option[Int]](_.filter)
val f3 = lift2[Either[String, Int]](_.fold)
Implementation:
class Lift0[T] {
def apply[R](f: T => R): T => R = f
}
class Lift1[T] {
def apply[A, R](f: (T) => (A) => R): (T, A) => R =
f(_)(_)
}
class Lift2[T] {
def apply[A1, A2, R](f: (T) => (A1, A2) => R): (T, A1, A2) => R =
f(_)(_,_)
}
// ... etc. ...
object Lift {
def lift0[T] = new Lift0[T]
def lift1[T] = new Lift1[T]
def lift2[T] = new Lift2[T]
// ... etc. ...
}
Passing in filter as partially applied method seems to do the job:
scala> class Lift[T] {
| def apply[R](f: T => R): T => R = f
| }
defined class Lift
scala> object Lift {
| def apply[T] = new Lift[T]
| }
defined module Lift
scala> val ls = Lift[String](_.length)
ls: (String) => Int = <function1>
scala> val los = Lift[Option[Int]](_.filter _)
los: (Option[Int]) => ((Int) => Boolean) => Option[Int] = <function1>