I understand it would be difficult to change now without breaking existing code, but I'm wondering why it was done that way in the first place.
Why not just:
sealed trait Either[+A, +B]
case class Left[A](x: A) extends Either[A, Nothing]
case class Right[B](x: B) extends Either[Nothing, B]
Is there some drawback here that I'm failing to see...?
Not sure how relevant this answer really is to Scala, but it certainly is in Haskell which is evidently where Scala's Either was borrowed from and so that's probably the best historical reason for why Scala did it this way.
Either is the canonical coproduct, i.e. for any types A and B you have
The type EitherA,B ≈ A ⊕ B
Two coprojections LeftA,B : A -> A⊕B and RightA,B : B -> A⊕B
such that for any type Y and any functions fA : A -> Y and fB : B -> Y, there exists exactly one function f : A⊕B -> Y with the property that fA = f ∘ LeftA,B and fB = f ∘ RightA,B.
To formulate this mathematically, it is quite helpful to have the information which particular Left you're working with explicit, because else the domains of the morphisms would be all unclear. In Scala this may be unnecessary because of implicit covariant conversion, but not in maths and not in Haskell.
In Haskell it isn't really an issue at all, because type inference will automatically do what's needed:
GHCi, version 8.6.5: http://www.haskell.org/ghc/ :? for help
Loaded GHCi configuration from /tmp/haskell-stack-ghci/2a3bbd58/ghci-script
Prelude> let right2 = Right 2
Prelude> let left42 = Left 42.0
Prelude> (+) <$> right2 <*> left42
Left 42.0
Unlike, apparently, in Scala, Haskell just leaves the unspecified second argument of left42 as a type variable (unless the monomorphism restriction is enabled), so you can later use it in any context requiring some Either Double R for any type R. Of course it's possible to make that explicit too
right2 :: Either a Int
right2 = Right 2
left42 :: Either Double a
left42 = Left 42
main :: IO ()
main = print $ (+) <$> right2 <*> left42
which surely is possible in Scala just as well.
There's no meaningful drawback that I've found to your scheme. For the last eight years or so I've used my own variant of Either which is exactly as you describe under another name (Ok[+Y, +N] with Yes[+Y] and No[+N] as the alternatives). (Historical note: I started when Either was not right-biased, and wanted something that was; but then I kept using my version because it was more convenient to have only half the types.)
The only case I've ever found where it matters is when you pattern match out one branch and no longer have access to the type information of the other branch.
def foo[A, B: Typeclass](e: Either[A, B]) =
implicitly[Typeclass[B]].whatever()
// This works
myEither match {
case l: Left[L, R] => foo(l)
case r: Right[L, R] => foo(r)
}
def bar[N, Y: Typeclass](o: Ok[N, Y]) =
implicitly[Typeclass[Y]].whatever()
// This doesn't work
myOk match {
case y: Yes[Y] => bar(y) // This is fine
case n: No[N] => bar(n) // Y == Nothing!
}
However, I never do this. I could just use o to get the right type. So it doesn't matter! Everything else is easier (like pattern matching and changing one case and not the other...you don't need case Left(l) => Left(l) which rebuilds the Left for no reason except to switch the type of the uninhabited branch).
There are other cases (e.g. setting types in advance) that seem like they should be important, but in practice are almost impossible to make matter (e.g. because covariance will find the common supertype anyway, so what you set doesn't constrain anything).
So I think the decision was made before there was enough experience with the two ways to do it, and the wrong choice was made. (It's not a very wrong choice; Either is still decent.)
Related
Given the following code
sealed trait Fruit
case class Apple(color: String) extends Fruit
case class Orange(color: String) extends Fruit
def getAppleColor(apple: Apple) = apple.color
def getOrangeColor(orange: Orange) = orange.color
val myMap: Map[String, Fruit] = Map(
"myApple" -> Apple("red"),
"myOrange" -> Orange("orange"),
)
val myMapOfFunctions: Map[String, Apple with Orange => String] = Map(
"myAppleColorFun" -> getAppleColor,
"myOrangeColorFun" -> getOrangeColor,
)
Why myMapOfFunctions is not a Map[String, Fruit => String] similarly to myMap? I guess because it is about functions but I'd like better to understand why. Thanks!
I am just trying to understand why the compiler says that the type of the map is Apple with Orange and not Fruit
Okay, the good thing is that this is "easy" to explain.
The bad thing is that it may not be as easy to understand.
Let's take a couple of steps back, and let's simplify the code a little by using an if instead of a collection.
When you do something like this:
val foo = if (bar) x else y
The compiler has to infer the type of foo, for doing that it will first get / infer the types of x and y; let's call those X & Y respectively and then compute the LUB (least upper bound) between both, resulting in a new type Z which will be the type assigned to foo
This makes sense because X <: Z and Y <: Z and thus Liskov is respected.
Quick note, if X and Y are the same types A then the LUB is just A
Another quick one, if X is a subtype of Y then the LUB is simply Y
Let's see those applied to simple types:
val fruit = if (true) Apple(color = "red") else Orange(color = "green")
Here, one branch has the type Apple and the other Orange and the LUB between both is Fruit.
Everything has been straightforward until this point.
Now, let's spice the things up a little:
val optApple: Option[Apple] = Apple(color = "red")
val optOrange: Option[Orange] = Orange(color = "green")
val optFruit = if (true) optApple else optOrange
Here one branch is Option[Apple] and the other is Option[Orange], we know that the result will be Option[Fruit], but why?
Well, because Option was defined to be covariant on its type parameter thus Option[Fruit] is a supertype of both branches; and mainly the LUB.
Okay, but what happens with functions?
// Implementations do not matter.
val appleFunction : Apple => Apple = ???
val orangeFunction: Orange => Orange = ???
val fruitFunction = if (true) appleFunction else orangeFunction
In this case, the LUB will be (Apple with Orange) => Fruit ... but why?
Well, the return is easy since functions are also covariant on their returns which means the LUB will again be Fruit
But, why is the input not like that? Well, because functions are contravariant on their inputs, thus for a function f to be a subtype of another function g, the type of the input of f must be a super type of the input of g; i.e. it goes in the opposite order which is why it is called contravariance.
So that explains why the compiler inferred that type.
But, you may be wondering what this variance business is, why it matters, and why there is one that seems counterintuitive. But that is outside of the scope of this question & answer.
Nevertheless, I can share a couple of resources that may be useful:
https://www.youtube.com/watch?v=aUmj7jnXet4&t=53s
https://www.youtube.com/watch?v=b1ftkK1zhxI&t=3s
Covariant type A accepted in contravariant position of function argument A => B
I was recently reading Category Theory for Programmers and in one of the challenges, Bartosz proposed to write a function called memoize which takes a function as an argument and returns the same one with the difference that, the first time this new function is called, it stores the result of the argument and then returns this result each time it is called again.
def memoize[A, B](f: A => B): A => B = ???
The problem is, I can't think of any way to implement this function without resorting to mutability. Moreover, the implementations I have seen uses mutable data structures to accomplish the task.
My question is, is there a purely functional way of accomplishing this? Maybe without mutability or by using some functional trick?
Thanks for reading my question and for any future help. Have a nice day!
is there a purely functional way of accomplishing this?
No. Not in the narrowest sense of pure functions and using the given signature.
TLDR: Use mutable collections, it's okay!
Impurity of g
val g = memoize(f)
// state 1
g(a)
// state 2
What would you expect to happen for the call g(a)?
If g(a) memoizes the result, an (internal) state has to change, so the state is different after the call g(a) than before.
As this could be observed from the outside, the call to g has side effects, which makes your program impure.
From the Book you referenced, 2.5 Pure and Dirty Functions:
[...] functions that
always produce the same result given the same input and
have no side effects
are called pure functions.
Is this really a side effect?
Normally, at least in Scala, internal state changes are not considered side effects.
See the definition in the Scala Book
A pure function is a function that depends only on its declared inputs and its internal algorithm to produce its output. It does not read any other values from “the outside world” — the world outside of the function’s scope — and it does not modify any values in the outside world.
The following examples of lazy computations both change their internal states, but are normally still considered purely functional as they always yield the same result and have no side effects apart from internal state:
lazy val x = 1
// state 1: x is not computed
x
// state 2: x is 1
val ll = LazyList.continually(0)
// state 1: ll = LazyList(<not computed>)
ll(0)
// state 2: ll = LazyList(0, <not computed>)
In your case, the equivalent would be something using a private, mutable Map (as the implementations you may have found) like:
def memoize[A, B](f: A => B): A => B = {
val cache = mutable.Map.empty[A, B]
(a: A) => cache.getOrElseUpdate(a, f(a))
}
Note that the cache is not public.
So, for a pure function f and without looking at memory consumption, timings, reflection or other evil stuff, you won't be able to tell from the outside whether f was called twice or g cached the result of f.
In this sense, side effects are only things like printing output, writing to public variables, files etc.
Thus, this implementation is considered pure (at least in Scala).
Avoiding mutable collections
If you really want to avoid var and mutable collections, you need to change the signature of your memoize method.
This is, because if g cannot change internal state, it won't be able to memoize anything new after it was initialized.
An (inefficient but simple) example would be
def memoizeOneValue[A, B](f: A => B)(a: A): (B, A => B) = {
val b = f(a)
val g = (v: A) => if (v == a) b else f(v)
(b, g)
}
val (b1, g) = memoizeOneValue(f, a1)
val (b2, h) = memoizeOneValue(g, a2)
// ...
The result of f(a1) would be cached in g, but nothing else. Then, you could chain this and always get a new function.
If you are interested in a faster version of that, see #esse's answer, which does the same, but more efficient (using an immutable map, so O(log(n)) instead of the linked list of functions above, O(n)).
Let's try(Note: I have change the return type of memoize to store the cached data):
import scala.language.existentials
type M[A, B] = A => T forSome { type T <: (B, A => T) }
def memoize[A, B](f: A => B): M[A, B] = {
import scala.collection.immutable
def withCache(cache: immutable.Map[A, B]): M[A, B] = a => cache.get(a) match {
case Some(b) => (b, withCache(cache))
case None =>
val b = f(a)
(b, withCache(cache + (a -> b)))
}
withCache(immutable.Map.empty)
}
def f(i: Int): Int = { print(s"Invoke f($i)"); i }
val (i0, m0) = memoize(f)(1) // f only invoked at first time
val (i1, m1) = m0(1)
val (i2, m2) = m1(1)
Yes there is pure functional ways to implement polymorphic function memoization. The topic is surprisingly deep and even summons the Yoneda Lemma, which is likely what Bartosz had in mind with this exercise.
The blog post Memoization in Haskell gives a nice introduction by simplifying the problem a bit: instead of looking at arbitrary functions it restricts the problem to functions from the integers.
The following memoize function takes a function of type Int -> a and
returns a memoized version of the same function. The trick is to turn
a function into a value because, in Haskell, functions are not
memoized but values are. memoize converts a function f :: Int -> a
into an infinite list [a] whose nth element contains the value of f n.
Thus each element of the list is evaluated when it is first accessed
and cached automatically by the Haskell runtime thanks to lazy
evaluation.
memoize :: (Int -> a) -> (Int -> a)
memoize f = (map f [0 ..] !!)
Apparently the approach can be generalised to function of arbitrary domains. The trick is to come up with a way to use the type of the domain as an index into a lazy data structure used for "storing" previous values. And this is where the Yoneda Lemma comes in and my own understanding of the topic becomes flimsy.
How to implement the function normalize(type: Type): Type such that:
A =:= B if and only if normalize(A) == normalize(B) and normalize(A).hashCode == normalize(B).hashCode.
In other words, normalize must return equal results for all equivalent Type instances; and not equal nor equivalent results for all pair of non equivalent inputs.
There is a deprecated method called normalize in the TypeApi, but it does not the same.
In my particular case I only need to normalize types that represent a class or a trait (tpe.typeSymbol.isClass == true).
Edit 1: The fist comment suggests that such a function might not be possible to implement in general. But perhaps it is possible if we add another constraint:
B is obtained by navigating from A.
In the next example fooType would be A, and nextAppliedType would be B:
import scala.reflect.runtime.universe._
sealed trait Foo[V]
case class FooImpl[V](next: Foo[V]) extends Foo[V]
scala> val fooType = typeOf[Foo[Int]]
val fooType: reflect.runtime.universe.Type = Foo[Int]
scala> val nextType = fooType.typeSymbol.asClass.knownDirectSubclasses.iterator.next().asClass.primaryConstructor.typeSignature.paramLists(0)(0).typeSignature
val nextType: reflect.runtime.universe.Type = Foo[V]
scala> val nextAppliedType = appliedType(nextType.typeConstructor, fooType.typeArgs)
val nextAppliedType: reflect.runtime.universe.Type = Foo[Int]
scala> println(fooType =:= nextAppliedType)
true
scala> println(fooType == nextAppliedType)
false
Inspecting the Type instances with showRaw shows why they are not equal (at least when Foo and FooImpl are members of an object, in this example, the jsfacile.test.RecursionTest object):
scala> showRaw(fooType)
val res2: String = TypeRef(SingleType(SingleType(SingleType(ThisType(<root>), jsfacile), jsfacile.test), jsfacile.test.RecursionTest), jsfacile.test.RecursionTest.Foo, List(TypeRef(ThisType(scala), scala.Int, List())))
scala> showRaw(nextAppliedType)
val res3: String = TypeRef(ThisType(jsfacile.test.RecursionTest), jsfacile.test.RecursionTest.Foo, List(TypeRef(ThisType(scala), scala.Int, List())))
The reason I need this is difficult to explain. Let's try:
I am developing this JSON library which works fine except when there is a recursive type reference. For example:
sealed trait Foo[V]
case class FooImpl[V](next: Foo[V]) extends Foo[V]
That happens because the parser/appender it uses to parse and format are type classes that are materialized by an implicit macro. And when an implicit parameter is recursive the compiler complains with a divergence error.
I tried to solve that using by-name implicit parameter but it not only didn't solve the recursion problem, but also makes many non recursive algebraic data type to fail.
So, now I am trying to solve this problem by storing the resolved materializations in a map, which also would improve the compilation speed. And that map key is of type Type. So I need to normalize the Type instances, not only to be usable as key of a map, but also to equalize the values generated from them.
If I understood you well, any equivalence class would be fine. There is no preference.
I suspect you didn't. At least "any equivalence class would be fine", "There is no preference" do not sound good. I'll try to elaborate.
In math there is such construction as factorization. If you have a set A and equivalence relation ~ on this set (relation means that for any pair of elements from A we know whether they are related a1 ~ a2 or not, equivalence means symmetricity a1 ~ a2 => a2 ~ a1, reflexivity a ~ a, transitivity a1 ~ a2, a2 ~ a3 => a1 ~ a3) then you can consider the factor-set A/~ whose elements are all equivalence classes A/~ = { [a] | a ∈ A} (the equivalence class
[a] = {b ∈ A | b ~ a}
of an element a is a set consisting of all elements equivalent (i.e. ~-related) to a).
The axiom of choice says that there is a map (function) from A/~ to A i.e. we can select a representative in every equivalence class and in such way form a subset of A (this is true if we accept the axiom of choice, if we don't then it's not clear whether we get a set in such way). But even if we accept the axiom of choice and therefore there is a function A/~ -> A this doesn't mean we can construct such function.
Simple example. Let's consider the set of all real numbers R and the following equivalence relation: two real numbers are equivalent r1 ~ r2 if their difference is a rational number
r2 - r1 = p/q ∈ Q
(p, q≠0 are arbitrary integers). This is an equivalence relation. So it's known that there is a function selecting a single real number from any equivalence class but how to define this function explicitly for a specific input? For example what is the output of this function for the input being the equivalence class of 0 or 1 or π or e or √2 or log 2...?
Similarly, =:= is an equivalence relation on types, so it's known that there is a function normalize (maybe there are even many such functions) selecting a representative in every equivalence class but how to prefer a specific one (how to define or construct the output explicitly for any specific input)?
Regarding your struggle against implicit divergence. It's not necessary that you've selected the best possible approach. Sounds like you're doing some compiler work manually. How do other json libraries solve the issue? For example Circe? Besides by-name implicits => there is also shapeless.Lazy / shapeless.Strict (not equivalent to by-name implicits). If you have specific question about deriving type classes, overcoming implicit divergence maybe you should start a different question about that?
Regarding your approach with HashMap with Type keys. I'm still reminding that we're not supposed to rely on == for Types, correct comparison is =:=. So you should build your HashMap using =:= rather than ==. Search at SO for something like: hashmap custom equals.
Actually I guess your normalize sounds like you want some caching. You should have a type cache. Then if you asked to calculate normalize(typ) you should check whether in the cache there is already a t such that t =:= typ. If so you should return t, otherwise you should add typ to the cache and return typ.
This satisfies your requirement: A =:= B if and only if normalize(A) == normalize(B) (normalize(A).hashCode == normalize(B).hashCode should follow from normalize(A) == normalize(B)).
Regarding transformation of fooType into nextAppliedType try
def normalize(typ: Type): Type = typ match {
case TypeRef(pre, sym, args) =>
internal.typeRef(internal.thisType(pre.typeSymbol), sym, args)
}
Then normalize(fooType) == nextAppliedType should be true.
I was trying to do some stuff last night around accepting and calling a generic function (i.e. the type is known at the call site, but potentially varies across call sites, so the definition should be generic across arities).
For example, suppose I have a function f: (A, B, C, ...) => Z. (There are actually many such fs, which I do not know in advance, and so I cannot fix the types nor count of A, B, C, ..., Z.)
I'm trying to achieve the following.
How do I call f generically with an instance of (A, B, C, ...)? If the signature of f were known in advance, then I could do something involving Function.tupled f or equivalent.
How do I define another function or method (for example, some object's apply method) with the same signature as f? That is to say, how do I define a g for which g(a, b, c, ...) type checks if and only if f(a, b, c, ...) type checks? I was looking into Shapeless's HList for this. From what I can tell so far, HList at least solves the "representing an arbitrary arity args list" issue, and also, Shapeless would solve the conversion to and from tuple issue. However, I'm still not sure I understand how this would fit in with a function of generic arity, if at all.
How do I define another function or method with a related type signature to f? The biggest example that comes to mind now is some h: (A, B, C, ...) => SomeErrorThing[Z] \/ Z.
I remember watching a conference presentation on Shapeless some time ago. While the presenter did not explicitly demonstrate these things, what they did demonstrate (various techniques around abstracting/genericizing tuples vs HLists) would lead me to believe that similar things as the above are possible with the same tools.
Thanks in advance!
Yes, Shapeless can absolutely help you here. Suppose for example that we want to take a function of arbitrary arity and turn it into a function of the same arity but with the return type wrapped in Option (I think this will hit all three points of your question).
To keep things simple I'll just say the Option is always Some. This takes a pretty dense four lines:
import shapeless._, ops.function._
def wrap[F, I <: HList, O](f: F)(implicit
ftp: FnToProduct.Aux[F, I => O],
ffp: FnFromProduct[I => Option[O]]
): ffp.Out = ffp(i => Some(ftp(f)(i)))
We can show that it works:
scala> wrap((i: Int) => i + 1)
res0: Int => Option[Int] = <function1>
scala> wrap((i: Int, s: String, t: String) => (s * i) + t)
res1: (Int, String, String) => Option[String] = <function3>
scala> res1(3, "foo", "bar")
res2: Option[String] = Some(foofoofoobar)
Note the appropriate static return types. Now for how it works:
The FnToProduct type class provides evidence that some type F is a FunctionN (for some N) that can be converted into a function from some HList to the original output type. The HList function (a Function1, to be precise) is the Out type member of the instance, or the second type parameter of the FnToProduct.Aux helper.
FnFromProduct does the reverse—it's evidence that some F is a Function1 from an HList to some output type that can be converted into a function of some arity to that output type.
In our wrap method, we use FnToProduct.Aux to constrain the Out of the FnToProduct instance for F in such a way that we can refer to the HList parameter list and the O result type in the type of our FnFromProduct instance. The implementation is then pretty straightforward—we just apply the instances in the appropriate places.
This may all seem very complicated, but once you've worked with this kind of generic programming in Scala for a while it becomes more or less intuitive, and we'd of course be happy to answer more specific questions about your use case.
I have two functions (not these have been edited since the original -- some of the answers below are responding to the original ones which returned a sequence of ()):
def foo1[A](ls: Iterable[A]) : Iterator[A] =
for (List(a, b) <- ls sliding 2) yield a
def foo2[A](ls: Iterable[A]) : Iterator[A] =
for (a::b::Nil <- ls sliding 2) yield a
which I naively thought were the same. But Scala gives this waning only for the first one:
warning: non variable type-argument A in type pattern List[A]
is unchecked since it is eliminated by erasure
I think I understand why it gives that error for the first one: Scala thinks that I'm trying to use the type as a condition on the pattern, ie a match against List[B](_, _) should fail if B does not inherit from A, except that this can't happen because the type is erased in both cases.
So two questions:
1) Why does the second one not give the same warning?
2) Is it possible to convince Scala that the type is actually known at compile time, and thus can't possibly fail to match?
edit: I think this answers my first question. But I'm still curious about the second one.
edit: agilesteel mentioned in a comment that
for (List(a, b) <- List(1,2,3,4) sliding 2) yield ()
produces no warning. How is that different from foo1 (shouldn't the [Int] parameter be erased just the same as the [A] parameter is)?
I'm not sure what is happening here, but the static type of Iterable[A].sliding is Iterator[Iterable[A]], not Iterator[List[A]] which would be the static type of List[A].sliding.
You can try receiving Seq instead of Iterable, and that work too. EDIT Contrary to what I previously claimed, both Iterable and Seq are co-variant, so I don't know what's different. END EDIT The definition of sliding is pretty weird too:
def sliding [B >: A] (size: Int): Iterator[Iterable[A]]
See how it requires a B, superclass of A, that never gets used? Contrast that with an Iterator.sliding, for which there's no problem:
def sliding [B >: A] (size: Int, step: Int = 1): GroupedIterator[B]
Anyway, on to the second case:
for (a::b::Nil <- ls sliding 2) yield a
Here you are decomposing the list twice, and for each decomposition the type of head is checked against A. Since the type of head is not erased, you don't have a problem. This is also mostly a guess.
Finally, if you turn ls into a List, you won't have a problem. Short of that, I don't think there's anything you can do. Otherwise, you can also write this:
def foo1[A](ls: Iterable[A]) : Iterator[A] =
for (Seq(a, b) <- ls.iterator sliding 2) yield a
1) The second one does not produce a warning probably because you are constructing the list (or the pattern) by prepending elements to the Nil object, which extends List parameterising it with Nothing. And since everything is Nothing, there is nothing to be worried about ;) But I'm not sure, really guessing here.
2) Why don't you just use:
def foo[A](ls: Iterable[A]) =
for (list <- ls sliding 2) yield ()