I had spent weeks on trying to understand the idea behind "lifting" in scala.
Originally, it was from the example related to Chapter 4 of book "Functional Programming in Scala"
Then I found below topic "How map work on Options in Scala?"
The selected answer specify that:
def map[B](f: A => B): Option[B] = this match (Let's considered this as (*) )
So, from above code, I assume that function "map" is derived from function match. Hence, the mechanism behind "map"
is a kind of pattern matching to provide a case selection between Some, and None
Then, I created below examples by using function map for Seq, Option, and Map (Let's considered below examples as (**) )
Example 1: map for Seq
val xs = Seq(1, 2, 3)
xs.map(println)
Example 2: map for Option
val a:Option[Int] = Some(5)
a.map(println)
val b:Option[Int] = None
b.map(println)
Example 3: map for Map
val capitals = Map("France" -> "Paris", "Japan" -> "Tokyo")
capitals.map(println)
From (*) and (**), I could not know whether "map" is a pattern matching or an iteration, or both.
Thank you for helping me to understand this.
#Jwvh provided a more programming based answer but I want to dig a little bit deeper.
I certainly appreciate you trying to understand how things work in Scala, however if you really want to dig that deep, I am afraid you will need to obtain some basic knowledge of Category Theory since there is no "idea behind lifting in scala" but just the "idea behind lifting"
This is also why functions like "map" can be very confusing. Inherently, programmers are taught map etc. as operations on collections, where as they are actually operations that come with Functors and Natural Transformations (this is normally referred to as fmap in Category Theory and also Haskell).
Before I move on, the short answer is it is a pattern matching in the examples you gave and in some of them it is both. Map is defined specifically to the case, the only condition is that it maintains functoriality
Attention: I will not be defining every single term below, since I would need to write a book to build up to some of the following definitions, interested readers are welcome to research them on their own. You should be able to get some basic understanding by following the types
Let's consider these as Functors, the definition will be something along the lines of this:
In (very very) short, we consider types as objects in the category of our language. The functions between these types (type constructors) are morphisms between types in this category. The set of these transformations are called Endo-Functors (take us from the category of Scala and drop us back in the category of Scala). Functors have to have a polymorphic (which actually has a whole different (extra) definition in category theory) map function, that will take some object A, through some type constructor turn it into object B.
implicit val option: Functor[Option] = new Functor[Option] {
override def map[A,B](optA: Option[A])(f: (A) => B): Option[B] = optA match{
case Some(a) => Some(f(a))
case _ => None
}
}
implicit val seq: Functor[Seq[_]] = new Functor[Seq[_]] {
override def map[A,B](sA: Seq[A])(f: (A) => B): Seq[B] = sA match{
case a :: tail => Seq(f(a), map(tail)(f))
case Nil => Nil
}
}
As you can see in the second case, there is a little bit of both (more of a recursion than iteration but still).
Now before the internet blows up on me, I will say you cant pattern match on Seq in Scala. It works here because the default Seq is also a List. I just provided this example because it is simpler to understand. The underlying definition something along the lines of that.
Now hold on a second. If you look at these types, you see that they also have flatMap defined on them. This means they are something more special than plain Functors. They are Monads. So beyond satisfying functoriality, they obey the monadic laws.
Turns out Monad has a different kind of meaning in the core scala, more on that here: What exactly makes Option a monad in Scala?
But again very very short, this means that we are now in a category where the endofunctors from our previous category are the objects and the mappings between them are morphisms (natural transformations), this is slightly more accurate because if you think about it when you take a type and transform it, you take (carry over) all of it's internal type constructors (2-cell or internal morphisms) with it, you do not only take this sole idea of a type without it's functions.
implicit val optionMonad: Monad[Option] = new Monad[Option] {
override def flatMap[A, B](optA: Option[A])(f: (A) => Option[B]): Option[B] = optA match{
case Some(a) => f(a)
case _ => None
}
def pure[A](a: A): Option[A] = Some(a)
//You can define map using pure and flatmap
}
implicit val seqMonad: Monad[Seq[_]] = new Monad[Seq[_]] {
override def flatMap[A, B](sA: Seq[A])(f: (A) => Seq[B]): Seq[B] = sA match{
case x :: xs => f(a).append(flatMap(tail)(f))
case Nil => Nil
}
override def pure[A](a: A): Seq[A] = Seq(a)
//Same warning as above, also you can implement map with the above 2 funcs
}
One thing you can always count on is map being having pattern match (or some if statement). Why?
In order to satisfy the identity laws, we need to have some sort of a "base case", a unit object and in many cases (such as Lists) those types are gonna be what we call either a product or coproduct.
Hopefully, this did not confuse you further. I wish I could get into every detail of this but it would simply take pages, I highly recommend getting into categories to fully understand where these come from.
From the ScalaDocs page we can see that the type profile for the Standard Library map() method is a little different.
def map[B](f: (A) => B): Seq[B]
So the Standard Library map() is the means to transition from a collection of elements of type A to the same collection but the elements are type B. (A and B might be the same type. They aren't required to be different.)
So, yes, it iterates through the collection applying function f() to each element A to create each new element B. And function f() might use pattern matching in its code, but it doesn't have to.
Now consider a.map(println). Every element of a is sent to println which returns Unit. So if a is List[Int] then the result of a.map(println) is List[Unit], which isn't terribly useful.
When all we want is the side effect of sending information to StdOut then we use foreach() which doesn't create a new collection: a.foreach(println)
Function map for Option isn't about pattern matching. The match/case used in your referred link is just one of the many ways to define the function. It could've been defined using if/else. In fact, that's how it's defined in Scala 2.13 source of class Option:
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable {
self =>
...
final def map[B](f: A => B): Option[B] =
if (isEmpty) None else Some(f(this.get))
...
}
If you view Option like a "collection" of either one element (Some(x)) or no elements (None), it might be easier to see the resemblance of how map transforms an Option versus, say, a List:
val f: Int => Int = _ + 1
List(42).map(f)
// res1: List[Int] = List(43)
List.empty[Int].map(f)
// res2: List[Int] = List()
Some(42).map(f)
// res3: Option[Int] = Some(43)
None.map(f)
// res4: Option[Int] = None
Related
I have the following data structure:
class MyDaSt[A]{
def map[B: ClassTag](f: A => B) = //...
}
I'd like to implement a Functor instance for to be able to use ad-hoc polymorphism. The obvious attempt would be as follows:
implicit val mydastFunctor: Functor[MyDaSt] = new Functor[MyDaSt] {
override def map[A, B](fa: MyDaSt[A])(f: A => B): MyDaSt[B] = fa.map(f) //compile error
}
It obviously does not compile because we did not provide an implicit ClassTag[B]. But would it be possible to use map only with functions f: A => B such that there is ClassTag[B]. Otherwise compile error. I mean something like that:
def someFun[A, B, C[_]: Functor](cc: C[A], f: A => B) = cc.map(f)
val f: Int => Int = //...
val v: MyDaSt[Int] = //...
someFunc(v, f) //fine, ClassTag[Int] exists and in scope
I cannot change its implementation in anyway, but I can create wrappers (which does not look helpful through) or inheritance. I'm free to use shapeless of any version.
I currently think that shapeless is a way to go in such case...
I'll expand on what comments touched:
Functor
cats.Functor describes an endofunctor in a category of Scala types - that is, you should be able to map with a function A => B where A and B must support any Scala types.
What you have is a mathematical functor, but in a different, smaller category of types that have a ClassTag. These general functors are somewhat uncommon - I think for stdlib types, only SortedSet can be a functor on a category of ordered things - so it's fairly unexplored territory in Scala FP right now, only rumored somewhat in Scalaz 8.
Cats does not have any tools for abstracting over such things, so you won't get any utility methods and ecosystem support. You can use that answer linked by #DmytroMitin if you want to roll your own
Coyoneda
Coyoneda can make an endofunctor on Scala types from any type constructor F[_]. The idea is simple:
have some initial value F[Initial]
have a function Initial => A
to map with A => B, you don't touch initial value, but simply compose the functions to get Initial => B
You can lift any F[A] into cats.free.Coyoneda[F, A]. The question is how to get F[A] out.
If F is a cats.Functor, then it is totally natural that you can use it's native map, and, in fact, there will not be any difference in result with using Coyoneda and using F directly, due to functor law (x.map(f).map(g) <-> x.map(f andThen g)).
In your case, it's not. But you can tear cats.free.Coyoneda apart and delegate to your own map:
def coy[A](fa: MyDaSt[A]): Coyoneda[MyDaSt, A] = Coyoneda.lift(fa)
def unCoy[A: ClassTag](fa: Coyoneda[MyDaSt, A]): MyDaSt[A] =
fa.fi.map(fa.k) // fi is initial value, k is the composed function
Which will let you use functions expecting cats.Functor:
def generic[F[_]: Functor, A: Show](fa: F[A]): F[String] = fa.map(_.show)
unCoy(generic(coy(v))) // ok, though cumbersome and needs -Ypartial-unification on scala prior to 2.13
(runnable example on scastie)
An obvious limitation is that you need to have a ClassTag[A] in any spot you want to call unCo - even if you did not need it to create an instance of MyDaSt[A] in the first place.
The less obvious one is that you don't automatically have that guarantee about having no behavioral differences. Whether it's okay or not depends on what your map does - e.g. if it's just allocating some Arrays, it shouldn't cause issues.
I have this two functions
def pattern(s: String): Option[Pattern] =
try {
Some(Pattern.compile(s))
} catch {
case e: PatternSyntaxException => None
}
and
def mkMatcher(pat: String): Option[String => Boolean] =
pattern(pat) map (p => (s: String) => p.matcher(s).matches)
Map is the higher-order function that applies a given function to each element of a list.
Now I am not getting that how map is working here as per above statement.
Map is the higher-order function that applies a given function to each element of a list.
This is an uncommonly restrictive definition of map.
At any rate, it works because it was defined by someone who did not hold to that.
For example, that someone wrote something akin to
sealed trait Option[+A] {
def map[B](f: A => B): Option[B] = this match {
case Some(value) => Some(f(value))
case None => None
}
}
as part of the standard library. This makes map applicable to Option[A]
It was defined because it makes sense to map many kinds of data structures not just lists.
Mapping is a transformation applied to the elements held by the data structure.
It applies a function to each element.
Option[A] can be thought of as a trivial sequence. It either has zero or one elements. To map it means to apply the function on its element if it has one.
Now it may not make much sense to use this facility all of the time, but there are cases where it is useful.
For example, it is one of a few distinct methods that, when present enable enable For Expressions to operate on a type. Option[A] can be used in for expressions which can be convenient.
For example
val option: Option[Int] = Some(2)
val squared: Option[Int] = for {
n <- option
if n % 2 == 0
} yield n * n
Interestingly, this implies that filter is also defined on Option[A].
If you just have a simple value it may well be clearer to use a less general construct.
Map is working the same way that it does with other collections types like List and Vector. It applies your function to the contents of the collection, potentially changing the type but keeping the collection type the same.
In many cases you can treat an Option just like a collection with either 0 or 1 elements. You can do a lot of the same operations on Option that you can on other collections.
You can modify the value
var opt = Option(1)
opt.map(_ + 3)
opt.map(_ * math.Pi)
opt.filter(_ == 1)
opt.collect({case i if i > 0 => i.toString })
opt.foreach(println)
and you can test the value
opt.contains(3)
opt.forall(_ > 0)
opt.exists(_ > 0)
opt.isEmpty
Using these methods you rarely need to use a match statement to unpick an Option.
I'm trying to define "immutable setter traits", and generic functions for those.
I have a working implementation, but i'm bit disturbed about the "unchecked" warnings from the pattern matching. I'm not really sure what i can do about it.
type Point = (Double, Double)
trait Sizable[A] {
this: A =>
def size: Point
/* immutable object value setter, returns a new of the same object*/
def size(point: Point): A with Sizable[A]
}
def partitionToSizable[T](elements: List[T]):
(List[T], List[T with Sizable[T]]) =
elements.foldLeft((List[T](), List[T with Sizable[T]]()))((p, c) =>
c match {
case a: T with Sizable[T] => (p._1, p._2 ++ List(a))
case a => (p._1 ++ List(a), p._2)
})
The code above demonstrates the problem.
I'm not even sure how big of an issue is that T being unchecked, since all the elements in the list will have a type of T, and the point of the pattern matching is not to determine if it's type is T since we already know that.
In theory Sizable will always have type of T because of the signature of it's enclosing function.
If it's there is no other solution i'd at least like to suppress the warning. #unchecked annotations does not seem to suppress the warning.
If i modify case a: T with Sizable[T] to case a: Sizable[_] it will not compile, since the result type will obviously not confirm to T.
ClassTags or TypeTags might solve the warning, but i suspect they are not necessary really. (also that might have a performance overhead and TypeTags don't work with Scala.js)
I think just case a: Sizable[T #unchecked] => a.size((15,20)) should work.
The main problem here is in partitionToSizable. It's taking a List[T] and then trying to partition it into a List[T] and a List[T with Sizable[T]]. You are taking a list of a single type and then saying that one type T is actually two different types T and T with Sizable[T]. This indicates an issue with your type design. I would recommend solving that first.
One solution might be to recognise that things that are sizable and things that are not sizable should be represented as two different types. Then you can use the following signature:
def partitionToSizable[T1, T2 <: Sizable[T2]](
elements: List[Either[T1, T2]]): (List[T1], List[T2])
Then you don't need to cast; you can pattern match or fold on the Either.
In my team, I often see teammates writing
list.filter(_.isInstanceOf[T]).map(_.asInstanceOf[T])
but this seems a bit redundant to me.
If we know that everything in the filtered list is an instance of T then why should we have to explicitly cast it as such?
I know of one alternative, which is to use match.
eg:
list.match {
case thing: T => Some(thing)
case _ => None
}
but this has the drawback that we must then explicitly state the generic case.
So, given all the above, I have 2 questions:
1) Is there another (better?) way to do the same thing?
2) If not, which of the two options above should be preferred?
You can use collect:
list collect {
case el: T => el
}
Real types just work (barring type erasure, of course):
scala> List(10, "foo", true) collect { case el: Int => el }
res5: List[Int] = List(10)
But, as #YuvalItzchakov has mentioned, if you want to match for an abstract type T, you must have an implicit ClassTag[T] in scope.
So a function implementing this may look as follows:
import scala.reflect.ClassTag
def filter[T: ClassTag](list: List[Any]): List[T] = list collect {
case el: T => el
}
And using it:
scala> filter[Int](List(1, "foo", true))
res6: List[Int] = List(1)
scala> filter[String](List(1, "foo", true))
res7: List[String] = List(foo)
collect takes a PartialFunction, so you shouldn't provide the generic case.
But if needed, you can convert a function A => Option[B] to a PartialFunction[A, B] with Function.unlift. Here is an example of that, also using shapeless.Typeable to work around type erasure:
import shapeless.Typeable
import shapeless.syntax.typeable._
def filter[T: Typeable](list: List[Any]): List[T] =
list collect Function.unlift(_.cast[T])
Using:
scala> filter[Option[Int]](List(Some(10), Some("foo"), true))
res9: List[Option[Int]] = List(Some(10))
but this seems a bit redundant to me.
Perhaps programmers in your team are trying to shield that piece of code from someone mistakenly inserting a type other then T, assuming this is some sort of collection with type Any. Otherwise, the first mistake you make, you'll blow up at run-time, which is never fun.
I know of one alternative, which is to use match.
Your sample code won't work because of type erasure. If you want to match on underlying types, you need to use ClassTag and TypeTag respectively for each case, and use =:= for type equality and <:< for subtyping relationships.
Is there another (better?) way to do the same thing?
Yes, work with the type system, not against it. Use typed collections when you can. You haven't elaborated on why you need to use run-time checks and casts on types, so I'm assuming there is a reasonable explanation to that.
If not, which of the two options above should be preferred?
That's a matter of taste, but using pattern matching on types can be more error-prone since one has to be aware of the fact that types are erased at run-time, and create a bit more boilerplate code for you to maintain.
In languages like SML, Erlang and in buch of others we may define functions like this:
fun reverse [] = []
| reverse x :: xs = reverse xs # [x];
I know we can write analog in Scala like this (and I know, there are many flaws in the code below):
def reverse[T](lst: List[T]): List[T] = lst match {
case Nil => Nil
case x :: xs => reverse(xs) ++ List(x)
}
But I wonder, if we could write former code in Scala, perhaps with desugaring to the latter.
Is there any fundamental limitations for such syntax being implemented in the future (I mean, really fundamental -- e.g. the way type inference works in scala, or something else, except parser obviously)?
UPD
Here is a snippet of how it could look like:
type T
def reverse(Nil: List[T]) = Nil
def reverse(x :: xs: List[T]): List[T] = reverse(xs) ++ List(x)
It really depends on what you mean by fundamental.
If you are really asking "if there is a technical showstopper that would prevent to implement this feature", then I would say the answer is no. You are talking about desugaring, and you are on the right track here. All there is to do is to basically stitch several separates cases into one single function, and this can be done as a mere preprocessing step (this only requires syntactic knowledge, no need for semantic knowledge). But for this to even make sense, I would define a few rules:
The function signature is mandatory (in Haskell by example, this would be optional, but it is always optional whether you are defining the function at once or in several parts). We could try to arrange to live without the signature and attempt to extract it from the different parts, but lack of type information would quickly come to byte us. A simpler argument is that if we are to try to infer an implicit signature, we might as well do it for all the methods. But the truth is that there are very good reasons to have explicit singatures in scala and I can't imagine to change that.
All the parts must be defined within the same scope. To start with, they must be declared in the same file because each source file is compiled separately, and thus a simple preprocessor would not be enough to implement the feature. Second, we still end up with a single method in the end, so it's only natural to have all the parts in the same scope.
Overloading is not possible for such methods (otherwise we would need to repeat the signature for each part just so the preprocessor knows which part belongs to which overload)
Parts are added (stitched) to the generated match in the order they are declared
So here is how it could look like:
def reverse[T](lst: List[T]): List[T] // Exactly like an abstract def (provides the signature)
// .... some unrelated code here...
def reverse(Nil) = Nil
// .... another bit of unrelated code here...
def reverse(x :: xs ) = reverse(xs) ++ List(x)
Which could be trivially transformed into:
def reverse[T](list: List[T]): List[T] = lst match {
case Nil => Nil
case x :: xs => reverse(xs) ++ List(x)
}
// .... some unrelated code here...
// .... another bit of unrelated code here...
It is easy to see that the above transformation is very mechanical and can be done by just manipulating a source AST (the AST produced by the slightly modified grammar that accepts this new constructs), and transforming it into the target AST (the AST produced by the standard scala grammar).
Then we can compile the result as usual.
So there you go, with a few simple rules we are able to implement a preprocessor that does all the work to implement this new feature.
If by fundamental you are asking "is there anything that would make this feature out of place" then it can be argued that this does not feel very scala. But more to the point, it does not bring that much to the table. Scala author(s) actually tend toward making the language simpler (as in less built-in features, trying to move some built-in features into libraries) and adding a new syntax that is not really more readable goes against the goal of simplification.
In SML, your code snippet is literally just syntactic sugar (a "derived form" in the terminology of the language spec) for
val rec reverse = fn x =>
case x of [] => []
| x::xs = reverse xs # [x]
which is very close to the Scala code you show. So, no there is no "fundamental" reason that Scala couldn't provide the same kind of syntax. The main problem is Scala's need for more type annotations, which makes this shorthand syntax far less attractive in general, and probably not worth the while.
Note also that the specific syntax you suggest would not fly well, because there is no way to distinguish one case-by-case function definition from two overloaded functions syntactically. You probably would need some alternative syntax, similar to SML using "|".
I don't know SML or Erlang, but I know Haskell. It is a language without method overloading. Method overloading combined with such pattern matching could lead to ambiguities. Imagine following code:
def f(x: String) = "String "+x
def f(x: List[_]) = "List "+x
What should it mean? It can mean method overloading, i.e. the method is determined in compile time. It can also mean pattern matching. There would be just a f(x: AnyRef) method that would do the matching.
Scala also has named parameters, which would be probably also broken.
I don't think that Scala is able to offer more simple syntax than you have shown in general. A simpler syntax may IMHO work in some special cases only.
There are at least two problems:
[ and ] are reserved characters because they are used for type arguments. The compiler allows spaces around them, so that would not be an option.
The other problem is that = returns Unit. So the expression after the | would not return any result
The closest I could come up with is this (note that is very specialized towards your example):
// Define a class to hold the values left and right of the | sign
class |[T, S](val left: T, val right: PartialFunction[T, T])
// Create a class that contains the | operator
class OrAssoc[T](left: T) {
def |(right: PartialFunction[T, T]): T | T = new |(left, right)
}
// Add the | to any potential target
implicit def anyToOrAssoc[S](left: S): OrAssoc[S] = new OrAssoc(left)
object fun {
// Use the magic of the update method
def update[T, S](choice: T | S): T => T = { arg =>
if (choice.right.isDefinedAt(arg)) choice.right(arg)
else choice.left
}
}
// Use the above construction to define a new method
val reverse: List[Int] => List[Int] =
fun() = List.empty[Int] | {
case x :: xs => reverse(xs) ++ List(x)
}
// Call the method
reverse(List(3, 2, 1))