Is cons a method or a class? - scala

Marting Odersky in his book writes:
Class ::, pronounced “cons” for “construct,” represents non-empty lists.
and
The list construction methods :: and ::: are special. Because they end
in a colon, they are bound to their right operand. That is, an
operation such as x :: xs is treated as the method call xs.::(x), not
x.::(xs). In fact, x.::(xs) would not make sense, as x is of the list
element type, which can be arbitrary, so we cannot assume that this
type would have a :: method. For this reason, the :: method should
take an element value and yield a new list
So is :: a method or a class?

It is both a class and a method. Cons is a type parametised class. List[A] has two sub classes: Cons and Nil. As Cons is a class it can be created by its constructor as follows:
val s = new ::[Int](4, Nil)
Cons is a case class and we use the constructor when we do a pattern match. Cons is also a method on the list class that is implemented in its two sub classes. Hence we can use the cons method on the cons class that we have created above.
val s1 = s.::(5)
Part of the confusion may arise, because we normally create Lists using the apply method of the List object:
val s2 = List(1, 2, 3)
Normally the apply method of an object returns a new instance of the Class with the same name as an object. This is however just convention. In this particular case the List Object returns a new instance of the Cons subclass. The List class itself is a sealed abstract class so can not be instantiated. So the above apply method does the following:
val s2 = 1 :: 2 :: 3 :: Nil
Any method that ends in a ':' is a method on the operand to its right so it can be rewritten as
val s2 = 1 :: (2 :: (3 :: Nil)) //or as
val s2 = Nil.::(3).::(2).::(1)
So the Cons(::) method on the Nil object takes the 3 as a parameter and produces an anonymous instantiation of the Cons class with 3 as its head and a reference to the Nil object as its tail. Lets call this anonymous object c1. The Cons method is then called on c1 taking 2 as its parameter returning a new anonymous Cons instantiation lets call it c2, which has 2 for its head and a reference to c1 as its tail. Then finally the cons method on the c2 object takes the 1 as a parameter and returns the named object s2 with 1 as its head and a reference to c2 as its tail.
A second point of confusion is that the REPL and Scala worksheets use a class' toString methods to display results. So the worksheet gives us:
val s3 = List(5, 6, 7) // s3 : List[Int] = List(5, 6, 7)
val s4 = List() // s4 : List[Nothing] = List()
val s5: List[Int] = List() // s5 : List[Int] = List()
s3.getClass.getName // res3: String = scala.collection.immutable.$colon$colon
s4.getClass.getName // res4: String = scala.collection.immutable.Nil$
s5.getClass.getName // res5: String = scala.collection.immutable.Nil$
As said above List is sealed so new sub sub classes can not be created as Nil is an object and Cons is final. As Nil is an object it can't be parametrised. Nil inherits from List[Nothing]. At first glance that doesn't sound that useful but remember these data structures are immutable, so we can never add to them directly and Nothing is a sub class of every other class. So we can add the Nil class (indirectly) to any List without problem. The Cons class has two members a head and another List. Its a rather neat solution when you clock it.
I'm not sure if this has any practical use but you can use Cons as a type:
var list: ::[Int] = List (1,2,3).asInstanceOf[::[Int]]
println("Initialised List")
val list1 = Nil
list = list1.asInstanceOf[::[Int]] //this will throw a java class cast exception at run time
println("Reset List")

Short answer: both.
There is a subclass of list called ::, but you won't refer to it explicitly very often.
When you write e.g. 1 :: 2 :: Nil, the :: is a method on List, which creates an instance of the class :: behind the scenes.
:: is best thought of not as a method or a class, though, but as a constructor in the algebraic data type (ADT) sense. Wikipedia calls ADT constructors "quasi-functional entit[ies]", which makes them sound more complicated than they are, but isn't necessarily a bad way to think about them.
List has two constructors, :: (called cons in some languages) and Nil. The Wikipedia article I've linked above gives a good introduction to the idea of lists as algebraic data types.
It's worth noting that in some languages (like Haskell) ADT constructors don't have their own types associated with them—they are just functions that create an instance of the ADT.
This is typically effectively the case in Scala as well, where it's fairly rare to refer to the type of an ADT constructor like ::. It is possible, though—we can write this:
def foo(xs: ::[Int]) = ???
Or this (where Some is one of the constructors for the Option ADT).
def bar(s: Some[Int]) = ???
But this is in general not very useful, and could be considered an artifact of the way Scala implements algebraic data types.

Related

Why does a list initialised with Nil and cons (::) not have a NULL value at the end?

Today is my first day of learning Scala, and one of the things I have learnt is that a common way to initialise lists combines Nil (an empty list) with :: (prepend method).
Let's say I initialise a list this way:
val myList = List("A", "B", "C")
which is the same as:
val myList = "A" :: "B" :: "C" :: Nil
I understand that you can read the second chunk as the following:
Start with an empty list
Add "C" to the beginning of that list. The list is no longer empty.
Add "B" to the beginning of the list.
Add "A" to the beginning of the list.
Assign the list to the immutable List collection. The data type of each list element is inferred (String).
What I don't understand is why doesn't Nil act the same as NULL? Why is there no NULL value at the end of the list when it is initialised in this way?
(Forgive me, OOP and FP are not my thing but I'd like to learn.)
If you want to be able to call methods on all lists including empty list (for example Nil.length), Nil must be an object, not null. Otherwise you'll have NullPointerException instead of 0.
Also it's necessary for type safety. Nil is a value of type List[Nothing], i.e. of List[A] for all A (because List is covariant). And null is a value of type Null i.e. of all reference types (because Null is a subtype of any reference type), not only of list types.
Too many things to learn. You first need to understand what is an object, what is a class, what is an interface / trait, what is a value, what is a type. Then with those, you may need to learn about ADTs.
I will try to give a quick answer to that.
The List datatype is defined as
sealed trait List[+A]
final case class ::[+A](head: A, tail: List[A]) extends List[A]
final case object Nil extends List[Nothing]
So here we can see three things: a trait, a class and an object.
A class is a blueprint of memory for creating objects (also called instances). It is not a value, for using it you have to call its constructor (in this case, it receives two values, the head and the tail) to get a new value of that class.
An object is like an anonymous class that is already instantiated, so it is already a value, which you can use as it is.
A trait is a like a class that can not be instantiated, it is said to be abstract. It is used to define common behaviors that all subclasses have to follow.
Also, when the trait is sealed and the classes and objects are cases, we call those ADTs (algebraic data types).
Do not let the fancy name to scare you, it just means that the trait represents a single type (om this case the List) and all the cases represents the parts of such type.
ADTs are formed of products and sums, sums represents alternatives (it is an A or a B) and products represents conjunctions (it is an A and a B).
In the case of the List we say that it is either the empty list (Nil) or a cons (::) which has its own head and tail, where the tail is another list.
With all that, and a the knowledge that Scala has a syntactic trick to allow symbolic names to be used in the middle, you can see that this:
val list = 1 :: 2 :: 3 :: Nil
Is the same as:
val list = new ::(
head = 1,
tail = new ::(
head = 2,
tail = new ::(
head = 3,
tail = Nil
)
)
)
Bonus:
There are more types than classes.
null its a value of type Null (the only value to habit such type), which is a subtype of all the AnyRef types. It is, however, a special kind of value; because it will fail if you try to operate with it in any way. My advice, forget that it exists, it was a mistake and it is only there for Java interop. If you need to model the absence of a value use Option.

How does Scala transform case classes to be accepted as functions?

I am trying to understand how a case class can be passed as an argument to a function which accepts functions as arguments. Below is an example:
Consider the below function
def !![B](h: Out[B] => A): In[B] = { ... }
If I understood correctly, this is a polymorphic method which has a type parameter B and accepts a function h as a parameter. Out and In are other two classes defined previously.
This function is then being used as shown below:
case class Q(p: boolean)(val cont: Out[R])
case class R(p: Int)
def g(c: Out[Q]) = {
val rin = c !! Q(true)_
...
}
I am aware that currying is being used to avoid writing the type annotation and instead just writing _. However, I cannot grasp why and how the case class Q is transformed to a function (h) of type Out[B] => A.
EDIT 1 Updated !! above and the In and Out definitions:
abstract class In[+A] {
def future: Future[A]
def receive(implicit d: Duration): A = {
Await.result[A](future, d)
}
def ?[B](f: A => B)(implicit d: Duration): B = {
f(receive)
}
}
abstract class Out[-A]{
def promise[B <: A]: Promise[B]
def send(msg: A): Unit = promise.success(msg)
def !(msg: A) = send(msg)
def create[B](): (In[B], Out[B])
}
These code samples are taken from the following paper: http://drops.dagstuhl.de/opus/volltexte/2016/6115/
TLDR;
Using a case class with multiple parameter lists and partially applying it will yield a partially applied apply call + eta expansion will transform the method into a function value:
val res: Out[Q] => Q = Q.apply(true) _
Longer explanation
To understand the way this works in Scala, we have to understand some fundamentals behind case classes and the difference between methods and functions.
Case classes in Scala are a compact way of representing data. When you define a case class, you get a bunch of convenience methods which are created for you by the compiler, such as hashCode and equals.
In addition, the compiler also generates a method called apply, which allows you to create a case class instance without using the new keyword:
case class X(a: Int)
val x = X(1)
The compiler will expand this call to
val x = X.apply(1)
The same thing will happen with your case class, only that your case class has multiple argument lists:
case class Q(p: boolean)(val cont: Out[R])
val q: Q = Q(true)(new Out[Int] { })
Will get translated to
val q: Q = Q.apply(true)(new Out[Int] { })
On top of that, Scala has a way to transform methods, which are a non value type, into a function type which has the type of FunctionX, X being the arity of the function. In order to transform a method into a function value, we use a trick called eta expansion where we call a method with an underscore.
def foo(i: Int): Int = i
val f: Int => Int = foo _
This will transform the method foo into a function value of type Function1[Int, Int].
Now that we posses this knowledge, let's go back to your example:
val rin = c !! Q(true) _
If we just isolate Q here, this call gets translated into:
val rin = Q.apply(true) _
Since the apply method is curried with multiple argument lists, we'll get back a function that given a Out[Q], will create a Q:
val rin: Out[R] => Q = Q.apply(true) _
I cannot grasp why and how the case class Q is transformed to a function (h) of type Out[B] => A.
It isn't. In fact, the case class Q has absolutely nothing to do with this! This is all about the object Q, which is the companion module to the case class Q.
Every case class has an automatically generated companion module, which contains (among others) an apply method whose signature matches the primary constructor of the companion class, and which constructs an instance of the companion class.
I.e. when you write
case class Foo(bar: Baz)(quux: Corge)
You not only get the automatically defined case class convenience methods such as accessors for all the elements, toString, hashCode, copy, and equals, but you also get an automatically defined companion module that serves both as an extractor for pattern matching and as a factory for object construction:
object Foo {
def apply(bar: Baz)(quux: Corge) = new Foo(bar)(quux)
def unapply(that: Foo): Option[Baz] = ???
}
In Scala, apply is a method that allows you to create "function-like" objects: if foo is an object (and not a method), then foo(bar, baz) is translated to foo.apply(bar, baz).
The last piece of the puzzle is η-expansion, which lifts a method (which is not an object) into a function (which is an object and can thus be passed as an argument, stored in a variable, etc.) There are two forms of η-expansion: explicit η-expansion using the _ operator:
val printFunction = println _
And implicit η-expansion: in cases where Scala knows 100% that you mean a function but you give it the name of a method, Scala will perform η-expansion for you:
Seq(1, 2, 3) foreach println
And you already know about currying.
So, if we put it all together:
Q(true)_
First, we know that Q here cannot possibly be the class Q. How do we know that? Because Q here is used as a value, but classes are types, and like most programming languages, Scala has a strict separation between types and values. Therefore, Q must be a value. In particular, since we know class Q is a case class, object Q is the companion module for class Q.
Secondly, we know that for a value Q
Q(true)
is syntactic sugar for
Q.apply(true)
Thirdly, we know that for case classes, the companion module has an automatically generated apply method that matches the primary constructor, so we know that Q.apply has two parameter lists.
So, lastly, we have
Q.apply(true) _
which passes the first argument list to Q.apply and then lifts Q.apply into a function which accepts the second argument list.
Note that case classes with multiple parameter lists are unusual, since only the parameters in the first parameter list are considered elements of the case class, and only elements benefit from the "case class magic", i.e. only elements get accessors implemented automatically, only elements are used in the signature of the copy method, only elements are used in the automatically generated equals, hashCode, and toString() methods, and so on.

How do I make the contents of a Scala List implicitly available?

I have a List of items (in this case Spray Marshallers) which I want to make implicitly available in the current context. Just to clarify: I don't want the List itself to be implicitly available, but each element of the list. Does anyone have trick to do this? Or is this actually not possible since the exact implicits have to be available at compile time?
In this particular case, if I understood correctly, you need to have all your specific marshallers available in a context, hence you need to provide each of them implicitly.
You should put them into an object and define them as implicit values.
object Protocol {
implicit val marshaller1: ...
implicit val marshaller2: ...
}
Then, by importing the Protocol object, you would have all your marshallers implicitly available.
And to answer to your question, yes, implicits are resolved at compile time. Hence, whenever you need to use a specific marshaller, the implicit resolution algorithm will look for a value of the specific marshaller type. This is why you cannot get it from a list, let's say, and you shouldn't want this either way.
You can certainly achieve this with shapeless:
import shapeless._
import shapeless.ops.hlist.Selector
case class A()
case class B()
case class C()
val list : A :: B :: C :: HNil = A() :: B() :: C() :: HNil
def screamIfAccessible[T](implicit selector: Selector[A :: B :: C :: HNil ,T]) = {
selector(list) // Actually picks the T type element
// from the HList to work with
println("YAY!")
}
screamIfAccessible[A]
screamIfAccessible[B]
Shapeless - Generic programming with scala
Any method in your code that needs access to an element of type A from the list(or in this case HList) needs to have the Selector implicit argument

How do I check if a Scala HigherKinded TypeTag is an Array?

I'm trying to convert a type tag into a java class that maintains/persists normally-erased type parameters. There are quite a few libraries that benefit from conversions like these (such as Jackson, and Guice). I'm currently trying to migrate Manifest based code to TypeTag since Manifests are insufficient for some corner cases.
The JVM treats Arrays special in comparison to other data types. The difference between a classOf[Int] and classOf[Array[Int]] is that the method Class.isArray() will return true for the latter.
The Manifest implementation was simple. Manifest.erasure was a Class instance where isArray() was already valid/true.
The TypeTag implementation is trickier. There is no quick and easy erasure method. In fact the 'similar' TypeTag variant, RuntimeMirror.runtimeClass, prefers not to handle creating any Array based classes on our behalf. Read the documentation:
Note: If the Scala symbol is ArrayClass, a ClassNotFound exception is
thrown because there is no unique Java class corresponding to a Scala
generic array
To work around this I try to detect if it is an Array. If it is an array, then I manually create the class object. However, I've come across an additional edge case when Array has an unknown type argument.
First let me show you an example that is not a HigherKinded type.
import scala.reflect.runtime.universe._
class A[T]
val innerType = typeOf[A[Array[_]]].asInstanceOf[TypeRefApi].args.head
innerType <:< typeOf[Array[_]] // Returns true.
So far so good.
class B[T[_]]
val innerType = typeOf[B[Array]].asInstanceOf[TypeRefApi].args.head
innerType <:< typeOf[Array[_]] // Returns false.
I can't create a typeOf[Array] since it complains about the missing parameter. How can I detect that B has an type parameter of Array?
Also, what would the class instance look like in this case? Is it an Array[Object]?
Decompose again:
scala> innerType match { case TypeRef(pre, sym, args) => sym == definitions.ArrayClass }
res13: Boolean = true
That might get you part-way.
Another way is to compare typeConstructors:
import scala.reflect.runtime.universe._
class B[T[_]]
val innerType = typeOf[B[Array]].asInstanceOf[TypeRefApi].args.head
innerType.typeConstructor =:= typeOf[Array[_]].typeConstructor
innerType: reflect.runtime.universe.Type = Array
res4: Boolean = true
This also works in general when we need to detect the Type is an Array (of any type).
The try to get erased type of such innerType (to compare) fails for Array (while works for others HigherKinded types):
class B[T[_]]
val innerType = typeOf[B[Array]].typeArgs.head
innerType.typeArgs
innerType.erasure // fails
innerType.erasure =:= typeOf[Array[_]].erasure
defined class B
innerType: reflect.runtime.universe.Type = Array
res4: List[reflect.runtime.universe.Type] = List()
java.util.NoSuchElementException: head of empty list
at scala.collection.immutable.Nil$.head(List.scala:431)
at scala.collection.immutable.Nil$.head(List.scala:428)
at scala.reflect.internal.transform.Erasure$ErasureMap.apply(Erasure.scala:126)
at scala.reflect.internal.transform.Transforms$class.transformedType(Transforms.scala:43)
at scala.reflect.internal.SymbolTable.transformedType(SymbolTable.scala:16)
at scala.reflect.internal.Types$TypeApiImpl.erasure(Types.scala:225)
at scala.reflect.internal.Types$TypeApiImpl.erasure(Types.scala:218)
... 36 elided

differences between List(), Array() and new List(), new Array() in Scala

I know that when you type:
val list = List(2,3)
you are accessing the apply method of the List object which returns a List. What I can't understand is why is this possible when the List class is abstract and therefore cannot be directly instanciated(new List() won't compile)?
I'd also like to ask what is the difference between:
val arr = Array(4,5,6)
and
val arr = new Array(4, 5, 6)
The List class is sealed and abstract. It has two concreate implementations
Nil which represents an empty list
::[B] which represents a non empty list with head and tail. ::[B] in the documentation
When you call List.apply it will jump through some hoops and supply you with an instance of the ::[B] case class.
About array: new Array(4, 5, 6) will throw a compile error as the constructor of array is defined like this: new Array(_length: Int). The apply method of the Array companion object uses the arguments to create a new instance of an Array (with the help of ArrayBuilder).
I started writing that the easy way to determine this is to look at the sources for the methods you're calling, which are available from the ScalaDoc. However, the various levels of indirection that are gone through to actually build a list give lie to the term 'easy'! It's worth having a look through if you want, starting from the apply method in the List object which is defined as follows:
override def apply[A](xs: A*): List[A] = xs.toList
You may or may not know that a parameter of the form xs : A* is treated internally as a Seq, which means that we're calling the toList method on a Seq, which is defined in TraversableOnce. This then delegates to a generic to method, which looks for an implicit
CanBuildFrom which actually constructs the list. So what you're getting back is some implementation of List which is chosen by the CanBuildFrom. What you actually get is a scala.collection.immutable.$colon$colon, which implements a singly-linked list.
Luckily, the behaviour of Array.apply is a little easier to look up:
def apply[T: ClassTag](xs: T*): Array[T] = {
val array = new Array[T](xs.length)
var i = 0
for (x <- xs.iterator) { array(i) = x; i += 1 }
array
}
So, Array.apply just delegates to new Array and then sets elements appropriately.