I am reading some Scala code. What does the -> mean in the following context?
var queries = { "Select apple from farm" -> None, "Select orange from fram" -> None, "Select blueberry from anotherFarm" -> Some( #randomStuff ) }
It looks like a list of lambda functions but I thought it should be => in that case instead of ->.
Also,
what does this single line code mean?
def onConnection(id) = { application ! turnOnApplication(id) }
Specifically, I am confused with the use of !. It doesn't seem to be a "NOT" as it is in most languages
The -> symbol is one way to define a tuple in Scala. The below are all equivalent:
val apples1 = "Select apple from farm" -> None
val apples2 = ("Select apple from farm" -> None)
val apples3 = ("Select apple from farm", None)
As for the !:
def onConnection(id) = { application ! turnOnApplication(id) }
! in Scala can be the negation operator, but ! in the above code snippet looks like tell from Akka (Akka is the main actor library for Scala). This pattern is used to send a message to an actor. So if application is a reference to an actor, the code snippet sends the result of turnOnApplication(id) to the application actor. From the linked documentation:
"!" means “fire-and-forget”, e.g. send a message asynchronously and return immediately. Also known as tell.
The thin arrow -> is Tuple syntax. It's just a different way of writing Tuples. I.e.
val x: (Int, String) = 3 -> "abc"
Is the same as writing:
val x: (Int, String) = (3, "abc")
The arrow syntax is done by providing an implicit class ArrowAssoc which defines a method def ->[B](y: B): (A, B). ArrowAssoc is part of Predef which is inserted into every Scala source file. You can find the docs here.
The bracket syntax meanwhile is syntactic sugar done by the compiler.
You can form tuple using two syntaxes
1) Using comma
val tuple = (1, 2)
2) Using -> (arrow)
val tuple = 1 -> 2
Scala repl
scala> val tuple = (1, 2)
tuple: (Int, Int) = (1,2)
scala> val tuple = 1 -> 2
tuple: (Int, Int) = (1,2)
Finding-symbols defines -> as Method provided by implicit conversion. Just look at the methods tagged with implicit that receive, as parameter, an object of type that is receiving the method. For example:
"a" -> 1 // Look for an implicit from String, AnyRef, Any or type parameter
In the above case, -> is defined in the class ArrowAssoc through the method any2ArrowAssoc that takes an object of type A, where A is an unbounded type parameter to the same method.
tutorialPoint definde ! as It is called Logical NOT Operator. Use to reverses the logical state of its operand. If a condition is true then Logical NOT operator will make false.
Related
I don't know why pattern d is bad in this list below.
Why need expicit type declaration?
def adder1(m:Int,n:Int) = m + n
val a = adder1(2,_) //OK
val b = adder1(_,2) //OK
def adder2(m:Int)(n:Int) = m + n
val c = adder2(2)(_) //OK
val d = adder2(_)(2) //NG:missing parameter type
val e = adder2(_:Int)(2) //OK
I just want to know the reason pattern d needs parameter type.
Very welcome just showing citation language spec.
So I believe this comes from the concept of Partial Application.
Intuitively, partial function application says "if you fix the first arguments of the function, you get a function of the remaining arguments"
...
Scala implements optional partial application with placeholder, e.g. def add(x: Int, y: Int) = {x+y}; add(1, _: Int) returns an incrementing function. Scala also support multiple parameter lists as currying, e.g. def add(x: Int)(y: Int) = {x+y}; add(1) _.
Lets take a look at adder2
From the REPL:
scala> def adder2(m:Int)(n:Int) = m + n
def adder2(m: Int)(n: Int): Int
Lets get a value to represent this:
scala> val adder2Value = adder2
^
error: missing argument list for method adder2
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing `adder2 _` or `adder2(_)(_)` instead of `adder2`.
Ok, let's try:
val adder2Value = adder2 _
val adder2Value: Int => (Int => Int) = $Lambda$1382/0x0000000840703840#4b66a923
Ahha!
In English: "A function that takes an Int and returns a function that takes an Int and returns an Int"
How can we bind the second argument using this signature? How can we access the inner function unless we first have gone through the outer one?
As far as I know, this is not possible to do using this signature, unless you explicitly define the type of your first argument.
(But what about adder2(_)(_)?)
scala> adder2(_)(_)
^
error: missing parameter type for expanded function ((<x$1: error>, x$2) => adder2(x$1)(x$2))
^
error: missing parameter type for expanded function ((<x$1: error>, <x$2: error>) => adder2(x$1)(x$2))
(Maybe this hints at our solution?)
Notice what happens if we explicitly define both arguments:
val adder2Value2= adder2Value (_:Int) (_:Int)
val adder2Value2: (Int, Int) => Int = $Lambda$1394/0x000000084070d840#32f7d983
This is much more manageable, we can now fix either argument, and get a simplified partial function:
scala> val adder2FirstArg = adder2Value (_:Int) (10)
val adder2FirstArg: Int => Int = $Lambda$1395/0x000000084070d040#47f5ddf4
scala> val adder2SecondArg = adder2Value (5) (_:Int)
val adder2SecondArg: Int => Int = $Lambda$1396/0x000000084070c840#21ed7ce
So what's really going on here?
When you bind an argument to a value, you have explicitly expressed the type (maybe it's inferred, but it's definitely that type, in this case, Ints). It's sugar so we don't need to write it. But under the hood, these are composed functions, and how they are composed is very important. To be able to match and simplify the function signature, the compiler requires us to provide this information in an outside-in manner. Otherwise, we need to give it some help to get there.
EDIT:
I think that this question serves as more of a Scala language spec. puzzle exercise, however. I can't think of any good reason, from a design perspective, for which you would need to implement a curried function in such a way that you cannot order the parameters such that the last parameters are the ones being inferred.
I have in scala a
case class foo(a: Array[String], b: Map[String,Any])
I am trying to run unit tests for this but the assertEquals while storing the foo elements(actual & expected) in a Arrays.
So the last line is using assertEquals(expected.deep, actual.deep).
The map b is showing correctly, but the assertEquals is trying the match the hashcode of the Array a instead of the content. the error is get is something like this:
Array(foo([Ljava.lang.string#235543a70,Map("bar" -> "bar")))
The overall code looks like
case class Foo(a: Array[String], b: Map[String, Any])
val foo = Foo(Array("1"), Map("ss" -> 1))
val foo2 = Foo(Array("1"), Map("ss" -> 1))
org.junit.Assert.assertEquals(Array(foo).deep, Array(foo2).deep)
How do you suggest making this work?
Case classes in scala come with their own hashCode and equals methods that should not be overridden. This implementation checks equality of the content. But it relies on "content" based hashCode and equals implementations of types used within the case class.
Unfortunately, this is not the case for Arrays, which means that Arrays cannot be checked by content with default equals method.
The easiest solution is to use data collections that check equality based on the content, like Seq.
case class Foo(a: Seq[String], b: Map[String, Any])
val foo = Foo(Seq("1"), Map("ss" -> 1))
val foo2 = Foo(Seq("1"), Map("ss" -> 1))
org.junit.Assert.assertEquals(foo, foo2)
Calling deep won't help you in this case as it scans and converts nested Array only.
println(Array(foo, Array("should be converted and printed")))
println(Array(foo, Array("should be converted and printed")).deep)
produces
[Ljava.lang.Object;#188715b5
Array(Foo([Ljava.lang.String;#6eda5c9,Map(ss -> 1)), Array(should be converted and printed))
I have a class, where all of its functions have the same arity and same type of output. (Why? Each function is a separate processor that is applied to a Spark DataFrame and yields another DataFrame).
So, the class looks like this:
class Processors {
def p1(df: DataFrame): DataFrame {...}
def p2(df: DataFrame): DataFrame {...}
def p3(df: DataFrame): DataFrame {...}
...
}
I then apply all the methods of a given DataFrame by mapping over Processors.getClass.getMethod, which allows me to add more processors without changing anything else in the code.
What I'd like to do is define a template to the methods under Processors which will restrict all of them to accept only one DataFrame and return a DataFrame. Is there a way to do this?
Implementing a restriction on what kind of functions can be added to a "list" is possible by using an appropriate container class instead of a generic class to hold the methods that are restricted. The container of restricted methods can then be part of some new class or object or part of the main program.
What you lose below by using containers (e.g. a Map with string keys and restricted values) to hold specific kinds of functions is compile-time checking of the names of the methods. e.g. calling triple vs trilpe
The restriction of a function to take a type T and return that same type T can be defined as a type F[T] using Function1 from the scala standard library. Function1[A,B] allows any single-parameter function with input type A and output type B, but we want these input/output types to be the same, so:
type F[T] = Function1[T,T]
For a container, I will demonstrate scala.collection.mutable.ListMap[String,F[T]] assuming the following requirements:
string names reference the functions (doThis, doThat, instead of 1, 2, 3...)
functions can be added to the list later (mutable)
though you could choose some other mutable or immutable collection class (e.g. Vector[F[T]] if you only want to number the methods) and still benefit from the restriction of what kind of functions future developers can include into the container.
An abstract type can be defined as:
type TaskMap[T] = ListMap[String, F[T]]
For your specific application you would then instantiate this as:
val Processors:TaskMap[Dataframe] = ListMap(
"p1" -> ((df: DataFrame) => {...code for p1 goes here...}),
"p2" -> ((df: DataFrame) => {...code for p2 goes here...}),
"p3" -> ((df: DataFrame) => {...code for p3 goes here...})
)
and then to call one of these functions you use
Processors("p2")(someDF)
For simplicity of demonstration, let's forget about Dataframes for a moment and consider whether this scheme works with integers.
Consider the short program below. The collection "myTasks" can only contain functions from Int to Int. All of the lines below have been tested in the scala interpreter, v2.11.6, so you can follow along line by line.
import scala.collection.mutable.ListMap
type F[T] = Function1[T,T]
type TaskMap[T] = ListMap[String, F[T]]
val myTasks: TaskMap[Int] = ListMap(
"negate" -> ((x:Int)=>(-x)),
"triple" -> ((x:Int)=>(3*x))
)
we can add a new function to the container that adds 7 and name it "add7"
myTasks += ( "add7" -> ((x:Int)=>(x+7)) )
and the scala interpreter responds with:
res0: myTasks.type = Map(add7 -> <function1>, negate -> <function1>, triple -> <function1>)
but we can't add a function named "half" because it would return a Float, and a Float is not an Int and should trigger a type error
myTasks += ( "half" -> ((x:Int)=>(0.5*x)) )
Here we get this error message:
scala> myTasks += ( "half" -> ((x:Int)=>(0.5*x)) )
<console>:12: error: type mismatch;
found : Double
required: Int
myTasks += ( "half" -> ((x:Int)=>(0.5*x)) )
^
In a compiled application, this would be found at compile time.
How to call the functions stored this way is a bit more verbose for single calls, but can be very convenient.
Suppose we want to call "triple" on 10.
We can't write
triple(10)
<console>:9: error: not found: value triple
Instead it is
myTasks("triple")(10)
res4: Int = 30
Where this notation becomes more useful is if you have a list of tasks to perform but only want to allow tasks listed in myTasks.
Suppose we want to run all the tasks on the input data "10"
myTasks mapValues { _ apply 10 }
res9: scala.collection.Map[String,Int] =
Map(add7 -> 17, negate -> -10, triple -> 30)
Suppose we want to triple, then add7, then negate
If each result is desired separately, as above, that becomes:
List("triple","add7","negate") map myTasks.apply map { _ apply 10 }
res11: List[Int] = List(30, 17, -10)
But "triple, then add 7, then negate" could also be describing a series of steps to do 10, i.e. we want -((3*10)+7)" and scala can do that too
val myProgram = List("triple","add7","negate")
myProgram map myTasks.apply reduceLeft { _ andThen _ } apply 10
res12: Int = -37
opening the door to writing an interpreter for your own customizable set of tasks because we can also write
val magic = myProgram map myTasks.apply reduceLeft { _ andThen _ }
and magic is then a function from int to int that can take aribtrary ints or otherwise do work as a function should.
scala> magic(1)
res14: Int = -10
scala> magic(2)
res15: Int = -13
scala> magic(3)
res16: Int = -16
scala> List(10,20,30) map magic
res17: List[Int] = List(-37, -67, -97)
Is this what you mean?
class Processors {
type Template = DataFrame => DataFrame
val p1: Template = ...
val p2: Template = ...
val p3: Template = ...
def applyAll(df: DataFrame): DataFrame =
p1(p2(p3(df)))
}
Below is my HashMap:
val params1 = collection.mutable.HashMap[String, Any]()
params1 += "forCourseId" -> "2"
println(params1.isDefinedAt("forCourseId"))
println(params1.contains("forCourseId"))
What is difference between isDefinedAt() & contains() method?
I need to check whether key is present or not
main concern is , it will not throw null pointer exception.
You can check the Scala source code. In MapLike.scala you'll see that isDefinedAt is actually just calling contains, meaning that they are truly identical:
def isDefinedAt(key: A) = contains(key)
The only real difference is that contains is specific to the Map interface (specifically it's declared on GenMapLike), while isDefinedAt is found on all PartialFunction classes.
val m: Map[Int,Int] = Map(1 -> 2)
val pf: PartialFunction[Int,Int] = { case 1 => 1 }
m.isDefinedAt(1) // ok
m.contains(1) // ok
pf.isDefinedAt(1) // ok
pf.contains(1) // doesn't compile
According to the Scala Doc isDefinedAt is equivalent to contains.
This method, which implements an abstract method of trait PartialFunction, is equivalent to contains.
http://www.scala-lang.org/api/current/#scala.collection.mutable.HashMap
I am new to scala, and today when I came across this akka source code I was puzzled:
def traverse[A, B](in: JIterable[A], fn: JFunc[A, Future[B]],
executor: ExecutionContext): Future[JIterable[B]] = {
implicit val d = executor
scala.collection.JavaConversions.iterableAsScalaIterable(in).foldLeft(
Future(new JLinkedList[B]())) { (fr, a) ⇒
val fb = fn(a)
for (r ← fr; b ← fb) yield { r add b; r }
}
}
Why the code is written using implicit parameters intentionally? Why can't it be written as:
scala.collection.JavaConversions.iterableAsScalaIterable(in).foldLeft(
Future(new JLinkedList[B](),executor))
without decalaring a new implicit variable d? Is there any advantage of doing this? For now I only find implicits increase the ambiguity of the code.
I can give you 3 reasons.
1) It hides boilerplate code.
Lets sort some lists:
import math.Ordering
List(1, 2, 3).sorted(Ordering.Int) // Fine. I can tell compiler how to sort ints
List("a", "b", "c").sorted(Ordering.String) // .. and strings.
List(1 -> "a", 2 -> "b", 3 -> "c").sorted(Ordering.Tuple2(Ordering.Int, Ordering.String)) // Not so fine...
With implicit parameters:
List(1, 2, 3).sorted // Compiller knows how to sort ints
List(1 -> "a", 2 -> "b", 3 -> "c").sorted // ... and some other types
2) It alows you to create API with generic methods:
scala> (70 to 75).map{ _.toChar }
res0: scala.collection.immutable.IndexedSeq[Char] = Vector(F, G, H, I, J, K)
scala> (70 to 75).map{ _.toChar }(collection.breakOut): String // You can change default behaviour.
res1: String = FGHIJK
3) It allows you to focus on what really matters:
Future(new JLinkedList[B]())(executor) // meters: what to do - `new JLinkedList[B]()`. don't: how to do - `executor`
It's not so bad, but what if you need 2 futures:
val f1 = Future(1)(executor)
val f2 = Future(2)(executor) // You have to specify the same executor every time.
Implicit creates "context" for all actions:
implicit val d = executor // All `Future` in this scope will be created with this executor.
val f1 = Future(1)
val f2 = Future(2)
3.5) Implicit parameters allows type-level programming . See shapeless.
About "ambiguity of the code":
You don't have to use implicits, alternatively you can specify all parameters explicitly. It looks ugly sometimes (see sorted example), but you can do it.
If you can't find which implicit variables are used as parameters you can ask compiler:
>echo object Test { List( (1, "a") ).sorted } > test.scala
>scalac -Xprint:typer test.scala
You'll find math.this.Ordering.Tuple2[Int, java.lang.String](math.this.Ordering.Int, math.this.Ordering.String) in output.
In the code from Akka you linked, it is true that executor could be just passed explicitly. But if there was more than one Future used throughout this method, declaring implicit parameter would definitely make sense to avoid passing it around many times.
So I would say that in the code you linked, implicit parameter was used just to follow some code style. It would be ugly to make an exception from it.
Your question intrigued me, so I searched a bit on the net. Here's what I found on this blog: http://daily-scala.blogspot.in/2010/04/implicit-parameters.html
What is an implicit parameter?
An implicit parameter is a parameter to method or constructor that is marked as implicit. This means that if a parameter value is not supplied then the compiler will search for an "implicit" value defined within scope (according to resolution rules.)
Why use an implicit parameter?
Implicit parameters are very nice for simplifying APIs. For example the collections use implicit parameters to supply CanBuildFrom objects for many of the collection methods. This is because normally the user does not need to be concerned with those parameters. Another example is supplying an encoding to an IO library so the encoding is defined once (perhaps in a package object) and all methods can use the same encoding without having to define it for every method call.