I have higher order function:
def calculateFusionRiskContract(postCalc: DataFrame => DataFrame)
How i can set default value for postCalc parameter that just return parameter of postCalc? without any calculation?
def calculateFusionRiskContract(postCalc: DataFrame => DataFrame = ???)
The function that "just returns the parameter" is identity
def calculateFusionRiskContract(postCalc: DataFrame => DataFrame = identity) = ???
Is there a scala identity function?
Related
case class User(id: Int)
def Transform(u: User): String = u.id.toString()
def Process[A](f: A => String): String = "hello"
val u = User(123)
println(Process(u => Transform(u))) // Scenerio#1 works fine
println(Process(Transform(u))) // error
println(Process[User](Transform(u))) // error
I'm a little confused as to why I need the => in Scenerio#1
I have a function Process that takes a function. The function requires:
a parameter of type A
return value of String.
Now I have a function named Transform that takes a parameter of type A (a User) and returns a string.
Why can't I pass in:
Process(Transform(u))
I am passing it a function that meets the requirements am I not? (I guess not but I don't understand!)
I guess I still don't understand what the following notation really means:
Process(u => Transform(u))
As you already noticed, Transform is of type User => String, therefore, you can just pass it as a parameter to Process:
val process1: String = Process(u => Transform(u))
val process2: String = Process(Transform)
val process3: String = Process[User](Transform)
All of the above are exactly the same, and outputs hello.
def Process[A](f: A => String) method in your code is expecting a function definition as argument whose input parameter type should A and return type should be String.
Case 1:
println(Process(u => Transform(u)))
You are passing function definition so it is working fine as per expectation.
Case 2:
println(Process(Transform(u))) // error
println(ProcessUser) // error
You are not passing function definition here as expected by Process function instead you are passing the function call Transform(u) which will return the value and pass as argument to Process function.
You can refer high order function in scala using this link (https://docs.scala-lang.org/tour/higher-order-functions.html).
I have code below and I wanted to know what does Seq[String] = List() mean?
Does it mean it takes sequence of strings and converts it into List()?
def somefuncname(input: Seq[String] = List()): Unit = {
//Some Code
}
First try to understand the below function signature.
def somefuncname(input: Seq[String]): Unit = {
//Some Code
}
The above code is a function declaration. Its a function which takes 1 argument called input which is of type Seq[String]. That means it takes sequence or list of strings as input and returns nothing Unit
Now, what does = mean?
= after the input argument of the function means default value for the function argument. If you are not interested in passing a custom "sequence of strings" then you can rely on the default argument of already passed.
Now, what does List() mean?
List() returns sequence of 0 elements or empty sequence. That means function is taking empty elements as default argument
alternatively you can also pass Seq() as default argument. It also means empty sequence
def somefuncname(input: Seq[String] = Seq()): Unit = {
//Some Code
}
Now to use the function in any of the following ways
somefuncname() // Now input is empty sequence of strings
somefuncname(Seq("apple", "cat"))
somefuncname(List("apple", "cat"))
input is of type Seq[String] and it has a default value of empty list (List()).
Having a default value means so that if you call the function without passing an argument it would get the default value
I am following this tutorial and I see this code:
println("\nStep 1: How to define a higher order function which takes another function as parameter")
def totalCostWithDiscountFunctionParameter(donutType: String)(quantity: Int)(f: Double => Double): Double = {
println(s"Calculating total cost for $quantity $donutType")
val totalCost = 2.50 * quantity
f(totalCost)
}
println("\nStep 2: How to define and pass a def function to a higher order function")
def applyDiscount(totalCost: Double): Double = {
val discount = 2 // assume you fetch discount from database
totalCost - discount
}
println(s"Total cost of 5 Glazed Donuts with discount def function = ${totalCostWithDiscountFunctionParameter("Glazed Donut")(5)(applyDiscount(_))}")
println("\nStep 3: How to define and pass a val function to a higher order function")
val applyDiscountValueFunction = (totalCost: Double) => {
val discount = 2 // assume you fetch discount from database
totalCost - discount
}
println(s"Total cost of 5 Glazed Donuts with discount val function = ${totalCostWithDiscountFunctionParameter("Glazed Donut")(5)(applyDiscountValueFunction)}")
The author says:
The function has a by-name parameter which is expected to be a function that has a parameter of type Double and will also return a type of Double.
Is that true? What would be a by-value parameter here? Is the function lazily evaluated which makes it a by-name parameter? Is that true?
Why does the author use the wildcard when passing in applyDiscount but not when passing in applydiscountValueFunction? Both work without the wildcard operator.
applyDiscount(_) transforms the method into a function, using placeholder syntax for anonymous functions.
This process can be automated by the compiler (using a technique called eta-expansion) whenever a function type is expected and a method is passed instead, which is exactly the case in the example.
(for a more thorough discussion, refer to this answer: Underscore usage when passing a function in Scala)
So you're right in saying that this
totalCostWithDiscountFunctionParameter("Glazed Donut")(5)(applyDiscount)
would work anyway, because the compiler will transform the applyDiscount to a function automatically.
As per the by-name parameter, what the author is calling by-name parameter (the parameter f) is really just a parameter of type Double => Double, so it seems he's using incorrect terminology.
A by-name parameter in Scala is expressed using => like this:
def someMethod(someParam: => Double): Double = // ...
In this example someParam is a by name parameter, meaning it won't be evaluated at call-site.
I want to define a UDF in scala spark like the pseudo code below:
def transformUDF（size:Int）:UserDefinedFunction = udf((input:Seq[T]){
if (input != null)
Vectors.dense(input.map(_.toDouble).toArray)
else
Vectors.dense(Array.fill[Double](size)(0.0))
})
if input is not null, cast every element to Double Type.
if input is null, return a all-zero vector.
And I want T to be limited to numeric type, like java.lang.Number in Java. But it seems that Seq[java.lang.Number] cannot work with the toDouble.
Is there any appropriate way?
As mentioned in my working comment as
def transformUDF: UserDefinedFunction = udf((size: Int, input:Seq[java.lang.Number]) => {
if (input != null)
Vectors.dense(input.map(_.doubleValue()).toArray)
else
Vectors.dense(Array.fill[Double](size)(0.0))
})
You don't need to create a new column, you can just pass it to the udf function as
dataframe.withColumn("newCol", transformUDF(lit(the size you want), dataframe("the column you want to transform")))
I am really new to Scala and trying to study it.
I don't know how to access or using the parameter of higher order function. For example:
def higherOrderFunc(f: Int => Boolean): String = {
/* Logic to print parameter is here */
"Hello"
}
val func = higherOrderFunc(x => x > 1)
How can I print the value of x before I return value "Hello"
You can't. The argument does not exist in this context; it'd need to be passed into the higher-order function along with the anonymous function.