my code as follow:
var args = "arg1,arg2" //come from external and also many e.g. arg3,arg4 ...
df.select(args.split(","):_*)
then got the errors:
:31: error: no `: *' annotation allowed here (such
annotations are only allowed in arguments to *-parameters)
df.select(args.split(","):*)
any body could give help? thanks.
Well... the use of varargs syntax (: _*) is allowed only for functions which expect variable arguments.
scala> def iAcceptVarArgs(strings: String*) = println(strings)
// iAcceptVarArgs: (strings: String*)Unit
scala> iAcceptVarArgs("str1", "str2")
// WrappedArray(str1, str2)
scala> iAcceptVarArgs(List("str1", "str2"): _*)
// List(str1, str2)
It will not work for functions which do not expect variable arguments,
scala> def iDoNotAcceptVarArgs(s: String) = println(List(s))
// iDoNotAcceptVarArgs: (s: String)Unit
scala> iDoNotAcceptVarArgs(List("str1"): _*)
// <console>:14: error: no `: _*' annotation allowed here
// (such annotations are only allowed in arguments to *-parameters)
// iDoNotAcceptVarArgs(List("str1"): _*)
^
Since Dataframe.select has following signature,
def select(col: String, cols: String*): DataFrame
It means that the first argument is fixed and only second argument onwards can be varargs
You should use pattern-matching in this case,
val args = "arg1,arg2"
val dfSelection = args.split(",").toList match {
case a1 :: tail => df.select(a1, tail: _*)
case Nil => df // well... what to do here ?
}
can you try this
var args = "arg1,arg2" //come from external and also many e.g. arg3,arg4 ...
var x = args.split(",").toList;
df.select(x.head,x.tail:_*)
Related
For example, I have 2 method definition as below with and without bracket.
def foo1: String = "bar"
def foo2():String = "bar"
In case of method parameter, I can create higher ordered functions for both as below:
def bar(fn: =>String) = ??? // bar(foo1)
def bar(fn: ()=>String) = ??? // bar(foo2)
However in case of generic type, I'm only able to use foo2.
val list: List[() => String] = List(foo2)
The following does not work.
val list: List[() => String] = List(foo1) //foo1 type mismatch.
val list: List[ => String] = List(foo1) //Invalid syntax.
So, my question is what is the correct type definition for parameterless and bracketless method?
Your first definition of bar def bar(fn: => String) = ??? does not describe a function parameter with no arguments, it describes a lazy parameter.
So if you have a function which produces a string, and you wish to defer the execution of this function, you can pass it as a lazy param into your function. This allows the function bar in this case, to either execute the function and produce the String, or to pass this code to another function. Take for example an if statement, it takes in 2 functions, and only executes 1 of them based on the predicate.
Now to answer your question, you can either use the _ symbol to turn your method into a function (eta expansion), or you can just use the lambda notation to manually make your foo an function.
def foo1 = "bar"
val list: List[() => String] = List(foo1) // crashes since foo1 is not a fn its
// just a string
val list: List[() => String] = List(() => foo1) // works
val foo1Eta = foo1 _
val list: List[() => String] = List(foo1Eta) // works
val foo2 = () => "bar"
val list: List[() => String] = List(foo2) // works
def foo3 = () => "bar"
val list: List[() => String] = List(foo3) // also works
To pass a parameterless def as a function parameter, you need to turn it into a function. You do this with this syntax:
val list: List[() => String] = List(foo1 _)
I'm trying to invoke a function using the scala reflection api in v2.11.6:
import scala.reflect.runtime.{universe => ru}
def f(i: Int) = i + 2
val fn: Any = (f _)
val ref = ru.runtimeMirror(ru.getClass.getClassLoader).reflect(fn)
val apply = ref.symbol.typeSignature.member(ru.TermName("apply"))
When using ref.reflectMethod(apply.asMethod) it complains about multiple alternatives to apply on ref. Examining apply.asTerm.alternatives reveals two methods, one with signature (x$1: Int)Int and the other with (v1: T1)R. Calling
ref.reflectMethod(apply.asTerm.alternatives(1).asInstanceOf[ru.MethodSymbol])(1)
(with the second alternative) returns the correct result (3). However calling the first alternative raises an exception: java.lang.IllegalArgumentException: object is not an instance of declaring class
What are those alternatives and how can I make sure to always invoke the proper one? There also seems to be a problem with invoking a Function2 or higher with this method, so what is the correct way to do it?
The reason why there are overloaded apply methods is that Function1 is #specialized for primitive types.
I don't know if there is a better way to distinguish them, but the following seems to work, looking for the alternative whose argument erases to AnyRef (instead of a primitive such as Int):
def invoke(fun1: Any, arg1: Any): Any = {
import scala.reflect.runtime.{universe => ru}
val mirror = ru.runtimeMirror(ru.getClass.getClassLoader)
val ref = mirror.reflect(fn)
val applies = ref.symbol.typeSignature.member(ru.TermName("apply"))
val syms = apply.alternatives.map(_.asMethod)
val sym = syms.find { m =>
m.paramLists match {
case (arg :: Nil) :: Nil
if arg.asTerm.typeSignature.erasure =:= ru.typeOf[AnyRef] => true
case _ => false
}
} getOrElse sys.error("No generic apply method found")
ref.reflectMethod(sym)(arg1)
}
// Test:
val fn: Any = (i: Int) => i + 2
invoke(fn, 1) // res: 3
I have a requirement in which I want to implicitly convert and Iterable to Option. The requirement is if the Iterable is empty I return None otherwise I return Some(iterable).
The code below
implicit def toOption[T](iterable: Iterable[T]): Option[Iterable[T]] = if (iterable.isEmpty) None else Some(iterable)
val opt = List(6,7,8).toOption
I am getting compile error 'value toOption is not a member of
List[Int]'
What am I missing here?
Implicit conversion alone is not going to help you if you want to do this. What you need here is an implicit class :
object Rich {
implicit class RichIterable[T](it: Iterable[T]){
def toOption: Option[Iterable[T]] = if(it.isEmpty) None else Some(it)
}
}
scala> import Rich._
import Rich._
scala> List(1,2,3).toOption
res0: Option[Iterable[Int]] = Some(List(1, 2, 3))
scala> List().toOption
res1: Option[Iterable[Nothing]] = None
You have the calling syntax wrong (toOption is not a method on list this way, it's a top level method/function) and you need to be less specific in your type (so it will work with implicit conversions etc):
implicit def toOption[I <% Iterable[_]](iterable: I): Option[I] = if (iterable.isEmpty) None else Some(iterable)
val opt1: Option[List[Int]] = List(6,7,8)
val opt2: Option[List[Int]] = List()
Here I used a view (<%), meaning as long as the argument can be coerced or transformed to the Iterable type, it's ok.
Also you can also write the assignment like this:
val opt2 = List(): Option[List[Int]]
And here is some documentation on views:
http://docs.scala-lang.org/tutorials/tour/views.html
Can someone explain how and when to use the triple caret ^^^ (vs the double caret ^^) when designing scala parser combinators? And also when / how to use the parser.into() method (>>).
I'll begin with an example using Scala's Option type, which is similar in some important ways to Parser, but can be easier to reason about. Suppose we have the following two values:
val fullBox: Option[String] = Some("13")
val emptyBox: Option[String] = None
Option is monadic, which means (in part) that we can map a function over its contents:
scala> fullBox.map(_.length)
res0: Option[Int] = Some(2)
scala> emptyBox.map(_.length)
res1: Option[Int] = None
It's not uncommon to care only about whether the Option is full or not, in which case we can use map with a function that ignores its argument:
scala> fullBox.map(_ => "Has a value!")
res2: Option[String] = Some(Has a value!)
scala> emptyBox.map(_ => "Has a value!")
res3: Option[String] = None
The fact that Option is monadic also means that we can apply to an Option[A] a function that takes an A and returns an Option[B] and get an Option[B]. For this example I'll use a function that attempts to parse a string into an integer:
def parseIntString(s: String): Option[Int] = try Some(s.toInt) catch {
case _: Throwable => None
}
Now we can write the following:
scala> fullBox.flatMap(parseIntString)
res4: Option[Int] = Some(13)
scala> emptyBox.flatMap(parseIntString)
res5: Option[Int] = None
scala> Some("not an integer").flatMap(parseIntString)
res6: Option[Int] = None
This is all relevant to your question because Parser is also monadic, and it has map and flatMap methods that work in very similar ways to the ones on Option. It also has a bunch of confusing operators (which I've ranted about before), including the ones you mention, and these operators are just aliases for map and flatMap:
(parser ^^ transformation) == parser.map(transformation)
(parser ^^^ replacement) == parser.map(_ => replacement)
(parser >> nextStep) == parser.flatMap(nextStep)
So for example you could write the following:
object MyParser extends RegexParsers {
def parseIntString(s: String) = try success(s.toInt) catch {
case t: Throwable => err(t.getMessage)
}
val digits: Parser[String] = """\d+""".r
val numberOfDigits: Parser[Int] = digits ^^ (_.length)
val ifDigitsMessage: Parser[String] = digits ^^^ "Has a value!"
val integer: Parser[Int] = digits >> parseIntString
}
Where each parser behaves in a way that's equivalent to one of the Option examples above.
Given the following function:
def foo()(implicit count: Int): (String => Seq[String]) = {
s => for (i <- 1 until count) yield s
}
Calling apply() on the result explicitly works:
implicit val count = 5
val x = foo().apply("x") // <- works fine
And setting the result to a val, which you then call as a function, works:
val f: String => Seq[String] = foo()
f("y") // <- works fine
But trying to do it all in one line, without apply, confuses the compiler into thinking you're passing the implicit explicitly:
val z = foo()("z") // type mismatch; found: String("z"), required: Int
Is there a way to do this without either the explicit apply or the intermediate val? For instance, is it possible somehow to move the implicit declaration into the returned anonymous function?
scala> (foo() _)("z")
res10: Seq[String] = Vector(z, z, z, z)