I may have not asked the question correctly but let say I want to use the name of whatever parameter i pass in to my function, to have my variable within that function to be named ;
def myFunc(dfName: DataFrame): Unit = {
val "{dfName}_concatenated" = dfName
}
so if i passed in myFunc(testDf) the variable inside should be named testDf_concatenated
Values and variables have to be named at compile time and this process you're looking for needs to be done at runtime, so you can not do it this way, try using a simple name and the value would be a Map("{whatever}_concatenated" -> ...).
I am really not sure what is your question, will that work for you?
def myFunc(dfName: String, anotherParam: Int): Unit = {
val dfNameConcatenated = s"${dfName}_concatenated"
}
Related
I try to write a method which should use a variable from the surrounding scope. The problem is that I cannot access the part of the code where the variable is defined. Something like this:
object Example extends App {
val myvar = 1.0 // cannot change that code
def myMethod()(implicit value:Double) = {
print(value)
}
myMethod()
}
This fails because myMethod cannot find a suitable implicit for value.
is there a way to "mark" value as implicit after it has been defined, other than defining a new implicit variable pointing to value?
Background: We use Spark-Notebook where the SparkContext (named sc) is created automatically. As sc is a commonly known name for this variable in the community, we would prefer not to introduce another variable name.
If you just don't want to have an instance of SparkContext that isn't labeled as sc you could assign it to an implicit-underscore variable like this:
implicit val _: SparkContext = sc
That would create an implicit variable of the correct type in scope for your function to use.
Alternatively you can always pass implicit variables explicitly, in your example above you could do
myMethod()(myvar)
and that is the solution that I would use.
I have defined two partial functions (hashes), which I expect to take an optional second Boolean parameter:
def SHA1 = hash(MessageDigest.getInstance("SHA-1"))_
def MD5 = hash(MessageDigest.getInstance("MD5"))_
private def hash(algorithm:HashAlgorithm)(s:String, urlencode:Boolean = false) = {
val form = if (urlencode) "%%%02X" else "%02X"
(algorithm.digest(s.getBytes) map(form format _)).mkString
}
When I call the function with both parameters, it compiles, but with just one parameter I get a compilation error:
// First 3 tests are fine
val test1 = hash(MessageDigest.getInstance("SHA-1"))("foo", true)
val test2 = hash(MessageDigest.getInstance("SHA-1"))("foo")
val test3 = SHA1("foo", true)
// not enough arguments for method apply: (v1: String, v2: Boolean)String in trait Function2. Unspecified value parameter v2.
val test4 = SHA1("foo")
I just refactored this to use partial functions, and before I refactored I could force the hash function to use the default without any problem.
Any ideas why the partial function implementation fails to permit default arguments? Am I doing something wrong using both partial functions and currying together?
When you use partial application to generate a function, you lose the ability to call the default. A method is a static thing, so the compiler knows where to look up the default value; a function can be passed around into different contexts, so the compiler will not in general have the information it needs to be able to apply the default parameter.
To think about it another way, functions only know how many arguments they have. There's just one method, apply, that you pass parameters into; otherwise you'd need some way (different types, presumably) to distinguish, for example, Function2-that-must-take-two-parameters and Function2-that-can-be-called-with-one-parameter-also-because-there-is-a-stored-default.
In scala, I have the following code:
def isDifferentGroup(artifact2: DefaultArtifact) = getArtifact(artifact1Id).getGroupId != artifact2.getGroupId
val artifacts = getArtifacts().filter(isSameGroup)
the function isDifferentGroup is accessing the external String variable artifactId (closure).
I'd like to avoid computing getArtifact(artifactId) for each item in the list.
I could do as follows:
val artifact1: DefaultArtifact = getArtifact(artifact1Id)
def isDifferentGroup(artifact2: DefaultArtifact) = artifact1.getGroupId != artifact2.getGroupId
val artifacts = getArtifacts().filter(isSameGroup)
however, we are creating a variable artifact1 outside the fonction isDifferentGroup, and that is ugly, because this variable is used only inside the fonction isDifferentGroup.
how to solve it?
one possibility would be to make a partial function as follows:
def isDifferentGroup(artifact1: DefaultArtifact)(artifact2: DefaultArtifact) = artifact1.getGroupId != artifact2.getGroupId
val artifacts = getArtifacts().filter(isDifferentGroup(getArtifact(artifact1Id)))
however, I have to move the code getArtifact(artifactId) outside the isDifferentGroup function, and I don't want this.
how to solve it?
Everything that is processed in a function body is evaluated every time the function is called, so you can't tell that a part of it will be statically evaluated and shared (without resorting to using some kind of external cache). So you have to separate the function body from values you want to evaluate in advance and use inside the function.
However, you can enclose such values in a block so that forms a compact block. The best thing I can think of is to declare a val with a function type, such as
val isDifferentGroup: DefaultArtifact => Boolean = {
val gid = getArtifact(artifact1Id).getGroupId
(artifact2: DefaultArtifact) => {
(gid != artifact2.getGroupId)
}
}
This way, you can explicitly state what part is evaluated statically only once in the main val block (here gid) and what part is evaluated in response to an artifact2 argument. And you can call isDifferentGroup just as if it were a method:
println(isDifferentGroup(someArtifact))
This is basically just a different way of creating an encapsulating class like
val isDifferentGroup: DefaultArtifact => Boolean =
new Function1[DefaultArtifact,Boolean] {
val gid = getArtifact(artifact1Id).getGroupId
override def apply(artifact2: DefaultArtifact): Boolean =
(gid != artifact2.getGroupId);
}
You can even declare it as a lazy val, in which case gid is evalauted at most once, the first time the function is called.
Why not just create a class to encapsulate both a (private) value and functionality?
This code will probably not compile, just to illustrate it:
class Artifact(artifact1Id: Id) {
private val artifact1: DefaultArtifact = getArtifact(artifact1Id)
def isDifferentGroup(artifact2: DefaultArtifact) =
artifact1.getGroupId != artifact2.getGroupId
}
I have a class which takes a function
case class FunctionParser1Arg[T, U](func:(T => U))
def testFunc(name1:String):String = name1
val res = FunctionParser1Arg(testFunc)
I would like to know the type signature information on the function from inside the case class. I want to know both the parameter name and type. I have had success in finding the type using the runtime mirror objects, but not the name. Any suggestions?
Ok, let's say you got the symbol for the instance func points to:
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{currentMirror => m}
val im = m reflect res.func // Instance Mirror
You can get the apply method from its type members:
val apply = newTermName("apply")
val applySymbol = im.symbol.typeSignature member apply
And since we know it's a method, make it a method symbol:
val applyMethod = applySymbol.asMethod
It's parameters can be found through paramss, and we know there's only one parameter on one parameter list, so we can get the first parameter of the first parameter list:
val param = applyMethod.paramss(0)(0)
Then what you are asking for is:
val name = param.name.decoded // if you want "+" instead of "$plus", for example
val type = param.typeSignature
It's possible that you think that's the wrong answer because you got x$1 instead of name1, but what is passed to the constructor is not the named function testFunc, but, instead, an anonymous function representing that method created through a process called eta expansion. You can't find out the parameter name of the method because you can't pass the method.
If that's what you need, I suggest you use a macro instead. With a macro, you'll be able to see exactly what is being passed at compile time and get the name from it.
I would like to write a class looking like this:
class Store[+A](dest: Symbol)(implicit c: String => A) extends Action(dest) {
override def update(options: HashMap[Symbol,Any], arg: String): Unit = {
options += ((dest -> c(arg)))
}
}
object Store {
def apply[A](dest: Symbol)(c: String=>A) = new Store[A](dest)(c)
def apply[A](dest: Symbol) = new Store[A](dest)
}
Doing so, I have a few problems:
Using implicit with strings causes no end of trouble
Anyway, the system doesn't find the implicit if they are defined in my module, they would need to be defined in the module creating the class
the second apply method of the Store object just doesn't compile as A will be erased so the compiler has no way of finding a conversion from String to A
How would you create such an object that convert a string to some other type? I wouldn't want the user of the library to enter the type rwice (i.e. by specifying both the type and the conversion function).
I don't understand what you are trying with the second apply. To me, it looks like the first apply should have the implicit keyword, and you'd be done with it. You can either pass the parameter explicitly, or leave it out if an implicit is present. Also, you wouldn't need to pass c explicitly, since you'd already have it implicitly in the scope of the first apply.
I'd venture the second apply doesn't compile because there's no implicit String => A available in the scope of object Store.