When I use :load in the spark-shell it appears as though lines are read separately and thus companion objects are not read in the same "source" file. :paste does not appear to take arguments.
Previously I was building and loading a jar with my code into the spark-shell but was hoping to run it as a script for simplicity. Does anyone have a favorite workaround?
A sufficiently recent shell will have :paste file.
Or, as a workaround, link the templates this way to :load them:
class C(i: Int) {
def c = { println("C..."); i }
}; object C {
def apply(i: Int = 42) = new C(i)
}
Or,
scala> (new $intp.global.Run) compile List("C.scala")
scala> new C().c
C...
res1: Int = 42
More API:
scala> import reflect.io._
import reflect.io._
scala> import reflect.internal.util._
import reflect.internal.util._
scala> val code = File("C.scala").slurp
code: String =
"
class C(i: Int) { def c = { println("C..."); i } }
object C { def apply(i: Int = 42) = new C(i) }
"
scala> $intp interpret code
defined class C
defined object C
res0: scala.tools.nsc.interpreter.IR.Result = Success
scala> C()
res1: C = C#f2f2cc1
Similarly,
scala> $intp interpret s"object X { $code }"
defined object X
res0: scala.tools.nsc.interpreter.IR.Result = Success
scala> X.C()
res1: X.C = X$C#7d322cad
My startup script defines:
implicit class `interpreter interpolator`(val sc: StringContext) { def i(args: Any*) = $intp interpret sc.s(args: _*) }
for
scala> i"val x = 42"
x: Int = 42
res0: scala.tools.nsc.interpreter.IR.Result = Success
This compile trick doesn't appear to work with "script" files. It expects a source file compilable by scalac, where all val and def declarations are inside a type.
So, an alternative hack that will work with :load is to write the case class and companion object inside another object. Here I just pasted the code, without using :paste, but it works with :load, too.
scala> object O {
| case class C(s: String)
| object C {
| def apply() = new C("<no string>")
| }
| }
defined module O
scala> O.C()
res0: O.C = C(<no string>)
Related
In scala def is used to define a method and val, var are used for defining variables.
Consider the following code:
scala> def i = 3
i: Int
scala> i.getClass()
res0: Class[Int] = int
scala> val v = 2
v: Int = 2
scala> v.getClass()
res1: Class[Int] = int
scala> println(v)
2
scala> println(i)
3
scala> i+v
res4: Int = 5
scala> def o = () => 2+3
o: () => Int
scala> o.getClass()
res5: Class[_ <: () => Int] = class $$Lambda$1139/1753607449
Why does variable definition work using def? If it is defining a function that returns an Int then why does getClass show Int instead of a function object?
Unlike val or var declaration, def i = 3 is not variable declaration. You are defining a method/function which returns a constant 3 and i does not take any parameters.
declaration using val and var get evaluated immediately but in case of lazy val and def evaluation happens when called explicitly.
i is a not argument function. In order to get rid of confusion you could declare it using empty parenthesis as well
def i() = 3
Difference between lazy val and def is
lazy val is lazily evaluated and the result is cached. That means further
def declaration is evaluated every time you call method name.
Example using Scala REPL
scala> lazy val a = { println("a evaluated"); 1}
a: Int = <lazy>
scala> def i = { println("i function evaluated"); 2}
i: Int
scala> a
a evaluated
res0: Int = 1
scala> a
res1: Int = 1
scala> a
res2: Int = 1
scala> i
i function evaluated
res3: Int = 2
scala> i
i function evaluated
res4: Int = 2
scala> i
i function evaluated
res5: Int = 2
Notice that a is evaluated only once and further invocations of a return the cached result i.e lazy val is evaluated once when it is called and the result is stored forever. So you see println output once
Notice function is evaluated every time it is invoked. In this case you see println output every time you invoke the function
General Convention
There's a convention of using an empty parameter list when the method has side effects and leaving them off when its pure.
edited
scala> def i = 1
i: Int
scala> :type i
Int
scala> :type i _
() => Int
EDIT: My answer addresses the question of revision #3.
It is quite useful to look on the code in the middle of the compilation process where you can look on what your code is actually translated to. The following simple program:
object TestApp {
def definedVal = 3
val valVal = 3
lazy val lazyValVal = 3
def main(args: Array[String]) {
println(definedVal)
println(valVal)
println(lazyValVal)
}
}
is translated to the following (using -Xprint:mixin compiler option):
[[syntax trees at end of mixin]] // test.scala
package <empty> {
object TestApp extends Object {
#volatile private[this] var bitmap$0: Boolean = false;
private def lazyValVal$lzycompute(): Int = {
{
TestApp.this.synchronized({
if (TestApp.this.bitmap$0.unary_!())
{
TestApp.this.lazyValVal = 3;
TestApp.this.bitmap$0 = true;
()
};
scala.runtime.BoxedUnit.UNIT
});
()
};
TestApp.this.lazyValVal
};
def definedVal(): Int = 3;
private[this] val valVal: Int = _;
<stable> <accessor> def valVal(): Int = TestApp.this.valVal;
lazy private[this] var lazyValVal: Int = _;
<stable> <accessor> lazy def lazyValVal(): Int = if (TestApp.this.bitmap$0.unary_!())
TestApp.this.lazyValVal$lzycompute()
else
TestApp.this.lazyValVal;
def main(args: Array[String]): Unit = {
scala.this.Predef.println(scala.Int.box(TestApp.this.definedVal()));
scala.this.Predef.println(scala.Int.box(TestApp.this.valVal()));
scala.this.Predef.println(scala.Int.box(TestApp.this.lazyValVal()))
};
def <init>(): TestApp.type = {
TestApp.super.<init>();
TestApp.this.valVal = 3;
()
}
}
}
From the output above it is possible to conclude the following:
definedVal is actually a method.
valVal is a field which is initialized in the constructor and has an automatically generated accessor.
For the lazy field lazyValVal compiler generates compute method which is called only once when the field is accessed the first time.
There is few different concept. Call by name, call by value and call by need. All def is essentially calls by name. What do you mean variable definition using def ??
Looks like duplicate to me:
Call by name vs call by value in Scala, clarification needed
More details in wiki: https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name
consider the following code:
object Foo{def foo(a:Int):List[(String, Int)] = ???}
class Bar{def bar(a:Int, b:Any):Option[(String, Long)] = ???}
Given either the object or class, I need to first find the method names (does not seem to be that difficult).
After that, for each method, I want to find a string description of the Scala return types (not the Java ones) . So for instance, for Foo.foo, I would need the String List[(String, Int)] and for Bar.bar, I would need the String Option[(String, Long)].
I saw this and this tutorial but could not figure it out.
EDIT: Here is what I tried based on the comments:
class RetTypeFinder(obj:AnyRef) {
import scala.reflect.runtime.{universe => ru}
val m = ru.runtimeMirror(getClass.getClassLoader)
val im = m.reflect(obj)
def getRetType(methodName:String) = {
ru.typeOf[obj.type].declaration(ru.TermName(methodName)).asMethod.returnType
}
}
object A { def foo(a:Int):String = ??? } // define dummy object
class B { def bar(a:Int):String = ??? } // define dummy class
val a = new RetTypeFinder(A)
a.getRetType("foo") // exception here
val b = new RetTypeFinder(new B)
b.getRetType("bar") // exception here
The error I get is:
scala.ScalaReflectionException: <none> is not a method
at scala.reflect.api.Symbols$SymbolApi$class.asMethod(Symbols.scala:228)
at scala.reflect.internal.Symbols$SymbolContextApiImpl.asMethod(Symbols.scala:84)
at cs.reflect.Test.getRetCls(Test.scala:11)
...
However, this works (tried in REPL):
import scala.reflect.runtime.{universe => ru}
val m = ru.runtimeMirror(getClass.getClassLoader)
object A { def foo(a:Int):String = ??? } // define dummy object
val im = m.reflect(A)
ru.typeOf[A.type].declaration(ru.TermName("foo")).asMethod.returnType
class B { def bar(a:Int):String = ??? } // define dummy class
val im = m.reflect(new B)
ru.typeOf[B].declaration(ru.TermName("bar")).asMethod.returnType
I need to use it in the first way, where I don't know in advance what objects/classes will be passed. Any help will be appreciated.
Once you have a universe.Type you can use the way from the comments to get the return type of one of its methods:
import scala.reflect.runtime.{universe => ru}
def getRetTypeOfMethod(tpe: ru.Type)(methodName: String) =
tpe.member(ru.TermName(methodName)).asMethod.returnType
To get a universe.Type the easiest way is to capture in an implicit TypeTag:
class RetTypeFinder[T <: AnyRef](obj: T)(implicit tag: ru.TypeTag[T]) {
def getRetType(methodName: String) = {
val tpe = tag.tpe
getRetTypeOfMethod(tpe)(methodName)
}
}
But if you don't have a TypeTag, but just an object of type AnyRef, you can go through a mirror to reflect it. The resulting Type will have some information lost due to Java's type erasure, but it would still be enough to get the return type of a method by name, because that's supported by JVM reflection:
class RetTypeFinder2(obj: AnyRef) {
def getRetType(methodName: String) = {
val mirror = ru.runtimeMirror(getClass.getClassLoader)
val tpe = mirror.reflect(obj).symbol.info
getRetTypeOfMethod(tpe)(methodName)
}
}
Both methods work fine for your problem:
scala> new RetTypeFinder(A).getRetType("foo")
res0: reflect.runtime.universe.Type = String
scala> new RetTypeFinder2(A).getRetType("foo")
res1: reflect.runtime.universe.Type = String
scala> new RetTypeFinder(new B).getRetType("bar")
res2: reflect.runtime.universe.Type = String
scala> new RetTypeFinder2(new B).getRetType("bar")
res3: reflect.runtime.universe.Type = String
What does the compiler see differently in each class declaration below that causes typeOf to behave differently. (i.e. what's the property of the class declaration that causes it to fail).
import org.junit.Test
import scala.reflect.runtime.universe._
case class Person1(name: String)
class ReflectTest {
case class Person2(name: String)
#Test
def constructorTest(): Unit = {
case class Person3(name: String)
typeOf[Person1] // Yep
typeOf[Person2] // No problem
typeOf[Person3] // No typetag information available :(
}
}
It's a free type. And you can't refer to it with a prefix.
scala> import reflect.runtime.universe._
import reflect.runtime.universe._
scala> class C { class D }
defined class C
scala> val c = new C
c: C = C#29626d54
scala> val x = { final class X ; weakTypeTag[X] }
x: reflect.runtime.universe.WeakTypeTag[_ <: AnyRef] = WeakTypeTag[X]
scala> val t = typeTag[C]
t: reflect.runtime.universe.TypeTag[C] = TypeTag[C]
scala> val w = typeTag[c.D]
w: reflect.runtime.universe.TypeTag[c.D] = TypeTag[c.D]
scala> val v = typeTag[C#D]
v: reflect.runtime.universe.TypeTag[C#D] = TypeTag[C#D]
scala> (t.tpe.typeSymbol, t.tpe.typeSymbol.isStatic)
res1: (reflect.runtime.universe.Symbol, Boolean) = (class C,true)
scala> (x.tpe.typeSymbol, x.tpe.typeSymbol.isStatic)
res2: (reflect.runtime.universe.Symbol, Boolean) = (free type X,false)
scala> (w.tpe.typeSymbol, w.tpe.typeSymbol.isStatic)
res3: (reflect.runtime.universe.Symbol, Boolean) = (class D,false)
scala> (v.tpe.typeSymbol, v.tpe.typeSymbol.isStatic)
res4: (reflect.runtime.universe.Symbol, Boolean) = (class D,false)
scala> reflect.runtime.universe.internal.asFreeType(x.tpe.typeSymbol)
res5: reflect.runtime.universe.FreeTypeSymbol = free type X
Other things you can't do with local classes, as mentioned in the spec:
scala> val x = { private class X ; weakTypeTag[X] }
<console>:1: error: illegal start of statement (no modifiers allowed here)
val x = { private class X ; weakTypeTag[X] }
^
scala> import c.D
import c.D
I'd look more but it's late and my REPL broke...
scala> val y = { final class X { class Y } ; val x = new X ; import x.Y ; weakTypeTag[Y] }
warning: there was one feature warning; re-run with -feature for details
java.lang.AssertionError: assertion failed: x.type
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.adaptToNewRun(TypeMaps.scala:1106)
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.apply(TypeMaps.scala:1150)
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.apply(TypeMaps.scala:1079)
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.apply(TypeMaps.scala:1148)
at scala.reflect.internal.tpe.TypeMaps$TypeMap.mapOver(TypeMaps.scala:162)
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.apply(TypeMaps.scala:1197)
at scala.reflect.internal.tpe.TypeMaps$adaptToNewRunMap$.apply(TypeMaps.scala:1171)
at scala.reflect.internal.Symbols$Symbol.adaptInfos(Symbols.scala:1629)
at scala.reflect.internal.Symbols$Symbol.rawInfo(Symbols.scala:1581)
at scala.tools.nsc.typechecker.Typers$Typer.isStale(Typers.scala:504)
at scala.tools.nsc.typechecker.Typers$Typer.reallyExists(Typers.scala:496)
at scala.tools.nsc.typechecker.Typers$Typer.typedSelectInternal$1(Typers.scala:4712)
at scala.tools.nsc.typechecker.Typers$Typer.typedSelect$1(Typers.scala:4676)
I'm new to the reflection API.
I'd like to get a reference to an object from its name. I've made it to the point where I can get a reference using the class name of the object.
$ scala
Welcome to Scala version 2.11.7 ...
scala> case object Foo { val x = 5 }
defined object Foo
scala> import scala.reflect.runtime.{universe => ru}
import scala.reflect.runtime.{universe=>ru}
scala> val m = ru.runtimeMirror(getClass.getClassLoader)
m: reflect.runtime.universe.Mirror...
scala> val f = m.reflectModule(m.staticModule(Foo.getClass.getName)).instance.asInstanceOf[Foo.type]
f: Foo.type = Foo
scala> f.x
res0: Int = 5
Works just fine. However, trying to use the computed type name as a string doesn't work:
scala> m.staticModule(Foo.getClass.getName)
res2: reflect.runtime.universe.ModuleSymbol = object iw$Foo$
scala> Foo.getClass.getName
res1: String = Foo$
scala> m.staticModule("Foo$")
scala.ScalaReflectionException: object Foo$ not found.
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:162)
at scala.reflect.internal.Mirrors$RootsBase.staticModule(Mirrors.scala:22)
... 33 elided
What am I missing here? Thanks.
This problem appears in REPL only. Try the following in REPL:
scala> Foo.getClass.getName.length
res5: Int = 25
So, the 'Foo$' is not the full name of class Foo
scala> new String(Foo.getClass.getName.getBytes("UTF-8").map(b => if(b==36) '?'.toByte else b), "UTF-8")
res6: String = ?line3.?read??iw??iw?Foo?
And you can call with no problem:
scala>m.staticModule("$line3.$read$$iw$$iw$Foo$")
res7: reflect.runtime.universe.ModuleSymbol = object iw$Foo$
See also: https://issues.scala-lang.org/browse/SI-9335
How do I avoid the initialization (lines 5 and 6) here?
import scala.collection._
def newHash = mutable.Map[String,String]()
def newHoH = mutable.Map[String,mutable.Map[String,String]]()
var foo = mutable.Map[String,mutable.Map[String,mutable.Map[String,String]]]()
foo("bar") = newHoH //line 5
foo("bar")("baz") = newHash //line 6
foo("bar")("baz")("whee") = "duh"
I tried withDefaultValue with a simpler example but obviously I did it wrong:
/***
scala> var foo = mutable.Map[String,mutable.Map[String,String]]().withDefaultValue(mutable.Map(""->""))
foo: scala.collection.mutable.Map[String,scala.collection.mutable.Map[String,String]] = Map()
scala> foo("bar")("baz") = "duh"
scala> foo("b")("baz") = "der"
scala> foo("bar")("baz")
res7: String = der
*/
The withDefault method won't work here. A Map created this way returns a new map everytime there is no key, so calling mymap("foo")("bar") = "ok" will assign "ok" into a temporarily created map, but the next time you call mymap("foo")("bar"), a non-existent "foo" key on mymap will result in creating a new map, which will not contain the mapping "foo" -> "bar".
Instead, consider creating an anonymous map. I show a solution with only 1 nestings:
‡ scala-version 2.10.1
Welcome to Scala version 2.10.1 (Java HotSpot(TM) Server VM, Java 1.7.0_21).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import collection._
import collection._
scala> :paste
// Entering paste mode (ctrl-D to finish)
def newHash = mutable.Map[String,String]().withDefault(_ => "")
def newHoH = new mutable.Map[String,mutable.Map[String,String]]() {
val m = mutable.Map[String, mutable.Map[String, String]]()
def +=(kv: (String, mutable.Map[String, String])) = { m += kv; this }
def -=(k: String) = { m -= k; this }
def get(k: String) = m.get(k) match {
case opt # Some(v) => opt
case None =>
val v = newHash
m(k) = v
Some(v)
}
def iterator = m.iterator
}
// Exiting paste mode, now interpreting.
newHash: scala.collection.mutable.Map[String,String]
newHoH: scala.collection.mutable.Map[String,scala.collection.mutable.Map[String,String]]{val m: scala.collection.mutable.Map[String,scala.collection.mutable.Map[String,String]]}
scala> val m = newHoH
m: scala.collection.mutable.Map[String,scala.collection.mutable.Map[String,String]]{val m: scala.collection.mutable.Map[String,scala.collection.mutable.Map[String,String]]} = Map()
scala> m("foo")("bar") = "ok"
scala> m("foo")("bar")
res1: String = ok