What kind of constructs need 'scalac' compile and how to make an equivalent that will work in interpreter?
Edit: I want to use scala instead of python as a scripting language. (with #!/usr/bin/scala)
You ought to be able to do anything in the REPL that you can do in outside code. Keep in mind that:
Things that refer to each other circularly need to be inside one block. So the following can't be entered as-is; you have to wrap it inside some other object:
class C(i : Int) { def succ = C(i+1) }
object C { def apply(i: Int) = new C(i) }
The execution environment is somewhat different, so benchmarking timings will not always come out the same way as if you run them from compiled code.
You enter the execution path a different way; if you want to call a main method, though, you certainly can from inside the REPL.
You can't just cut-and-paste an entire library into the REPL and have it work exactly like the library did; the REPL has a different structure than normal packages do. So drop the "package" declarations during testing.
EDIT: after reading your question again, I have to admit, that I didn't really answer it ;). But maybe it still helps.
I know of two limitations of interpreter (or REPL), when it comes to loading scala files (in order to interactively test them).
You can't load scala files with package definitions in them. REPL complains and does not load all class is the scala file to be loaded. It read that it has to do with the fact that files that are loaded into the REPL are treated as an object . . . which of course can't have any package definitions in them.
REPL is strange (or a little bit unpredictable) when there are class files of loaded scala files on the classpath. Check out this question by myself and especially my 2 last comments to the second answer.
Furthermore there is a problem with circular dependencies, that I don't know a workaround for: Suppose there is a Class A that uses Class B which again needs A to do it's job. Of course you can't define A since there is no definition of B and vice versa. Providing a dummy for one of those doesn't work either:
scala> class A {
| def alterString(s:String) = s
| def printStuff(s:String) = println(alterString(s))
| }
defined class A
scala> class B {
| val prefix = "this is a test: "
| def doJob() = new A() printStuff "1 2 3"
| }
defined class B
scala> class A {
| def alterString(s:String) = new B().prefix + s
| def printStuff(s:String) = println(alterString(s))
| }
defined class A
scala> new B().doJob()
1 2 3
scala>
Although I already provided a newer definition of A, class B still used the one that was present when I defined it.
I think everything in Scala will work from the interpreter as well (which I believe just calls the compiler under the hood anyway).
Do you suspect something to not work? Or is this a trick/interview question (I suppose anything that want to directly interact with the class loader or class files may behave differently in the two environments, but I have no idea really).
Related
I would like to extract from a given Scala project, the call graph of all methods which are part of the project's own source.
As I understand, the presentation compiler doesn't enable that, and it requires going down all the way down to the actual compiler (or a compiler plugin?).
Can you suggest complete code, that would safely work for most scala projects but those that use the wackiest dynamic language features? for the call graph, I mean a directed (possibly cyclic) graph comprising class/trait + method vertices where an edge A -> B indicates that A may call B.
Calls to/from libraries should be avoided or "marked" as being outside the project's own source.
EDIT:
See my macro paradise derived prototype solution, based on #dk14's lead, as an answer below. Hosted on github at https://github.com/matanster/sbt-example-paradise.
Here's the working prototype, which prints the necessary underlying data to the console as a proof of concept. http://goo.gl/oeshdx.
How This Works
I have adapted the concepts from #dk14 on top boilerplate from macro paradise.
Macro paradise lets you define an annotation that will apply your macro over any annotated object in your source code. From there you have access to the AST that the compiler generates for the source, and scala reflection api can be used to explore the type information of the AST elements. Quasiquotes (the etymology is from haskell or something) are used to match the AST for the relevant elements.
More about Quasiquotes
The generally important thing to note is that quasiquotes work over an AST, but they are a strange-at-first-glance api and not a direct representation of the AST (!). The AST is picked up for you by paradise's macro annotation, and then quasiquotes are the tool for exploring the AST at hand: you match, slice and dice the abstract syntax tree using quasiquotes.
The practical thing to note about quasiquotes is that there are fixed quasiquote templates for matching each type of scala AST - a template for a scala class definition, a template for a scala method definition, etc. These tempaltes are all provided here, making it very simple to match and deconstruct the AST at hand to its interesting constituents. While the templates may look daunting at first glance, they are mostly just templates mimicking the scala syntax, and you may freely change the $ prepended variable names within them to names that feel nicer to your taste.
I still need to further hone the quasiquote matches I use, which currently aren't perfect. However, my code seems to produce the desired result for many cases, and honing the matches to 95% precision may be well doable.
Sample Output
found class B
class B has method doB
found object DefaultExpander
object DefaultExpander has method foo
object DefaultExpander has method apply
which calls Console on object scala of type package scala
which calls foo on object DefaultExpander.this of type object DefaultExpander
which calls <init> on object new A of type class A
which calls doA on object a of type class A
which calls <init> on object new B of type class B
which calls doB on object b of type class B
which calls mkString on object tags.map[String, Seq[String]](((tag: logTag) => "[".+(Util.getObjectName(tag)).+("]")))(collection.this.Seq.canBuildFrom[String]) of type trait Seq
which calls map on object tags of type trait Seq
which calls $plus on object "[".+(Util.getObjectName(tag)) of type class String
which calls $plus on object "[" of type class String
which calls getObjectName on object Util of type object Util
which calls canBuildFrom on object collection.this.Seq of type object Seq
which calls Seq on object collection.this of type package collection
.
.
.
It is easy to see how callers and callees can be correlated from this data, and how call targets outside the project's source can be filtered or marked out. This is all for scala 2.11. Using this code, one will need to prepend an annotation to each class/object/etc in each source file.
The challenges that remain are mostly:
Challenges remaining:
This crashes after getting the job done. Hinging on https://github.com/scalamacros/paradise/issues/67
Need to find a way to ultimately apply the magic to entire source files without manually annotating each class and object with the static annotation. This is rather minor for now, and admittedly, there are benefits for being able to control classes to include and ignore anyway. A preprocessing stage that implants the annotation before (almost) every top level source file definition, would be one nice solution.
Honing the matchers such that all and only relevant definitions are matched - to make this general and solid beyond my simplistic and cursory testing.
Alternative Approach to Ponder
acyclic brings to mind a quite opposite approach that still sticks to the realm of the scala compiler - it inspects all symbols generated for the source, by the compiler (as much as I gather from the source). What it does is check for cyclic references (see the repo for a detailed definition). Each symbol supposedly has enough information attached to it, to derive the graph of references that acyclic needs to generate.
A solution inspired by this approach may, if feasible, locate the parent "owner" of every symbol rather than focus on the graph of source files connections as acyclic itself does. Thus with some effort it would recover the class/object ownership of each method. Not sure if this design would not computationally explode, nor how to deterministically obtain the class encompassing each symbol.
The upside would be that there is no need for macro annotations here. The downside is that this cannot sprinkle runtime instrumentation as the macro paradise rather easily allows, which could be at times useful.
It requires more precise analysis, but as a start this simple macro will print all possible applyies, but it requires macro-paradise and all traced classess should have #trace annotation:
class trace extends StaticAnnotation {
def macroTransform(annottees: Any*) = macro tracerMacro.impl
}
object tracerMacro {
def impl(c: Context)(annottees: c.Expr[Any]*): c.Expr[Any] = {
import c.universe._
val inputs = annottees.map(_.tree).toList
def analizeBody(name: String, method: String, body: c.Tree) = body.foreach {
case q"$expr(..$exprss)" => println(name + "." + method + ": " + expr)
case _ =>
}
val output = inputs.head match {
case q"class $name extends $parent { ..$body }" =>
q"""
class $name extends $parent {
..${
body.map {
case x#q"def $method[..$tt] (..$params): $typ = $body" =>
analizeBody(name.toString, method.toString, body)
x
case x#q"def $method[..$tt]: $typ = $body" =>
analizeBody(name.toString, method.toString, body)
x
}
}
}
"""
case x => sys.error(x.toString)
}
c.Expr[Any](output)
}
}
Input:
#trace class MyF {
def call(param: Int): Int = {
call2(param)
if(true) call3(param) else cl()
}
def call2(oaram: Int) = ???
def cl() = 5
def call3(param2: Int) = ???
}
Output (as compiler's warnings, but you may output to file intead of println):
Warning:scalac: MyF.call: call2
Warning:scalac: MyF.call: call3
Warning:scalac: MyF.call: cl
Of course, you might want to c.typeCheck(input) it (as now expr.tpe on found trees is equals null) and find which class this calling method belongs to actually, so the resulting code may not be so trivial.
P.S. macroAnnotations give you unchecked tree (as it's on earlier compiler stage than regular macroses), so if you want something typechecked - the best way is surround the piece of code you want to typecheck with call of some your regular macro, and process it inside this macro (you can even pass some static parameters). Every regular macro inside tree produced by macro-annotation - will be executed as usual.
Edit
The basic idea in this answer was to bypass the (pretty complex) Scala compiler completely, and extract the graph from the generated .class files in the end. It appeared that a decompiler with sufficiently verbose output could reduce the problem to basic text manipulation. However, after a more detailed examination it turned out that this is not the case. One would just get back to square one, but with obfuscated Java code instead of the original Scala code. So this proposal does not really work, although there is some rationale behind working with the final .class files instead of intermediate structures used internally by the Scala compiler.
/Edit
I don't know whether there are tools out there that do it out of the box (I assume that you have checked that). I have only a very rough idea what the presentation compiler is. But if all that you want is to extract a graph with methods as nodes and potential calls of methods as edges, I have a proposal for a quick-and-dirty solution. This would work only if you want to use it for some sort of visualization, it doesn't help you at all if you want to perform some clever refactoring operations.
In case that you want to attempt building such a graph-generator yourself, it might turn out much simpler than you think. But for this, you need to go all the way down, even past the compiler. Just grab your compiled .class files, and use something like the CFR java decompiler on it.
When used on a single compiled .class file, CFR will generate list of classes that the current class depends on (here I use my little pet project as example):
import akka.actor.Actor;
import akka.actor.ActorContext;
import akka.actor.ActorLogging;
import akka.actor.ActorPath;
import akka.actor.ActorRef;
import akka.actor.Props;
import akka.actor.ScalaActorRef;
import akka.actor.SupervisorStrategy;
import akka.actor.package;
import akka.event.LoggingAdapter;
import akka.pattern.PipeToSupport;
import akka.pattern.package;
import scala.Function1;
import scala.None;
import scala.Option;
import scala.PartialFunction;
...
(very long list with all the classes this one depends on)
...
import scavenger.backend.worker.WorkerCache$class;
import scavenger.backend.worker.WorkerScheduler;
import scavenger.backend.worker.WorkerScheduler$class;
import scavenger.categories.formalccc.Elem;
Then it will spit out some horribly looking code, that might look like this (small excerpt):
public PartialFunction<Object, BoxedUnit> handleLocalResponses() {
return SimpleComputationExecutor.class.handleLocalResponses((SimpleComputationExecutor)this);
}
public Context provideComputationContext() {
return ContextProvider.class.provideComputationContext((ContextProvider)this);
}
public ActorRef scavenger$backend$worker$MasterJoin$$_master() {
return this.scavenger$backend$worker$MasterJoin$$_master;
}
#TraitSetter
public void scavenger$backend$worker$MasterJoin$$_master_$eq(ActorRef x$1) {
this.scavenger$backend$worker$MasterJoin$$_master = x$1;
}
public ActorRef scavenger$backend$worker$MasterJoin$$_masterProxy() {
return this.scavenger$backend$worker$MasterJoin$$_masterProxy;
}
#TraitSetter
public void scavenger$backend$worker$MasterJoin$$_masterProxy_$eq(ActorRef x$1) {
this.scavenger$backend$worker$MasterJoin$$_masterProxy = x$1;
}
public ActorRef master() {
return MasterJoin$class.master((MasterJoin)this);
}
What one should notice here is that all methods come with their full signature, including the class in which they are defined, for example:
Scheduler.class.schedule(...)
ContextProvider.class.provideComputationContext(...)
SimpleComputationExecutor.class.fulfillPromise(...)
SimpleComputationExecutor.class.computeHere(...)
SimpleComputationExecutor.class.handleLocalResponses(...)
So if you need a quick-and-dirty solution, it might well be that you could get away with just ~10 lines of awk,grep,sort and uniq wizardry to get nice adjacency lists with all your classes as nodes and methods as edges.
I've never tried it, it's just an idea. I cannot guarantee that Java decompilers work well on Scala code.
I know programmers are supposed to wrap their code in an application object:
object Hello extends App {
println("Hello, World")
}
It is required in Eclipse, if I ever want to get any output. However, when I tried to write some code (very casually) in Emacs, I write like this:
class Pair[+T](val first: T, val second: T)
trait Friend[-T] {
def befriend(someone: T)
}
def makeFriendWith(s: Student, f: Friend[Student]) {
f.befriend(s)
}
It seems like there is no universal object or class that wraps over the function makeFriendWith. Is Scala like JavaScript, everything is attached to a global object? If not, what is this function attached to?
Also why can this work in console (I complied it with scala command and it worked) but does not work in Eclipse? What's the use of the Application object?
Scala doesn't have top-level defs, but your script can be run by either the REPL or the scala script runner.
The precise behavior of your script depends on which way you run it.
The REPL can run scripts line-by-line or whole hog. (Compare :paste and :paste -raw versus :load or -i init.script and the future option -I init.script.)
There is an issue about sensitive scripting. The script runner should realize if you're trying to run an App.
There is another effort to make scripting a compiler phase that is easily customized. Scroll to Scripter.scala for code comments about its current heuristics.
In short, your defs must be wrapped in a top-level entity, but exactly how that happens is context-dependent.
There was a recent effort to make an alternative baked-in wrapping scheme available for the REPL.
None of this is mandated by the language spec, any more than special rules pertaining to sbt build files are defined by the language.
You can define methods like this only in the console, which (behind the scenes) automatically wraps them in an anonymous class for you.
Outside of the console, there's no such luxury.
As a JVM language, Scala cannot truly create any top-level entities other than classes and interfaces.
It does, however, have the notion of a "package object" which creates the illusion of value entites (val, var and def) not enclosed in a class or trait.
See http://www.scala-lang.org/docu/files/packageobjects/packageobjects.html for information on package objects.
You can run code like this directly in Eclipse: use Scala worksheet. IntelliJ IDEA Scala plugin supports it as well.
With intention of reducing the boilerplate for the end-user when deriving instances of a certain typeclass (let's take Showable for example), I aim to write a macro, which will autogenerate the instance names. E.g.:
// calling this:
Showable.derive[Int]
Showable.derive[String]
// should expand to this:
// implicit val derivedShowableInstance0 = new Showable[Int] { ... }
// implicit val derivedShowableInstance1 = new Showable[String] { ... }
I tried to approach the problem the following way, but the compiler complained that the expression should return a < no type > instead of Unit:
object Showable {
def derive[a] = macro Macros.derive[a]
object Macros {
private var instanceNameIndex = 0
def derive[ a : c.WeakTypeTag ]( c : Context ) = {
import c.universe._
import Flag._
val newInstanceDeclaration = ...
c.Expr[Unit](
ValDef(
Modifiers(IMPLICIT),
newTermName("derivedShowableInstance" + nameIndex),
TypeTree(),
newInstanceDeclaration
)
)
}
}
}
I get that a val declaration is not exactly an expression and hence Unit might not be appropriate, but then how to make it right?
Is this at all possible? If not then why, will it ever be, and are there any workarounds?
Yes, that's right. Declarations/definitions aren't expressions, so they need to be wrapped into Unit-returning blocks to become ones. Typically Scala does this automatically, but in this particular case you need to do it yourself.
However if you wrap a definition in a block, then it becomes invisible from the outside. That's the limitation of the current macro system that strictly follows the metaphor of "macro application is very much similar to a typed function call". Function calls don't bring new members into scope, so neither do def macros - both for technical and philosophical reasons. As shown in the example #3 of my recent "What Are Macros Good For?" talk, by using structural types def macros can work around this limitation, but this doesn't look particularly related to your question, so I'll omit the details.
On the other hand, there are some ideas how to overcome this limitation with new macro flavors. Macro annotations show that it's technically feasible for the macro engine to hook into new member creation, but we'd like to get more experience with macro annotations before bringing them into trunk. Some details about this can be found in my "Philosophy of Scala Macros" presentation. This can be useful in your situation, but I still won't go into details, because I think there's a much better solution for this particular case (feel free to ask for elaboration in comments, though!).
What I'd like to recommend you is to use materialization as described in http://docs.scala-lang.org/overviews/macros/implicits.html. With materializing macros you can automatically generate type class instances for the user without having the user write any code at all. Would that fit your use case?
I've been working with Scala for a while now and have written a 10,000+ line program with it, but I'm still confused by some of the inner workings. I came to Scala from Python after already having intimate familiarity with Java, C and Lisp, but even so it's been slow going, and a huge problem is the frustrating difficulty I've often found when trying to investigate the inner workings of objects/types/classes/etc. using the Scala REPL as compared with Python. In Python you can investigate any object foo (type, object in a global variable, built-in function, etc.) using foo to see what the thing evaluates to, type(foo) to show its type, dir(foo) to tell you the methods you can call on it, and help(foo) to get the built-in documentation. You can even do things like help("re") to find out the documentation on the package named re (which holds regular-expression objects and methods), even though there isn't an object associated with it.
In Scala, you can try and read the documentation online, go look up the source code to the library, etc., but this can often be very difficult for things where you don't know where or even what they are (and it's often a big chunk to bite off, given the voluminous type hierarchy) -- stuff is floating around in various places (package scala, Predef, various implicit conversions, symbols like :: that are nearly impossible to Google). The REPL should be the way to explore directly, but in reality, things are far more mysterious. Say that I've seen a reference to foo somewhere, but I have no idea what it is. There's apparently no such thing as a "guide to systematically investigating Scala thingies with the REPL", but the following is what I've pieced together after a great deal of trial and error:
If foo is a value (which presumably includes things stored in variables plus companion objects and other Scala objects), you can evaluate foo directly. This ought to tell you the type and value of the result. Sometimes the result is helpful, sometimes not.
If foo is a value, you can use :type foo to get its type. (Not necessarily enlightening.) If you use this on a function call, you get the type of the return value, without calling the function.
If foo is a value, you can use foo.getClass to get its class. (Often more enlightening than the previous, but how does an object's class differ from its type?)
For a class foo, you can use classOf[foo], although it's not obvious what the result means.
Theoretically, you can use :javap foo to disassemble a class -- which should be the most useful of all, but fails entirely and uniformly for me.
Sometimes you have to piece things together from error messages.
Example of failure using :javap:
scala> :javap List
Failed: Could not find class bytes for 'List'
Example of enlightening error message:
scala> assert
<console>:8: error: ambiguous reference to overloaded definition,
both method assert in object Predef of type (assertion: Boolean, message: => Any)Unit
and method assert in object Predef of type (assertion: Boolean)Unit
match expected type ?
assert
^
OK, now let's try a simple example.
scala> 5
res63: Int = 5
scala> :type 5
Int
scala> 5.getClass
res64: java.lang.Class[Int] = int
Simple enough ...
Now, let's try some real cases, where it's not so obvious:
scala> Predef
res65: type = scala.Predef$#3cd41115
scala> :type Predef
type
scala> Predef.getClass
res66: java.lang.Class[_ <: object Predef] = class scala.Predef$
What does this mean? Why is the type of Predef simply type, whereas the class is scala.Predef$? I gather that the $ is the way that companion objects are shoehorned into Java ... but Scala docs on Google tell me that Predef is object Predef extends LowPriorityImplicits -- how can I deduce this from the REPL? And how can I look into what's in it?
OK, let's try another confusing thing:
scala> `::`
res77: collection.immutable.::.type = ::
scala> :type `::`
collection.immutable.::.type
scala> `::`.getClass
res79: java.lang.Class[_ <: object scala.collection.immutable.::] = class scala.collection.immutable.$colon$colon$
scala> classOf[`::`]
<console>:8: error: type :: takes type parameters
classOf[`::`]
^
scala> classOf[`::`[Int]]
res81: java.lang.Class[::[Int]] = class scala.collection.immutable.$colon$colon
OK, this left me hopelessly confused, and eventually I had to go read the source code to make sense of this all.
So, my questions are:
What's the recommended best way from the true Scala experts of using the REPL to make sense of Scala objects, classes, methods, etc., or at least investigate them as best as can be done from the REPL?
How do I get :javap working from the REPL for built-in stuff? (Shouldn't it work by default?)
Thanks for any enlightenment.
You mentioned an important point which Scala lacks a bit: the documentation.
The REPL is a fantastic tool, but it is not as fantastic at it can be. There are too much missing features and features which can be improved - some of them are mentioned in your post. Scaladoc is a nice tool, too, but is far away to be perfect. Furthermore lots of code in the API is not yet or too less documented and code examples are often missing. The IDEs are full ob bugs and compared to the possibilities Java IDEs show us they look like some kindergarten toys.
Nevertheless there is a gigantic difference of Scalas current tools compared to the tools available as I started to learn Scala 2-3 years ago. At that time IDEs compiled permanently some trash in the background, the compiler crashed every few minutes and some documentation was absolutely nonexistent. Frequently I got rage attacks and wished death and corruption to Scala authors.
And now? I do not have any of these rage attacks any more. Because the tools we currently have are great although the are not perfect!
There is docs.scala-lang.org, which summarizes a lot of great documentation. There are Tutorials, Cheat-sheets, Glossaries, Guides and a lot of more great stuff. Another great tools is Scalex, which can find even the weirdest operator one can think of. It is Scalas Hoogle and even though it is not yet as good as his great ideal, it is very useful.
Great improvements are coming with Scala2.10 in form of Scalas own Reflection library:
// needs Scala2.10M4
scala> import scala.reflect.runtime.{universe => u}
import scala.reflect.runtime.{universe=>u}
scala> val t = u.typeOf[List[_]]
t: reflect.runtime.universe.Type = List[Any]
scala> t.declarations
res10: Iterable[reflect.runtime.universe.Symbol] = SynchronizedOps(constructor List, method companion, method isEmpty, method head, method tail, method ::, method :::, method reverse_:::, method mapConserve, method ++, method +:, method toList, method take, method drop, method slice, method takeRight, method splitAt, method takeWhile, method dropWhile, method span, method reverse, method stringPrefix, method toStream, method removeDuplicates)
Documentation for the new Reflection library is still missing, but in progress. It allows one to use scalac in an easy way inside of the REPL:
scala> u reify { List(1,2,3) map (_+1) }
res14: reflect.runtime.universe.Expr[List[Int]] = Expr[List[Int]](immutable.this.List.apply(1, 2, 3).map(((x$1) => x$1.$plus(1)))(immutable.this.List.canBuildFrom))
scala> import scala.tools.reflect.ToolBox
import scala.tools.reflect.ToolBox
scala> import scala.reflect.runtime.{currentMirror => m}
import scala.reflect.runtime.{currentMirror=>m}
scala> val tb = m.mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = scala.tools.reflect.ToolBoxFactory$ToolBoxImpl#32f7fa37
scala> tb.parseExpr("List(1,2,3) map (_+1)")
res16: tb.u.Tree = List(1, 2, 3).map(((x$1) => x$1.$plus(1)))
scala> tb.runExpr(res16)
res18: Any = List(2, 3, 4)
This is even greater when we want to know how Scala code is translated internally. Formerly wen need to type scala -Xprint:typer -e "List(1,2,3) map (_+1)"
to get the internally representation. Furthermore some small improvements found there way to the new release, for example:
scala> :type Predef
scala.Predef.type
Scaladoc will gain some type-hierarchy graph (click on type-hierarchy).
With Macros it is possible now, to improve error messages in a great way. There is a library called expecty, which does this:
// copied from GitHub page
import org.expecty.Expecty
case class Person(name: String = "Fred", age: Int = 42) {
def say(words: String*) = words.mkString(" ")
}
val person = Person()
val expect = new Expecty()
// Passing expectations
expect {
person.name == "Fred"
person.age * 2 == 84
person.say("Hi", "from", "Expecty!") == "Hi from Expecty!"
}
// Failing expectation
val word1 = "ping"
val word2 = "pong"
expect {
person.say(word1, word2) == "pong pong"
}
/*
Output:
java.lang.AssertionError:
person.say(word1, word2) == "pong pong"
| | | | |
| | ping pong false
| ping pong
Person(Fred,42)
*/
There is a tool which allows one to find libraries hosted on GitHub, called ls.implicit.ly.
The IDEs now have some semantic highlighting, to show if a member is a object/type/method/whatever. The semantic highlighting feature of ScalaIDE.
The javap feature of the REPL is only a call to the native javap, therefore it is not a very featue-rich tool. You have to fully qualify the name of a module:
scala> :javap scala.collection.immutable.List
Compiled from "List.scala"
public abstract class scala.collection.immutable.List extends scala.collection.AbstractSeq implements scala.collection.immutable.LinearSeq,scala.Product,scala.collection.LinearSeqOptimized{
...
Some time ago I have written a summary of how Scala code is compiled to Bytecode, which offers a lot of things to know.
And the best: This is all done in the last few months!
So, how to use all of these things inside of the REPL? Well, it is not possible ... not yet. ;)
But I can tell you that one day we will have such a REPL. A REPL which shows us documentation if we want to see it. A REPL which let us communicate with it (maybe like lambdabot). A REPL which let us do cool things we still cannot imagine. I don't know when this will be the case, but I know that a lot of stuff was done in the last years and I know even greater stuff will be done in the next years.
Javap works, but you are pointing it to scala.Predef.List, which is a type, not a class. Point it instead to scala.collection.immutable.List.
Now, for the most part just entering a value and seeing what the result's type is is enough. Using :type can be helpful sometimes. I find that use getClass is a really bad way of going about it, though.
Also, you are sometimes mixing types and values. For example, here you refer to the object :::
scala> `::`.getClass res79: java.lang.Class[_ <: object
scala.collection.immutable.::] = class
scala.collection.immutable.$colon$colon$
And here you refer to the class :::
scala> classOf[`::`[Int]] res81: java.lang.Class[::[Int]] = class
scala.collection.immutable.$colon$colon
Objects and classes are not the same thing, and, in fact, there's a common pattern of objects and classes by the same name, with a specific name for their relationship: companions.
Instead of dir, just use tab completion:
scala> "abc".
+ asInstanceOf charAt codePointAt codePointBefore codePointCount
compareTo compareToIgnoreCase concat contains contentEquals endsWith
equalsIgnoreCase getBytes getChars indexOf intern isEmpty
isInstanceOf lastIndexOf length matches offsetByCodePoints regionMatches
replace replaceAll replaceFirst split startsWith subSequence
substring toCharArray toLowerCase toString toUpperCase trim
scala> "abc".compareTo
compareTo compareToIgnoreCase
scala> "abc".compareTo
def compareTo(String): Int
If you enter the power mode, you'll get way more information, but that's hardly for beginners. The above shows types, methods, and method signatures. Javap will decompile stuff, though that requires you to have a good handle on bytecode.
There's other stuff in there -- be sure to look up :help, and see what's available.
Docs are only available through the scaladoc API. Keep it open on the browser, and use its search capability to quickly find classes and methods. Also, note that, as opposed to Java, you don't need to navigate through the inheritance list to get the description of the method.
And they do search perfectly fine for symbols. I suspect you haven't spent much time on scaladoc because other doc tools out there just aren't up to it. Javadoc comes to mind -- it's awful browsing through packages and classes.
If you have specific questions Stack Overflow style, use Symbol Hound to search with symbols.
Use the nightly Scaladocs: they'll diverge from whatever version you are using, but they'll always be the most complete. Besides, right now they are far better in many respects: you can use TAB to alternate between frames, with auto-focus on the search boxes, you can use arrows to navigate on the left frame after filtering, and ENTER to have the selected element appear on the right frame. They have the list of implicit methods, and have class diagrams.
I've made do with a far less powerful REPL, and a far poorer Scaladoc -- they do work, together. Granted, I skipped to trunk (now HEAD) just to get my hands on tab-completion.
Note that scala 2.11.8 New tab-completion in the Scala REPL can facilitate the type exploration/discovery.
It now includes:
CamelCase completion:
try:
(l: List[Int]).rroTAB,
it expands to:
(l: List[Int]).reduceRightOption
Find members by typing any CamelCased part of the name:
try:
classOf[String].typTAB,
to get getAnnotationsByType, getComponentType and others
Complete bean getters without typing get:
try:
(d: java.util.Date).dayTAB
Press TAB twice to see the method signature:
try:
List(1,2,3).partTAB,
which completes to:
List(1,2,3).partition;
press TAB again to display:
def partition(p: Int => Boolean): (List[Int], List[Int])
You need to pass fully qualified class name to javap.
First take it using classOf:
scala> classOf[List[_]]
res2: java.lang.Class[List[_]] = class scala.collection.immutable.List
Then use javap (doesn't work from repl for me: ":javap unavailable on this platform.") so example is from a command line, in repl, I believe, you don't need to specify classpath:
d:\bin\scala\scala-2.9.1-1\lib>javap -classpath scala-library.jar "scala.collection.immutable.List"
But I doubt this will help you. Probably you're trying to use techniques you used to use in dynamic languages. I extremely rarely use repl in scala (while use it often in javascript). An IDE and sources are my all.
What is a nice approach for collecting and using useful Scala utility functions across projects. The focus here on really simple, standalone functions like:
def toBinary(i: Int, digits: Int = 8) =
String.format("%" + digits + "s", i.toBinaryString).replace(' ', '0')
def concat(ss: String*) = ss filter (_.nonEmpty) mkString ", "
concat: (ss: String*)String
This question is basic, I know ;-) but, I've learned that there is always an optimum way to do something. For example, reusing code from within the Scala interactive shell, Idea, Eclipse, with or without SBT, having the library hosed on GitHub, ect, could quickly introduce optimal, and non-optimal approaches to such a simple problem.
You might want to put such methods in a package object.
You could also put them in a normal object and import everything in the object when you need those methods.
object Utilities {
def toBinary(i: Int, digits: Int = 8) = // ...
}
// Import everything in the Utilities object
import Utilities._
If you want it trivially accessible from everywhere, your best bet is actually to stick it inside the scala-library jar. Personally, I have all my custom stuff in /jvm/S.jar (or something like that) and add it to the classpath every time I need it.
Repacking the Scala library jar is really easy--unpack it, move your class hierarchy in, and pack it up again. (You should have it inside some package and/or package object.)