trait PublicApi{
def sayHi(from:String,content:String)
}
I know that it is impossible in java to get "from" and "content" in runtime
but can scala manifest help me out here?? like defining
trait PublicApi{
def sayHi(from:String,content:String)(m:Manifest)
}
invoking m.methodErasure.getArgumentName(0) would return "from"
In its current implementation, ClassManifest is all about type reification, and would know nothing about parameter names.
Unless you:
add meta data for parameter names at compile time
or use a cut down version of ASM (Java bytecode manipulation and analysis framework) to extract debug information from a class at runtime
, as paranamer does (like Monkey mentions in the comments), you won't have any information on the parameter name (at least not before Java 8.0, even though it was initially mentioned for Java6!: the proposal is still in progress)
The Scala "signature" DOES contain these parameter names, as you rightly surmise. After all, they have to be stored somewhere for named/default parameters to work. You can also use scalap to see these names.
Feel free to take a look at my preliminary work on a reflection library.
It's a work in progress, and I make absolutely no guarantees as to the current feature set or correctness, but may give you some idea as to what's involved - you might even be able to use it for extracting parameter names in its current state. This is also something I'm actively working on, so you can expect it to improve with the passage of time...
Related
Is there anyway we can read scala doc comments using reflection. My requirement is to read the #group tag value and use it for counting how many functions are there for each group
No, you can't use Scala reflection to access documentation comments. The reason is simple: comments are, almost by definition, not part of the program. Therefore, it is logically impossible for them to be available via reflection.
In Python, for example, documentation is available from the running program (in fact, even without using reflection), because the documentation is not hidden away in comments, but rather simply assigned to a field of the object that is being documented. Many Lisps (e.g. Clojure), and also Ioke and Seph work that way, too.
In Newspeak, what they call "comments" is available using reflection, but that's because what they call "comments" are not really comments, it is more like arbitrary metadata that can be attached to objects. It is in fact more similar to an annotation in Scala than a comment.
In Scala, documentation is written in comments, and comments are not part of the program (they are literally equivalent to whitespace in the Scala Language Specification), and therefore, cannot possibly be part of the program and thus cannot possibly be accessed via reflection.
I have read various old StackOverflow discussions on this general topic but there is still one part of the puzzle which appears, to me at least, to be missing.
It is simply this: what is the actual mechanism by which the anonymous function is serialized? And, where could we find its source code?
Or is it all just magic?
Other relevant SO articles (the third of these itself points to some useful articles outside StockOverflow):
Serialization of Scala Functions
Why Scala can serialize...
How to serialize functions in Scala
I'm going to answer my own question with what, I believe is the correct answer. The reason I'm doing it this way is that it seems to me that this aspect of serialization is never explained and it does appear to work just by magic. I essentially confirmed (to my satisfaction) the answer as part of the research I was doing to ensure that my question above was indeed appropriate.
But the main reason I'm offering my own answer is that I invite knowledgeable users either to agree with it, to correct it, to expand upon it, or to destroy it. Here goes...
It's all magic. No, I'm just kidding. But essentially the mechanism, once Scala has taken the step of representing the anonymous function as a Class, is entirely provided for by Java. In addition, we, the programmer, need to ensure that an anonymous function is as much pure code as possible: no references to any objects that might not be serializable. The secret sauce is to be found in the Java class: ObjectStreamClass. Which, in turn, is invoked by the Java serialization classes: ObjectInputStream and ObjectOutputStream.
Essentially the serialized bytes contain the full pathname of the class, its serialVersionUID, and whatever other relevant information is necessary. When deserializing, the system will simply look up the class in the appropriate classpath and return a reference to it. This obviously assumes that the deserializing system has the class in its classpath. The mechanism for that is a little beyond the scope of my research but it's clear that in a system like Spark, it should be easy to arrange.
No (additional) compilation/decompilation of byte code is necessary as the classLoader has everything necessary. I'm slightly surprised to find the ObjectStreamClass in java.io rather than in the reflection package, but I suppose there's an argument for it being there, given the tight coupling with ObjectInputStream and ObjectOutputStream.
One thing to keep in mind is that while we think in terms of serializing/deserializing objects, rather than classes, what we are dealing with here is an object of type Class.
One more thing to note is that in Scala 2.12, anonymous functions are now implemented differently: as Java8 lambdas. This has broken the mechanism described above in a rather serious way. So serious, that Spark is currently having trouble supporting Scala 2.12. The holdup appears to be this issue: SPARK-14540.
When I use scalac to compile file.scala, I end up with 2 outputs, file.class and file$.class. What is the difference between these files and which is the appropriate one to then run? I get distinctly different error messages between executing "scala file" vs "scala file$".
Scala objects get compiled to classes ending in "$" because you're allowed to have an "ordinary" class with the same name. But the object's methods are also exposed as static methods on the "ordinary" class, so that they can be called under the names you would expect. This is an artifact of trying to represent the scala semantics in a way that make sense to Java / the JVM, and I would encourage you to regard it as an implementation detail rather than something important.
(#MattPutnam's answer is correct that anonymous classes, including closures, are compiled to class files with $es in their name, but that's not what's causing your file$.class in this particular instance)
Use scala file. If you're interested in the implementation details you might also want to try java -cp /path/to/scala-library.jar file.
file$.class is some inner anonymous class. In Java they're very explicit, but they can be easy to miss in Scala. If you use any method that takes a function, there's an implicit anonymous class there. Post the code and I'll point it out.
I have a relatively large Scala code base that does not use named parameters for any function/class calls. Rather than going in and manually entering it, which would be a very tedious process, I was looking at a formatter to do the job. The best I found is scalariform, but I'm not sure whether I can even write a rule for something so complex.
I'm curious if anyone has ran into a similar problem and found a powerful formatter.
The Scala Refactoring library might be something you could use. You will need some knowledge of Scala's Abstract Syntax Tree representation.
Why do you want to use named parameters throughout your code base? I like IntelliJ's default which is to suggest to name boolean arguments (only).
I'm having a hard time deciphering Scala API documentation.
For example, I've defined a timestamp for use in a database.
def postedDate = column[Timestamp]("posted_date", O NotNull, O Default new Timestamp(Calendar.getInstance.getTimeInMillis), O DBType("timestamp"))
If I hadn't read several examples, of which none were in the API doc, how could I construct this statement? From the Column documentation how could I know the parameters?
I guessed it had something to do with TimestampTypeMapperDelegate but it is still not crystal clear how to use it.
The first thing to note from the scaladoc for Column is that it is abstract, so you probably want to deal directly with one if its subclasses. For example, NamedColumn.
Other things to note are that it has a type parameter and the constructor takes an implicit argument of a TypeMapper of the same parameter type. The docs for TypeMapper provide an example of how to create a custom one, but if you look at the subclasses, there are plenty of provided ones (such as timestamp). The fact that the argument is declared as implicit suggests that there could be one in scope, and if so, it will automatically be used as the parameter without explicitly stating that. If there isn't an implicit in scope that satisfies the requirement, you'll have to provide it.
The next think to note is that a TypeMapper is a trait that extends a function with an argument of a BasicProfile and a TypeMapperDelegate result. Basically what's going on here is the definition of a type mapper is separated from the implementation. This is done to support multiple flavors of database. If look at the subclasses of BasicProfile, it will become apparent that ScalaQuery supports quite a few, and as we know, their implementations are sometimes quite different.
If you chase the docs for a while, you end up at the BasicTypeMapperDelegates trait that has a bunch of vals in it with delegates for each of the basic types (including timestamps).
BasicTable defines a method called column (which you've found), and the intent of the column method is to shield you from having to know anything about TypeMappers and Delegates as long as you are using standard types.
So, I guess to answer your question about whether there is enough information in the API docs, I'd personally say yes, but the docs could be enhanced with better descriptions of classes, objects, traits and methods.
All that said, I've always found that leveraging examples, API docs, and even the source code of the project provides a robust way of getting up to speed on most open source projects. To be quite blunt, many of these projects (including ScalaQuery) have saved me countless hours of work, but probably cost the author(s) countless hours of personal time to create and make available. These are not necessarily commercial products, and we as consumers shouldn't hold them to the same standards that we hold for-fee products. If you find docs inadequate, contribute!