I have a jar file contains scala files which I'd like to version somehow. Sometimes I can use git SHA inserted in METADATA but it's not always the case.
As a "fallback" I thought doing md5 on the bytecodes themselfs (*.class files). For that I'll need to make sure same code given to maven package will produce same bytecode. I found out here that:
Different versions of Scala may produce slightly different bytecode.
which is fine by me, I'm asking if there are more variables involved, mainly time as in jar creation date presented in bytecode somehow.
Thanks!
Related
I don't get how standard library like libc is linked.I use MingW compiler.
I see that there is no libc.dll file in its bin folder.Then how libc is linked?
How does compiler know difference between custom and dynamic library?
We use build tools because they are a practical way to compile code and create executables, deployables etc.
For example. Consider that you have a large Java application consisting of thousands of separate source files split over multiple packages. Suppose that it has a number of JAR file dependencies, some of them for libraries that you have developed, and others for external libraries that can be downloaded from standard places.
You could spend hours manually downloading external JAR files and putting them in the right place. Then you could manually run javac for each of the source files, and jar in multiple times. Each time in the correct directory with the correct command line arguments ... and in the correct order.
And each time you change the source code ... repeat the relevant parts of the above process.
And make sure you don't make a mistake which will cause you to waste time finding test failures, runtime errors, etc caused by not building correctly.
Or ... you could use a build tool that takes care of it all. And does it correctly each time.
In summary, the reasons we use build tools are:
They are less work than doing it by hand
They are more accurate
The results are more reproducible.
I want to know why compiler can't do it?
Because compilers are not designed to perform the entire build process. Just like your oven is not designed to cook a three course meal and serve it up at the dinner table.
A compiler is (typically) designed to compile individual source code files. There is more to building than doing that.
I want to see the control flow graph generated by the Scala compiler. Is there a way to do so? I tried searching online but only found Eclipse plugins for Java like the one from here www.drgarbage.com but none for Scala.
Thanks
EDIT: I took the .class file generated by scalac and opened that with the dr garbage plug in to see the bytecode visualized as a control flow graph. But scalac makes 3 different .class files: Foo, Foo$, and Foo$delayedInit$body. I see a bunch of disconnected graphs and only one of the graphs in Foo$ looks reasonable. I tried searching online for the difference between the 3 .class files but couldn't find anything.
I didn't realize that the IR (intermediate representation) for scala in the backend was actually called icode. The option in the compiler -Xprint-icode actually shows the IR separated into basic blocks. This was what I was looking for.
A compiler plugin can do just that. However, it requires abstracting away some internals introduced by the compiler - which are more specific than what you'd expect from how a project's source code looks. You can use this plugin to get the raw information extracted for you, whereas the re-abstraction from the raw data is still work-in-progress, and you'd have to sbt publishLocal before you can include this plugin in your sbt definition.
I'm building a tool that will receive unpredictable data structure, and I want to generate case class to accomplish the structure of the received data.
I'm trying to figure out if it's possible to generate case class at runtime? This structure will be know only at runtime.
It's something similar to what macro does, but in runtime.
I've found this project on the internet
mars
Which is very close to what I want to do ,but I couldn't find if it was successful of not.
Another way of doing it is generate the code, compile and put the result in the classpath, like IScala is doing to use the code in an iterative way. But I don't think that this will scale.
Does anybody has already done something like runtime code generation?
This question was also posted in scala-user mailing list
UPDATE: (as per the comments)
If all you want is throw-away code generated at runtime to be fed into to a library that cannot work with just lists and maps, and not code to be stored and used later, it would make sense to look for solutions to this problem for Java or JVM. That is, unless the library requires some Scala specific features not available to vanilla JVM bytecode (Scala adds some extras to the bytecode, which Java code doesn't need/have).
what is the benefit of generating statically typed code dynamically? as opposed to using a dynamic data structure.
I would not attempt that at all. Just use a structure such as nested lists and maps.
Runtime code generation is one of the purposes of the Mars Project. Mars is under development, at the moment there is no release version. Mars requires its own toolchain to expand macros at runtime and should use several features unique to scala.meta (http://scalameta.org/), for example, AST interpretation and AST persistence. Currently we are working on ASTs typechecking in scala-reflect, required for runtime macros expansion.
Is it possible for a Compiler (for ex. javac) to scan your whole project for unused methods and variables before compilation and then compiles the project without those unused methods and variables such that you end up with fewer bytes of executable code.
If this would be a compiler optimization, I would create one huge library that contains all my helper methods and import it in all my projects and not worry that it being so huge could effect my Software size.
I understand this could be impossible, if you do not have the source code of those libraries you are using(importing), but I am speaking of the case where you have the source code.
Is there a tool/IDE plugin that does something similar? I would think this could be also done in one step ahead of the compilation.
Java's compiler doesn't do this natively, but you can use a tool like ProGuard or any number of other Java optimizers to remove unused code.
But, in your case, why don't you just compile your big software library once and put it on your classpath? That way you don't have to duplicate it at all.
sbt maintains dependencies between tasks, and the resulting graph can be reasoned about fairly easily. On the other hand, skimming the source code, it seems like the incremental compilation logic is a lot more opaque. I'd like to be able to do the following things:
Say the equivalent of "if I modified this interface [in this way], what would get invalidated?"
Build a graph of how modifying different class interfaces affects the rest of the build. Graphing scala import dependencies isn't a particularly good approximation of this, given how complicated implicit dependencies can get in Scala. It seems like sbt must maintain this information in some form or another to do incremental compilation, so I "just" need to figure out how to access it and hope that it's in a form suitable for my use case.
Are either of these feasible? I'm not opposed to writing sbt plugins, but would appreciate hints about how to proceed.
Edit: it looks like Relation's usesInternalSrc(dep: File): Set[File] could be promising. Does that capture all of sbt's dependency knowledge?
Edit 2: even more promising, there's a DotGraph object inside the sbt source tree. It has no documentation and google doesn't have any human-readable text about it. If I can figure out how to use it I'll post an answer.
Sample console-project session:
> val (s, a) = runTask(compile in Compile, currentState)
> DotGraph.sources(a.relations, file("source-graph"), Nil)
source-graph is a directory that will contain two dot files, one with source dependencies and one with binary. You can alternatively directly interact with a.relations of type Relations, as suggested in the question, and which does capture all of sbt's dependency knowledge. In 0.13 there will also be information about which dependencies are due to inheriting from something in another source file.
In terms of how modifying a source file affects invalidation, it is very coarse grained. Any change to any non-private signature marks a source as changed. In 0.12 and earlier, this will at least invalidate direct dependencies and maybe more. In 0.13, this will invalidate direct dependencies only, except for inherited dependencies, which are transitively invalidated. There is currently no way to see what will be invalidated when a source file's non-private API is modified except by doing it.