Is there a simple way to follow the Scala convention and append the Scala binary version to the archive name and artifactID of Scala libraries in Gradle? Even better if it is possible to compile the library against multiple Scala versions.
The convention that you are referring to isn't built into the Scala plugin, but setting jar.archiveName should do the trick. Likewise, building multiple variants isn't currently a first-class feature, but can be achieved in various ways, for example by declaring a separate source set per variant, with each source set pointing to the same source and resource directories. A proof of concept that also handles variants in dependencies (similar to what's offered by sbt) was presented in my Polyglot Gradle talk at Gradle Exchange 2013.
Related
The underlying mechanism used to indicate which version of Scala a library was compiled against is to append _<scala-version> to the library's name. This fairly simple approach allows interoperability with users of Maven, Ant and other build tools.
-- sbt Documentation: Cross-Build Publishing Conventions
While this is a simple approach, the interoperability with Maven and other build tools leaves something to be desired. Because the artifactId is different (e.g. scalatest_2.9.0 and scalatest_2.10.0), Maven treats them as different artifacts. Maven's dependency resolution mechanism is thus compromised and multiple versions of the same artifact (built against different scala versions) can wind up on the classpath.
Why not put the scala version in the classifier? This seems to be one of the primary intended use cases for the classifier:
The classifier allows [Maven] to distinguish artifacts that were built from the same POM but differ in their content. As a motivation for this element, consider for example a project that offers an artifact targeting JRE 1.5 but at the same time also an artifact that still supports JRE 1.4. The first artifact could be equipped with the classifier jdk15 and the second one with jdk14 such that clients can choose which one to use.
-- Maven Documentation: POM Reference
Appending version to the name is a historical decision that was made long time ago so it'll likely not going to change since many libraries are published with the convention already.
Having said that, as Seth noted, there was a discussion to review this topic a few years ago when sbt 0.12 shortened "_2.10.0" postfix to "_2.10" to take advantage of Scala library's binary compatibility between the minor versions. Here's Mark from [0.12] plan:
By cross versioning, I mean the practice of including some part of the Scala version in the module ID to distinguish an artifact generated by compiling the same source code against different Scala versions. I do not mean the ability to build against multiple Scala versions using +task, which will stay; I am just referring to the cross version convention.
[snip]
It has always been a hack to encode this in the inflexible pom.xml format and I think it may be best to move away from this for projects built against Scala 2.10 and later. However, perhaps this is better than any ad hoc solutions that might take its place. I don't see users of other build tools doing this, so I expect nothing would replace it.
Somewhere down the thread Josh suggested:
(1) Scala classifiers. These can be custom strings and can be specified with dependencies. At least, IIRC this should work.
Here's Mark's response:
What do mean by "can be specified with dependencies"? There is only one pom for all of the classifiers, right? How can you declare different dependencies for each classifier?
Here are some more interesting remark on classifiers from Geoff Reedy
I too thought that classifiers would be the perfect way to deal with
this issue especially in light of the suggestion in the maven docs that
classifiers java-1.4 and java-1.5 be used to distiguish between jars
appropriate for the respective platform. The fatal flaw seems to be
transitive dependency management. That is, there's no way to choose the
transitive dependency set based on the classifier used to require the
module. We'd need to be able to say that when you're using this module
with the scala-2.10 classifier it brings its own dependencies using the
scala-2.10 classifier and when used with the 2.9 classifier brings in
its own deps with the scala-2.9 classifier.
I think with the jvm versions it's possible to make this work because
jvm versioning has special support in the profile activation which can
can control dependencies.
I am trying to build my Scala source code with SBT targeting same Scala version but different versions of libraries. My questions are:
Does SBT support this? (seems unlikely)
If not, is there a way to achieve this without messing the build script too much?
Details:
I am building for clusters with different versions of Spark and other libraries. This seems to be a common user case in such a scenario.
Possible solutions I have found:
Use different minor versions of Scala, then dispatch different versions of library with cross-building. Very hackish.
Use dummy directories for multiple projects, redirecting their source paths to the real source directory. This seems most promising but I have not tried it yet.
No sbt does not support this, but if you want to do that then you can use shading.
like if I want to use two different versions of guava that was supported by two different libraries then use guava shading like:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.google.guava**" -> "shadeio.#1").inAll
)
scala-2.11 folder appeared after recent update of IDEA and Scala plugin.
What should it be used for?
Usually such directories are used for binary version-dependent code. For example, macros in 2.10 are not source-compatible with macros in 2.11, so if you're building your project for different binary versions and you're using macros, it makes sense to put code which is only valid for the specific version in different source roots. SBT then will use the appropriate directory when compiling for 2.10 or 2.11.
If you're using SBT, though, you would need to set such thing up manually in the build definition. If you're not using SBT, then probably IDEA plugin was updated to handle such things by itself.
I am using scala in android. And I want to include some basic library to the project, and I found there are 2 jar one is scala-compiler and another is scala-library. What's the difference between them, and how should we choose one?
Scala-compiler - is a compiler itself, which (simply saying) takes .scala files and compiles them into the .class files. You don't need it to run already compiled .jar/.class file if you don't interpret scala-code in runtime (which usually you don't). Scala-compiler.jar is used by your built-tool/ide to compile your scala code. Sometimes third-party libraries may also want it as transitive dependency.
Scala-library - is a library that contains scala API (built-in functions, collections, concurrency etc.). Usually (99,9%) you need this.
Since you have only source compatibility between Scala-versions you unfortunately need to compile libraries like scalatest or scalamock for each scala version they support. What puzzles me is that the libraries are provided with loads of artifacts (scalatest_2.9.0, scalatest_2.9.1, scalatest_2.10 and so forth) - one for each scala version, such that the maven repository is littered with many artefacts that are built from the same source. My instinct tells me rather to use one artifact with a classifier for each scala version. (In fact, the maven pom reference mentions that this was sometimes done with jdk14 and jdk15 classifiers for artifacts, which seems similar to me.) So, why did the Scala people go for the many artifact overkill :-) instead?
I may be wrong, but if a classifier's purpose is to "distinguish artifacts that were built from the same POM but differ in their content", then I see a very good reason not to use them for scala versioning: scala versions are not just binary incompatible, they can very well be source incompatible.
By example, when upgrading scala from 2.7 to 2.8, I had to make some significant changes to the code base. If I wanted to keep both a scala 2.7 and 2.8 version at the same time, I would have needed to create a parallel branch, and both branches would definitely not have the same source code.
When I read "from the same POM", I understand that it means from the same source code too, which would clearly not be the case with those two branches of code.
Another more important reason is that a classifier is essentially a single string, which is already used for many things. More or less standard classifiers include "sources", "javadoc" or "resources". The meanings of these classifiers are the same in scala project, and are totally orthogonal to the scala version, as I'll try to show.
Maven's documentation suggests using classifiers such as "jdk15" or "jdk14" to denote which version of the jvm the binary artifact was compiled against.
Given that java code is backward compatible, in principle the artifacts with both classifiers ("jdk15" or "jdk14") are compiled from the same source code. This is why you don't need to duplicate classifiers for "sources" artifact, or in other words you don't need to have a classifier named "sources-jdk14" and "sources-jdk15".
But you cannot apply the same rationale to the scala version: given that you might need different source code whether you compile against scala 2.7 or scala 2.8, you would indeed need two different artifacts with classifiers such as "sources-scala2.7" or "source-scala2.8". So we have composite classifiers already.
As for binary artifacts, you would also not only need to distinguish between the target jvm version (remember, you can compile your scala code to target different jvm versions) but also the scala version it was compiled against. So you would end up with something like "jdk14-scala2.7" or "jdk14-scala2.8" or "jdk15-scala2.7" or "jdk15-scala2.8". Yet another set of composite classifiers.
So the take home message is that the scala version really is a separate way of classifying artifacts, that is totally orthognal to all the existing classifiers.
Yes, we could really use composite classifiers as above (such as "sources-scala2.7") but then we would not be using standard classifiers, which is confusing enough in itself, but would also require to modify all the tooling around classifiers: what if I use a build tool that has no knowledge of scala (only java) but knows how to automatically publish a "source" artifact? Will I need to modify this build tool so that he knows to publish a "sources-scala2.7" artifact instead? On the other hand, if I encode the scala version in the (base) artifact name and give that to the build tool, everything works as usual and I get a an artifact with the "source" classifier.
All in all, and contrary to immediate intuition, encoding the scala version in the name allows for better integration in the existing java build ecosystem.
Scala provides inter-version compatibility of its bytecode output (.class files) only across patch-releases (third component of Major.Minor.Patch version spec).
Maven has no place to properly encode this as a first-class property of the artifact, so it has to be encoded by convention in the name.
Sadly...