I'm trying to cross build a project for (2.11, 2.12) where some of the subprojects should not be built for 2.12 because their transitive dependencies are not yet released for 2.12. Specifically Spark for Scala 2.12. The root project's aggregate looks like
lazy val root = (project in file(".")).
aggregate(vegas, spark, flink, macros).
settings(commonSettings: _*).
settings(noPublishSettings: _*)
Is there some way to detect the scalaBinaryVersion in the #aggregate and aggregate a different set of projects if the crossbuild is trying to produce a 2.12 artifact?
There appears to be no direct way to do it. As a workaround, you may want to get a similar effect by making spark's libraryDependencies empty and skipping compile and publish when scalaBinaryVersion is 2.12:
// tested on sbt 1.1.0
lazy val spark = (project in file("spark"))
.settings(
// ... other settings ...
// Empty out libraryDependencies when scalaBinaryVersion is 2.12.
libraryDependencies :=
(if (scalaBinaryVersion.value == "2.12") Seq.empty else libraryDependencies.value),
// Skip compilation and publishing when scalaBinaryVersion is 2.12.
skip in compile := scalaBinaryVersion.value == "2.12",
skip in publish := scalaBinaryVersion.value == "2.12"
)
The skip task key allows us to skip some task. From inspect skip:
Task: Boolean
For tasks that support it (currently only compile, update, and publish), setting skip to true will force the task to not to do its work. This exact semantics may vary by task.
However, in contrast to compile and publish, skip in update := scalaBinaryVersion.value == "2.12" does not work here. From sbt Reference Manual:
Overriding all of the above, skip in update := true will tell sbt to never perform resolution. ... Also, (note that) update itself will immediately fail if resolution has not been allowed to run since the last clean.
Related
I am in the process of writing a library that does monitoring/OpenTracing and I am attempting to use sbt-aspectj so that users of the library don't need to manually instrument their code. I am currently however getting an issue with creating an sbt-project representing such a library.
The idea is that I want an external library as indicated in this sample here https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/external however that external library is dependant on an external dependency (i.e. akka-actors). Basically I am trying to combine both https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/external and https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/jar. I have created a sample project here https://github.com/mdedetrich/sbt-aspectj-issue to indicate the problem I am having however below is the relevant sample
lazy val root = (project in file("."))
.enablePlugins(SbtAspectj)
.settings(
name := RootName,
version := Version,
// add akka-actor as an aspectj input (find it in the update report)
// aspectjInputs in Aspectj ++= update.value.matching(
// moduleFilter(organization = "com.typesafe.akka", name = "akka-actor*")),
// replace the original akka-actor jar with the instrumented classes in runtime
// fullClasspath in Runtime := aspectjUseInstrumentedClasses(Runtime).value,
// only compile the aspects (no weaving)
aspectjCompileOnly in Aspectj := true,
// ignore warnings (we don't have the target classes at this point)
aspectjLintProperties in Aspectj += "invalidAbsoluteTypeName = ignore",
// replace regular products with compiled aspects
products in Compile ++= (products in Aspectj).value,
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion
)
)
lazy val test = (project in file("test"))
.enablePlugins(SbtAspectj)
.settings(
aspectjBinaries in Aspectj ++= update.value.matching(
moduleFilter(organization = Organization, name = s"$RootName*")),
aspectjInputs in Aspectj ++= update.value.matching(
moduleFilter(organization = "com.typesafe.akka", name = "akka-actor*")),
fullClasspath in Runtime := aspectjUseInstrumentedClasses(Runtime).value,
// weave this project's classes
aspectjInputs in Aspectj += (aspectjCompiledClasses in Aspectj).value,
products in Compile := (products in Aspectj).value,
products in Runtime := (products in Compile).value,
libraryDependencies ++= Seq(
Organization %% RootName % Version
)
)
The idea is that we publish the root project using root/publishLocal and the test project is just designed to include root as a libraryDependency so we can see if the aspect-j is working properly.
The problem is simple that I am unable to get it working. The current code at https://github.com/mdedetrich/sbt-aspectj-issue publishes with root/publishLocal (not sure if its correct though) however when I then do test/run I get this
[info] Weaving 2 inputs with 1 AspectJ binary to /home/mdedetrich/github/sbt-aspectj-issue/test/target/scala-2.13/aspectj/classes...
[error] stack trace is suppressed; run last test / Compile / packageBin for the full output
[error] (test / Compile / packageBin) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] Total time: 1 s, completed Dec 29, 2019 4:31:27 PM
sbt:sbt-aspectj-issue>
Which seems to be an issue with having duplicate akka-actor entries. I tried toggling various entries in build.sbt but didn't manage to get it working.
EDIT: This was also posted as a github issue here https://github.com/sbt/sbt-aspectj/issues/44
Generally, you can exclude the META-INF directories from the external libraries woven.
mappings in (Compile, packageBin) := {
(mappings in (Compile, packageBin)).value
.filterNot(_._2.startsWith("META-INF/"))
}
But for akka libraries, there is another problem. In each akka library, there is a reference.conf, which contains the fallback configuration for the provided features. This will also lead to conflicts like the META-INF did. But it cannot be just excluded like the META-INF, because they are essential for akka to work properly.
If you exclude them, you'll have to provide all the required akka configurations in your application.conf, or a merged (not simply concatenate) reference.conf in your project. It's not trivial, and subject to version change of akka.
Another solution would be weaving and repackaging the akka libraries individually, so the reference.conf can be kept in the repackaged libraries. The project layout and build script will a bit more complicated, but also be easier to maintain if you have plan to upgrade to newer versions of akka in the future.
Following the hints of the post explaining the basics of migrating to scalajs and this page about cross-compilations, I decided to add cross compilation to my standalone dependency-free scala library by doing the following changes:
I added a file project/plugins.sbt with the content
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.16")
I added scalaVersion in ThisBuild := "2.11.8" in build.sbt because else just scalaVersion was using 2.10
I also added in the build.sbt the following content to ensure that I can keep the same directory structure, since I don't have any particular files for the JVM or for Javascript:
lazy val root = project.in(file(".")).
aggregate(fooJS, fooJVM).
settings(
publish := {},
publishLocal := {}
)
lazy val foo = crossProject.crossType(CrossType.Pure).in(file(".")).
settings(version := "0.1").
jvmSettings(
// Add JVM-specific settings here
).
jsSettings(
// Add JS-specific settings here
)
lazy val fooJVM = foo.jvm
lazy val fooJS = foo.js
But now, after I published the project locally using sbt publish-local the projects depending on this library do not work anymore, i.e. they don't see the classes that this library was offering and raise errors.
I looked into .ivy2/local/.../foo/0.1/jars and the JAR went from 1MB to 1KB, so the errors make sense.
However, how can I make sure the JVM jar file is compiled correctly?
Further informations
The jar time does not change anymore, it looks like there had been some miscompilation. I deleted the .ivy2 cache, but now sbt publish-local always finishes with success but does not regenerate the files.
Ok I found the solution myself.
I needed to remove the publishLocal := {} from the build, and now all the projects depending on my library work fine.
I am upgrading one of the my Scala projects to Scala 2.11.7, but it has some dependent projects. They are using 2.10 so it is referring to a lot of dependent libraries with 2.10 Scala version (ex: com.novus:salat-core_2.10:1.9.9). I want to exclude which and all having "_2.10-" instead of writing one by one.
Is this possible?
1: Using scala version and providing sbt to choose correct version with %%
scalaVersion := "2.11.7"
val scalaz = "org.scalaz" %% "scalaz-core" % "7.1.0"
2: To exclude explicitly dependencies which has 2.10 build version use custom methods
// exclude from all with rule which check whether artifact name contains 2.10
def excludeFromAll(items: Seq[ModuleID], group: String, artifact: String) =
items.map(x => if(x.name.contains("_2.10")) x.exclude(group, artifact))
//all your dependencies
val deps = Seq(dependencies) //library Dependencies.
//exlusion
val appDependencies = excludeFromAll(deps, _, _)
I'm experiencing a problem when running latest play framework 2.3.
It compiles just fine, although when I do activator run this error happens:
java.lang.NoSuchMethodError: scala.Predef$.ArrowAssoc(Ljava/lang/Object;)Ljava/lang/Object;
Full error log
I explicitly tried scalaVersion in every build.sbt file and it is the same.
I tried several things like activator clean, full removal os sbt caches and local repo sbt stuff, updating dependencies to latest version but no success.
I have scala version defined.
My current dependencies are:
I tried with both %% and force _2.11 in the name of the dependency.
Dependency List
Other important files
build.sbt
Common.scala
Dependencies.scala
When I fully clean caches it downloads scala 2.10.4 for no reason:
in this download log it says the sbt need the old scala.
Any idea why is this?
Any ideas?
Firstly, it does not matter which version of Scala is used to generate your build. The build system can use version 2.10.4, and that does not prevent your code from using version 2.11.1.
To look at the issue with your scala version, you should consider that settings added directly in build.sbt are added to the root project, but not to other projects. You can see this with a minimal project such as:
$ cat build.sbt
scalaVersion := "2.11.1"
lazy val subproj = project in (file("subproj"))
Then the output of sbt looks like this:
> show scalaVersion
[info] subproj/*:scalaVersion
[info] 2.10.4
[info] sbttest/*:scalaVersion
[info] 2.11.1
So, how can this be fixed?
We can create a lazy val containing a Seq[Setting[_]], and add it to the sub-project definition with the settings() method.
Here's an example build.sbt which adds the same settings to all projects:
$ cat build.sbt
lazy val root = project.in(file("."))
.aggregate(subproj)
.dependsOn(subproj)
.settings(commonSettings: _*)
lazy val subproj = project.in(file("subproj"))
.settings(commonSettings: _*)
lazy val commonSettings = Seq(
scalaVersion := "2.11.1"
)
The _* is required because settings() is defined as requiring Setting[_]* rather than Seq[Setting[_]], so _* converts between the two types. See also: What does param: _* mean in Scala?
And the output from sbt is:
> show scalaVersion
[info] subproj/*:scalaVersion
[info] 2.11.1
[info] root/*:scalaVersion
[info] 2.11.1
This approach can easily be applied to your build.
You're defining your dependencies in the wrong way.
When using SBT, don't use:
"org.mydep" % "mydep_2.11" % "1.0.0"
Instead use:
"org.mydep" %% "mydep" % "1.0.0"
The %% operator automatically appends the current Scala version of the current build. If you use dependencies built against other Scala versions, you're going to be getting the weird errors.
Also make sure you explicitly specify your Scala version somewhere:
scalaVersion := "2.11.1"
The problem was related to some manually added jars that I discovered in the lib folder, that were causing issues with the dependencies of the project defined in the build.sbt.
The only way to find out was to generate a activator dist and look for similar dependencies with different versions since in the dependency graph there were no conflicts
I need to build a single jar, including dependencies, for one of my sub-projects so that it can be used as a javaagent.
I have a multi-module sbt project and this particular module is the lowest level one (it's also pure Java).
Can I (e.g. with sbt-onejar, sbt-proguard or sbt assembly) override how the lowest level module is packaged?
It looks like these tools are really designed to be a post-publish step, but I really need a (replacement or additional) published artefact to include the dependencies (but only for this one module).
UPDATE: Publishing for sbt-assembly are instructions for a single project, and doesn't easily translate into multi-project.
Publishing for sbt-assembly are instructions for a single project, and doesn't easily translate into multi-project.
People have been publishing fat JAR using sbt-assembly & sbt-release without issues. Here's a blog article from 2011: Publishing fat jar created by sbt-assembly. It boils down to adding addArtifact(Artifact(projectName, "assembly"), sbtassembly.AssemblyKeys.assembly) to your build.sbt (note that the blog is a little out of date AssemblyKeys is now a member of sbtassembly directly).
For sbt 0.13 and above, I prefer to use build.sbt for multi-projects too, so I'd write it like:
import AssemblyKeys._
lazy val commonSettings = Seq(
version := "0.1-SNAPSHOT",
organization := "com.example",
scalaVersion := "2.10.1"
)
val app = (project in file("app")).
settings(commonSettings: _*).
settings(assemblySettings: _*).
settings(
artifact in (Compile, assembly) ~= { art =>
art.copy(`classifier` = Some("assembly"))
}
).
settings(addArtifact(artifact in (Compile, assembly), assembly).settings: _*)
See Defining custom artifacts:
addArtifact returns a sequence of settings (wrapped in a SettingsDefinition). In a full build configuration, usage looks like:
...
lazy val proj = Project(...)
.settings( addArtifact(...).settings : _* )
...