I am using the SBT native packager plugin (https://github.com/sbt/sbt-native-packager) for a project composed of multiple modules.
In my SBT settings I have:
lazy val settings = packageArchetype.java_application ++ Seq(
...
// Java is required to install this application
debianPackageDependencies in Debian ++= Seq("java2-runtime"),
// Include the module JAR in the ZIP file
mappings in Universal <+= (packageBin in Compile) map { jar =>
jar -> ("lib/" + jar.getName)
}
)
The problem is that the generated ZIP, or DEB for example, do not seem to include my project's modules dependencies. There is only the final module JAR, and the libraries used in it, but not the modules that it depends on.
Do you know how could I fix that?
Found a solution to my problem:
I needed to add exportJars := true in my settings for all my internal dependencies to be embedded in the package.
Related
Disclaimer: I am new to sbt and Scala so I might be missing obvious things.
My objective here is to use the Scala compiler as a library from my main project. I was initially doing that by manually placing the scala jars in a libs directory in my project and then including that dir in my classpath. Note that at the time I wasn't using sbt. Now, I want to use sbt and also download the scala sources from github, build the scala jars and then build my project. I start by creating 2 directories: myProject and myProject/project. I then create the following 4 files:
The sbt version file:
// File 1: project/build.properties
sbt.version=0.13.17
The plugins file (not relevant to this question):
// File 2: project/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-buildinfo" % "0.7.0")
The build.sbt file:
// File 3: build.sbt
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
organization := "me",
scalaVersion := "2.11.12",
version := "0.1.0-SNAPSHOT"
)),
name := "a name"
).dependsOn(ScalaDep)
lazy val ScalaDep = RootProject(uri("https://github.com/scala/scala.git"))
My source file:
// File 4: Test.scala
import scala.tools.nsc.MainClass
object Test extends App {
println("Hello World !")
}
If I run sbt inside myProject then sbt will download the scala sources from github and then try to compile them. The problem is that the base-directory is still myProject. This means that if the scala sbt source files refer to something that is in the scala base-directory they won't find it. For example, the scala/project/VersionUtil.scala file tries to open the scala/versions.properties file that lies in the scala base-directory.
Question: How can I set sbt to download a github repo and then build it using that project's base-directory instead of mine's (by that I mean the base-directory of myProject in the above example) ??
Hope that makes sense.
I would really appreciate any feedback on this.
Thanks in advance !
In the Scala ecosystem you usually depend on binary artifacts (libraries) that are published in Maven or Ivy repositories. Virtually all Scala projects publish binaries, including the compiler. So all you have to do is add the line below to your project settings:
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
dependsOn is used for dependencies between sub-projects in the same build.
For browsing sources you could use an IDE. IntelliJ IDEA can readily import Sbt projects and download/attach sources for library dependencies. Eclipse has an Sbt plugin that does the same. Ensime also, etc. Or just git clone the repository.
I need to build a single jar, including dependencies, for one of my sub-projects so that it can be used as a javaagent.
I have a multi-module sbt project and this particular module is the lowest level one (it's also pure Java).
Can I (e.g. with sbt-onejar, sbt-proguard or sbt assembly) override how the lowest level module is packaged?
It looks like these tools are really designed to be a post-publish step, but I really need a (replacement or additional) published artefact to include the dependencies (but only for this one module).
UPDATE: Publishing for sbt-assembly are instructions for a single project, and doesn't easily translate into multi-project.
Publishing for sbt-assembly are instructions for a single project, and doesn't easily translate into multi-project.
People have been publishing fat JAR using sbt-assembly & sbt-release without issues. Here's a blog article from 2011: Publishing fat jar created by sbt-assembly. It boils down to adding addArtifact(Artifact(projectName, "assembly"), sbtassembly.AssemblyKeys.assembly) to your build.sbt (note that the blog is a little out of date AssemblyKeys is now a member of sbtassembly directly).
For sbt 0.13 and above, I prefer to use build.sbt for multi-projects too, so I'd write it like:
import AssemblyKeys._
lazy val commonSettings = Seq(
version := "0.1-SNAPSHOT",
organization := "com.example",
scalaVersion := "2.10.1"
)
val app = (project in file("app")).
settings(commonSettings: _*).
settings(assemblySettings: _*).
settings(
artifact in (Compile, assembly) ~= { art =>
art.copy(`classifier` = Some("assembly"))
}
).
settings(addArtifact(artifact in (Compile, assembly), assembly).settings: _*)
See Defining custom artifacts:
addArtifact returns a sequence of settings (wrapped in a SettingsDefinition). In a full build configuration, usage looks like:
...
lazy val proj = Project(...)
.settings( addArtifact(...).settings : _* )
...
I want to create a standalone version of my application and was wondering how i could exclude
an unmanaged *.jar file to be packaged. It's the "mariaDB4j-2.0-SNAPSHOT.jar" file I solely
use in tests which is about 56MB huge.
I tried to put the jar file into a custom directory 'test/lib'. Unfortunately, this did not exclude mariaDB4j from packaging.
unmanagedBase <<= baseDirectory { base => base / "test/lib" }
unmanagedJars in Test <<= unmanagedBase map { base => (base ** "mariaDB4j-2.0-SNAPSHOT.jar").classpath }
Any thoughts on this?
Cheers
Oliver
I think you want to add to the testing classpath.
Two things:
You can check out what's on the classpath using show test:fullClasspath to make sure your jar is on there. Using inspect test:fullClasspath will show you what the dependencies used for testing are.
I think you can directly add your jar to the classpath via:
fullClasspath in Test += Attributed.blank(baseDirectory.value / "test/lib/mariaDB4j-2.0-SNAPSHOT")
Hope that helps!
This works, but it looks a little overstated. Changing the base directory of the unmanaged dependencies, include the file to the test's and exclude it from compile.
unmanagedBase <<= baseDirectory { base => base / "test/lib" }
unmanagedJars in Test <<= unmanagedBase map { base => (base ** "mariaDB4j-2.0-SNAPSHOT.jar").classpath }
excludeFilter in unmanagedJars in Compile := "mariaDB4j-2.0-SNAPSHOT.jar"
excludeFilter in unmanagedJars in Compile ~= { _ || "mariaDB4j-2.0-SNAPSHOT.jar" }
don't use unmanaged dependencies
if you want to keep the jar in your source repository just use a file based maven repository in your source tree with
resolvers += "Private Maven Repository" at file(".").toURI.toURL+"/repository"
then mvn install MariaDB4j locally and copy resulting stuff from maven cache to $yourproject/repository
and use the dependency like a regular managed dependency
I am attempting to use sbt with the following plugin https://github.com/siasia/xsbt-proguard-plugin. So far I haven't had any issues with the plugin, apart from the fact that proguard puts all of the unmanaged jars into the final min.jar file (causing problems with multiple jar's that conflict). Proguard has the proguardLibraryJars flag which allows you to specify jars for proguard to exclude
Essentially I want to add all of the jars from the TaskKey unamangedJars to proguardLibraryJars using the plugin, i.e. do something like this
lazy val proguard = proguardSettings ++ Seq(
proguardOptions := Seq(
keepMain("com.test.FacebookPostScheduler"),
keepMain("org.postgresql.Driver")
),
proguardLibraryJars <++= unmanagedClasspath
)
The problem is the above obviously doesn't compile at this line
proguardLibraryJars <++= unmanagedClasspath
with the
No implicit for Append.Values[Seq[java.io.File], sbt.Keys.Classpath] found
error.
How would you code what I am attempting to do using the latest SBT (0.11.3-2) using a Build.scala (not a build.sbt)
I have a public repository of an SBT plugin that manages to pass jars to proguard. It doesn't use the proguard plugin but the code might help you figure how to gather dependencies.
https://github.com/tlazaro/xsbt-plugin-deployer/blob/master/src/main/scala/Deployer.scala
Look for:
private def getDepsJars(project: ProjectRef, bs: BuildStructure) = forAllProjects(project, bs) {p =>
artifactPath in Compile in packageBin in p get bs.data
}
That might give you a way to get started. It gathers every needed jar which is what you usually want, not just the unmanaged.
Alternatively you can just use this plugin and maybe collaborate. The code is a bit sloppy, it not intended to be released yet. The plugin does some other neat stuff like compressing everything with pack200 into a jar and having a custom ClassLoader that loads the compressed classes from there in runtime.
adamw/xsbt-proguard-plugin, which is the successor of siasia/xsbt-proguard-plugin seems to have the very option:
By default Proguard will be instructed to include everything except classes from the Java runtime. To treat additional libraries as external (i.e. to add them to the list of -libraryjars passed to Proguard), do the following. Here comes the example how to select a module named "httpclient" from the library dependencies:
proguardLibraryJars <++= (update) map (_.select(module = moduleFilter(name = "httpclient")))
I am trying to figure out how idea will recognize thrid party dependencies when using SBT. When I use the sbt plugin gen-idea it seems to download all the necessary dependencies which get put into my ~/.ivy/ directory as expected. How can intellij use these deps?
EDIT:
One thing I noticed is if I make a new idea project instead of just a module then this works? Any idea why this would be? I would like to be able to have multiple sbt modules in the same project.
The sbt-idea plugin works with multi-module sbt project. We have been using it since somewhere around sbt-0.10.0, and currently are at sbt-0.11.2. It seems like you have the dependency part of the build file set up ok, so here's an example of how we do the project setup from a full specification Build.scala file:
object Vcaf extends Build {
import Resolvers._
import Dependencies._
import BuildSettings._
lazy val vcafDb = Project(
id = "vcaf-db",
base = file("./vcaf-db"),
dependencies = Seq(),
settings = buildSettings ++ /* proguard */ SbtOneJar.oneJarSettings ++ Seq(libraryDependencies := dbDeps, resolvers := cseResolvers)
)
lazy val vcaf = Project(
"vcaf",
file("."),
dependencies = Seq(vcafDb),
aggregate = Seq(vcafDb),
settings = buildSettings ++ Seq(libraryDependencies := vcafDeps, resolvers := cseResolvers) ++ webSettings
)
}
In the example, the vcaf-db project is in the a folder within the vcaf project folder. The vcaf-db project does not have it's own build.sbt or Build.scala file. You'll notice that we are specifying libraryDependencies for each project, which may or may not be your missing link.
As ChrisJamesC mentioned, you need to do a "reload" from within SBT (or exit sbt and come back in) to pick up changes to your build definition. After the project is reloaded, you should be able to do a "gen-idea no-classifiers no-sbt-classifiers" and get an intellij project that has the main project, modules, and library access as defined in the build file.
Hope it helps!
If you want multiple SBT modules in one IDEA project, you can use sbt multi-project builds (aka subprojects). Just create a master project that refers to the modules as sub-projects, then run gen-idea on the master. To specify dependencies among the modules you have to use Build.scala (not build.sbt), as in jxstanford's answer or like this:
lazy val foo = Project(id = "foo", base = file("foo"))
lazy val bar = Project(id = "bar", base = file("bar")) dependsOn(foo)
One level of subprojects works fine (with the dependencies correctly reflected in the resulting IDEA project), but nested subprojects don't seem to work. Also, it seems to be an sbt restriction that the subprojects must live in subdirectories of the master project (i.e., file("../foo") is not allowed).
See also How to manage multiple interdependent modules with SBT and IntelliJ IDEA?.