Remove entry from classpath after compile - scala

I have a legacy war project that depends on a jar project, the jar project needs to add a few unmanaged jars to the classpath for compilation. But these jars should not be packaged in the war. So my question is how do I remove these entries from the fullClasspath. The following won't work:
val excludeFilter = "(servlet-api.jar)|(gwt-dev.jar)|(gwt-user.jar)"
val filteredCP = cp.flatMap({ entry =>
val jar = entry.data.getName()
if (jar.matches(excludeFilter)) {
Nil
} else {
Seq(entry)
}
})
fullClasspath in Runtime = filteredCP
I am pretty sure there must be simple way to do this but so far it has eluded me.
Edit: Based on Pablo's sugestion to use the managed classpath instead of the unmanaged I can rephrase the question as: how do you add local jars to the managedClasspath. My jars are placed in a local folder with a (very) nonstandard layout:
lib/testng.jar
lib/gwt/2.3/gwt-user.jar
lib/jetty/servlet.jar
So basically I am looking for something like:
libraryDependencies += "testng" % "provided->test"
libraryDependencies += "gwt" % "2.3" % "gwt-user" % "provided->compile"
libraryDependencies += "jetty" % "servlet" % "provided->default"
allowing me to grab jars from my own local lib folder.

Some information is provided on the Classpaths page, but it is not very clear or detailed. The information is also available using the inspect command, described on the Inspecting Settings page.
Basically, for a configuration X, in a short-hand notation:
// complete, exported classpath, such as used by
// 'run', 'test', 'console', and the war task
fullClasspath in X =
dependencyClasspath in X ++ exportedProducts in X
// classpath only containing dependencies,
// used by 'compile' or 'console-quick', for example
dependencyClasspath in X =
externalDependencyClasspath in X ++ internalDependencyClasspath in X
// classpath containing only dependencies external to the build
// (as opposed to the inter-project dependencies in internalDependencyClasspath)
externalDependencyClasspath in X =
unmanagedClasspath in X ++ managedClasspath in X
// the manually provided classpath
unmanagedClasspath in X =
unmanagedJars for X and all configurations X extends, transitively
So, normally, when you want to add unmanaged libraries, you add them to unmanagedJars. For example, if you add libraries to unmanagedJars in Compile, then sbt will correctly include the libraries on the unmanagedClasspath for Compile, Runtime, and Test.
However, you want explicit control here. Add the libraries only to the unmanagedClasspath you want the jars on, which is unmanagedClasspath in Compile. For example, in sbt 0.11.0+:
unmanagedClasspath in Compile <++= baseDirectory map { base =>
val lib = base / "lib"
Seq(
lib / "testng.jar",
lib / "gwt/2.3/gwt-user.jar",
lib / "jetty/servlet.jar"
)
}
Assuming the war plugin uses the Runtime classpath, those jars will only show up on the compile classpath and not in the war.

sbt supports ivy-like configurations, and implements maven basic scopes.
If you want to use some jars in your compilation classpath but don't want to ship them, I guess the provided scope is for you:
libraryDependencies += "org.example" % "example" % "1.0" % "provided->compile"

Related

Creating an aspectJ library using sbt-aspect

I am in the process of writing a library that does monitoring/OpenTracing and I am attempting to use sbt-aspectj so that users of the library don't need to manually instrument their code. I am currently however getting an issue with creating an sbt-project representing such a library.
The idea is that I want an external library as indicated in this sample here https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/external however that external library is dependant on an external dependency (i.e. akka-actors). Basically I am trying to combine both https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/external and https://github.com/sbt/sbt-aspectj/tree/master/src/sbt-test/weave/jar. I have created a sample project here https://github.com/mdedetrich/sbt-aspectj-issue to indicate the problem I am having however below is the relevant sample
lazy val root = (project in file("."))
.enablePlugins(SbtAspectj)
.settings(
name := RootName,
version := Version,
// add akka-actor as an aspectj input (find it in the update report)
// aspectjInputs in Aspectj ++= update.value.matching(
// moduleFilter(organization = "com.typesafe.akka", name = "akka-actor*")),
// replace the original akka-actor jar with the instrumented classes in runtime
// fullClasspath in Runtime := aspectjUseInstrumentedClasses(Runtime).value,
// only compile the aspects (no weaving)
aspectjCompileOnly in Aspectj := true,
// ignore warnings (we don't have the target classes at this point)
aspectjLintProperties in Aspectj += "invalidAbsoluteTypeName = ignore",
// replace regular products with compiled aspects
products in Compile ++= (products in Aspectj).value,
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion
)
)
lazy val test = (project in file("test"))
.enablePlugins(SbtAspectj)
.settings(
aspectjBinaries in Aspectj ++= update.value.matching(
moduleFilter(organization = Organization, name = s"$RootName*")),
aspectjInputs in Aspectj ++= update.value.matching(
moduleFilter(organization = "com.typesafe.akka", name = "akka-actor*")),
fullClasspath in Runtime := aspectjUseInstrumentedClasses(Runtime).value,
// weave this project's classes
aspectjInputs in Aspectj += (aspectjCompiledClasses in Aspectj).value,
products in Compile := (products in Aspectj).value,
products in Runtime := (products in Compile).value,
libraryDependencies ++= Seq(
Organization %% RootName % Version
)
)
The idea is that we publish the root project using root/publishLocal and the test project is just designed to include root as a libraryDependency so we can see if the aspect-j is working properly.
The problem is simple that I am unable to get it working. The current code at https://github.com/mdedetrich/sbt-aspectj-issue publishes with root/publishLocal (not sure if its correct though) however when I then do test/run I get this
[info] Weaving 2 inputs with 1 AspectJ binary to /home/mdedetrich/github/sbt-aspectj-issue/test/target/scala-2.13/aspectj/classes...
[error] stack trace is suppressed; run last test / Compile / packageBin for the full output
[error] (test / Compile / packageBin) java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
[error] Total time: 1 s, completed Dec 29, 2019 4:31:27 PM
sbt:sbt-aspectj-issue>
Which seems to be an issue with having duplicate akka-actor entries. I tried toggling various entries in build.sbt but didn't manage to get it working.
EDIT: This was also posted as a github issue here https://github.com/sbt/sbt-aspectj/issues/44
Generally, you can exclude the META-INF directories from the external libraries woven.
mappings in (Compile, packageBin) := {
(mappings in (Compile, packageBin)).value
.filterNot(_._2.startsWith("META-INF/"))
}
But for akka libraries, there is another problem. In each akka library, there is a reference.conf, which contains the fallback configuration for the provided features. This will also lead to conflicts like the META-INF did. But it cannot be just excluded like the META-INF, because they are essential for akka to work properly.
If you exclude them, you'll have to provide all the required akka configurations in your application.conf, or a merged (not simply concatenate) reference.conf in your project. It's not trivial, and subject to version change of akka.
Another solution would be weaving and repackaging the akka libraries individually, so the reference.conf can be kept in the repackaged libraries. The project layout and build script will a bit more complicated, but also be easier to maintain if you have plan to upgrade to newer versions of akka in the future.

Multiproject SBT build bad symbolic reference scala-reflect runtime

I've been tasked with rewriting an old ant build script to SBT. As it happens, our suite is built up of 3 modules:
A Play 2.3 front-end webserver;
A back-end for retrieving data from various other systems;
A middle module containing some shared classes for database access and business logic.
Below an excerpt of my Build.scala file can be found:
val sharedSettings = Seq(
organization := <organization here>,
version := "1.2.5",
scalaVersion := "2.11.1",
libraryDependencies ++= libraries,
unmanagedJars in Compile ++= baseDirectory.value / "lib",
unmanagedJars in Compile ++= baseDirectory.value / "src",
unmanagedJars in Compile ++= baseDirectory.value / "test"
)
lazy val middle = project.settings(sharedSettings: _*)
lazy val back = project.settings(sharedSettings: _*).dependsOn(middle)
However, when I try to compile the source, I get the following error:
bad symbolic reference to scala.reflect.runtime encountered in class file 'ValueConverter.class'. Cannot access term runtime in package scala.reflect. The current classpath may be missing a definition for scala.reflect.runtime, or ValueConverter.class may have been compiled against a version that's incompatible with the one found on the current classpath.
The source code is organized in the following structure:
back
src
test
lib
middle
src
test
lib
front
src
test
lib
Here each lib folder contains some manually maintained libraries (which is why we want to move to sbt).
Any ideas on how to solve this?
In the end, I gave up on trying to get the compiler to understand the additional libraries. Eventually, I added those dependencies that were available using sbt, to the sbt managed libraries. This apparently works well.

How to exclude unnecessary unmanaged dependencies from packaging?

I want to create a standalone version of my application and was wondering how i could exclude
an unmanaged *.jar file to be packaged. It's the "mariaDB4j-2.0-SNAPSHOT.jar" file I solely
use in tests which is about 56MB huge.
I tried to put the jar file into a custom directory 'test/lib'. Unfortunately, this did not exclude mariaDB4j from packaging.
unmanagedBase <<= baseDirectory { base => base / "test/lib" }
unmanagedJars in Test <<= unmanagedBase map { base => (base ** "mariaDB4j-2.0-SNAPSHOT.jar").classpath }
Any thoughts on this?
Cheers
Oliver
I think you want to add to the testing classpath.
Two things:
You can check out what's on the classpath using show test:fullClasspath to make sure your jar is on there. Using inspect test:fullClasspath will show you what the dependencies used for testing are.
I think you can directly add your jar to the classpath via:
fullClasspath in Test += Attributed.blank(baseDirectory.value / "test/lib/mariaDB4j-2.0-SNAPSHOT")
Hope that helps!
This works, but it looks a little overstated. Changing the base directory of the unmanaged dependencies, include the file to the test's and exclude it from compile.
unmanagedBase <<= baseDirectory { base => base / "test/lib" }
unmanagedJars in Test <<= unmanagedBase map { base => (base ** "mariaDB4j-2.0-SNAPSHOT.jar").classpath }
excludeFilter in unmanagedJars in Compile := "mariaDB4j-2.0-SNAPSHOT.jar"
excludeFilter in unmanagedJars in Compile ~= { _ || "mariaDB4j-2.0-SNAPSHOT.jar" }
don't use unmanaged dependencies
if you want to keep the jar in your source repository just use a file based maven repository in your source tree with
resolvers += "Private Maven Repository" at file(".").toURI.toURL+"/repository"
then mvn install MariaDB4j locally and copy resulting stuff from maven cache to $yourproject/repository
and use the dependency like a regular managed dependency

Simple build tool (SBT) package WAR

I have a multiple project configuration. One of the projects needs to be build as a war file for future deployment. I used this sbt-plugin: https://github.com/JamesEarlDouglas/xsbt-web-plugin to build the war. However I need it to include depended projects as jars.
In maven I included other modules as a dependency in my WAR module and they appeared eventually in the WAR:lib directory. It seems that sbt-web-plugin default behavior is not to include them
What I mean is:
This is a part of my parent.scala file
lazy val dataPopulator = Project(
"data-populator",
file("data-populator"),
settings = buildSettings ++ Seq (libraryDependencies ++= dataPopulatorDeps))
lazy val warProject = Project(id = "rest-ws",
base = file("rest-ws"),
settings = buildSettings ++ Seq (libraryDependencies)) dependsOn(dataPopulator)
The dataPopulator Project above when packaged creates a jar.
The warProject Project above have a specific build.sbt in it's directory
rest-ws/build.sbt:
seq(webSettings :_*)
name := "main-ws"
libraryDependencies += "org.mortbay.jetty" % "jetty" % "6.1.22" % "container"
When I run the package command (added by the web plugin) it creates a war, the problem is that this war doesn't include dataPopulator jar while it depends on it during compile time.
Anyone have a suggestion how to include generated jar artifacts from some modules into another project that is packaged as war ?

Adding unmanagedJars to proguardLibraryJars in SBT

I am attempting to use sbt with the following plugin https://github.com/siasia/xsbt-proguard-plugin. So far I haven't had any issues with the plugin, apart from the fact that proguard puts all of the unmanaged jars into the final min.jar file (causing problems with multiple jar's that conflict). Proguard has the proguardLibraryJars flag which allows you to specify jars for proguard to exclude
Essentially I want to add all of the jars from the TaskKey unamangedJars to proguardLibraryJars using the plugin, i.e. do something like this
lazy val proguard = proguardSettings ++ Seq(
proguardOptions := Seq(
keepMain("com.test.FacebookPostScheduler"),
keepMain("org.postgresql.Driver")
),
proguardLibraryJars <++= unmanagedClasspath
)
The problem is the above obviously doesn't compile at this line
proguardLibraryJars <++= unmanagedClasspath
with the
No implicit for Append.Values[Seq[java.io.File], sbt.Keys.Classpath] found
error.
How would you code what I am attempting to do using the latest SBT (0.11.3-2) using a Build.scala (not a build.sbt)
I have a public repository of an SBT plugin that manages to pass jars to proguard. It doesn't use the proguard plugin but the code might help you figure how to gather dependencies.
https://github.com/tlazaro/xsbt-plugin-deployer/blob/master/src/main/scala/Deployer.scala
Look for:
private def getDepsJars(project: ProjectRef, bs: BuildStructure) = forAllProjects(project, bs) {p =>
artifactPath in Compile in packageBin in p get bs.data
}
That might give you a way to get started. It gathers every needed jar which is what you usually want, not just the unmanaged.
Alternatively you can just use this plugin and maybe collaborate. The code is a bit sloppy, it not intended to be released yet. The plugin does some other neat stuff like compressing everything with pack200 into a jar and having a custom ClassLoader that loads the compressed classes from there in runtime.
adamw/xsbt-proguard-plugin, which is the successor of siasia/xsbt-proguard-plugin seems to have the very option:
By default Proguard will be instructed to include everything except classes from the Java runtime. To treat additional libraries as external (i.e. to add them to the list of -libraryjars passed to Proguard), do the following. Here comes the example how to select a module named "httpclient" from the library dependencies:
proguardLibraryJars <++= (update) map (_.select(module = moduleFilter(name = "httpclient")))