I am trying to create an executable jar using SBT assembly plugin.
I am ending up with below error :
[error] (app/*:assembly) deduplicate: different file contents found in the following:
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-continuation/jars/jetty-continuation-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-http/jars/jetty-http-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-io/jars/jetty-io-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-security/jars/jetty-security-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-server/jars/jetty-server-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-servlet/jars/jetty-servlet-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-util/jars/jetty-util-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-webapp/jars/jetty-webapp-8.1.8.v20121106.jar:about.html
[error] /Users/rajeevprasanna/.ivy2/cache/org.eclipse.jetty/jetty-xml/jars/jetty-xml-8.1.8.v20121106.jar:about.html
[error] Total time: 2562 s, completed Dec 5, 2013 12:03:25 PM
After reading wikis of assembly plugin, i have added merge strategy in build.scala file. Seems it is not working. I am not sure whether it is right fix or not. Can someone suggest me the right strategy.
Below is the code which i have in build.scala file :
mergeStrategy in assembly <<= (mergeStrategy in assembly) {
(old) => {
case "about.html" => MergeStrategy.discard
case "logback.xml" => MergeStrategy.first //case PathList("logback.xml") => MergeStrategy.discard
case x => old(x)
}
}
I have coded plugin integration with my app as per this doc : Standalone deployment of Scalatra servlet
I tried diffrent strategies like MergeStrategy.rename and MergeStrategy.deduplicate. But nothing works..
Looking for help...
Your MergeStrategy looks correct. The only unhandled conflicts are "about.html" in the jetty jars, so case "about.html" => MergeStrategy.discard should just do it.
If you're still getting the error, I suspect that re-wiring of the mergeStrategy in assembly setting is either not going in, or going in the wrong order. The only way to know for sure is to see your Build.scala. #Stefan Ollinger's answer to your linked question for example sets up the project as follows:
lazy val project = Project("myProj", file(".")).
settings(mySettings: _*).
settings(myAssemblySettings:_*)
Could you post your Build.scala on gist if possible?
Related
I have a scala-spark project and I have created a fat jar using sbt assembly. An error raised when I tried to run jar, "Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: json. Please find packages at http://spark.apache.org/third-party-projects.html". I added this code block to build.sbt;
assemblyMergeStrategy in assembly := {
case "META-INF/services/org.apache.spark.sql.sources.DataSourceRegister" => MergeStrategy.concat
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case x => MergeStrategy.first
}
It did not work, I would be very grateful for any help
Best regards
I am trying to pack one of the modules of my application into a war.
I have chosen xsbt-web-plugin to help me out.
I have prepared the sbt, I guess correctly:
lazy val `my-project` = (project in file("my-project"))
...
.enablePlugins(TomcatPlugin)
But during sbt package I get this error:
[info] Packaging /home/siatkowskim/Documents/....target/scala-2.11/my-project_2.11-1.2-SNAPSHOT.war ...
[error] java.util.zip.ZipException: duplicate entry: META-INF/MANIFEST.MF
I am familiar with sbt-assembly but I see no way of deduplication here.
How can I even debug, where is it duplicated from? Or how to solve this duplication?
Turned out I had MANIFEST.MF file in my classpath.
I do not know what it was for, but removing it solved the problem.
I have the same issue, yet I had no evident MANIFEST.MF file in my classpath. I can only suppose that it was coming from the multitude of .jar files included.
The following solved the problem:
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs # _*) =>
(xs map {_.toLowerCase}) match {
case ("manifest.mf" :: Nil) | ("index.list" :: Nil) | ("dependencies" :: Nil) => MergeStrategy.discard
case _ => MergeStrategy.last
}
}
See here to understand what the double colon notation means.
Every now and then I start things from scratch to make sure I know what all the intricate details of setting up a project are. I have a simple app like the following but I get some dependency issues that don't seem to point me anywhere. I'm using scala verions 2.11.
My SBT:
name := "Helios"
version := "1.0"
scalaVersion := "2.11.8"
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
libraryDependencies ++= Seq(
"com.typesafe.akka" % "akka-actor" % "2.0.2",
"com.typesafe.akka" % "akka-slf4j" % "2.0.5")
My Sample Class
import com.echostar.ese.helios.core.Asset
import akka.actor._
class NSPSG extends Actor {
def receive = {
case a: Asset => {
println(s"NSPSG Received asset: ${a}")
}
case _ => println("Unexpected message received")
}
}
(Asset class is just a case class with id and title in it.)
Error Message:
C:\PROJECTS\active\Helios>sbt compile
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
[info] Loading global plugins from C:\Users\dana.murad\.sbt\0.13\plugins
[info] Loading project definition from C:\PROJECTS\active\Helios\project
[info] Set current project to Helios (in build file:/C:/PROJECTS/active/Helios/)
[info] Updating {file:/C:/PROJECTS/active/Helios/}helios...
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[info] Compiling 3 Scala sources to C:\PROJECTS\active\Helios\target\scala-2.11\classes...
[error] missing or invalid dependency detected while loading class file 'package.class'.
[error] Could not access type ScalaObject in package scala,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
[error] A full rebuild may help if 'package.class' was compiled against an incompatible version of scala.
[error] missing or invalid dependency detected while loading class file 'Actor.class'.
[error] Could not access type ScalaObject in package scala,
[error] because it (or its dependencies) are missing. Check your build definition for
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
[error] A full rebuild may help if 'Actor.class' was compiled against an incompatible version of scala.
[error] C:\PROJECTS\active\Helios\src\main\scala\com\echostar\ese\helios\workers\NSPSG.scala:9: illegal inheritance;
[error] self-type com.echostar.ese.helios.workers.NSPSG does not conform to akka.actor.Actor's selftype akka.actor.Actor
[error] class NSPSG extends Actor {
[error] ^
[error] three errors found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Apr 4, 2016 9:19:45 AM
My main App is just a println for now. It's not even calling this actor.
Am I using the wrong version of akka with scala 2.11? -Ylog-classpath didn't help
Don't know what fixed it but here's a list of things I did and the project compiles now.
Changed akka dependency line to this (added double percent and changed version back to 2.4)
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.4.0"
)
Removed my Run configuration and added it back again. ( I think the path to main showed as red (invalid) so redoing it helped resolve it. I changed my package name and I don't think intelliJ picked up the rename well in build-conf)
Started another test project from IntelliJ menu (Akka Main in Scala) and that took a bit to download all dependencies. So maybe my project needed those and wasn't download them?
Removed commented out lines (got rid of a semi-colon). I don't think this did anything but full disclosure, technically I did touch the code even though my actor definition is exactly the same.
Can somebody provide me with an example build.sbt file showing how to use revolver with multiple projects?
Specifically where should the Revolver.settings statement go, at the top level or in one or more of the sub-projects? The top level does not work for me and I cannot add it to a sub-project without the build breaking.
For what it's worth when I add the settings at the project top-level I get the following error
> project backend
> re-start 2551
[error] Expected ID character
[error] Not a valid command: re-start
[error] Expected project ID
[error] Expected configuration
[error] Expected ':' (if selecting a configuration)
[error] Expected key
[error] Expected '::'
[error] Not a valid key: re-start (similar: state, startYear, cross-target)
[error] re-start 2551
[error] ^
Turns out you need to create a second build.sbt file in the sub-projects directory, and put the settings for it there. An example of this stub build file is below:
Revolver.settings
javaOptions in run ++= Seq("-Djava.library.path=lib")
javaOptions in Revolver.reStart ++= Seq("-Dconfig.resource=dev.conf", "-Djava.library.path=src/universal/lib")
mainClass in Revolver.reStart := Some("com.example.Main")
Here is the attempt to exclude the javax.servlet classes:
libraryDependencies ++= Seq(
..
("org.apache.spark" % "spark-core_2.10" % sparkVersion \
% "compile->default" withSources()).exclude("org.mortbay.jetty", "servlet-api"),
Here is my attempt at a MergeStrategy:
mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) => {
..
case PathList("javax", "servlet", xs # _*) => MergeStrategy.singleOrError
..
Following shows that multiple copies of the javax.servlet classes are being loaded:
[error] (*:assembly) singleOrError: found multiple files for same
target path: [error]
C:\Users\s80035683.ivy2\cache\org.mortbay.jetty\servlet-api-2.5\jars\servlet-api-2.5-6.1.14.jar:javax/servlet/Filter.class
[error]
C:\Users\s80035683.ivy2\cache\javax.servlet\servlet-api\jars\servlet-api-2.5.jar:javax/servlet/Filter.class
[error]
C:\Users\s80035683.ivy2\cache\org.eclipse.jetty.orbit\javax.servlet\orbits\javax.servlet-3.0.0.v201112011016.jar:javax/servlet/Filter.class
[error]
C:\Users\s80035683.ivy2\cache\org.mortbay.jetty\servlet-api\jars\servlet-api-2.5-20110124.jar:javax/servlet/Filter.class
Note: this is not a DUPLICATE since similar questions do NOT address this issue. For example +the answer to the following question ("highest version selected by default") is NOT working
How could SBT choose highest version amongst dependencies?
You can use sbt-dependency-graph and see from where the servlet-api is included as a transitive dependency.
You can do that by using what-depends-on <organization> <module> <revision> command.
Knowing that you can exclude the transitive dependency.
This is easier than excluding class from a specific jar (unless you don't care which jar, and you just want to take it from any jar).
You can also check my answer to another question on how to write a custom MergeStrategy, which would know from which jar the class was comming.