I have a Boot object with below definition
object Boot extends App with xxxService {}
And add below lines in plugins.sbt to enable sbt-assembly
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.15.0")
then run this command to build up the whole project:
sbt clean compile assembly
In the genearted jar META-INF/MAINFEST.MF file, it does not have Main-Class generated, if I specify main class in build.sbt like mainClass in assembly := Some("com.XXX.Boot"), it worked.
My sbt version is 1.3.2, jvm is zulu8, scala is 2.12.13
From enter link description here it says, sbt-assembly will autodetect main class, then how does my example failed?
I update sbt-assembly into below
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "1.2.0")
it can auto detect the main class without specify in build.sbt...
I guess this is might be related with sbt and scala verion...
my colleague can run the config in the question correctly...
Related
I am working on a Scala project on IntelliJ with SBT as my build tool. I started working sbt build recently.
This is my project structure:
This is my build.sbt file:
name := "AnalyzeTables"
version := "0.1"
scalaVersion := "2.11.8"
// https://mvnrepository.com/artifact/org.postgresql/postgresql
libraryDependencies += "org.postgresql" % "postgresql" % "42.1.4"
// https://mvnrepository.com/artifact/commons-codec/commons-codec
libraryDependencies += "commons-codec" % "commons-codec" % "1.13"
// https://mvnrepository.com/artifact/org.apache.commons/commons-lang3
libraryDependencies += "org.apache.commons" % "commons-lang3" % "3.8.1"
// https://mvnrepository.com/artifact/log4j/log4j
//libraryDependencies += "log4j" % "log4j" % "1.2.17"
I have Class.forName("org.postgresql.Driver) in my code to connect to database and query. Along with that, I have password decryption & logger added in the code.
I am running the jar in the below format:
scala <jarname> <argument I use in my code>
The problem here is if I just submit in the way I mentioned above, I see ClassNotFoundException for postgres driver. So I add it to the classpath of the jar and submit it as below.
scala -cp /path/postgresql-42.1.4.jar <jarname> <argument I use in my code>
Now I get exception for Logger. So I add it to classpath again and it becomes:
scala -cp /path/postgresql-42.1.4.jar:/path/log4j-1.2.17.jar <jarname> <argument I use in my code>
Now I get exception for commons-codec, so I added that as well.
scala -cp /path/postgresql-42.1.4.jar:/path/log4j-1.2.17.jar:/path/commons-codec-1.13.jar
Now that jar is running fine and I can see the result.
So I have the dependencies added to the build.sbt file. I also did the below operation:
project structure -> Modules -> Dependencies -> + -> jars -> Add all the missing jars that are giving problems
If I remove all the -cp parameters and submit the jar with just: scala <jarname> <argument> it goes back again to ClassNotFoundException to postgres jar.
So what is the point of adding dependencies on build.sbt file and then add them in the classpath again ?
Is there ay setting I am missing or am I looking it in the wrong way ?
Edit:
After suggestions I created a new project & copied all the code into it and added the plugin addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5") in a new file plugins.sbt file which I created in the dir /project/ as shown in the image below.
I can see the plugin in the sbt-plugins dir. But when I build the jar once again and export, it still shows 11kb jar instead of a fat jar.
What is the best way to make a SBT project work in offline environment?
There is any ways to compile it outside (in network env),
e.g uber jar with all its dependencies, and then in the offline env
just run it?
For example in Java Maven,
We can compile it with uber jar with all dependencies,
and then in the offline env just java -jar MyJar.jar ...
Thanks.
Best bet for creating an uber/fat jar is the sbt assembly plugin. Include it in your project by putting something like the following in project/assembly.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.6")
Then just running sbt assembly should create a jar with all the dependencies in target/scala_$SCALA_VERSION. You may need to designate a main class for the jar if you want to run it with java -jar (as opposed to java -cp $jarfile $mainclass):
mainClass in assembly := Some("package.mainClass")
I'm currently facing a problem with deploying an uber-jar to a Spark Streaming application, where there are congruent JARs with different versions which are causing spark to throw run-time exceptions. The library in question is TypeSafe Config.
After attempting many things, my solution was to defer to shading the provided dependency so it won't clash with the JAR provided by Spark at run-time.
Hence, I went to the documentation for sbt-assembly and under shading, I saw the following example:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("org.apache.commons.io.**" -> "shadeio.#1")
.inLibrary("commons-io" % "commons-io" % "2.4", ...).inProject
)
Attempting to shade over com.typesafe.config, I tried applying the following solution to my build.sbt:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.typesafe.config.**" -> "shadeio.#1").inProject
)
I assumed it was supposed to rename any reference to TypeSafe Config in my project. But, this doesn't work. It matches multiple classes in my project and causing them to be removed from the uber jar. I see this when trying to run sbt assembly:
Fully-qualified classname does not match jar entry:
jar entry: ***/Identifier.class
class name: **/Identifier.class
Omitting ***/OtherIdentifier.class.
Fully-qualified classname does not match jar entry:
jar entry: ***\SparkBaseJobRunner$$anonfun$1.class
class name: ***/SparkBaseJobRunner$$anonfun$1.class
I also attempted using:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.typesafe.config.**" -> "shadeio.#1")
.inLibrary("com.typesafe" % "config" % "1.3.0")
This did finish the assemblying process of the uber JAR, but didn't have the desired run time effect.
I'm not sure I fully comprehend the effect shading has on my build process with sbt.
How can I shade over references to com.typesafe.config in my project so when I invoke the library at run-time Spark will load my shaded library and avoid the clash caused by versioning?
I'm running sbt-assembly v0.14.1
Turns out this was a bug in sbt-assembly where shading was completely broken on Windows. This caused source files to be removed from the uber JAR, and for tests to fail as the said classes were unavailable.
I created a pull request to fix this. Starting version 0.14.3 of SBT, the shading feature works properly. All you need to do is update to the relevant version in plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
In order to shade a specific JAR in your project, you do the following:
assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.typesafe.config.**" -> "my_conf.#1")
.inLibrary("com.typesafe" % "config" % "1.3.0")
.inProject
)
This will rename the com.typesafe.config assembly to be packaged inside my_conf. You can then view this using jar -tf on your assembly (omitted irrelevant parts for brevity):
***> jar -tf myassembly.jar
my_conf/
my_conf/impl/
my_conf/parser/
Edit
I wrote a blog post describing the issue and the process that led to it for anyone interested in a more in-depth explanation.
My build.sbt:
libraryDependencies += "org" %% "A" % "0.0.1"
So I run sbt on this file:
> sbt
I know that there is Main class in 'A', let's say "mainRun.scala". But I don't how to run it from my project.
How should I run it from SBT?
By default, sbt does not look into your dependencies for the auto detection of a main class. You can can however force it to use a specific class, either on the command line with
> runMain pack.MainClass
or via the sbt setting
mainClass := Some("pack.MainClass")
I added the Mailer plugin to my Play Framework 2 project. It compiles and works fine, but IntelliJ can't resolve any of its classes. I would normally just add the jar as a module in my IntelliJ project settings, but I don't have a jar. As far as I understand, the plugin is automatically being pulled from some repository. So how do I make IntelliJ aware of it?
I added this to conf/play.plugins
1500:com.typesafe.plugin.CommonsMailerPlugin
And this as a dependency in project/build.scala
"com.typesafe" %% "play-plugins-mailer" % "2.0.4"
I resolve problems like this with the sbt-idea SBT plugin. Just add this to your project/plugins.sbt file:
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.2.0")
Then, whenever you change your project dependencies, run sbt idea and your IntelliJ project will be updated.
I had to delete the reference to com.github.mpeltonen in my plugins.sbt file first. Then as Bill said, I needed to run play idea. If I tried to run play idea without deleting the reference I got this exception:
java.lang.NoSuchMethodError: org.sbtidea.SbtIdeaPlugin$.ideaSettings()Lscala/collection/Seq;