Running multiple Scala apps from one jar on JVM - scala

I have a Scala application that successfully runs on the JVM using an uber jar via the command: java -jar myapp.jar. I need to create a separate, but related Scala job that utilizes many of the same objects/functions/dependencies as the first, making it a great candidate to keep in the same code repository & uber jar. Please note that these Scala jobs do not utilize Spark, so spark2-submit is out of the equation.
Question: How can I run 2 separate Scala jobs from the same uber jar on the JVM? (I am using Scala 2.11.8 and SBT for jar assembly)
Additional Context:
I've already looked into related StackOverflow discussions, namely this post about specifying Java classes using java -cp myapp.jar MyClass and this post, which only presented the solution of running the Scala equivalent using scala -classpath myapp.jar MyClass.
While the scala -classpath solution may have worked for the OP of the second linked discussion, I'll be deploying my code to an environment that doesn't have executables for scala or sbt, only java.
Let's say these are the 2 Scala jobs I want to run:
// MyClass.scala
package mypackage
object MyClass {
def main(args: Array[String]): Unit = {
println("Hello, World!")
}
}
// MyClass2.scala
package mypackage
object MyClass2 {
def main(args: Array[String]): Unit = {
println("Hello, World! This is the second job!")
}
}
Is there a way to run Scala code using java -cp myapp.jar MyClass?
I've tried this and receive the following error:
Error: Could not find or load main class MyClass
The main alternative I can think of would be to create a Scala object that serves as a main entry point and takes a parameter to determine which job gets run. I'd like to avoid that solution if possible, but it would allow me to continue using java -jar myapp.jar, which has been working fine.

You need to use a fully qualified name for the App instance:
java -cp myapp.jar mypackage.MyClass

Related

Compile Akka actor files using scalac command

I wrote a sample compiler plugin for Scala. I tested it on two following sample files:
scalac -Xplugin: Test.jar *.scala
A:
package pkg {
class A {
def compute(): Int = 42
}
}
B:
import pkg._
class B {
def fun(): Unit = {
new A().compute()
}
}
I need to test this plugin on Akka scala files(https://github.com/akka/akka/tree/v2.6.10/akka-actor/src/main/scala/akka), but it shows thousands of errors because it can not compile the Akka files. In another word, how can I compile Akka actor files with this command:
scalac *.scala
To compile the Akka project directly with scalac you have to add all of the options sbt injects and somehow provide the additional sources that sbt generates at build time(e.g. the Version and more).
You can still easily inject a compiler plugin re-using the sbt build.
You can take inspiration on how it is done here for Scala 3 plugin, or, similarly, here for a Scala 2 one.

java.lang.NoSuchMethodError in Scala SBT Shell after executing JAR [duplicate]

This question already has answers here:
How to run jar generated by package (possibly with other jars under lib)?
(3 answers)
Closed 2 years ago.
after packaging and executing a Scala application (build.sbt with version 2.12.0, but in fact having 2.13.3 installed) with SBT (version 1.3.13), I get the following error:
Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.wrapRefArray([Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray;
at org.example.GreetWorld$.printMessage(GreetWorld.scala:5)
The source file GreetWorld.scala that caused the error looks like this:
package org.example
object GreetWorld {
def printMessage(theMessage:String):Unit = {
println(s"${theMessage} from me")
}
}
The main file that is invoking the file above looks like this:
package org.example
object HelloWorld {
def main(args: Array[String]) = {
GreetWorld.printMessage("Hello")
}
}
Does anybody know the root cause? At first I thought it has to do with the SBT shell picking Java 11, but even after changing my Windows' JAVA_HOME to Java 8, I still get the same error. Compiling and running it in SBT Shell works fine. Only the JAR execution fails.
The error is pretty simple. When you run package you create a JAR which only contains the classes corresponding to your source code, nothing more. And your code depends on the Scala stdlib, so if you try to run it with java - jar it will fail with a class path error.
You have 4 solutions:
Run the JAR using scala directly. However, you need to use the same major version it was used to compile.
Put the Scala library jar in the classpath when running. This is basically the same as above. Thus again, you have to use the same major version.
Create an uber jar that already has the Scala stdlib (as well as any other dependency) in it, using sbt-assembly.
Create a native distributable that will set up everything for you, using sbt-native-packager.
For local development and testing option 1 is usually the best one.
For simple projects option 3 is, IMHO, the simplest alternative.
And for very complex projects option 4 is very popular.

object scala in compiler mirror not found - running Scala compiler programatically [no sbt - no IDE] [duplicate]

I'm trying to run a Scala application packed as JAR (including dependencies) but this fails until the Scala library is added by using the -Xbootclasspath/p option.
Failing invocation:
java -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
After the application did some of its intended output, the console shows:
Exception in thread "main"
scala.reflect.internal.MissingRequirementError: object scala.runtime
in compiler mirror not found.
at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
at scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
at scala.reflect.internal.Mirrors$RootsBase.getRequiredPackage(Mirrors.scala:175)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage$lzycompute(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass$lzycompute(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr$lzycompute(Definitions.scala:1015)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr(Definitions.scala:1014)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1144)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1143)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1252)
at scala.tools.nsc.Global$Run.(Global.scala:1290)
at extract.ScalaExtractor$Compiler$2$.(ScalaExtractor.scala:24)
Working invocation:
java -Xbootclasspath/p:/path/to/home/.sbt/boot/scala-2.10.2/lib/scala-library.jar -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
The strange thing about it is that the application-assembly-1.0.jar was built so that it includes all dependencies including the Scala library. When one extracts the JAR file it can be verified that the class files in the scala.runtime package have been included.
Creation of the JAR file
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.9.1") was added to project/plugins.sbt and the assembly target was invoked. A JAR file of about 25MB results.
Building the JAR with proguard shows the same runtime behavior as seen with assembly's JAR file.
Application code that triggers the MissingRequirementError
Some application code works fine and the previously described exception is triggered as soon as the new Run from the following fragment executes.
import scala.reflect.internal.util.BatchSourceFile
import scala.reflect.io.AbstractFile
import scala.reflect.io.Path.jfile2path
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
…
import scala.tools.nsc._
object Compiler extends Global(new Settings()) {
new Run // This is line 24 from the stack trace!
def parse(path: File) = {
val code = AbstractFile.getFile(path)
val bfs = new BatchSourceFile(code, code.toCharArray)
val parser = new syntaxAnalyzer.UnitParser(new CompilationUnit(bfs))
parser.smartParse()
}
}
val ast = Compiler.parse(file)
Among others, scala-library, scala-compiler and scala-reflect are defined as dependencies in build.sbt.
For the curios / background information
The aim of the application is to aid in localization of Java and Scala programs. The task of the code fragment above is to get an AST from a Scala file in order to find method calls in there.
The questions
Given the Scala library is included in the JAR file, why is necessary to call the JAR using -Xbootclasspath/p:scala-library.jar?
Why do other parts of the application run just fine even though scala.runtime is reported as missing later?
The easy way to configure the settings with familiar keystrokes:
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
def main(args: Array[String]) {
val s = new Settings
s processArgumentString "-usejavacp"
val g = new Global(s)
val r = new g.Run
}
That works for your scenario.
Even easier:
java -Dscala.usejavacp=true -jar ./scall.jar
Bonus info, I happened to come across the enabling commit message:
Went ahead and implemented classpaths as described in email to
scala-internals on the theory that at this point I must know what I'm
doing.
** PUBLIC SERVICE ANNOUNCEMENT **
If your code of whatever kind stopped working with this commit (most
likely the error is something like "object scala not found") you can
get it working again with either of:
passing -usejavacp on the command line
set system property "scala.usejavacp" to "true"
Either of these will alert scala that you want the java application
classpath to be utilized by scala as well.

"scala.runtime in compiler mirror not found" but working when started with -Xbootclasspath/p:scala-library.jar

I'm trying to run a Scala application packed as JAR (including dependencies) but this fails until the Scala library is added by using the -Xbootclasspath/p option.
Failing invocation:
java -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
After the application did some of its intended output, the console shows:
Exception in thread "main"
scala.reflect.internal.MissingRequirementError: object scala.runtime
in compiler mirror not found.
at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
at scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
at scala.reflect.internal.Mirrors$RootsBase.getRequiredPackage(Mirrors.scala:175)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage$lzycompute(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass$lzycompute(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr$lzycompute(Definitions.scala:1015)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr(Definitions.scala:1014)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1144)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1143)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1252)
at scala.tools.nsc.Global$Run.(Global.scala:1290)
at extract.ScalaExtractor$Compiler$2$.(ScalaExtractor.scala:24)
Working invocation:
java -Xbootclasspath/p:/path/to/home/.sbt/boot/scala-2.10.2/lib/scala-library.jar -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
The strange thing about it is that the application-assembly-1.0.jar was built so that it includes all dependencies including the Scala library. When one extracts the JAR file it can be verified that the class files in the scala.runtime package have been included.
Creation of the JAR file
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.9.1") was added to project/plugins.sbt and the assembly target was invoked. A JAR file of about 25MB results.
Building the JAR with proguard shows the same runtime behavior as seen with assembly's JAR file.
Application code that triggers the MissingRequirementError
Some application code works fine and the previously described exception is triggered as soon as the new Run from the following fragment executes.
import scala.reflect.internal.util.BatchSourceFile
import scala.reflect.io.AbstractFile
import scala.reflect.io.Path.jfile2path
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
…
import scala.tools.nsc._
object Compiler extends Global(new Settings()) {
new Run // This is line 24 from the stack trace!
def parse(path: File) = {
val code = AbstractFile.getFile(path)
val bfs = new BatchSourceFile(code, code.toCharArray)
val parser = new syntaxAnalyzer.UnitParser(new CompilationUnit(bfs))
parser.smartParse()
}
}
val ast = Compiler.parse(file)
Among others, scala-library, scala-compiler and scala-reflect are defined as dependencies in build.sbt.
For the curios / background information
The aim of the application is to aid in localization of Java and Scala programs. The task of the code fragment above is to get an AST from a Scala file in order to find method calls in there.
The questions
Given the Scala library is included in the JAR file, why is necessary to call the JAR using -Xbootclasspath/p:scala-library.jar?
Why do other parts of the application run just fine even though scala.runtime is reported as missing later?
The easy way to configure the settings with familiar keystrokes:
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
def main(args: Array[String]) {
val s = new Settings
s processArgumentString "-usejavacp"
val g = new Global(s)
val r = new g.Run
}
That works for your scenario.
Even easier:
java -Dscala.usejavacp=true -jar ./scall.jar
Bonus info, I happened to come across the enabling commit message:
Went ahead and implemented classpaths as described in email to
scala-internals on the theory that at this point I must know what I'm
doing.
** PUBLIC SERVICE ANNOUNCEMENT **
If your code of whatever kind stopped working with this commit (most
likely the error is something like "object scala not found") you can
get it working again with either of:
passing -usejavacp on the command line
set system property "scala.usejavacp" to "true"
Either of these will alert scala that you want the java application
classpath to be utilized by scala as well.

AbstractMethodError when running standalone jar built with SBT and ProGuard

I've written a simple Scala application that I'd like to distribute in the form of a standalone, executable jar to servers without the Scala runtime. Everything works fine when invoked through SBT run, but not java -jar.
When I run the jar through java, I get the following unhandled exception:
Exception in thread "main" java.lang.AbstractMethodError: java.util.logging.Handler.publish(Ljava/util/logging/LogRecord;)V
at java.util.logging.Logger.log(Logger.java:458)
at net.lag.logging.Logger.log(Logger.scala:108)
at net.lag.logging.Logger.log(Logger.scala:91)
at net.lag.logging.Logger.info(Logger.scala:121)
at com.rentawebgeek.sitewiki.SiteWiki$.main(SiteWiki.scala:29)
at com.rentawebgeek.sitewiki.SiteWiki.main(SiteWiki.scala)
Exception in thread "Thread-0" java.lang.AbstractMethodError: java.util.logging.Handler.close()V
at java.util.logging.LogManager.resetLogger(LogManager.java:682)
at java.util.logging.LogManager.reset(LogManager.java:665)
at java.util.logging.LogManager$Cleaner.run(LogManager.java:223)
I'm using Configgy and it's Logger, and, per the javadocs for AbstractMethodError, thought it might be related to Scala/SBT using a different Java version than what I'm invoking from my shell. However, java -version and $JAVA_HOME/bin/java -version (what /usr/local/bin/scala uses) both match up as 1.6.0_22.
My ProGuard options are:
//program entry point
override def mainClass: Option[String] = Some("com.rentawebgeek.sitewiki.SiteWiki")
//proguard
override def proguardOptions = List(
"-keepclasseswithmembers public class * { public static void main(java.lang.String[]); }",
"-dontoptimize",
"-dontobfuscate",
"-keep class *",
proguardKeepLimitedSerializability,
proguardKeepAllScala,
"-keep interface scala.ScalaObject"
)
override def proguardInJars = Path.fromFile(scalaLibraryJar) +++ super.proguardInJars
How can I resolve this error? Or find another way to build an executable jar from an SBT project for a Scala-less deployment?
Check out what call you are making at line 29 in SiteWiki.scala; that is the offending call. You're probably calling a trait/class there with an abstract method. Most probably the method that should implement the abstract method is ripped away by proguard (or there the Scala override doesn't match up (I've seen that happen)).
If the line is long to find the offending call; try to decompose over multiple lines.
The latest ProGuard releases contain a sample configuration for processing a Scala application plus the Scala runtime:
http://proguard.sourceforge.net/manual/examples.html#scala
If that doesn't work, the output of -printconfiguration and the console output might help finding the root cause.