SBT: Plugin dependencies and project classpath - scala

How does one add a external dependency to a SBT plugin and make it available on both the project and plugin classpath ?:
Specifically I have a simple plugin that should run our TestNG test suites and do some post processing. Here is a simplified version:
import sbt._
import java.util.ArrayList
import Keys._
import org.testng._
object RunTestSuitesPlugin extends Plugin {
lazy val runTestSuites = TaskKey[Unit]("run-test-suites", "runs TestNG test suites")
lazy val testSuites = SettingKey[Seq[String]]("test-suites", "list of test suites to run")
class JavaListWrapper[T](val seq: Seq[T]) {
def toJavaList = seq.foldLeft(new java.util.ArrayList[T](seq.size)) { (al, e) => al.add(e); al }
}
implicit def listToJavaList[T](l: Seq[T]) = new JavaListWrapper(l)
def runTestSuitesTask = runTestSuites <<= (target, streams, testSuites) map {
(targetDirectory, taskStream, suites) =>
import taskStream.log
log.info("running test suites: " + suites)
runSuites(suites)
}
private def runSuites(testSuites: Seq[String]) = {
var tester = new TestNG
tester.setTestSuites(testSuites.toJavaList)
tester.run()
}
def testSuiteSettings = {
inConfig(Compile)(Seq(
runTestSuitesTask,
testSuites := Seq("testsuites/mysuite.xml"),
libraryDependencies += "org.testng" % "testng" % "5.14"))
}
}
The problem is that when I add this plugin to a project and run it with run-test-suites then it fails with java.lang.NoClassDefFoundError: org/testng/TestNG even though show full-classpath shows that testng.jar is on the classpath.
So somehow the classpath used when executing the plugin differs from the one in my project, so how do I make a plugin dependency appear in both places ?

I'll try an answer, but I'm not very familiar with the inner details of sbt.
Normally, the path for the build system (as opposed to your program) is under project, as explained here. That would typically be in a project/plugins.sbt. Sounds right, as there is no reason that the application you develop should be concerned by what libraries your build system uses, nor the other way round.
When your plugin run the application code, that may not be so simple and there could well be classpath/classloader issues. I'm not sure that it will work. Normally, your plugin should implement a testing Framework rather than define its own task. Documentation of testing for sbt is limited.
A testing framework should implement org.scalatools.testing.Framework, in test-interface. Your build will take it into account after you add
testFrameworks += new TestFramework("full.class.name")
When you run the normal test command, it let every framework recognize the test classes it deals with (two criteria available: extending some base class or having some annotation) and run them. The framework run in the build, it is given a class loader to access the application code.
You may have a look at the framework implementation for junit (shipped with sbt). Also there is a TestNG implementation. I don't know it, according to its doc, it is a little bit unorthodox, hopefully it will work for you.

The error was fixed by adding TestNG directly to unmanagedJars in Compile in the project that uses the plugin.
I have not found any resources explaining the structure of the SBT class path during plugin execution so any attempt at explaining why this step is necessary will be greatly appreciated.

Related

Compile Akka actor files using scalac command

I wrote a sample compiler plugin for Scala. I tested it on two following sample files:
scalac -Xplugin: Test.jar *.scala
A:
package pkg {
class A {
def compute(): Int = 42
}
}
B:
import pkg._
class B {
def fun(): Unit = {
new A().compute()
}
}
I need to test this plugin on Akka scala files(https://github.com/akka/akka/tree/v2.6.10/akka-actor/src/main/scala/akka), but it shows thousands of errors because it can not compile the Akka files. In another word, how can I compile Akka actor files with this command:
scalac *.scala
To compile the Akka project directly with scalac you have to add all of the options sbt injects and somehow provide the additional sources that sbt generates at build time(e.g. the Version and more).
You can still easily inject a compiler plugin re-using the sbt build.
You can take inspiration on how it is done here for Scala 3 plugin, or, similarly, here for a Scala 2 one.

object scala in compiler mirror not found - running Scala compiler programatically [no sbt - no IDE] [duplicate]

I'm trying to run a Scala application packed as JAR (including dependencies) but this fails until the Scala library is added by using the -Xbootclasspath/p option.
Failing invocation:
java -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
After the application did some of its intended output, the console shows:
Exception in thread "main"
scala.reflect.internal.MissingRequirementError: object scala.runtime
in compiler mirror not found.
at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
at scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
at scala.reflect.internal.Mirrors$RootsBase.getRequiredPackage(Mirrors.scala:175)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage$lzycompute(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass$lzycompute(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr$lzycompute(Definitions.scala:1015)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr(Definitions.scala:1014)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1144)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1143)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1252)
at scala.tools.nsc.Global$Run.(Global.scala:1290)
at extract.ScalaExtractor$Compiler$2$.(ScalaExtractor.scala:24)
Working invocation:
java -Xbootclasspath/p:/path/to/home/.sbt/boot/scala-2.10.2/lib/scala-library.jar -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
The strange thing about it is that the application-assembly-1.0.jar was built so that it includes all dependencies including the Scala library. When one extracts the JAR file it can be verified that the class files in the scala.runtime package have been included.
Creation of the JAR file
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.9.1") was added to project/plugins.sbt and the assembly target was invoked. A JAR file of about 25MB results.
Building the JAR with proguard shows the same runtime behavior as seen with assembly's JAR file.
Application code that triggers the MissingRequirementError
Some application code works fine and the previously described exception is triggered as soon as the new Run from the following fragment executes.
import scala.reflect.internal.util.BatchSourceFile
import scala.reflect.io.AbstractFile
import scala.reflect.io.Path.jfile2path
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
…
import scala.tools.nsc._
object Compiler extends Global(new Settings()) {
new Run // This is line 24 from the stack trace!
def parse(path: File) = {
val code = AbstractFile.getFile(path)
val bfs = new BatchSourceFile(code, code.toCharArray)
val parser = new syntaxAnalyzer.UnitParser(new CompilationUnit(bfs))
parser.smartParse()
}
}
val ast = Compiler.parse(file)
Among others, scala-library, scala-compiler and scala-reflect are defined as dependencies in build.sbt.
For the curios / background information
The aim of the application is to aid in localization of Java and Scala programs. The task of the code fragment above is to get an AST from a Scala file in order to find method calls in there.
The questions
Given the Scala library is included in the JAR file, why is necessary to call the JAR using -Xbootclasspath/p:scala-library.jar?
Why do other parts of the application run just fine even though scala.runtime is reported as missing later?
The easy way to configure the settings with familiar keystrokes:
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
def main(args: Array[String]) {
val s = new Settings
s processArgumentString "-usejavacp"
val g = new Global(s)
val r = new g.Run
}
That works for your scenario.
Even easier:
java -Dscala.usejavacp=true -jar ./scall.jar
Bonus info, I happened to come across the enabling commit message:
Went ahead and implemented classpaths as described in email to
scala-internals on the theory that at this point I must know what I'm
doing.
** PUBLIC SERVICE ANNOUNCEMENT **
If your code of whatever kind stopped working with this commit (most
likely the error is something like "object scala not found") you can
get it working again with either of:
passing -usejavacp on the command line
set system property "scala.usejavacp" to "true"
Either of these will alert scala that you want the java application
classpath to be utilized by scala as well.

"scala.runtime in compiler mirror not found" but working when started with -Xbootclasspath/p:scala-library.jar

I'm trying to run a Scala application packed as JAR (including dependencies) but this fails until the Scala library is added by using the -Xbootclasspath/p option.
Failing invocation:
java -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
After the application did some of its intended output, the console shows:
Exception in thread "main"
scala.reflect.internal.MissingRequirementError: object scala.runtime
in compiler mirror not found.
at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
at scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
at scala.reflect.internal.Mirrors$RootsBase.getRequiredPackage(Mirrors.scala:175)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage$lzycompute(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackage(Definitions.scala:181)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass$lzycompute(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.RuntimePackageClass(Definitions.scala:182)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr$lzycompute(Definitions.scala:1015)
at scala.reflect.internal.Definitions$DefinitionsClass.AnnotationDefaultAttr(Definitions.scala:1014)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1144)
at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1143)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1187)
at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1252)
at scala.tools.nsc.Global$Run.(Global.scala:1290)
at extract.ScalaExtractor$Compiler$2$.(ScalaExtractor.scala:24)
Working invocation:
java -Xbootclasspath/p:/path/to/home/.sbt/boot/scala-2.10.2/lib/scala-library.jar -jar /path/to/target/scala-2.10/application-assembly-1.0.jar
The strange thing about it is that the application-assembly-1.0.jar was built so that it includes all dependencies including the Scala library. When one extracts the JAR file it can be verified that the class files in the scala.runtime package have been included.
Creation of the JAR file
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.9.1") was added to project/plugins.sbt and the assembly target was invoked. A JAR file of about 25MB results.
Building the JAR with proguard shows the same runtime behavior as seen with assembly's JAR file.
Application code that triggers the MissingRequirementError
Some application code works fine and the previously described exception is triggered as soon as the new Run from the following fragment executes.
import scala.reflect.internal.util.BatchSourceFile
import scala.reflect.io.AbstractFile
import scala.reflect.io.Path.jfile2path
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
…
import scala.tools.nsc._
object Compiler extends Global(new Settings()) {
new Run // This is line 24 from the stack trace!
def parse(path: File) = {
val code = AbstractFile.getFile(path)
val bfs = new BatchSourceFile(code, code.toCharArray)
val parser = new syntaxAnalyzer.UnitParser(new CompilationUnit(bfs))
parser.smartParse()
}
}
val ast = Compiler.parse(file)
Among others, scala-library, scala-compiler and scala-reflect are defined as dependencies in build.sbt.
For the curios / background information
The aim of the application is to aid in localization of Java and Scala programs. The task of the code fragment above is to get an AST from a Scala file in order to find method calls in there.
The questions
Given the Scala library is included in the JAR file, why is necessary to call the JAR using -Xbootclasspath/p:scala-library.jar?
Why do other parts of the application run just fine even though scala.runtime is reported as missing later?
The easy way to configure the settings with familiar keystrokes:
import scala.tools.nsc.Global
import scala.tools.nsc.Settings
def main(args: Array[String]) {
val s = new Settings
s processArgumentString "-usejavacp"
val g = new Global(s)
val r = new g.Run
}
That works for your scenario.
Even easier:
java -Dscala.usejavacp=true -jar ./scall.jar
Bonus info, I happened to come across the enabling commit message:
Went ahead and implemented classpaths as described in email to
scala-internals on the theory that at this point I must know what I'm
doing.
** PUBLIC SERVICE ANNOUNCEMENT **
If your code of whatever kind stopped working with this commit (most
likely the error is something like "object scala not found") you can
get it working again with either of:
passing -usejavacp on the command line
set system property "scala.usejavacp" to "true"
Either of these will alert scala that you want the java application
classpath to be utilized by scala as well.

Error in scala + dispatch

I am trying to GET a response from an API, using scala and dispatch. However, I get this error, after building. I googled for a solution, and tried cleaning, and resarting eclipse, but the error wont go away. What seems to be the problem? I use eclipse Helios (ie 3.6) and Scala v2.8.1, with Scala IDE v1.0.0.201104170033, installed from the Eclipse market.
dispatch{dispatch.type}.Http{object dispatch.Http} of type object dispatch.Http does not take parameters
This is my code.
class getList {
def main(args: Array[String]){
Http("http://foo.com/" >>> System.out)
}
}
What am I doing wrong?
What libraries have you downloaded? Are you sure the dependencies are set correctly? I tried with dispatch_http, version 8.0 for Scala 2.8.1, and it worked.
What imports are you using? I used these imports to make it work:
import dispatch.Http
import dispatch.HandlerVerbs._
Finally... class getList??? I assume this is a result of cut&pasting from actual code, but you should strive to produce a compilable example of your problem. Scala doesn't run programs from class, only from object, and it follows Java style of having classes start with an uppercase letter.
Here's the minimal code I used with SBT to get a working example.
Initializing:
~/test$ sbt
Project does not exist, create new project? (y/N/s) y
Name: test
Organization: test
Version [1.0]:
Scala version [2.7.7]: 2.8.1
sbt version [0.7.4]:
~/test$ cat project/build/TestProject.scala
import sbt._
class TestProject(info: ProjectInfo) extends DefaultProject(info) {
val dvers = "0.8.0"
val http = "net.databinder" %% "dispatch-http" % dvers
}
~/test$ cat src/main/scala/GetList.scala
import dispatch.Http
import dispatch.HandlerVerbs._
object GetList {
def main(args: Array[String]){
Http("http://foo.com/" >>> System.out)
}
}
~/test# sbt update run

Run tests in broken project using SBT

When doing a serious refactor in a Java Eclipse project I will often break the build, but focus on getting one test to pass at a time. When running the tests Eclipse warns that the project cannot be compiled, but it will still run the tests it can compile.
Now I'm using SBT and would like to achieve the same thing with 'test-only', but it tries to compile the whole project, fails, and doesn't run the tests. How can I tell it to just compile the bits it can and run the tests.
You should add the following task to your project definition:
import sbt._
class Project(info: ProjectInfo) extends DefaultProject(info) {
lazy val justTest = testTask(testFrameworks, testClasspath, testCompileConditional.analysis, testOptions)
}
This is the same as the ordinary test task, but has no dependencies attached at the end. If you'd like it to have dependencies, call dependsOn on the testTask(...) expression and provide the tasks you want it to depend on.
testTask(testFrameworks, testClasspath, testCompileConditional.analysis, testOptions).dependsOn(testCompile, copyResources, copyTestResources)