Unable to get a resource when running an main class or test in sbt - scala

I'm trying to run a test that uses a resource in sbt. The test depends on a util that loads a resource via ClassLoader.getSystemResourceAsStream(...), which results unexpectedly in null. If I run the same test in Intellij or via bazel, the test succeeds. I additionally performed a test by creating a main class to list all resources according to the ResourceList example given in an answer to this question, and this confirmed that that file and many others were inaccessible at runtime.
These resources are not contained in the resources directories that sbt usually uses; they are contained in a jar file that is included in the lib directory. The sources in my project built by sbt depend heavily on this jar, and compilation is successful, so it appears the issue may be specific to resources. One thing I noticed is that the resource can be loaded if I use a ClassLoader object and call getResourceAsStream instead of using the static method ClassLoader.getSystemResourceAsStream.
Does anyone know how to resolve this issue (short of copying out all resources from the jar file in lib)?

Try running tests in forked JVM:
Test / fork := true
Unmanaged dependencies in lib/ should end up by default on all the classpaths, including test, without having to do anything special:
Dependencies in lib go on all the classpaths (for compile, test, run,
and console).
Note, when running tests in IntelliJ you might be using IntelliJ's internal build system as opposed to SBT shell, which could be the reason why it worked in IntelliJ. To run tests in IntelliJ via sbt shell, select Use sbt checkbox in Edit Configurations...
I would suggest using getClass.getResourceAsStream, as this anyways uses system ClassLoader as fallback:
public InputStream getResourceAsStream(String name) {
name = resolveName(name);
ClassLoader cl = getClassLoader0();
if (cl==null) {
// A system class.
return ClassLoader.getSystemResourceAsStream(name);
}
return cl.getResourceAsStream(name);
}

Related

Scala Play Rest service cannot find Controllers in routes file [Play 2.6, Scala]

I have recently revisited a project I have not worked on in over a year. Yesterday I was able to successfully run the REST service with no problems. Today while I was refactoring the location of certain controllers in this project I started to encounter errors related to controllers could not be found within a given package.
My routes file that looks like this:
UserController is defined as such:
However; when trying to compile this project, I receive a list of errors like: (redacted most, only included one controller for sample)
type UserController is not a member of package com.jkdev.controllers
[error] POST /users com.jkdev.controllers.UserController.createUser
Additionally, my Binders are no longer being detected by the routes file as well, so I am seeing errors like: [error] /Users/...../Developer/cashflows/metadata/conf/routes: object binders is not a member of package com.jkdev.
Like I mentioned, yesterday this was working, so I tried reverting to that commit and rebuilding, but this issue persists.
I have attempted to delete all target directories and recompile, ran sbt clean; cleanFiles. All of which provided nothing of value.
Overall this feels like a build error; but I changed nothing about the build file so I have no idea where to look next
The problem was:
I updated the intelliJ SBT preferences to use sbt shell for all builds,tests,runs, etc.
After deleting all of the ./target directories, doing SBT compile and run did not reproduce the target directories for my projects.
After updating IJ SBT preferences to not use SBT shell for builds; I was able to re-compile w/ IJ and reproduce the target directories.
Doing sbt run afterwards successfully launches the server.

In an sbt 0.13.7 project, compile the compiler-interfaces without compiling the project code

In a freshly checked out sbt ( 0.3.7 ) project and empty ivy cache, is it possible to trigger compilation of the compiler-interface(s) needed without compiling the project itself? I have poked around but haven't found a way.
Currently if a compiler-interface is required it will be created during compilation of the project. I would like to have this compiled directly in a separate command if possible. This would allow CircleCi to cache it saving 1-3 minutes with every build because it could be cached in the dependencies section of the circle.yml.
In sbt 0.13.12 compile:compileIncremental seems to do the trick. I ran inspect compile and inspected its dependencies to find the command.

Real SBT Classpath at Runtime

I have some test cases that need to look at the classpath to extract the paths of some files/directories in there. This works fine in the IDE.
The problem is that, when running SBT test, Properties.javaClassPath gives me /usr/share/sbt-launcher-packaging/bin/sbt-launch.jar.
The classpath is fine when I run show test:dependency-classpath. Is there a way to obtain that information from inside the running Scala/Java program? Or is there a way to toss it into a system property or environment variable?
By default the tests are run inside of the SBT process, so the classpath will look like it did when you started sbt (I guess sbt does some trixery to dynamicly load the classes for the tests, not sure). One way to do what you want is to run your tests in a forked jvm, that way sbt will start a new jvm to run the test suite and that should have the expected class path:
fork in Test := true
I have been working on understanding how the EmbeddedCassandra works in the spark-cassandra-connector project which uses the classpath to start up and control a Cassandra instance. Here is a line from their configuration that gets the correct classpath.
(compile in IntegrationTest) <<= (compile in Test, compile in IntegrationTest) map { (_, c) => c }
The entire source can be found here: https://github.com/datastax/spark-cassandra-connector/blob/master/project/Settings.scala
Information on the <<= operator can be found here: http://www.scala-sbt.org/0.12.2/docs/Getting-Started/More-About-Settings.html#computing-a-value-based-on-other-keys-values. I'm aware that this is not the current version of sbt, but the definition still holds.

Classpath problems when running JUnit tests from Eclipse with Gradle build

I have a Gradle project that declares a test-only dependency on an XML data file, and then loads the file from the classpath. When I run the tests directly in Gradle from the command line, everything works fine, but when I run "gradlew eclipse", refresh the project in Eclipse, and then try running the test from Eclipse (Debug As -> JUnit Test), the test fails because it's unable to find the XML file and the classpath (as accessed from the Properties context menu item on the process in the Debug view) shows no indication of the XML file being included on the classpath.
The behavior I'm seeing has some commonality with http://gradle.1045684.n5.nabble.com/gradle-junit-tests-resources-and-classpath-td4418753.html#a4420758, but Sean's problem was the reverse: his tests ran properly under Ant (but he never mentioned trying to run directly from the Eclipse JUnit plugin), but not under Gradle.
Here's the relevant part of build.gradle:
dependencies {
testCompile group: 'com.mycompany', name: 'MyConfigFile', version: '0.0.0+dirty', ext: 'xml' }
Because the only resources that URLClasspathLoader can load directly from the file system are JARs, I'm using the following static method to search the classpath for files that match the filename I need to load:
public static String getFullPathForResourceDirectlyOnClasspath(String nameFragment) {
ClassLoader cl = ClassLoader.getSystemClassLoader();
for (URL url: ((URLClassLoader)cl).getURLs()){
String fullPath = url.getFile();
if (fullPath.contains(nameFragment)) {
return fullPath;
}
}
return null;
}
I call that method as follows:
getFullPathForResourceDirectlyOnClasspath("/MyConfigFile-");
When I run that code from Gradle ("gradlew build"), it finds the file and my test succeeds. When I run it from Eclipse (Debug As -> JUnit Test), it fails to find that file on the classpath (because the Eclipse JUnit plugin doesn't put it there) and that call returns null.
I've tried changing the configuation from testCompile to compile to see if that made a difference, but it doesn't change anything (and perhaps tellingly, my .classpath doesn't have any entry for the XML file even when the compile configuration is selected).
Does anyone know of a way to make this work? Am I just missing something that should be obvious?
It seems that you are abusing the class path to pass a single argument (the absolute file path of the XML file) to a test. Instead, you should either put the XML file on the (test) class path in the correct way (it needs to go into a directory or Jar file that's listed on the class path) and load it correctly (e.g. with getClass().getClassLoader().getResource("some/resource.xml")), or pass the file path to the test as a system property. Naturally, the latter will be harder to make work for different environments (say Gradle build and IDEs).

Scala SBT: standalone jar

The answer: Making stand-alone jar with Simple Build Tool seems like what I need, but it did not have enough information for me, so this is a followup.
(1) How do I adapt the answer to my need? I don't understand what would need to be changed.
(2) What command do I run to create the standalone jar?
(3) Where can I find the jar after it has been created?
What I've tried:
Pasting the code in the linked answer verbatim into my: project/build/dsg.scala file. The file now has a
class ForkRun(info: ProjectInfo) extends DefaultProject(info)
(from before, used for running projects in a separate VM from SBT) and the new:
trait AssemblyProject extends BasicScalaProject
from the linked answer.
I also tried pasting the body (all defs and the lazy val of the AssemblyProject into the body of ForkRun.
To create a jar I ran package at the SBT prompt and get:
[info] Packaging ./target/scala_2.8.1/dsg_2.8.1-1.0.jar ...
[info] Packaging complete.
So I tried running the dsg_2.8.1-1.0.jar from the shell via:
java -jar dsg_2.8.1-1.0.jar
But I get:
Failed to load Main-Class manifest attribute from
dsg_2.8.1-1.0.jar
Could this be caused by having multiple entry points into my project? I select from a list when I execute run from the SBT prompt. Perhaps I need to specify the default when creating the package?
Here's a writeup I did on one way to make an executable jar with SBT:
http://janxspirit.blogspot.com/2011/01/create-executable-scala-jar-with-sbt.html
sbt-assembly is a sbt plugin to create a standalone jar of Scala sbt project with all of its dependencies.
Refer this post for more details with an example.