The problem:
I have 2 dependencies in Scala sbt project - org.slf4j:slf4j-api (direct, but used in other direct dependencies as well) and org.fusesource.hawtjni:hawtjni-generator (transitive). Both of them have the class org.slf4j.helpers.MessageFormatter. The issues arises when a third dependency tries to call MessageFormatter.someFunction(x), which should be called from slf4j-api, but instead it is called from hawtjni-generator.
What I have tried:
Cleaning the IDE (Intellij) files, as I supposed it could be the IDE not recognising them properly. Didn't work...
Adding exclusion rule for hawtjni-generator to all other dependencies, except the one that used it. Didn't work...
Edit:
A simplified dependency tree of the project looks like that:
[info] +-org.slf4j:slf4j-api:1.7.25
[info] +-ch.qos.logback:logback-classic:1.2.3
[info] | +-org.slf4j:slf4j-api:1.7.25
[info] |
[info] +-com.ltonetwork:leveldbjni.leveldbjni-osx:99-master-SNAPSHOT
[info] | +-org.fusesource.hawtjni:hawtjni-maven-plugin:1.18
[info] | +-org.fusesource.hawtjni:hawtjni-generator:1.18
The problem is with logback-classic (that's the third dependency) which wrongly calls hawtjni-generator, instead of slf4j-api.
Have you tried to use an alias for one of your imports?
import org.fusesource.hawtjni:hawtjni-generator
import org.slf4j.helpers.{MessageFormatter => MessageFormatterSlf4j}
MessageFormatterSlf4j.someFunctions(x)
MessageFormatter.someFunctions(x) // from fusesource
Scala import alias
Related
I'm new to sbt. I'd like to know how to get dependent jar files by Scala code, not executing sbt plugin.
In Gradle, it supports to get dependent jar files by Java code like the following (project is an instance of Project class):
Configuration config = project.getRootProject().getBuildscript().getConfigurations().detachedConfiguration();
Set<File> jars = config.resolve();
I'd like to know a way to do like that in sbt and Scala. Does anyone know this? I tried to use sbt.Project#dependencies, but it seems that one doesn't meet for this purpose.
You are looking for dependencyClasspathAsJars:
sbt > inspect dependencyClasspathAsJars
[info] Task: scala.collection.Seq[sbt.internal.util.Attributed[java.io.File]]
[info] Description:
[info] The classpath consisting of internal and external, managed and unmanaged dependencies, all as JARs.
[info] Provided by:
[info] ProjectRef(uri("file:/home/claudio/foo"), "foo") / Compile / dependencyClasspathAsJars
[info] Defined at:
[info] (sbt.Classpaths.classpaths) Defaults.scala:1800
[info] Dependencies:
...
As you can see, this is a Task that returns a scala.collection.Seq[sbt.internal.util.Attributed[java.io.File]] where Attributed is just a simple wrapper around arbitrary data: https://www.scala-sbt.org/1.x/api/sbt/internal/util/Attributed.html
sbt > show dependencyClasspathAsJars
[info] List(Attributed(/home/claudio/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.13.1.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/twirl-api_2.13/jars/twirl-api_2.13-1.5.0.jar),
Attributed(/home/claudio/.ivy2/cache/org.scala-lang.modules/scala-xml_2.13/bundles/scala-xml_2.13-1.2.0.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/play-server_2.13/jars/play-server_2.13-2.8.1.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/play_2.13/jars/play_2.13-2.8.1.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/build-link/jars/build-link-2.8.1.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/play-exceptions/jars/play-exceptions-2.8.1.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.play/play-streams_2.13/jars/play-streams_2.13-2.8.1.jar),
Attributed(/home/claudio/.ivy2/cache/org.reactivestreams/reactive-streams/jars/reactive-streams-1.0.3.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.akka/akka-stream_2.13/jars/akka-stream_2.13-2.6.3.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe.akka/akka-actor_2.13/jars/akka-actor_2.13-2.6.3.jar),
Attributed(/home/claudio/.ivy2/cache/com.typesafe/config/bundles/config-1.4.0.jar)
...)
If you want to process the value in any way, you probably want to write a custom task: https://www.scala-sbt.org/1.x/docs/Tasks.html
SBT scalariform plugin is formatting files more than once in multi project setup. Here is example https://github.com/Seetaramayya/sbt-scalariform-example
if you compile the project you can see something like this in the console
[info] Formatting 5 Scala sources ProjectRef(uri("file:/Users/seeta/projects/github/sbt-multi-scalariform/example/"), "root")(compile) ...
[info] Formatting 7 Scala sources ProjectRef(uri("file:/Users/seeta/projects/github/sbt-multi-scalariform/example/"), "module1")(compile) ...
[info] Formatting 7 Scala sources ProjectRef(uri("file:/Users/seeta/projects/github/sbt-multi-scalariform/example/"), "module2")(compile) ...
[info] Formatting 7 Scala sources ProjectRef(uri("file:/Users/seeta/projects/github/sbt-multi-scalariform/example/"), "main")(compile) ...
There are 5 scala files and 2 sbt files exist in the code base. Neither in SBT nor in scalariform documentation I could not find how to execute the task only once in multi module setup.
I would like to execute the task only once. Only option I see is enable plugin at root project level and disable at sub-projects but I did not like the option (I need to add disablePlugins in 15 sub-projects)
The issue seems to be that
Each subproject's scalariformFormat task in their different scopes
have the project root in their sourceDirectories setting. This just
leads to the whole project being crawled for scala sources and
formatted as many times as there are subprojects.
and has been addressed via scalariformWithBaseDirectory setting
val scalariformWithBaseDirectory = settingKey[Boolean]("Whether or not to format sources in project root (default: false)")
Thus setting withBaseDirectory=false in .scalariform.conf should give output
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "main")(compile) ...
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "module1")(compile) ...
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "module2")(compile) ...
Note, however, this will not format *.scala sources in the root project/ directory. To address that we could try keeping withBaseDirectory=false in .conf but switch it on for the root project like so
lazy val root = (project in file("."))
.aggregate(module1, module2, main)
.settings(scalariformWithBaseDirectory := true)
which outputs
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "module2")(compile) ...
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "module1")(compile) ...
[info] Formatting 1 Scala source ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "main")(compile) ...
[info] Formatting 5 Scala sources ProjectRef(uri("file:/Users/mario_galic/code/stackoverflow/sbt-scalariform-example/"), "root")(compile) ...
where we still get duplication but not as much as initially.
lihaoyi test$ tree
.
└── Foo.scala
0 directories, 1 file
lihaoyi test$ cat Foo.scala
object Main{
def main(args: Array[String]): Unit = {
println(getClass.getClassLoader.getResourceAsStream("java/lang/String.class"))
println(getClass.getClassLoader.getClass)
println(Thread.currentThread().getContextClassLoader.getResourceAsStream("java/lang/String.class"))
println(Thread.currentThread().getContextClassLoader.getClass)
}
}
lihaoyi test$ sbt run
[info] Loading global plugins from /Users/lihaoyi/.sbt/0.13/plugins
[info] Set current project to test (in build file:/Users/lihaoyi/Dropbox/Workspace/test/)
[info] Updating {file:/Users/lihaoyi/Dropbox/Workspace/test/}test...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/lihaoyi/Dropbox/Workspace/test/target/scala-2.10/classes...
[info] Running Main
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream#18e38ff2
class sbt.classpath.ClasspathUtilities$$anon$1
null
class sbt.classpath.ClasspathFilter
[success] Total time: 2 s, completed 29 May, 2017 4:14:11 PM
lihaoyi test$
Here, we can see that the getClass.getClassLoader and the Thread.currentThread.getContextClassLoader are returning different values. What's more, the Thread.currentThread.getContextClassLoader seems to be refusing to load java/lang/String.class, while the other can.
Notably, when I run the jar file using an external tool like scalac/scala, or java, both classloaders are able to load the classfile as a resource
lihaoyi test$ scalac Foo.scala
lihaoyi test$ scala Main
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream#1b28cdfa
class scala.reflect.internal.util.ScalaClassLoader$URLClassLoader
sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream#7229724f
class scala.reflect.internal.util.ScalaClassLoader$URLClassLoader
I would expect SBT to behave similarly: to have the Main.getClass.getClassLoader and the Thread.currentThread().getContextClassLoader both be able to load java/lang/String.class as a resource. What gives?
Some hints provided by Jason Zaugg (retronym)'s sbt launcher notes.
sbt/launcher is a small Scala application that bootstraps an arbitrary Scala program (typically, SBT) described a config file and sourced via Ivy dependency resolution.
This creates a child classloader containing Scala 2.10.6. A child of this contains SBT itself and xsbti/interface-0.13.11.jar.
SBT needs to use non-standard classloader delegation to selectively hide classes when creating child classloaders for plugin code, for the Scala compiler, or for user code.
Some more hints in the sbt 0.13 sources:
https://github.com/sbt/sbt-zero-thirteen/blob/v0.13.15/run/src/main/scala/sbt/Run.scala#L57-L62
https://github.com/sbt/sbt-zero-thirteen/blob/v0.13.15/util/classpath/src/main/scala/sbt/classpath/ClasspathUtilities.scala#L71-L72
def makeLoader(classpath: Seq[File], instance: ScalaInstance, nativeTemp: File): ClassLoader =
filterByClasspath(classpath, makeLoader(classpath, instance.loader, instance, nativeTemp))
def makeLoader(classpath: Seq[File], parent: ClassLoader, instance: ScalaInstance, nativeTemp: File): ClassLoader =
toLoader(classpath, parent, createClasspathResources(classpath, instance), nativeTemp)
Basically sbt is a kitchen sink of a Java application that has an arbitrary Scala versions and your code, and your test libraries along with the Oracle/OpenJDK's Java library. To construct a classpath that makes sense without loading them over and over again, it's creating a hierarchy of classloaders each filtered by some criteria. (I think)
Notably, one way to work around this issue is to set
(fork in run) := true,
(connectInput in run) := true,
(outputStrategy in run) := Some(StdoutOutput),
That seems to solve this problem, (Thread.currentThread().getContextClassLoader.getResourceAsStream("java/lang/String.class") now works) but introduces other unrelated problems (forked JVM takes a moment to boot, boots cold and takes time to warm up...)
I want to use the sbt-scrooge plugin, but its repo is unavailable now - http://koofr.github.com/.
I thought I'd include this plugin's source code directly in my own repo (as a git submodule).
I tried:
git submodule add https://github.com/bancek/sbt-scrooge.git project/sbt-scrooge
and added:
addSbtPlugin("net.koofr" % "sbt-scrooge" % "3.0.45")
to project/plugins.sbt. But it doesn't work - the following exception is thrown:
sbt.ResolveException: unresolved dependency: net.koofr#sbt-scrooge;3.0.45: not found
What's the right way to do that?
I know that I could checkout sbt-scrooge to the local filesystem, then sbt publish-local, and add the local ivy2 repo to sbt as a resolver. But I just want to know whether there are other ways to do this.
As explained here you can puth this in your project/plugins.sbt:
lazy val root = project.in(file(".")).dependsOn(scroogePlugin)
lazy val scroogePlugin = file("sbt-scrooge")
Or simply (without creating a local submodule):
lazy val root = project.in(file(".")).dependsOn(scroogePlugin)
lazy val scroogePlugin = uri("https://github.com/bancek/sbt-scrooge.git")
If you want to use a plugin it has to be available to sbt (and somehow finds its place in your local repository so addSbtPlugin can eventually find it or the project (sub)project of your sbt project should have it on the classpath).
Be adviced that not all plugins should be an integral part of a sbt project. Quite the contrary - they can be used in a project, but that doesn't necessarily mean they should be referenced by any project-specific files (within the project's directory), e.g. plugins to generate IDE-specific files. These plugins should be part of the global configuration in ~/.sbt under plugins.
There's also the issue of version mismatch between plugins and sbt. In your case, sbt-scrooge supports 0.12.2 (see project/build.properties) that might be unusable in sbt 0.13+.
With that said, I think the "right way" in your case since the sbt-scrooge plugin seems no longer maintained is to fork the project and maintain yourself in your own repository. sbt 0.13.1 is already the latest version, and the plugin may not yet support it. When the plugin gets new life with your fork other developers might benefit from the resurrection, too and having the sources attached to another project would only hinder reusability.
The answer to a similar question has helped me to offer a working solution that works with sbt 0.12.2 and without cloning the git repository.
$ cat project/build.properties
sbt.version=0.12.2
$ cat project/project/SbtScroogePlugin.scala
import sbt._
object SbtScroogePlugin extends Build {
lazy val plugins = Project("plugins", file(".")) dependsOn sbtScroogePlugin
lazy val sbtScroogePlugin = uri("https://github.com/bancek/sbt-scrooge.git")
}
$ cat sbt-scrooge.sbt
import net.koofr.sbt._
seq(CompileThriftScrooge.newSettings: _*)
With the project files above, sbt should be able to use the tasks and settings of the sbt-scrooge plugin.
$ sbt
[info] Loading global plugins from /Users/jacek/.sbt/plugins
[info] Loading project definition from /Users/jacek/sandbox/tmp/sample-project/project/project
[info] Loading project definition from /Users/jacek/.sbt/staging/52a2b7ff1377492a32ff/project
[info] Loading project definition from /Users/jacek/sandbox/tmp/sample-project/project
[info] Set current project to default-fe8e50 (in build file:/Users/jacek/sandbox/tmp/sample-project/)
> about
[info] This is sbt 0.12.2
[info] The current project is {file:/Users/jacek/sandbox/tmp/sample-project/}default-fe8e50
[info] The current project is built against Scala 2.9.2
[info] Available Plugins: org.sbtidea.SbtIdeaPlugin, com.timushev.sbt.updates.UpdatesPlugin, net.koofr.sbt.CompileThriftScrooge
[info] sbt, sbt plugins, and build definitions are using Scala 2.9.2
> scrooge-version
[info] 3.0.43
For the other tasks and settings, write scrooge- and hit TAB.
> scrooge-[TAB]
scrooge-build-options scrooge-cache-folder scrooge-fetch scrooge-gen
scrooge-jar scrooge-name scrooge-thrift-external-source-folder scrooge-thrift-include-folders
scrooge-thrift-namespace-map scrooge-thrift-output-folder scrooge-thrift-source-folder scrooge-thrift-sources
scrooge-unpack-deps scrooge-version
I'm trying to learn scala and play framework. I choosed IntelliJ Idea as ide. The problem is
editor gives warning on imports like ; "unused import statements"
import play.api._
import play.api.mvc._
Also give warnings on declerations like ; "Cannot resolve symbol Controller"
object Login extends Controller
How can i remove warnings?
Steps i followed;
Create template idea project with play command;
Add scala compiler and library into external libraries.
IntelliJ Idea Version : 12.1 Community Editon
This is how I did it (I'm using Play! 2.1.0):
C:\dev>play new community
_ _
_ __ | | __ _ _ _| |
| '_ \| |/ _' | || |_|
| __/|_|\____|\__ (_)
|_| |__/
play! 2.1.0 (using Java 1.7.0_15 and Scala 2.10.0), http://www.playframework.org
The new application will be created in C:\dev\community
What is the application name? [community]
>
Which template do you want to use for this new application?
1 - Create a simple Scala application
2 - Create a simple Java application
> 1
OK, application community is created.
Have fun!
C:\dev>cd community
C:\dev\community>play idea
[info] Loading project definition from C:\dev\community\project
[info] Set current project to community (in build file:/C:/dev/community/)
[info] Trying to create an Idea module community
[info] Updating {file:/C:/dev/community/}community...
[info] Done updating.
[info] Excluding folder target
[info] Created C:\dev\community/.idea/IdeaProject.iml
[info] Created C:\dev\community\.idea
[info] Excluding folder C:\dev\community\target\scala-2.10\cache
[info] Excluding folder C:\dev\community\target\resolution-cache
[info] Excluding folder C:\dev\community\target\streams
(commons-codec_commons-codec_1.6_test,List(commons-codec_commons-codec_1.3))
(org.apache.httpcomponents_httpcore_4.1.3_test,List(org.apache.httpcomponents_httpcore_4.0.1))
(org.apache.httpcomponents_httpclient_4.1.2_test,List(org.apache.httpcomponents_httpclient_4.0.1))
[info] Created C:\dev\community\.idea_modules/community.iml
[info] Created C:\dev\community\.idea_modules/community-build.iml
C:\dev\community>play compile
[info] Loading project definition from C:\dev\community\project
[info] Set current project to community (in build file:/C:/dev/community/)
[info] Compiling 5 Scala sources and 1 Java source to C:\dev\community\target\scala-2.10\classes...
The Unused import statement is because no classes are being used in that package. This line can safely be removed by optimizing import: Ctrl + Alt + o.
In this case I didn't add any scala libraries at all. I just compiled from the command line.