How I can list all sbt dependencies? - scala

I need to list all sbt dependencies in order to check if already exists a debian package (I also notice that there is a DEB package but it seems that external dependencies are not packaged).
At the moment I did a list of sbt dependencies with the following steps:
Install sbt manually
I created a simple script that extract all jar files in ~/.ivi2 directory (excluding sbt jar). Here the result of the execution:
Group;Artifact;Artifact+Version
org.scala-lang;jline;jline-2.10.5
org.scala-lang;scala-compiler;scala-compiler-2.10.5
org.scala-lang;scala-library;scala-library-2.10.5
org.scala-lang;scala-reflect;scala-reflect-2.10.5
com.jcraft;jsch;jsch-0.1.46
org.scalamacros;quasiquotes_2.10;quasiquotes_2.10-2.0.1
jline;jline;jline-2.11
com.thoughtworks.paranamer;paranamer;paranamer-2.6
org.json4s;json4s-ast_2.10;json4s-ast_2.10-3.2.10
org.json4s;json4s-core_2.10;json4s-core_2.10-3.2.10
org.scala-lang.modules;scala-pickling_2.10;scala-pickling_2.10-0.10.0
org.scala-tools.sbinary;sbinary_2.10;sbinary_2.10-0.4.2
org.fusesource.jansi;jansi;jansi-1.4
org.spire-math;json4s-support_2.10;json4s-support_2.10-0.6.0
org.spire-math;jawn-parser_2.10;jawn-parser_2.10-0.6.0
Do you think is the right way to list all sbt dependencies?

There is a nice sbt plugin for that:
https://github.com/jrudolph/sbt-dependency-graph
Simply adding to ~/.sbt/0.13/plugins/plugins.sbt:
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.8.0")
Calling sbt dependencyTree you can get an "ascii graph" like:
...
[info] | +-org.apache.lucene:lucene-spatial:4.10.2
[info] | | +-com.spatial4j:spatial4j:0.4.1
[info] | | +-org.apache.lucene:lucene-core:4.10.2
[info] | | +-org.apache.lucene:lucene-queries:4.10.2
[info] | | +-org.apache.lucene:lucene-core:4.10.2
[info] | |
[info] | +-org.apache.lucene:lucene-suggest:4.10.2
[info] | +-org.apache.lucene:lucene-analyzers-common:4.10.2
[info] | | +-org.apache.lucene:lucene-core:4.10.2
[info] | |
[info] | +-org.apache.lucene:lucene-core:4.10.2
[info] | +-org.apache.lucene:lucene-misc:4.10.2
[info] | | +-org.apache.lucene:lucene-core:4.10.2
[info] | |
[info] | +-org.apache.lucene:lucene-queries:4.10.2
[info] | +-org.apache.lucene:lucene-core:4.10.2
...

In case the dependency hierarchy provided by sbt-dependency-graph is not needed, the following might be useful:
sbt 'show dependencyClasspathFiles'

Just adding here how to install sbt-dependency-graph, I think that is relevant for the question.
IMPORTANT:
The answer is just the part related to sbt-dependency-graph. The complete answer (sbt+scala+homebrew+plugin) you may find here
In order to use the Snyk CLI to test Scala projects, you will need to install the Sbt dependency graph plugin.
Installing the Sbt dependency graph plugin for sbt 0.13
Prerequisites
Ensure you have installed Scala.
Ensure you have installed Sbt and ran sbt.
NOTE: The steps below will install the Sbt dependency plugin as a global plugin.
First navigate to the correct directory by typing the following
command: cd ~/.sbt
This will take you to the Sbt directory. From there you will need to
navigate to the 0.13 directory. Typing the ls command will show if
0.13 and/or 1.0 exists in the directory
Navigate to 0.13 by typing: cd 0.13 and then make a directory called
plugins by typing: mkdir plugins
Navigate to the new directory by typing: cd plugins and then proceed
to create a file called “plugins.sbt” by typing: touch plugins.sbt
Edit the plugins.sbt file via your preferred editor
Add the following line to the file:
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.10.0-RC1")
save the changes
Take the following steps for the 1.0 directory. Check if 1.0 exists
by typing ls in the sbt directory:
If the 1.0 does NOT exist in the sbt directory, type mkdir 1.0 in the sbt directory
If 1.0 exists in the directory, run the following command: cd ~/.sbt/1.0
Make a directory called “plugins” in that folder by typing: mkdir plugins
Copy the existing “plugins.sbt” file from the 0.13 directory to the current 1.0 directory by typing the following: cp ../0.13/plugins/plugins.sbt ./plugins
Validate that the plugin has been installed correctly by running the following command: sbt "-Dsbt.log.noformat=true" dependencyTree important This should be tested in the directory of the project and by running the command will generate the dependency graph. You can also run it each time you want to generate the dependency graph)

Related

Issue with installing CMAK - kafka manager

I am trying CMAK on ubuntu 20.4 and I am following instruction on https://github.com/yahoo/CMAK and https://codeforgeek.com/setting-up-kafka-management-for-kafka-cluster/. After running command ./sbt clean dist and trying to switch to directory /target/ in am not seeing universal directory. Is there any thing I am missing?
Please see below screenshot.
After running: ./sbt clean dist. I am getting below result.
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master# ./sbt clean dist
[info] Loading settings for project cmak-master-build from plugins.sbt ...
[info] Loading project definition from /opt/CMAK-master/project
[info] Loading settings for project root from build.sbt ...
[info] Set current project to cmak (in build file:/opt/CMAK-master/)
[success] Total time: 1 s, completed Apr 4, 2021, 10:40:07 AM
Warning: node.js detection failed, sbt will use the Rhino based
Trireme JavaScript engine instead to run JavaScript assets compilation, which in some cases may be orders of magnitude slower than using node.js.
[info] Wrote /opt/CMAK-master/target/scala-2.12/cmak_2.12-3.0.0.5.pom
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
[info] Main Scala API documentation to /opt/CMAK-master/target/scala-2.12/api...
[info] Compiling 136 Scala sources and 2 Java sources to /opt/CMAK-
master/target/scala-2.12/classes ...
model contains 640 documentable templates
[info] Main Scala API documentation successful.
[info] LESS compiling on 1 source(s)
./sbt: line 241: 7926 Killed "$#"
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master#
| => root / Web-assets / less 122s
| => root / Compile / compileIncremental 352s
After that when navigating to target directory, there is no universal directory.
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master# ls
LICENSE README.md app build.sbt conf img project public sbt
src target test
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master# cd target/
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-
master/target# ls
scala-2.12 streams web / less 122s
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master/target# ls
scala-2.12 streams web
root#ubuntu-s-1vcpu-1gb-blr1-01-testing-server:/opt/CMAK-master/target#

Why does the sbt ~compile command do not recompile on code change?

I currently have a problem with recompile on code change with sbt.
I was following the sbt reference 'sbt by example'
I installed sbt 1.2.8 and followed the instructions:
Create a minimum sbt build
$ mkdir foo-build
$ cd foo-build
$ touch build.sbt
Start sbt shell
$ sbt
[info] Loading global plugins from C:\Users\hce\.sbt\1.0\plugins
[info] Loading project definition from E:\learn\Scala\demo\foo-build\project
[info] Loading settings for project foo-build from build.sbt ...
[info] Set current project to foo-build (in build file:/E:/learn/Scala/demo/foo-build/)
[info] sbt server started at local:sbt-server-57c501e502d72a00d890
Recompile on code change (Note the ~ prefix before the compile command)
sbt:foo-build> ~compile
[success] Total time: 0 s, completed Jul 6, 2019 12:01:24 PM
1. Waiting for source changes in project foo-build... (press enter to interrupt)
Create a source file
Leave the previous command running. From a different shell or in your file manager create in the project directory the following nested directories: src/main/scala/example. Then, create Hello.scala in the example directory using your favorite editor as follows:
package example
object Hello extends App {
println("Hello")
}
This new file should be picked up by the running command. But it is not working on my system.
Expected Behaviour:
[info] Compiling 1 Scala source to /tmp/foo-build/target/scala-2.12/classes ...
[info] Done compiling.
[success] Total time: 2 s, completed May 6, 2018 3:53:42 PM
2. Waiting for source changes... (press enter to interrupt)
Here are some Information about my environment
$ java -version
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)
$ sbt sbtVersion
[info] Loading global plugins from C:\Users\hce\.sbt\1.0\plugins
[info] Loading project definition from E:\learn\Scala\demo\foo-build\project
[info] Loading settings for project foo-build from build.sbt ...
[info] Set current project to foo-build (in build file:/E:/learn/Scala/demo/foo-build/)
[info] 1.2.8
$ systeminfo.exe | grep '^OS'
OS Name: Microsoft Windows 10 Enterprise LTSC
OS Version: 10.0.17763 N/A Build 17763
OS Manufacturer: Microsoft Corporation
OS Configuration: Standalone Workstation
OS Build Type: Multiprocessor Free
What I already tried:
reinstall sbt
try it with windows commad line
try it mingw64 bash
What am I missing to run the sbt ~compile command correctly?
I found the answer.
The instruction contains an error.
We are supposed to put the src/main/scala/example directory in the project directory.
From a different shell or in your file manager create in the project directory the following nested directories: src/main/scala/example.
But this leads to the problem described. foo-build/project/ is for build definition code.
If I put the src/main/scala/example directory in the foo-build directory it is working.
I should have executed the run command beforehand, which gives the 'No main class detected' error. Which in turn helped to discover, that the directory structure was incorrect by reading the following stackoverflow question: how to set main class in sbt project .
My mistake. Sorry for bothering you.

Downloading Scala issues

So I need to download scala for one of my classes. I am using a mac pro and the command line to do so.
I downloaded sbt using homebrew which worked fine.
After these are the directions given:
Install the SBT plugin:
mkdir ~/.sbt
mkdir ~/.sbt/0.13
mkdir ~/.sbt/0.13/plugins
echo 'addSbtPlugin("edu.umass.cs" % "cmpsci220" % "3.0.0")' > ~/.sbt/0.13/plugins/plugins.sbt
To verify that everything is installed, I have to type 'sbt' to start SBT. Once typed, I should see output that looks like this:
[info] Loading global plugins from /Users/arjun/.sbt/0.13/plugins
[info] Updating {file:/Users/arjun/.sbt/0.13/plugins/}global-plugins...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] downloading https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/edu.umass.cs/cmpsci/scala_2.10/sbt_0.13/3.0.0/jars/cmpsci.jar ...
[info] [SUCCESSFUL ] edu.cs#cmpsci;3.0.0!cmpsci.jar (1261ms)$
However, My issue is that once I type 'sbt' to verify things I get this line:
[info] Set current project to home (in build file:/Users/home/)
It asks me for a command. I have tried 'run', 'test', 'update'. They all run fine but it keeps asking me a command.
My goal is to get to the 'sbt console' so I can run scala, import packages, scripts, etc..
Any ideas on what the issue is?
You need to be in the folder of your SBT project to do sbt run.
First cd into it, then sbt run to run it (or sbt test or whatever :))
Type console at the sbt prompt.
Edit:
So actually, why do you think this is what you want? Are you trying to run scala code that you're given? Or are you trying to write scala code? If the latter, you should download intellij and the scala plugin.
Please make sure there is a scala file in the directory you run sbt.
You may also check this for others: http://www.scala-sbt.org/0.13/docs/Hello.html

Runnig Twiiter Cassovary Examples in eclipse

I have imported the Twitter Cassovary into Eclipse form Git Hub but not sure how to run the examples.Please advise as I am new to eclipse as well as scala
I imported the project, opened a benchmark (cassovary-benchmarks/src/main/java/GraphGenerationBenchmarks) and used Run as->Scala application.
You can also do this from the command line. Start sbt in the main project directory, choose subproject cassovary-benchmarks and then use run. You will be asked which benchmark to run.
➜ twitter-cassovary git:(master) ✗ ls
README.md cassovary-examples project
cassovary-benchmarks cassovary-server sbt
cassovary-core lib_managed target
➜ twitter-cassovary git:(master) ✗ sbt
Loading /Users/szymon/bin/sbt-launch-lib.bash
[info] Loading global plugins from /Users/szymonmatejczyk/.sbt/0.13/plugins
[info] Loading project definition from /Users/szymon/oss/twitter-cassovary/project
[info] Set current project to cassovary (in build file:/Users/szymon/oss/twitter-cassovary/)
> project cassovary-benchmarks
[info] Set current project to cassovary-benchmarks (in build file:/Users/szymon/oss/twitter-cassovary/)
> run
[info] Compiling 12 Scala sources to /Users/szymon/oss/twitter-cassovary/cassovary-core/target/scala-2.11/classes...
[info] Compiling 56 Scala sources to /Users/szymon/oss/twitter-cassovary/cassovary-core/target/scala-2.11/classes...
[info] Compiling 2 Scala sources to /Users/szymon/oss/twitter-cassovary/cassovary- benchmarks/target/scala-2.11/classes...
[warn] Multiple main classes detected. Run 'show discoveredMainClasses' to see the list
Multiple main classes detected, select one to run:
[1] com.twitter.cassovary.PerformanceBenchmark
[2] com.twitter.cassovary.SubsetSamplingBenchmarks
[3] com.twitter.cassovary.GraphGenerationBenchmarks
[4] com.twitter.cassovary.utils.FastUtilsWrappingBenchmark
Enter number:

How to print dependency jars for use in an environment variable?

I wanted to print the dependency jars so that I can put them in a classpath variable.
I can fetch a classpath string for dependency jar files.
Curenttly, I can use compile:dependencyClasspath but it returns a format which needs to be manually transformed into a classpath string.
$ sbt "show compile:dependencyClasspath"
[info] Set current project to scala-snippets (in build file:/home/tuxdna/work/learn/mine/scala-snippets/)
[info] List(Attributed(/home/tuxdna/.ivy2/cache/org.apache.lucene/lucene-core/jars/lucene-core-3.0.1.jar), Attributed(/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-scala/jars/rxjava-scala-0.15.0.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.10.4-RC1.jar), Attributed(/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-core/jars/rxjava-core-0.15.0.jar), Attributed(/home/tuxdna/.ivy2/cache/com.typesafe.akka/akka-actor_2.10/jars/akka-actor_2.10-2.2.3.jar), Attributed(/home/tuxdna/.ivy2/cache/com.typesafe/config/bundles/config-1.0.2.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang.modules/scala-async_2.10/bundles/scala-async_2.10-0.9.0-M2.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.10.3.jar), Attributed(/home/tuxdna/.ivy2/cache/org.scala-lang/scala-actors/jars/scala-actors-2.10.2.jar))
[success] Total time: 0 s, completed Dec 17, 2013 4:00:05 PM
This is how I am transforming it into a classpath string
$ sbt "show compile:dependencyClasspath" | grep -B1 'success' | head -1 | ruby -e 's = STDIN.read; puts s[(s =~ /Attributed(.*?,)/)..-1].split(", ").map { |x| x.sub("Attributed(","").gsub(")", "")}.join(":")'
/home/tuxdna/.ivy2/cache/org.apache.lucene/lucene-core/jars/lucene-core-3.0.1.jar:/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-scala/jars/rxjava-scala-0.15.0.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.10.4-RC1.jar:/home/tuxdna/.ivy2/cache/com.netflix.rxjava/rxjava-core/jars/rxjava-core-0.15.0.jar:/home/tuxdna/.ivy2/cache/com.typesafe.akka/akka-actor_2.10/jars/akka-actor_2.10-2.2.3.jar:/home/tuxdna/.ivy2/cache/com.typesafe/config/bundles/config-1.0.2.jar:/home/tuxdna/.ivy2/cache/org.scala-lang.modules/scala-async_2.10/bundles/scala-async_2.10-0.9.0-M2.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-reflect/jars/scala-reflect-2.10.3.jar:/home/tuxdna/.ivy2/cache/org.scala-lang/scala-actors/jars/scala-actors-2.10.2.jar
How could similar functionality be achieved using SBT?
Also I tried to use sbt consoleProject to try and create a SBT Task.
Here is how:
$ sbt
sbt> consoleProject
scala> val dcc = (dependencyClasspath in Compile)
dcc: sbt.TaskKey[sbt.Keys.Classpath] = sbt.Scoped$$anon$2#3fa1115c
Now I can't figure out how to use SBT API to fetch the list of files. I have already tried to understand the Classpath from this page.
Any idea how to fetch the list of files?
Any task can define a separate log stream (named export) and dump information to it. All the classpath tasks do so.
sbt "export compile:dependencyClasspath"
sbt "export <...>" > foo should work, unless you have plugins writing to stdout directly rather than a logger.
This was broken prior to sbt 0.13.2 or 0.13.5 IIRC, but it should work now.