SBT stucks for an hour at downloadinng a library and showing speed = 0, here is the output:
sbt kafka/compile [process_args] java_version = '8'
# Executing command line: java
-XX:ReservedCodeCacheSize=128m
-Xms2g
-Xmx4g
-Xss4m
-XX:+UseG1GC
-XX:MaxMetaspaceSize=1g
-jar /usr/local/Cellar/sbt/1.2.8/libexec/bin/sbt-launch.jar kafka/compile
[info] Loading global plugins from /Users/minhthai/.sbt/1.0/plugins [info] Loading settings for project kafka_spark_streaming-build from assembly.sbt,plugins.sbt ... [info] Loading project definition from /Users/minhthai/coding/kafka_spark_streaming/project [info] Loading settings for project kafka_spark_streaming from build.sbt ... [info] Set current project to kafka_spark_streaming (in build file:/Users/minhthai/coding/kafka_spark_streaming/) [info] Fetching artifacts of kafka https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar
33.6% [... ] 1.5 MiB (0 B / s)
Here is my build file
ThisBuild / scalaVersion := "2.11.12"
ThisBuild / version := "0.0.1"
lazy val spark = project
.settings(
assembly / mainClass := Some("Main"),
assembly / assemblyJarName := "spark.jar",
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-sql" % "2.4.3",
"org.apache.spark" %% "spark-sql-kafka-0-10" % "2.4.3" % "provided"
)
)
lazy val kafka = project
.settings(
libraryDependencies ++= Seq(
"org.apache.kafka" % "kafka-clients" % "2.2.0"
)
)
I tried removing the cache at ~/.ivy2 and run again but the message is exactly the same, the download stops at 33.6%. I can still download other library and can manually download this jar file in browser.
So what can I do to resolve this issue? If no, is there a way to add this jar file manually?
Sbt has those kinds of downloading problems...restarting the process (Ctrl+C) and running again might help.
If it doesn't work, you can add the jar manually like this:
libraryDependencies += "org.scala-lang" % "scala-reflect" % "2.11.12" from "http://central.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar"
You can look at the sbt official documentation here.
I found the problem and it is silly. I use coursier plugin to handle dependencies and I should have deleted the cache of coursier instead of ivy2. On Mac, it is at ~/Library/Caches/Coursier/v1 (doc).
Related
Writing scala tests, I am encountering an error:
Symbol 'type org.scalatest.compatible.Assertion' is missing from the classpath.
This symbol is required by 'type org.scalatest.Assertion'.
Make sure that type Assertion is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'package.class' was compiled against an incompatible version of org.scalatest.compatible.
My build.sbt is very simple:
import sbt.Keys.libraryDependencies
ThisBuild / scalaVersion := "2.13.2"
ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / organization := "com.example"
ThisBuild / organizationName := "example"
lazy val root = (project in file("."))
.settings(
name := "myproj",
libraryDependencies += "org.scalactic" %% "scalactic" % "3.2.9",
libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.9" % "test",
libraryDependencies += "org.scalatest" %% "scalatest-funsuite" % "3.2.9" % "test",
libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.1.2",
)
This problem only occurs when I build my project within IntelliJ. Using sbt itself it compiles well.
sbt version in this project: 1.3.10
sbt script version: 1.3.6
Any idea of how to fix it in IntelliJ?
Most of the problems related to IntelliJ being broken I resolve by doing so:
sbt clean compile in terminal on root folder
View -> Tool Windows -> sbt and within tab Reload SBT project (right click on project)
As I want to execute the jar generated by my scala project in the Command Line Interface, I get the following problem:
Exception in thread "main" java.lang.NoClassDefFoundError: org/rogach/scallop/ScallopConf
Although in the dependency file I mentionned scallop dependency as follow
import sbt._
object Dependencies {
lazy val betterFiles = "com.github.pathikrit" %% "better-files" % "3.7.0"
lazy val scalaz = "org.scalaz" %% "scalaz-core" % "7.2.27"
lazy val scallop = "org.rogach" %% "scallop" % "3.1.5"
// -- Logging
lazy val scalaLogging = "com.typesafe.scala-logging" %% "scala-logging" % "3.9.2"
lazy val slf4jBackend = "org.slf4j" % "slf4j-simple" % "1.7.26"
// -- Testing
lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.5"
}
My build.sbt file is the following:
import Dependencies._
ThisBuild / scalaVersion := "2.12.5"
ThisBuild / sbtVersion := "1.2.6"
ThisBuild / version := "0.1.0-SNAPSHOT"
lazy val root = (project in file("."))
.settings(
name := "phenix-challenge",
libraryDependencies ++= Seq(
betterFiles,
scalaz,
scallop,
scalaLogging,
slf4jBackend % Runtime,
scalaTest % Test
)
)
If you have an Idea that could resolve my Issue please HELP!
Many thanks in advance
To execute the jar generated by your scala project in the Command Line Interface you can use sbt plugin to assembly a fat-jar including your libraries/dependencies. Having such jar you would be able to run your app via java -jar ...
There are several SBT plugins for build a fat-jar. Perhaps the easiest one would be the sbt-assembly.
Add this plugin to file project/plugins.sbt (create this file if needed):
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.9")
Now use sbt to build a fat-jar:
sbt assembly
Then run via java -jar YouMainClass
Another option is to use pure sbt to run Main class using command sbt run, then you do really need to build a fat-jar.
I have a problem, I want to build my scala project with sbt.
I have scala 2.11.8, sbt about print
[info] Loading project definition from /home/xxx
[info] Set current project to base (in build file:/home/xxx)
[info] This is sbt 0.13.8
[info] The current project is {file:/home/xxx}
[info] The current project is built against Scala 2.11.8
[info]
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.4
but still, when I run sbt compile I have exception with :
[error] during phase: typer
[error] library version: version 2.10.4
[error] compiler version: version 2.10.4
can someone help me?
Thanks
EDIT Forgot build.sbt file
build.sbt
name := "myproject"
version := "0.1"
scalaVersion := "2.11.8"
libraryDependencies ++= Seq(
"io.jvm.uuid" %% "scala-uuid" % "0.2.1",
"org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.4",
"org.scalatest" % "scalatest_2.11" % "2.2.6" % "test",
"com.github.nscala-time" %% "nscala-time" % "2.12.0",
"ch.qos.logback" % "logback-classic" % "1.1.7",
"com.typesafe.scala-logging" %% "scala-logging" % "3.4.0",
"org.scala-lang.modules" %% "scala-xml" % "1.0.4",
"org.scalacheck" %% "scalacheck" % "1.13.0" % "test"
)
I don't have any build.scala for this project
For Fedora 23. I initially installed sbt with the bintray repository. I had to remove it and re install sbt without the bintray repo.
I was not working so I re-re installed sbt avec bintray repository and it works. Some configuration somewhere was messed up I think...
I am creating a new Play Scala project within Eclipse + Scala IDE.
The original build.sbt file is:
name := """portal"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.11.1"
libraryDependencies ++= Seq(
jdbc,
cache,
ws
)
I've edited it to include some more dependencies:
libraryDependencies ++= Seq(
jdbc,
javaEbean,
cache,
ws,
"org.postgresql" % "postgresql" % "9.3-1100-jdbc4",
"org.scalatestplus" %% "play" % "1.1.0" % "test"
)
I can't figure out why SBT won't include Ebean, postgresql, nor scalatest in my classpath. Any help?
It happened that I needed to run the command
activator eclipse
again, and refresh the IDE.
How can I include mahout 0.9 libraries in play-framework 2.2.
I have added the jar files in the build path of eclipse but when I run the play app the following error is displayed for mahout imports
error: package org.apache.mahout.cf.taste.model does not exist
Include this line in build.sbt of play project
"org.apache.mahout" % "mahout-core" % "0.9"
Full build.sbt looks like this
name := <your_project_name>
version := "1.0-SNAPSHOT"
libraryDependencies ++= Seq(
javaJdbc,
javaEbean,
cache,
"org.apache.mahout" % "mahout-core" % "0.9"
)
play.Project.playJavaSettings