Can't use breeze even after I added dependencies to build.sbt - scala

I was following this tutorial on installing breeze, but I can't get it to work.
My directory structure:
myproject/
build.sbt
project/
Build.scala # This is empty
src/
main/
scala/
hello.scala
test/
scala/
my_tests.scala
My build.sbtlooks like this (It is mostly copied from the tutorial):
name := "My project"
libraryDependencies += "org.scalactic" %% "scalactic" % "3.0.0"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.0" % "test"
libraryDependencies ++= Seq(
// other dependencies here
"org.scalanlp" %% "breeze" % "0.12",
// native libraries are not included by default. add this if you want them (as of 0.7)
// native libraries greatly improve performance, but increase jar sizes.
// It also packages various blas implementations, which have licenses that may or may not
// be compatible with the Apache License. No GPL code, as best I know.
"org.scalanlp" %% "breeze-natives" % "0.12",
// the visualization library is distributed separately as well.
// It depends on LGPL code.
"org.scalanlp" %% "breeze-viz" % "0.12"
)
resolvers ++= Seq(
// other resolvers here
// if you want to use snapshot builds (currently 0.12-SNAPSHOT), use this.
"Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/",
"Sonatype Releases" at "https://oss.sonatype.org/content/repositories/releases/"
)
// or 2.11.5
scalaVersion := "2.11.8"
And my hello.scala file looks like this:
package mypackage
import breeze.linalg._
object Hello {
def main(args: Array[String]): Unit = {
println("Hello World")
val x = Dense.Vector.zeros[Double](5)
println(x)
}
}
The error that I get looks like this:
not found: value Dense
[error] val x = Dense.Vector.zeros[Double](5)
[error] ^
^
I know that I adding unit test related libraries to libraryDependencies correctly, because my unit tests work after I add them. But what am I doing wrong when adding dependencies for breeze? What steps should I take to narrow down the problem?

Related

SBT, how to add unmanaged JARs to IntelliJ?

I have build.sbt file:
import sbt.Keys.libraryDependencies
lazy val scalatestVersion = "3.0.4"
lazy val scalaMockTestSupportVersion = "3.6.0"
lazy val typeSafeConfVersion = "1.3.2"
lazy val scalaLoggingVersion = "3.7.2"
lazy val logbackClassicVersion = "1.2.3"
lazy val commonSettings = Seq(
organization := "com.stulsoft",
version := "0.0.1",
scalaVersion := "2.12.4",
scalacOptions ++= Seq(
"-feature",
"-language:implicitConversions",
"-language:postfixOps"),
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"ch.qos.logback" % "logback-classic" % logbackClassicVersion,
"com.typesafe" % "config" % typeSafeConfVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % "test",
"org.scalamock" %% "scalamock-scalatest-support" % scalaMockTestSupportVersion % "test"
)
)
unmanagedJars in Compile += file("lib/opencv-331.jar")
lazy val pimage = project.in(file("."))
.settings(commonSettings)
.settings(
name := "pimage"
)
parallelExecution in Test := true
It is working fine, if I use sbt run, but I cannot run from IntelliJ.
I receive error:
java.lang.UnsatisfiedLinkError: no opencv_java331 in java.library.path
I can add manually (File->Project Structure->Libraries->+ necessary dir).
My question is: is it possible to specify build.sbt that it will automatically create IntelliJ project with specified library?
I would say try to: drag and drop the dependency into the /lib which should be in the root directory of your project, if it's not there create it.
Run commands:
sbt reload
sbt update
Lastly you could try something like:
File -> Project Structure -> Modules -> then mark all the modules usually 1 to 3, delete them (don't worry won't delete your files) -> hit the green plus sign and select Import Module -> select root directory of your project and it should then refresh it
If none of these help, I'm out of ideas.

How to override dependency on certain task in sbt

I want to override dependency on project in certain Task.
I have a sbt multi-project which using spark.
lazy val core = // Some Project
val sparkLibs = Seq(
"org.apache.spark" %% "spark-core" % "1.6.1"
)
val sparkLibsProvided = Seq(
"org.apache.spark" %% "spark-core" % "1.6.1" % "provided"
)
lazy val main = Project(
id = "main",
base = file("main-project"),
settings = sharedSettings
).settings(
name := "main",
libraryDependencies ++= sparkLibs,
dependencyOverrides ++= Set(
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)
).dependsOn(core)
When I try to make fat jar to submit on my yarn cluster, I use https://github.com/sbt/sbt-assembly task. But in this case, I want to use sparkLibsProvided instead of sparkLibs something like:
lazy val sparkProvided = (project in assembly).settings(
dependencyOverrides ++= sparkLibsProvided.toSet
)
How can I properly override this dependency?
You can create a new project which is a dedicated project for creating your spark uber jar with the provided flag:
lazy val sparkUberJar = (project in file("spark-project"))
.settings(sharedSettings: _*)
.settings(
libraryDependencies ++= sparkLibsProvided,
dependencyOverrides ++= Set(
"com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)
)
And when you assemble in sbt, go to the said project first:
sbt project sparkUberJar
sbt assembly
This can be easily achieved by using the key provided specifically for what you want:
assemblyExcludedJars in assembly := {
val cp = (fullClasspath in assembly).value
cp filter {
_.data.getName == "spark-core-1.6.1.jar"
}
}
This approach is considered hacky, however, and it would be better if you managed to split your configuration into subprojects, as is also warned in official documentation here:
If you need to tell sbt-assembly to ignore JARs, you're probably doing it wrong. assembly task grabs deps JARs from your project's classpath. Try fixing the classpath first.

Creating a table in Cassandra using Phantom in Scala

I am following the tutorial here: http://blog.websudos.com/2015/04/04/a-series-on-phantom-part-1-getting-started-with-phantom/
Cassandra version: 2.1.8
PhatomVersion 1.10.1
Scala version: 2.11.2
sbt-version: 0.13.8
In addition to the code given in the article I have the following:
object App {
def main(args: Array[String]) {
val user = new User(UUID.fromString("00000000-0000-0000-0000-000000000000"), "test#test.com", "Dan", DateTime.now)
val resultSetFuture = Users.store(user)
Await.result(resultSetFuture, Duration.Inf)
}
}
When I run the program from sbt I get the following error (here is the head of the stack trace):
[error] (run-main-0) com.datastax.driver.core.exceptions.SyntaxError: line 1:157 no viable alternative at input 'CONSISTENCY' (..., 'Dan', 1437914728864) USING [CONSISTENCY]...)
com.datastax.driver.core.exceptions.SyntaxError: line 1:157 no viable alternative at input 'CONSISTENCY' (..., 'Dan', 1437914728864) USING [CONSISTENCY]...)
I have checked the csqlsh and the namespace has been created but no table has been created.
Any help much appreciated.
Here is the build.sbt in case it is useful:
name := "Something"
organization := "danmisun.github.com"
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.11.2"
crossScalaVersions := Seq("2.10.4", "2.11.2")
val PhantomVersion = "1.10.1"
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "2.2.1" % "test",
"org.scalacheck" %% "scalacheck" % "1.11.5" % "test",
"com.websudos" % "phantom_2.11" % PhantomVersion,
"com.websudos" % "phantom-dsl_2.11" % PhantomVersion,
"com.websudos" % "phantom-testkit_2.11" % PhantomVersion % "test, provided"
)
resolvers ++= Seq(
"Typesafe repository snapshots" at "http://repo.typesafe.com/typesafe/snapshots/",
"Typesafe repository releases" at "http://repo.typesafe.com/typesafe/releases/",
"Sonatype repo" at "https://oss.sonatype.org/content/groups/scala-tools/",
"Sonatype releases" at "https://oss.sonatype.org/content/repositories/releases",
"Sonatype snapshots" at "https://oss.sonatype.org/content/repositories/snapshots",
"Sonatype staging" at "http://oss.sonatype.org/content/repositories/staging",
"Java.net Maven2 Repository" at "http://download.java.net/maven/2/",
"Twitter Repository" at "http://maven.twttr.com",
Resolver.bintrayRepo("websudos", "oss-releases")
)
initialCommands := "import something._"
What does Users.store return? If it's returning Future[ResultSet], then wait for the future to complete. Try putting the await and check.
import scala.concurrent.{ Await, Future }
import scala.concurrent.duration.Duration
val resultSetFuture = Users.store(user)
Await.result(resultSetFuture, Duration.Inf)
First you need to create table: (comment after first run or you will receive "table already exists exception :P)
Await.result(Users.create.future(), 5000 millis)
Then remove .consistencyLevel_=(ConsistencyLevel.ALL) as phantom library seems to be very behind changes in cassandra which say that ConsistencyLevel is now defined per session not per request.
Now that should work but this library doesn't look too promising I would say.

Play 2.4.1, PlayEbean not found

After updating my Java project from 2.2 to 2.4, I followed the instructions on the Migration page, but am getting that error, saying the value PlayEbean was not found.
What am I doing wrong? As far as I can tell I only have to add that one line to the plugins.sbt file and it should work, right?
EDIT: I tried 2.4.2, exact same problem occured.
For clarity's sake: there is no build.sbt file. Only a Build.scala file and a BuildKeys.scala and BuildPlugin.scala file. Though those last 2 have no relation to this problem.
The files:
project/Build.scala:
import sbt._
import Keys._
import play.sbt.PlayImport._
import PlayKeys._
object BuildSettings {
val appVersion = "0.1"
val buildScalaVersion = "2.11.7"
val buildSettings = Seq (
version := appVersion,
scalaVersion := buildScalaVersion
)
}
object Resolvers {
val typeSafeRepo = "Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/"
val localRepo = "Local Maven Repositor" at "file://"+Path.userHome.absolutePath+"/.m2/repository"
val bintrayRepo = "scalaz-bintray" at "https://dl.bintray.com/scalaz/releases"
val sbtRepo = "Public SBT repo" at "https://dl.bintray.com/sbt/sbt-plugin-releases/"
val myResolvers = Seq (
typeSafeRepo,
localRepo,
bintrayRepo,
sbtRepo
)
}
object Dependencies {
val mindrot = "org.mindrot" % "jbcrypt" % "0.3m"
val libThrift = "org.apache.thrift" % "libthrift" % "0.9.2"
val commonsLang3 = "org.apache.commons" % "commons-lang3" % "3.4"
val commonsExec = "org.apache.commons" % "commons-exec" % "1.3"
val guava = "com.google.guava" % "guava" % "18.0"
val log4j = "org.apache.logging.log4j" % "log4j-core" % "2.3"
val jacksonDataType = "com.fasterxml.jackson.datatype" % "jackson-datatype-joda" % "2.5.3"
val jacksonDataformat = "com.fasterxml.jackson.dataformat" % "jackson-dataformat-xml" % "2.5.3"
val postgresql = "postgresql" % "postgresql" % "9.3-1103.jdbc41"
val myDeps = Seq(
// Part of play
javaCore,
javaJdbc,
javaWs,
cache,
// User defined
mindrot,
libThrift,
commonsLang3,
commonsExec,
guava,
log4j,
jacksonDataType,
jacksonDataformat,
postgresql
)
}
object ApplicationBuild extends Build {
import Resolvers._
import Dependencies._
import BuildSettings._
val appName = "sandbox"
val main = Project(
appName,
file("."),
settings = buildSettings ++ Seq (resolvers := myResolvers, libraryDependencies := myDeps)
)
.enablePlugins(play.PlayJava, PlayEbean)
.settings(jacoco.settings: _*)
.settings(parallelExecution in jacoco.Config := false)
.settings(javaOptions in Test ++= Seq("-Xmx512M"))
.settings(javaOptions in Test ++= Seq("-XX:MaxPermSize=512M"))
}
project/plugins.sbt:
// Use the Play sbt plugin for Play projects
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.4.1")
// The Typesafe repository
resolvers ++= Seq(
"Typesafe repository" at "http://repo.typesafe.com/typesafe/releases/",
"Local Maven Repositor" at "file://"+Path.userHome.absolutePath+"/.m2/repository",
"scalaz-bintray" at "https://dl.bintray.com/scalaz/releases",
"Public SBT repo" at "https://dl.bintray.com/sbt/sbt-plugin-releases/"
)
libraryDependencies ++= Seq(
"com.puppycrawl.tools" % "checkstyle" % "6.8",
"com.typesafe.play" %% "play-java-ws" % "2.4.1",
"org.jacoco" % "org.jacoco.core" % "0.7.1.201405082137" artifacts(Artifact("org.jacoco.core", "jar", "jar")),
"org.jacoco" % "org.jacoco.report" % "0.7.1.201405082137" artifacts(Artifact("org.jacoco.report", "jar", "jar"))
)
// Plugin for code coverage
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.1.6")
// Play enhancer - this automatically generates getters/setters for public fields
// and rewrites accessors of these fields to use the getters/setters. Remove this
// plugin if you prefer not to have this feature, or disable on a per project
// basis using disablePlugins(PlayEnhancer) in your build.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-play-enhancer" % "1.1.0")
// Play Ebean support, to enable, uncomment this line, and enable in your build.sbt using
// enablePlugins(SbtEbean). Note, uncommenting this line will automatically bring in
// Play enhancer, regardless of whether the line above is commented out or not.
addSbtPlugin("com.typesafe.sbt" % "sbt-play-ebean" % "1.0.0")
I have tried adding javaEbean to the myDeps variable, output remains the same.
Also, contrary to all the examples and tutorials, if I want to enable PlayJava, I have to do it via play.PlayJava. What is up with that?
For the error: not found: value PlayEbean, you must import play.ebean.sbt.PlayEbean in Build.scala,
Then you will have a not-found error for jacoco, you must import de.johoop.jacoco4sbt.JacocoPlugin.jacoco,
After that a NoClassDefFoundError, there you must upgrade SBT to 0.13.8 in project/build.properties,
Finally the postgresql dependency is incorrect and doesn't resolve.
The SBT part should work, in my case it fail later because I don't have eBeans in project.
Patch version:
diff a/project/Build.scala b/project/Build.scala
--- a/project/Build.scala
+++ b/project/Build.scala
## -1,3 +1,5 ##
+import de.johoop.jacoco4sbt.JacocoPlugin.jacoco
+import play.ebean.sbt.PlayEbean
import play.sbt.PlayImport._
import sbt.Keys._
import sbt._
## -35,7 +37,7 ##
val log4j = "org.apache.logging.log4j" % "log4j-core" % "2.3"
val jacksonDataType = "com.fasterxml.jackson.datatype" % "jackson-datatype-joda" % "2.5.3"
val jacksonDataformat = "com.fasterxml.jackson.dataformat" % "jackson-dataformat-xml" % "2.5.3"
- val postgresql = "postgresql" % "postgresql" % "9.3-1103.jdbc41"
+ val postgresql = "org.postgresql" % "postgresql" % "9.3-1103-jdbc41"
val myDeps = Seq(
// Part of play
diff a/project/build.properties b/project/build.properties
--- a/project/build.properties
+++ b/project/build.properties
## -1,1 +1,1 ##
-sbt.version=0.13.5
+sbt.version=0.13.8
EDIT: How did I end up with this: the latest versions of Scala plugin for IntelliJ IDEA allow better editing of SBT configs (than previously), but (for now) one need to make the SBT project build a first time to import it (i.e. commenting suspicious lines). Once the project is imported, one can use autocompletion, auto-import and other joys. I hope it will be usefull with crossScalaVersions. About that, keep in mind that Play 2.4 is Java 8+ only and Scala 2.10 doesn't support fully Java 8. (First section of the "Play 2.4 Migration Guide")

How to import Maven libraries with SBT?

I would like to import Maven libraries either with the Maven's XML file or SBT's Scala file. I guess there already are the same questions out, but I could't quite find any. Thank you!
You just treat remote Maven repositories normally. Unless you want to utilize your local .m2/repository. See below for an example Build.scala using both:
object myBuild extends Build {
lazy val mainProject = Project(
id="root",
base=file("."),
settings = Project.defaultSettings ++ Seq(
name := "Root project",
scalaVersion := "2.11.4",
version := "0.1",
resolvers ++= Seq(remoteMavenRepo, localMavenRepo),
libraryDependencies ++= List(
mavenLibrary1, mavenLibrary2
)
)
)
val remoteMavenRepo = "Sonatype Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots/"
val localMavenRepo = "Local Maven" at Path.userHome.asFile.toURI.toURL + ".m2/repository"
// if library folows scala version suffix convention, then we use %%
val mavenLibrary1 = "com.typesafe.slick" %% "slick" % "2.0.2"
// if it's a java library with no scala version suffix, then we use %
val mavenLibrary2 = "joda-time" % "joda-time" % "2.4"