How can I include mahout 0.9 libraries in play-framework 2.2.
I have added the jar files in the build path of eclipse but when I run the play app the following error is displayed for mahout imports
error: package org.apache.mahout.cf.taste.model does not exist
Include this line in build.sbt of play project
"org.apache.mahout" % "mahout-core" % "0.9"
Full build.sbt looks like this
name := <your_project_name>
version := "1.0-SNAPSHOT"
libraryDependencies ++= Seq(
javaJdbc,
javaEbean,
cache,
"org.apache.mahout" % "mahout-core" % "0.9"
)
play.Project.playJavaSettings
Related
SBT stucks for an hour at downloadinng a library and showing speed = 0, here is the output:
sbt kafka/compile [process_args] java_version = '8'
# Executing command line: java
-XX:ReservedCodeCacheSize=128m
-Xms2g
-Xmx4g
-Xss4m
-XX:+UseG1GC
-XX:MaxMetaspaceSize=1g
-jar /usr/local/Cellar/sbt/1.2.8/libexec/bin/sbt-launch.jar kafka/compile
[info] Loading global plugins from /Users/minhthai/.sbt/1.0/plugins [info] Loading settings for project kafka_spark_streaming-build from assembly.sbt,plugins.sbt ... [info] Loading project definition from /Users/minhthai/coding/kafka_spark_streaming/project [info] Loading settings for project kafka_spark_streaming from build.sbt ... [info] Set current project to kafka_spark_streaming (in build file:/Users/minhthai/coding/kafka_spark_streaming/) [info] Fetching artifacts of kafka https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar
33.6% [... ] 1.5 MiB (0 B / s)
Here is my build file
ThisBuild / scalaVersion := "2.11.12"
ThisBuild / version := "0.0.1"
lazy val spark = project
.settings(
assembly / mainClass := Some("Main"),
assembly / assemblyJarName := "spark.jar",
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-sql" % "2.4.3",
"org.apache.spark" %% "spark-sql-kafka-0-10" % "2.4.3" % "provided"
)
)
lazy val kafka = project
.settings(
libraryDependencies ++= Seq(
"org.apache.kafka" % "kafka-clients" % "2.2.0"
)
)
I tried removing the cache at ~/.ivy2 and run again but the message is exactly the same, the download stops at 33.6%. I can still download other library and can manually download this jar file in browser.
So what can I do to resolve this issue? If no, is there a way to add this jar file manually?
Sbt has those kinds of downloading problems...restarting the process (Ctrl+C) and running again might help.
If it doesn't work, you can add the jar manually like this:
libraryDependencies += "org.scala-lang" % "scala-reflect" % "2.11.12" from "http://central.maven.org/maven2/org/scala-lang/scala-reflect/2.11.12/scala-reflect-2.11.12.jar"
You can look at the sbt official documentation here.
I found the problem and it is silly. I use coursier plugin to handle dependencies and I should have deleted the cache of coursier instead of ivy2. On Mac, it is at ~/Library/Caches/Coursier/v1 (doc).
I have a spark streaming Scala project which uses Apache NiFi receiver. The projects runs fine under Eclipse/Scala IDE and now I want to package it for deployment now.
When I add it as
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
sbt assumes it's a Scala library and tries to resolve it.
How doe I add NiFi receiver and all it's dependencies to project's SBT file?
Also, is it possible to pint dependencies to local directories instead of sbt trying to resolve?
Thanks in advance.
Here is my sbt file contents:
name := "NiFi Spark Test"
version := "1.0"
scalaVersion := "2.10.5"
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2" % "provided"
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
libraryDependencies += "org.apache.nifi" %% "nifi-spark-receiver" % "0.3.0"
Double % are used for adding scala version as suffix to the maven artefact. It is required because different scala compiler versions produces incompatible bytecode. If you are would like to use java library from maven, then you should use single % character
libraryDependencies += "org.apache.nifi" % "nifi-spark-receiver" % "0.3.0"
I also found that I can put libraries the project depends on into the lib folder and they will be picked up during assembly.
I am new at scala. And for start I want to use Intellij 13.1.5 IDE.
However IDE can't attach sources. Here is how it looks for AnyVal:
Search at internet can't find any source.
I tried Attach sources and attach unpacked scala archive. It doesn't work either.
UPDATE:
Here is sbt configuration:
name := "scalatest-selenium"
version := "1.0"
scalaVersion := "2.11.1"
libraryDependencies ++= Seq(
"net.sourceforge.htmlunit" % "htmlunit" % "2.14",
"org.seleniumhq.selenium" % "selenium-java" % "2.42.2",
"org.scalacheck" % "scalacheck_2.10" % "1.11.4" % "test",
"org.scalatest" % "scalatest_2.11" % "2.2.0" % "test"
)
testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-u", "target/test-reports")
How to solve this trouble?
I get rid of this trouble at the following way:
removed the .sbt directory in your Home Folder.
When you run sbt again, the new folder is created in the correct format and the error goes away.
I am about to publish a Play project to github, but I would like that Scala developers don't need to install play separately if they already have sbt installed.
Would it be possible to compile and run a Play project with sbt if we add the correct dependencies ?
My working build.sbt for Play is here, but it doesn't work with sbt.
import play.Project._
name := "my_project_name"
version := "1.0"
libraryDependencies ++= Seq(
"com.netflix.rxjava" % "rxjava-scala" % "0.17.4",
"oauth.signpost" % "signpost-core" % "1.2.1.2",
"oauth.signpost" % "signpost-commonshttp4" % "1.2.1.2",
"org.apache.httpcomponents" % "httpclient" % "4.3.3",
"commons-io" % "commons-io" % "2.3"
)
playScalaSettings
Thanks!
You just should add the following line to project/plugins.sbt file:
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.2.3") // or your play version
play new does this by default.
You may check if the project is executable by sbt by executing sbt run from the terminal.
I am building a simple Scala project with SBT 0.11.
All the code files are in ~/MyProject/src/main/scala/
~/MyProject/build.sbt is the following
name := "MyProject"
version := "1.0"
scalaVersion := "2.9.1"
libraryDependencies ++= Seq(
"mysql" % "mysql-connector-java" % "5.1.+",
"c3p0" % "c3p0" % "0.9.1.2",
"org.apache.commons" % "commons-lang3" % "3.0.1",
"commons-lang" % "commons-lang" % "2.6",
"javassist" % "javassist" % "3.12.1.GA"
)
~/MyProject/project/Build.scala is the following
import sbt._
object MyProjectBuild extends Build {
lazy val MyProject = Project("MyProject", file("."))
}
This seems to work almost fine. The project does compile and run. The project name is set correctly (if I don't use Build.scala, then the name seems to appear something like "default-????", despite it being specified in build.sbt).
But the problem is that dependencies do not seem to work - update command doesn't download anything. How to fix this? Do I need to specify my dependencies in Build.scala rather than in build.sbt in this case?
Is it possible that you've already retrieved the project dependencies, but don't realize it because they are stored in the Ivy cache? You can view the managed classpath from the SBT console with the command
show managed-classpath
Recent versions of SBT do not store the managed dependencies in the project directory, unless the project is configured to do so. If you want, you can add the following to your build.sbt file:
retrieveManaged := true
This will create a ~/MyProject/lib_managed/ directory and contents.