so I'm trying to compile and run a scala project on a 512mb Digital Ocean server. Basically, the issue I'm running into is that when it starts compiling it pulls in all the dependencies, and ultimately runs out of memory. Is this normal? I know that the project I'd like to run doesn't need more than 512mb, and it would be a shame to have to increase the size just to make it compile. Am I doing something wrong here?
Here's my build.sbt, note this is using sbt.version 13.8
scalaVersion := "2.10.2"
libraryDependencies ++= {
val akkaV = "2.1.4"
val sprayV = "1.1.1"
Seq(
"com.typesafe.akka" %% "akka-actor" % akkaV,
"com.typesafe.akka" %% "akka-testkit" % akkaV % "test",
"org.specs2" %% "specs2-core" % "2.3.9" % "test",
"org.twitter4j" % "twitter4j-stream" % "4.0.2",
"io.spray" % "spray-can" % sprayV,
"io.spray" % "spray-routing" % sprayV,
"io.spray" % "spray-client" % sprayV,
"io.spray" % "spray-testkit" % sprayV,
"io.spray" % "spray-json_2.10" % "1.2.5",
"com.github.nscala-time" %% "nscala-time" % "1.4.0",
"com.codahale.metrics" % "metrics-core" % "3.0.1",
"org.elasticsearch" % "metrics-elasticsearch-reporter" % "2.0"
)
}
enablePlugins(JavaAppPackaging)
//settings(com.typesafe.sbt.SbtNativePackager.packageArchetype.java_application: _*)
Revolver.settings
You can use a trick. Sbt has requirements to virtual memory (it ignores jvm heap options with low values), so turn on swap space (it's disabled by default in DO images). There is no affects on performance if real memory is enough.
Related
I try to start the sample test NG test in the scala sbt framework. I am using below the dependencies.
"org.apache.spark" %% "spark-core" % sparkVer % Provided,
"org.apache.hadoop" % "hadoop-common" % sparkVer % Provided,
"org.apache.spark" % "spark-sql_2.11" % sparkVer % Provided,
"org.apache.spark" % "spark-hive_2.11" % sparkVer,
"org.scalactic" %% "scalactic" % scalatestVer,
"org.scalatest" %% "scalatest" % scalatestVer % Test,
"info.cukes" % "cucumber-scala_2.11" % cucumberVer,
"info.cukes" % "cucumber-junit" % cucumberVer,
"junit" % "junit" % "4.12",
"org.scalatest" % "scalatest_2.11" % "2.0" % "test",
"org.scalactic" %% "scalactic" % "3.0.1",
"org.scalatest" %% "scalatest" % "3.0.1" % "test",
[The image is ExampleSuite class as you can see I am not able to use it correctly1
Here is the link I follow up for this case http://www.scalatest.org/getting_started_with_testng_in_scala
Here is also sbt.version = 0.13.16
Any help really appreciate.
Add dependencies => "org.testng" % "testng" % "6.14.3" % Test, and "create TestNG XML" plugin upload from marketplace than resolve the issue.
i am using SBT to setup Akka Persistence but it's failing with Error :
Detected java.lang.NoSuchMethodError error, which MAY be caused by incompatible Akka versions on the classpath. Please note that a given Akka version MUST be the same across all modules of Akka that you are using, e.g. if you use akka-actor [2.5.6 (resolved from current classpath)] all other core Akka modules MUST be of the same version. External projects like Alpakka, Persistence plugins or Akka HTTP etc. have their own version numbers - please make sure you're using a compatible set of libraries.
Uncaught error from thread [TopsActivitiesSystem-akka.actor.default-dispatcher-16]: akka.persistence.Eventsourced.persist$(Lakka/persistence/Eventsourced;Ljava/lang/Object;Lscala/Function1;)V, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for for ActorSystem[TopsActivitiesSystem]
Detected java.lang.NoSuchMethodError error, which MAY be caused by incompatible Akka versions on the classpath. Please note that a given Akka version MUST be the same across all modules of Akka that you are using, e.g. if you use akka-actor [2.5.6 (resolved from current classpath)] all other core Akka modules MUST be of the same version. External projects like Alpakka, Persistence plugins or Akka HTTP etc. have their own version numbers - please make sure you're using a compatible set of libraries.
Uncaught error from thread [TopsActivitiesSystem-akka.actor.default-dispatcher-3]: akka.persistence.Eventsourced.persist$(Lakka/persistence/Eventsourced;Ljava/lang/Object;Lscala/Function1;)V, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for for ActorSystem[TopsActivitiesSystem]
java.lang.NoSuchMethodError: akka.persistence.Eventsourced.persist$(Lakka/persistence/Eventsourced;Ljava/lang/Object;Lscala/Function1;)V
My Configuration SBT Build file is :
scalaVersion := "2.12.4"
lazy val root = (project in file("."))
.configs(IntegrationTest)
def akkaVersion = "2.5.6"
def akkaHttpVersion = "10.0.10"
def logbackVersion = "1.2.3"
def ItAndTest = "it, test"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-stream" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster" % akkaVersion,
"com.typesafe.akka" %% "akka-cluster-tools" % akkaVersion,
"com.typesafe.akka" % "akka-cluster-metrics_2.12" % akkaVersion,
"com.redmart.fc" %% "fc-capacity-model" % "1.7.0",
"com.redmart.akka" %% "akka-downing" % "1.1.0",
"com.jsuereth" %% "scala-arm" % "2.0",
"com.typesafe" % "config" % "1.3.1",
"com.typesafe.scala-logging" %% "scala-logging" % "3.7.2",
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"ch.qos.logback" % "logback-classic" % logbackVersion,
"ch.qos.logback" % "logback-access" % logbackVersion,
"net.logstash.logback" % "logstash-logback-encoder" % "4.11",
"joda-time" % "joda-time" % "2.9.3",
"com.typesafe.akka" %% "akka-http" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-persistence-cassandra" % "0.58",
// "com.typesafe.akka" %% "akka-cluster" % akkaVersion,
// "com.typesafe.akka" %% "akka-cluster-tools" % akkaVersion,
"com.lightbend.akka" %% "akka-management-cluster-http" % "0.4",
// "org.iq80.leveldb" % "leveldb" % "0.9",
// "org.fusesource.leveldbjni" % "leveldbjni-all" % "1.8",
"io.spray" %% "spray-json" % "1.3.3",
"com.enragedginger" %% "akka-quartz-scheduler" % "1.6.1-akka-2.5.x",
// "com.softwaremill.macwire" %% "macros" % "2.2.2" % "provided",
// "com.softwaremill.macwire" %% "util" % "2.2.2",
"com.esotericsoftware" % "kryo" % "4.0.1",
"com.github.romix.akka" %% "akka-kryo-serialization" % "0.5.2",
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpVersion % ItAndTest,
"org.scalatest" %% "scalatest" % "3.0.4" % ItAndTest,
"org.mockito" % "mockito-core" % "2.10.0" % ItAndTest,
"com.github.dnvriend" %% "akka-persistence-inmemory" % "2.5.1.1" % ItAndTest
)
dependencyOverrides += "com.typesafe.akka" %% "akka-actor" % akkaVersion
dependencyOverrides += "com.typesafe.akka" %% "akka-stream" % akkaVersion
dependencyOverrides += "com.typesafe.akka" %% "akka-cluster" % akkaVersion
dependencyOverrides += "com.typesafe.akka" %% "akka-cluster-tools" % akkaVersion
dependencyOverrides += "com.typesafe.akka" %% "akka-persistence" % akkaVersion
dependencyOverrides += "com.typesafe.akka" %% "akka-slf4j" % akkaVersion
dependencyOverrides += "com.typesafe.akka" % "akka-cluster-metrics_2.12" % akkaVersion
Can Anyone explain the reason why it's failing? I am sure i am overriding to latest package. which is 2.5.6 using Dependency Overrides. i am Using SBT 0.13.
Use the current version (0.5) of akka-management-cluster-http:
"com.lightbend.akka" %% "akka-management-cluster-http" % "0.5"
Version 0.4 depends on an older version of Akka.
I have recently started using play with scala and akka and I am just playing with it. I have installed scala-ide, gst and hopefully everything that is needed. However, when working in the scala-ide(..eclipse?) my code editor is covered with a lot of "not found" errors that mainly come from sbt dependencies.
To be specific, my build.sbt file looks like this:
name := """testApp"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
scalaVersion := "2.11.1"
libraryDependencies ++= {
val akkaV = "2.3.6"
val sprayV = "1.3.2"
Seq(
"io.spray" %% "spray-can" % sprayV,
"io.spray" %% "spray-routing" % sprayV,
"io.spray" %% "spray-testkit" % sprayV % "test",
"com.typesafe.akka" %% "akka-actor" % akkaV,
"com.typesafe.akka" %% "akka-testkit" % akkaV % "test",
"com.typesafe.akka" %% "akka-contrib" % "2.3.7",
"org.webjars" % "bootstrap" % "3.0.0",
"org.webjars" % "knockout" % "2.3.0",
"org.webjars" % "requirejs" % "2.1.11-1",
"org.webjars" % "leaflet" % "0.7.2",
"org.webjars" % "rjs" % "2.1.11-1-trireme" % "test",
"org.webjars" % "squirejs" % "0.1.0" % "test",
"org.specs2" %% "specs2-core" % "2.3.11" % "test"
)
}
but still, scala-ide fails to load import spray.http._ or import akka.contrib.pattern.DistributedPubSubExtension (both are covered with errors when importing like "object contrib is not a member of package akka"). Sbt compiles the project with success and also I can easily run the app with ./activator run
I am using IntelliJ IDEA 13.1.2 with the Scala plugin version 0.36.431 on Windows 7 with sbt 0.13.1.
The following project definition build.sbt has no references to any Scala version other than 2.9.3.
import sbt._
import Keys._
import AssemblyKeys._
import NativePackagerKeys._
name := "simplews"
version := "0.1.0-SNAPSHOT"
val sparkVersion = "0.8.1-incubating"
scalaVersion := "2.9.3"
val akkaVersion = "2.0.5"
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.spark" % "spark-examples_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.spark" % "spark-tools_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.scalatest" % "scalatest_2.9.3" % "1.9.2" % "test" withSources(),
"org.apache.spark" % "spark-repl_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.kafka" % "kafka" % "0.7.2-spark",
"com.thenewmotion.akka" % "akka-rabbitmq_2.9.2" % "0.0.2" % "compile->default" withSources(),
"com.typesafe.akka" % "akka-actor" % akkaVersion % "compile->default" withSources(),
"com.typesafe.akka" % "akka-testkit" % akkaVersion % "compile->default" withSources(),
"com.rabbitmq" % "amqp-client" % "3.0.1" % "compile->default" withSources(),
"org.specs2" % "specs2_2.9.3" % "1.12.4.1" % "compile->default" withSources(),
"com.nebhale.jsonpath" % "jsonpath" % "1.2" % "compile->default" withSources(),
"org.mockito" % "mockito-all" % "1.8.5",
"junit" % "junit" % "4.11"
)
packagerSettings
packageArchetype.java_application
resolvers ++= Seq(
"Apache repo" at "https://repository.apache.org/content/repositories/releases",
"Cloudera repo" at "https://repository.cloudera.com/artifactory/repo/org/apache/kafka/kafka/0.7.2-spark/",
"akka rabbitmq" at "http://nexus.thenewmotion.com/content/repositories/releases-public",
"Local Repo" at Path.userHome.asFile.toURI.toURL + "/.m2/repository",
Resolver.mavenLocal
)
However as seen in the screenshot the debugger has jumped to scala 2.10.2. Note: the debugger is correctly going to 2.9.3 for some other debugging.
Here is project/plugins.sbt:
resolvers += "sbt-plugins" at "http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases"
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "0.7.0-RC2")
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
EDIT In order to reproduce it is necessary to do a mvn local install on one or two libraries that are not available in any public repo.
mvn org.apache.maven.plugins:maven-install-plugin:2.5.1:install-file -Dfile=c:\shared\kafka-0.7.2-spark.jar -DgroupId=org.apache.kafka -DartifactId=kafka -Dversion=0.7.2-spark -Dpackaging=jar
I had in any case not considered someone (om-nom-nom !) would attempt an exact reproduction - so had also omitted otherwise extraneous items like mergeStrategy and assemblyKeys.
A fully independent reproducible setup may be a bit in coming - I have been under rather quite heavy demands here.
This question is partially related to sbt-scalariform plugin - can't resolve settings. I managed to run scalariform from command line as SBT task.
Now the problem is with IDEA. When I open my build.sbt, which looks like this:
import scalariform.formatter.preferences._
name := """scheduling-backend"""
version := "1.0"
scalaVersion := "2.10.2"
resolvers += "spray repo" at "http://repo.spray.io"
resolvers += "spray nightlies" at "http://nightlies.spray.io"
resolvers += "SpringSource Milestone Repository" at "http://repo.springsource.org/milestone"
resolvers += "Neo4j Cypher DSL Repository" at "http://m2.neo4j.org/content/repositories/releases"
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % "2.3.0",
"com.typesafe.akka" %% "akka-slf4j" % "2.3.0",
"com.typesafe.akka" %% "akka-testkit" % "2.3.0" % "test",
"com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.0",
"io.spray" % "spray-can" % "1.3.0",
"io.spray" % "spray-routing" % "1.3.0",
"io.spray" % "spray-testkit" % "1.3.0" % "test",
"io.spray" %% "spray-json" % "1.2.5",
"ch.qos.logback" % "logback-classic" % "1.0.13",
"org.specs2" %% "specs2" % "1.14" % "test",
"org.springframework.scala" % "spring-scala" % "1.0.0.M2",
"org.springframework.data" % "spring-data-neo4j" % "3.0.0.RELEASE",
"org.springframework.data" % "spring-data-neo4j-rest" % "3.0.0.RELEASE",
"javax.validation" % "validation-api" % "1.1.0.Final",
"com.github.nscala-time" %% "nscala-time" % "0.8.0",
"org.neo4j" % "neo4j-kernel" % "2.0.1" % "test" classifier "tests",
"com.sun.jersey" % "jersey-core" % "1.9",
"org.mockito" % "mockito-all" % "1.9.5"
)
scalacOptions ++= Seq(
"-unchecked",
"-deprecation",
"-Xlint",
"-Ywarn-dead-code",
"-language:_",
"-target:jvm-1.7",
"-encoding", "UTF-8"
)
org.scalastyle.sbt.ScalastylePlugin.Settings
scalariformSettings
ScalariformKeys.preferences := ScalariformKeys.preferences.value
.setPreference(AlignParameters, true)
.setPreference(CompactControlReadability, true)
IDEA reports problems with my file.
I am getting Cannot resolve symbol scalariformSettings and Cannot resolve symbol ScalariformKeyseven if I everything works from terminal.
adding import com.typesafe.sbt.SbtScalariform._ to build.sbt seems to fix the error on 13.1.1 with scala plugin 0.33.403, but I have to admit it ignored the import at first and then randomly started to see it.