Error:- java.lang.NoClassDefFoundError: scala/Product$class - scala

Test project Build.sbt
scalaVersion := "2.12.8"
crossScalaVersions := Seq("2.11.12", "2.12.8")
val scalaTestVersion = "3.0.5"
val rocksDBVersion = "5.17.2"
val kafkaVersion = "2.2.0"
lazy val deps = Seq(
// "javax.ws.rs" % "javax.ws.rs-api" % "2.1" jar(),
"org.apache.kafka" % "kafka-clients" % kafkaVersion,
"org.apache.kafka" % "kafka-clients" % kafkaVersion classifier "test",
"org.apache.kafka" % "kafka-streams" % kafkaVersion,
"org.apache.kafka" % "kafka-streams" % kafkaVersion classifier "test",
"org.apache.kafka" %% "kafka" % kafkaVersion,
"org.apache.kafka" % "kafka-streams-test-utils" % kafkaVersion,
"org.scalatest" %% "scalatest" % scalaTestVersion % "test",
"org.rocksdb" % "rocksdbjni" % rocksDBVersion % "test"
)
Test Project plugins.sbt
addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.8")
This is my Test application sbt configuration file and when I run sbt package it will create a jar file for me then I have to use that jar in my other project.
Other project sbt file
scalaVersion := "2.12.6"
object Dependencies {
val deps = Seq(
"org.json4s" %% "json4s-native" % "3.5.3",
"ch.qos.logback" % "logback-classic" % "1.2.3",
"cool.graph" % "cuid-java" % "0.1.1",
"org.apache.kafka" % "kafka-streams" % "2.2.0",
"com.typesafe" % "config" % "1.3.3",
"org.scalaz" %% "scalaz-core" % "7.2.20",
"org.apache.commons" % "commons-lang3" % "3.8.1",
//Test
"org.scalatest" %% "scalatest" % "3.0.4" % Test,
"org.mockito" % "mockito-all" % "1.10.19" % Test,
"com.abc" %% "kafka-streams-test-kit" % "2.2.0" % Test
)
}
Other Project plugins.sbt
addSbtPlugin("se.marcuslonnberg" % "sbt-docker" % "1.5.0")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.9.0")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.6")
addSbtPlugin("com.github.gseitz" % "sbt-release" % "1.0.8")
As I'm new in Scala and I don't know what i did wrong in that as when i put my test scala jar into another project It will give me the error.
Error:-
java.lang.NoClassDefFoundError: scala/Product$class
[error] at com.shepherd.kafka.streams.test.kit.MockedStreams$Builder.<init>(MockedStreams.scala:17)
[error] at com.shepherd.kafka.streams.test.kit.MockedStreams$.apply(MockedStreams.scala:15)
[error] at com.shepherd.integration.WindowedThresholdSpec.$anonfun$new$4(WindowedThresholdSpec.scala:38)
[error] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
Can anyone please help me on this as I'm new also I have see this one java.lang.NoClassDefFoundError: scala/Product$class but this not help me so I have put my problem here.
Try with both the way like uploaded jar directly into the code and pull by S3 but the error is still exists.
resolvers += "Java.net Maven2 Repository" at "https://repo1.maven.org/maven2/"
resolvers += "S3" at "https://s3-eu-west-1.amazonaws.com/com.abc/" -> Test jar file
NOTE:- I have Set the same scala version in my both the application "2.12.6" but issue is still exists.

Related

New sbt application using the akka-http template, how to determine resolvers and add maven central?

I have a new sbt application that I built using the akka http g8 template.
I am trying to add reactivemongo 1.0 to my build and I am getting this error:
not found: https://repo1.maven.org/maven2/org/reactivemongo/reactivemongo_2.13/1.0/reactivemongo_2.13-1.0.pom
The documentation says this library is in maven central.
How can I determine which resolver my project is using by default currently in sbt?
Is it possible that this library is not built for scala 2.13.3 or 2.13.1?
How can I debug this type of error.
Thanks!
build.sbt:
import Dependencies._
lazy val akkaHttpVersion = "10.2.1"
lazy val akkaVersion = "2.6.10"
lazy val root = (project in file("."))
.settings(
inThisBuild(
List(
organization := "com.example",
scalaVersion := "2.13.3"
)
),
name := "akka-http",
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-http" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-actor-typed" % akkaVersion,
"com.typesafe.akka" %% "akka-stream" % akkaVersion,
"ch.qos.logback" % "logback-classic" % "1.2.3",
"com.softwaremill.macwire" %% "macros" % "2.3.3" % "provided",
"com.softwaremill.macwire" %% "util" % "2.3.3" % "provided",
"com.github.blemale" %% "scaffeine" % "3.1.0" % "compile",
"org.typelevel" %% "cats-core" % "2.1.1",
"com.lihaoyi" %% "scalatags" % "0.8.2",
"com.github.pureconfig" %% "pureconfig" % "0.13.0",
"org.reactivemongo" %% "reactivemongo" % "1.0",
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpVersion % Test,
"com.typesafe.akka" %% "akka-actor-testkit-typed" % akkaVersion % Test,
"org.scalatest" %% "scalatest" % "3.0.8" % Test
)
)
.enablePlugins(JavaAppPackaging)
Can you try replacing "org.reactivemongo" %% "reactivemongo" % "1.0" with "org.reactivemongo" %% "reactivemongo" % "1.0.0" % "provided"?
I copy it from Maven Repository https://mvnrepository.com/artifact/org.reactivemongo/reactivemongo_2.13/1.0.0

How to convert RDD[some case class] to csv file using scala?

I have an RDD[some case class] and I want to convert it to a csv file. I am using spark 1.6 and scala 2.10.5 .
stationDetails.toDF.coalesce(1).write.format("com.databricks.spark.csv").save("data/myData.csv")
gives error
Exception in thread "main" java.lang.ClassNotFoundException: Failed to find data source: com.databricks.spark.csv. Please find packages at http://spark-packages.org
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.lookupDataSource(ResolvedDataSource.scala:77)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:219)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:148)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:139)
I am not able to add the dependencies for "com.databricks.spark.csv" in my build.sbt file.
dependencies I added in build.sbt file are:
libraryDependencies ++= Seq(
"org.apache.commons" % "commons-csv" % "1.1",
"com.univocity" % "univocity-parsers" % "1.5.1",
"org.slf4j" % "slf4j-api" % "1.7.5" % "provided",
"org.scalatest" %% "scalatest" % "2.2.1" % "test",
"com.novocode" % "junit-interface" % "0.9" % "test"
)
I also tried this
stationDetails.toDF.coalesce(1).write.csv("data/myData.csv")
but it gives error : csv cannot be resolved.
Please change your build.sbt to below -
libraryDependencies ++= Seq(
"org.apache.commons" % "commons-csv" % "1.1",
"com.databricks" %% "spark-csv" % "1.4.0",
"com.univocity" % "univocity-parsers" % "1.5.1",
"org.slf4j" % "slf4j-api" % "1.7.5" % "provided",
"org.scalatest" %% "scalatest" % "2.2.1" % "test",
"com.novocode" % "junit-interface" % "0.9" % "test"
)

PlaySpecification not found in Play 2.6.3

I've imported Specs2 and everything looks good but some things that are not imported, among them the PlaySpecification trait.
I've tried to reload in sbt, to invalidate caches in Intellij ... But this trait is missing!
My built.sbt
name := """web2"""
version := "1.0-SNAPSHOT"
scalaVersion := "2.12.2"
lazy val root = (project in file(".")).enablePlugins(PlayScala, LauncherJarPlugin)
pipelineStages := Seq(digest)
libraryDependencies ++= Seq(
evolutions,
jdbc,
ehcache,
ws,
"com.softwaremill.macwire" %% "macros" % "2.3.0" % "provided",
"org.postgresql" % "postgresql" % "42.1.1",
"org.scalikejdbc" %% "scalikejdbc" % "3.0.0",
"org.scalikejdbc" %% "scalikejdbc-config" % "3.0.0",
"ch.qos.logback" % "logback-classic" % "1.2.3",
"de.svenkubiak" % "jBCrypt" % "0.4.1",
//"org.scalatestplus.play" %% "scalatestplus-play" % "3.0.+" % "test",
"org.mockito" % "mockito-core" % "2.7.22" % "test",
"org.specs2" %% "specs2-core" % "3.9.+" % "test"
)
resolvers += "scalaz-bintray" at "http://dl.bintray.com/scalaz/releases"
plugins.sbt
// The Play plugin
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.6.3")
// Web plugins
addSbtPlugin("com.typesafe.sbt" % "sbt-digest" % "1.1.2")
According to the documentation, replace "org.specs2" %% "specs2-core" % "3.9.+" % "test" with specs2 % Test in your dependencies:
libraryDependencies ++= Seq(
evolutions,
jdbc,
ehcache,
...
"org.mockito" % "mockito-core" % "2.7.22" % "test",
specs2 % Test
)

Why does IntelliJ IDEA debugger jump to wrong Scala version library?

I am using IntelliJ IDEA 13.1.2 with the Scala plugin version 0.36.431 on Windows 7 with sbt 0.13.1.
The following project definition build.sbt has no references to any Scala version other than 2.9.3.
import sbt._
import Keys._
import AssemblyKeys._
import NativePackagerKeys._
name := "simplews"
version := "0.1.0-SNAPSHOT"
val sparkVersion = "0.8.1-incubating"
scalaVersion := "2.9.3"
val akkaVersion = "2.0.5"
libraryDependencies ++= Seq(
"org.apache.spark" % "spark-core_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.spark" % "spark-examples_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.spark" % "spark-tools_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.scalatest" % "scalatest_2.9.3" % "1.9.2" % "test" withSources(),
"org.apache.spark" % "spark-repl_2.9.3" % sparkVersion % "compile->default" withSources(),
"org.apache.kafka" % "kafka" % "0.7.2-spark",
"com.thenewmotion.akka" % "akka-rabbitmq_2.9.2" % "0.0.2" % "compile->default" withSources(),
"com.typesafe.akka" % "akka-actor" % akkaVersion % "compile->default" withSources(),
"com.typesafe.akka" % "akka-testkit" % akkaVersion % "compile->default" withSources(),
"com.rabbitmq" % "amqp-client" % "3.0.1" % "compile->default" withSources(),
"org.specs2" % "specs2_2.9.3" % "1.12.4.1" % "compile->default" withSources(),
"com.nebhale.jsonpath" % "jsonpath" % "1.2" % "compile->default" withSources(),
"org.mockito" % "mockito-all" % "1.8.5",
"junit" % "junit" % "4.11"
)
packagerSettings
packageArchetype.java_application
resolvers ++= Seq(
"Apache repo" at "https://repository.apache.org/content/repositories/releases",
"Cloudera repo" at "https://repository.cloudera.com/artifactory/repo/org/apache/kafka/kafka/0.7.2-spark/",
"akka rabbitmq" at "http://nexus.thenewmotion.com/content/repositories/releases-public",
"Local Repo" at Path.userHome.asFile.toURI.toURL + "/.m2/repository",
Resolver.mavenLocal
)
However as seen in the screenshot the debugger has jumped to scala 2.10.2. Note: the debugger is correctly going to 2.9.3 for some other debugging.
Here is project/plugins.sbt:
resolvers += "sbt-plugins" at "http://scalasbt.artifactoryonline.com/scalasbt/sbt-plugin-releases"
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "0.7.0-RC2")
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
EDIT In order to reproduce it is necessary to do a mvn local install on one or two libraries that are not available in any public repo.
mvn org.apache.maven.plugins:maven-install-plugin:2.5.1:install-file -Dfile=c:\shared\kafka-0.7.2-spark.jar -DgroupId=org.apache.kafka -DartifactId=kafka -Dversion=0.7.2-spark -Dpackaging=jar
I had in any case not considered someone (om-nom-nom !) would attempt an exact reproduction - so had also omitted otherwise extraneous items like mergeStrategy and assemblyKeys.
A fully independent reproducible setup may be a bit in coming - I have been under rather quite heavy demands here.

sbt gen-idea scala version error

I receive the following error when running sbt gen-idea:
[error] {file:/home/jem/projects/app/}default-0df603/*:
update-classifiers: Version specified for dependency
org.scala-lang#scalap;2.10.0 differs from Scala version in project (2.10.1).
Despite this the idea files are still created and seem to work. But I worry that they may be incomplete somehow.
Here is my build.sbt:
import com.typesafe.startscript.StartScriptPlugin
organization := "com.github.synesso"
name := "app"
version := "0.1"
scalaVersion := "2.10.1"
seq(webSettings :_*)
classpathTypes ~= (_ + "orbit")
libraryDependencies ++= Seq(
"org.scalatra" % "scalatra_2.10" % "2.2.1",
"org.scalatra" % "scalatra-scalate_2.10" % "2.2.1",
"org.scalatra" % "scalatra-json_2.10" % "2.2.1",
"org.eclipse.jetty" % "jetty-server" % "9.0.1.v20130408",
"org.eclipse.jetty" % "jetty-server" % "9.0.1.v20130408" % "container",
"org.eclipse.jetty" % "jetty-webapp" % "9.0.1.v20130408",
"org.eclipse.jetty" % "jetty-webapp" % "9.0.1.v20130408" % "container",
"org.eclipse.jetty" % "jetty-webapp" % "8.1.7.v20120910" % "container,compile",
"org.eclipse.jetty.orbit" % "javax.servlet" % "3.0.0.v201112011016" % "container,compile" artifacts Artifact("javax.servlet", "jar", "jar"),
"org.json4s" % "json4s-jackson_2.10" % "3.2.4",
"org.slf4j" % "slf4j-simple" % "1.7.5",
"ch.qos.logback" % "logback-classic" % "1.0.11" % "runtime",
"com.google.api-client" % "google-api-client" % "1.14.1-beta",
"com.google.apis" % "google-api-services-plus" % "v1-rev62-1.14.1-beta",
"com.google.apis" % "google-api-services-oauth2" % "v1-rev33-1.14.1-beta",
"com.google.http-client" % "google-http-client-jackson2" % "1.14.1-beta"
)
resolvers += "Sonatype OSS Releases" at "http://oss.sonatype.org/content/repositories/releases/"
seq(StartScriptPlugin.startScriptForClassesSettings: _*)
I've tried mucking around with different versions of scala and the dependencies, but to no avail. What is going on here? Do I need to fix it? If so, how?
This is controlled by the checkExplicit value of the ivyScala setting. You can disable it with:
ivyScala ~= { (is: Option[IvyScala]) =>
for(i <- is) yield
i.copy(checkExplicit = false)
}
I'm surprised that is an error, since I'd normally expect it to be a warning. Also, this check is no longer on by default in 0.13.