I am getting the below error:
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/streaming/flume/FlumeUtils$
at SampleStream$.main(NetworkWordCount.scala:10)
at SampleStream.main(NetworkWordCount.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.ClassNotFoundException:
org.apache.spark.streaming.flume.FlumeUtils$
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
i refer below link
https://spark.apache.org/docs/1.4.1/streaming-flume-integration.html
You need to package spark-streaming-flume and its dependencies (except spark-core and spark-streaming which are provided by spark-submit) into the application JAR.
https://spark.apache.org/docs/1.6.1/streaming-flume-integration.html has a better description about deploying. You can use this doc as there is no API changes between 1.4 and 1.6.
You need to have all libraries loaded in classpath.
Add this to build.sbt file: libraryDependencies += "org.apache.spark" % "spark-streaming-flume_2.10" % "1.6.1"
Related
I have an Apache Spark 2.0 application written in Scala (2.11.12), built using SBT tool 1.2.8. When I'm trying to run the app in Intellij (2020.3.2 Ultimate), I get the following error -
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/util/concurrent/internal/InternalFutureFailureAccess
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
.....
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.internal.InternalFutureFailureAccess
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
On googling/searching Stackoverflow, it seems this is caused by some weird Guava dependency issues. I have these added to my Dependencies.scala
dependencies += "com.google.guava" % "guava" % "30.1-jre"
dependencies += "com.google.guava" % "failureaccess" % "1.0"
That didn't solve the issue. Also tried adding "com.google.guava" % "listenablefuture" % "1.0" to the dependencies, but that didn't help either. Tried doing File -> Invalidate Caches/Restart in Intellij but I still get the issue.
Could someone please help?
In my case, adding com.google.guava:failureaccess to external libraries to my project (File -> Project Structure -> Libraries) helped.
I am writing a plugin for DITA-OT PDF2. In this plugin I need to define extension functions for Saxon XSLT. I followed the tutorials from the dita-ot website:
https://www.dita-ot.org/3.3/topics/implement-saxon-extension-functions.html
and
https://www.dita-ot.org/3.3/topics/plugin-javalib.html
Note: I am using DITA OT 3.3.1
I have this feature in my plugin, pointing to the jar that contains my extension functions:
<feature extension="dita.conductor.lib.import" file="./xslt-extension.jar"/>
I get this error:
transform.topic2fo.main:
[pipeline] Static error in {ftd:lastApplicability()} in expression in xsl:variable/#select on line 30 column 79 of applicability.xsl:
[pipeline] XPST0017: Cannot find a 0-argument function named {https://orlandotechpubs.com/}lastApplicability()
[pipeline] Static error near {...eApplicability($applicabili...} in expression in xsl:value-of/#select on line 99 column 69 of applicability.xsl:
[pipeline] XPST0017: Cannot find a 1-argument function named {https://orlandotechpubs.com/}writeApplicability()
BUILD FAILED
C:\Users\d_bo\Desktop\test-dita-ot-pdf\test\sample_pdf.xml:9: The following error occurred while executing this line:
C:\Users\d_bo\Desktop\test-dita-ot-pdf\dita-ot-3.3.1\plugins\org.dita.base\build.xml:31: The following error occurred while executing this line:
C:\Users\d_bo\Desktop\test-dita-ot-pdf\dita-ot-3.3.1\plugins\org.dita.pdf2\build.xml:151: The following error occurred while executing this line:
C:\Users\d_bo\Desktop\test-dita-ot-pdf\dita-ot-3.3.1\plugins\org.dita.pdf2\build.xml:291: java.lang.RuntimeException: Failed to compile stylesheet 'C:\Users\d_bo\Desktop\test-dita-ot-pdf\dita-ot-3.3.1\plugins\org.dita.pdf2.axf\xsl\fo\topic2fo_shell_axf.xsl': net.sf.saxon.s9api.SaxonApiException: Errors were reported during stylesheet compilation
at org.dita.dost.module.XsltModule.execute(XsltModule.java:108)
at org.dita.dost.ant.ExtensibleAntInvoker.execute(ExtensibleAntInvoker.java:182)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
at org.apache.tools.ant.Task.perform(Task.java:350)
at org.apache.tools.ant.Target.execute(Target.java:449)
at org.apache.tools.ant.Target.performTasks(Target.java:470)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
at org.apache.tools.ant.Project.executeTarget(Project.java:1361)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.Main.runBuild(Main.java:834)
at org.apache.tools.ant.Main.startAnt(Main.java:223)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)
Caused by: javax.xml.transform.TransformerConfigurationException: net.sf.saxon.s9api.SaxonApiException: Errors were reported during stylesheet compilation
at net.sf.saxon.jaxp.SaxonTransformerFactory.newTemplates(SaxonTransformerFactory.java:158)
at org.dita.dost.module.XsltModule.execute(XsltModule.java:106)
... 56 more
Caused by: net.sf.saxon.s9api.SaxonApiException: Errors were reported during stylesheet compilation
at net.sf.saxon.s9api.XsltCompiler.compile(XsltCompiler.java:742)
at net.sf.saxon.jaxp.SaxonTransformerFactory.newTemplates(SaxonTransformerFactory.java:155)
... 57 more
Caused by: net.sf.saxon.trans.XPathException: Errors were reported during stylesheet compilation
at net.sf.saxon.style.StylesheetModule.loadStylesheet(StylesheetModule.java:260)
at net.sf.saxon.style.Compilation.compileSingletonPackage(Compilation.java:106)
at net.sf.saxon.s9api.XsltCompiler.compile(XsltCompiler.java:739)
... 58 more
I double checked everything and my extension functions are correctly defined. So I did some tests and apparently my jar is never added to the classpath. I created a simple ant target and launched again:
<target name="testJava">
<java classname="com.infotel.ftd.MyTest">
</java>
</target>
Here is what I get:
[java] Could not find com.infotel.ftd.MyTest. Make sure you have it in your classpath
[java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:141)
[java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:861)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:231)
[java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:135)
[java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:108)
[java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
[java] at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[java] at java.lang.reflect.Method.invoke(Method.java:498)
[java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
[java] at org.apache.tools.ant.Task.perform(Task.java:350)
[java] at org.apache.tools.ant.Target.execute(Target.java:449)
[java] at org.apache.tools.ant.Target.performTasks(Target.java:470)
[java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
[java] at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
[java] at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
[java] at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
[java] at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
[java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
[java] at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[java] at java.lang.reflect.Method.invoke(Method.java:498)
[java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
[java] at org.apache.tools.ant.Task.perform(Task.java:350)
[java] at org.apache.tools.ant.Target.execute(Target.java:449)
[java] at org.apache.tools.ant.Target.performTasks(Target.java:470)
[java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
[java] at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36)
[java] at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
[java] at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:437)
[java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[java] at java.lang.reflect.Method.invoke(Method.java:498)
[java] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:99)
[java] at org.apache.tools.ant.Task.perform(Task.java:350)
[java] at org.apache.tools.ant.Target.execute(Target.java:449)
[java] at org.apache.tools.ant.Target.performTasks(Target.java:470)
[java] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1388)
[java] at org.apache.tools.ant.Project.executeTarget(Project.java:1361)
[java] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
[java] at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
[java] at org.apache.tools.ant.Main.runBuild(Main.java:834)
[java] at org.apache.tools.ant.Main.startAnt(Main.java:223)
[java] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:284)
[java] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:101)
[java] Java Result: -1
How can I debug dita OT to see what is wrong with my configuration?
I finally nailed what was wrong with my DITA OT configuration.
I was using an ant build script to run DITA OT and when doing this, the plugin JARs are not automatically added to the classpath.
I switched to using the DITA OT command line. Now my JARs are seen by the ant scripts and my extension functions are accessible from the XSLT.
In addition, if someone else has a classpath problem like this, you can also check the file [DITA-OT]/config/env.bat (env.sh for lunix users). After installing a plugin that provides its own JAR libraries, the JAR files shall be present in this file:
set "CLASSPATH=%CLASSPATH%;%DITA_HOME%\plugins\org.dita.pdf2\lib\fo.jar"
set "CLASSPATH=%CLASSPATH%;%DITA_HOME%\plugins\com.infotel.orlando.pdf\.\xslt-extension.jar"
set "CLASSPATH=%CLASSPATH%;%DITA_HOME%\plugins\org.dita.eclipsehelp\lib\eclipsehelp.jar"
...
I added a file called plugins.sbt to my /project folder. I added the following line to this file
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")
In my build.scala I added the following line
object CoreBuild extends Build {
import sbtassembly.AssemblyKeys._
I did a sbt assembly I got a large jar file which I scp'ed to my target server.
Target server has
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)
when I say java -jar ./foo.jar com.abhi.Foo 10
I get an error
Exception in thread "main" java.lang.NoClassDefFoundError: scala/Function0
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scala.Function0
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
But I added the assembly plugin....
I had a problem, when I upgrade Slick 1.0.0 to 2.0.0:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Play fork run has failed due to:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at play.runsupport.Reloader$.startDevMode(Reloader.scala:223)
at play.forkrun.ForkRun$.startServer(ForkRun.scala:82)
at play.forkrun.ForkRun.run(ForkRun.scala:177)
at play.forkrun.ForkRun$$anonfun$settingUp$1.applyOrElse(ForkRun.scala:166)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at play.forkrun.ForkRun.aroundReceive(ForkRun.scala:152)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:254)
at akka.dispatch.Mailbox.run(Mailbox.scala:221)
at akka.dispatch.Mailbox.exec(Mailbox.scala:231)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.pollAndExecAll(ForkJoinPool.java:1253)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1346)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.lang.NoSuchMethodError: play.api.Logger$.init(Ljava/io/File;Lscala/Enumeration$Value;)V
at play.core.server.DevServerStart$$anonfun$mainDev$1.apply(DevServerStart.scala:88)
at play.core.server.DevServerStart$$anonfun$mainDev$1.apply(DevServerStart.scala:61)
at play.utils.Threads$.withContextClassLoader(Threads.scala:21)
at play.core.server.DevServerStart$.mainDev(DevServerStart.scala:60)
at play.core.server.DevServerStart$.mainDevHttpMode(DevServerStart.scala:50)
at play.core.server.DevServerStart.mainDevHttpMode(DevServerStart.scala)
... 20 more
What should I do?
Have you upgraded Play Framework to 2.5.x?
Update the Play version number in project/plugins.sbt to upgrade Play:
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.5.x")
See https://www.playframework.com/documentation/2.5.x/Migration25
I have created one small program and in order to test it I have write small Scala Test class. But when I tried to execute scala test I was getting below error, please advise,
java.lang.NoSuchMethodError: scala.collection.immutable.$colon$colon.hd$1()Ljava/lang/Object;
at org.scalatest.tools.Runner$.argTooShort$1(Runner.scala:1490)
at org.scalatest.tools.Runner$.parseReporterArgsIntoConfigurations(Runner.scala:1507)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:898)
at org.scalatest.tools.Runner$.run(Runner.scala:858)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:137)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
I got the resolution. Thanks all for you reply.
There was a problem with my Scatatest version.
I am using Scala version 11 and scalatest version is not compatible with with Scala version.
libraryDependencies += "org.scalatest" % "scalatest_2.11" % "2.2.1" % "test"
Above line added in .sbt file and refreshed. Now it works fine as expected.