I'm parsing a XML string to convert it to a JsonNode in Scala using a XmlMapper from the Jackson library. I code on a Databricks notebook, so compilation is done on a cloud cluster. When compiling my code I got this error java.lang.NoSuchMethodError: com.fasterxml.jackson.dataformat.xml.XmlMapper.coercionConfigDefaults()Lcom/fasterxml/jackson/databind/cfg/MutableCoercionConfig; with a hundred lines of "at com.databricks. ..."
I maybe forget to import something but for me this is ok (tell me if I'm wrong) :
import ch.qos.logback.classic._
import com.typesafe.scalalogging._
import com.fasterxml.jackson._
import com.fasterxml.jackson.core._
import com.fasterxml.jackson.databind.{ObjectMapper, JsonNode}
import com.fasterxml.jackson.dataformat.xml._
import com.fasterxml.jackson.module.scala._
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper
import java.io._
import java.time.Instant
import java.util.concurrent.TimeUnit
import javax.xml.parsers._
import okhttp3.{Headers, OkHttpClient, Request, Response, RequestBody, FormBody}
import okhttp3.OkHttpClient.Builder._
import org.apache.spark._
import org.xml.sax._
As I'm using Databricks, there's no SBT file for dependencies. Instead I installed the libs I need directly on the cluster. Here are the ones I'm using :
com.squareup.okhttp:okhttp:2.7.5
com.squareup.okhttp3:okhttp:4.9.0
com.squareup.okhttp3:okhttp:3.14.9
org.scala-lang.modules:scala-swing_3:3.0.0
ch.qos.logback:logback-classic:1.2.6
com.typesafe:scalalogging-slf4j_2.10:1.1.0
cc.spray.json:spray-json_2.9.1:1.0.1
com.fasterxml.jackson.module:jackson-module-scala_3:2.13.0
javax.xml.parsers:jaxp-api:1.4.5
org.xml.sax:2.0.1
The code causing the error is simply (coming from here : https://www.baeldung.com/jackson-convert-xml-json Chapter 5):
val xmlMapper: XmlMapper = new XmlMapper()
val jsonNode: JsonNode = xmlMapper.readTree(responseBody.getBytes())
with responseBody being a String containing a XML document (I previously checked the integrity of the XML). When removing those two lines the code is working fine.
I've read tons of articles or forums but I can't figure out what's causing my issue. Can someone please help me ? Thanks a lot ! :)
Welcome to dependency hell and breaking changes in libraries.
This usually happens, when various lib bring in different version of same lib. In this case it is Jackson.
java.lang.NoSuchMethodError: com.fasterxml.jackson.dataformat.xml.XmlMapper.coercionConfigDefaults()Lcom/fasterxml/jackson/databind/cfg/MutableCoercionConfig; means: One lib probably require Jackson version, which has this method, but on class path is version, which does not yet have this funcion or got removed bcs was deprecated or renamed.
In case like this is good to print dependency tree and check version of Jackson required in libs. And if possible use newer versions of requid libs.
Solution: use libs, which use compatible versions of Jackson lib. No other shortcut possible.
upgrading to 2.12.5 version fixed my issue.
this issue may also appear when there are multiple versions of jackson jars in project lib directory. you should remove the older versions.
Related
I have a lot of successful rest accesses in grails 2.x.x
I simply coded
import grails.plugins.rest.client.*
import grails.util.Holders
import org.codehaus.groovy.grails.web.json.JSONObject
import org.codehaus.groovy.grails.web.json.JSONArray
and Classes JSONObject, JSONArray, RestBuilder, RestResponse where available for further use.
What are the corresponding imports in 4.0.1 and what jars resp. what lines in build.gradle are necessary?
What are the corresponding imports in 4.0.1 and what jars resp. what
lines in build.gradle are necessary?
Grails 4 offers better options than to interact with the classes you asked about but to answer the question as asked...
org.grails.web.json.JSONObject is in grails-web-common-4.0.1.jar. Use import org.grails.web.json.JSONObject.
org.grails.web.json.JSONArray is in grails-web-common-4.0.1.jar. Use import org.grails.web.json.JSONArray.
grails.plugins.rest.client.RestBuilder is in grails-datastore-rest-client-6.1.12.RELEASE.jar. Use import grails.plugins.rest.client.RestBuilder.
grails.plugins.rest.client.RestResponse is in grails-datastore-rest-client-6.1.12.RELEASE.jar. Use import grails.plugins.rest.client.RestResponse.
Depending on what other dependencies you may have in your project those may or may not be pulled in transitively so you may not need to add them to your build.gradle directly. The most likely scenario is you won't need to add anything to pull in grails-web-common-4.0.1.jar but you probably will need to pull in grails-datastore-rest-client-6.1.12.RELEASE.jar which can be done by adding the following to your build.gradle:
compile "org.grails:grails-datastore-rest-client:6.1.12.RELEASE"
If you want to pull in grails-web-common explicitly you could use the following:
compile "org.grails:grails-web-common:4.0.1"
If you are using the BOM correctly, you could simplify that with the following:
compile "org.grails:grails-web-common"
I hope that helps.
I'm having trouble importing magellan-1.0.4-s_2.11 in spark notebook. I've downloaded the jar from https://spark-packages.org/package/harsha2010/magellan and have tried placing SPARK_HOME/bin/spark-shell --packages harsha2010:magellan:1.0.4-s_2.11 in the Start of Customized Settings section of the spark-notebook file of the bin folder.
Here are my imports
import magellan.{Point, Polygon, PolyLine}
import magellan.coord.NAD83
import org.apache.spark.sql.magellan.MagellanContext
import org.apache.spark.sql.magellan.dsl.expressions._
import org.apache.spark.sql.Row
import org.apache.spark.sql.types._
And my errors...
<console>:71: error: object Point is not a member of package org.apache.spark.sql.magellan
import magellan.{Point, Polygon, PolyLine}
^
<console>:72: error: object coord is not a member of package org.apache.spark.sql.magellan
import magellan.coord.NAD83
^
<console>:73: error: object MagellanContext is not a member of package org.apache.spark.sql.magellan
import org.apache.spark.sql.magellan.MagellanContext
I then tried to import the new library like any other library by placing it into the main script like so:
$lib_dir/magellan-1.0.4-s_2.11.jar"
This didn't work and I'm left scratching my head wondering what I've done wrong. How do I import libraries such as magellan into spark notebook?
Try evaluating something like
:dp "harsha2010" % "magellan" % "1.0.4-s_2.11"
It will load the library into Spark, allowing it to be imported - assuming it can be obtained though the Maven repo. In my case it failed with a message:
failed to load 'harsha2010:magellan:jar:1.0.4-s_2.11 (runtime)' from ["Maven2 local (file:/home/dev/.m2/repository/, releases+snapshots) without authentication", "maven-central (http://repo1.maven.org/maven2/, releases+snapshots) without authentication", "spark-packages (http://dl.bintray.com/spark-packages/maven/, releases+snapshots) without authentication", "oss-sonatype (https://oss.sonatype.org/content/repositories/releases/, releases+snapshots) without authentication"] into /tmp/spark-notebook/aether/b2c7d8c5-1f56-4460-ad39-24c4e93a9786
I think file was to big and connection was interrupted before whole file could be downloaded.
Workaround
So I downloaded the JAR manually from:
http://dl.bintray.com/spark-packages/maven/harsha2010/magellan/1.0.4-s_2.11/
and copied it into the:
/tmp/spark-notebook/aether/b2c7d8c5-1f56-4460-ad39-24c4e93a9786/harsha2010/magellan/1.0.4-s_2.11
And then :dp command worked. Try Calling it first, and if it will fail copy JAR into the right path to make things work.
Better solution
I should investigate why download failed to fix it in the first place... or put that library in my local M2 repo. But that should get you going.
I would suggest to check this:
https://github.com/spark-notebook/spark-notebook/blob/master/docs/metadata.md#import-download-dependencies
and
https://github.com/spark-notebook/spark-notebook/blob/master/docs/metadata.md#add-spark-packages
I think the :dp magic command is depreciated, instead you should add your custom dependencies in the notebook metadata. You can go in the menu Edit > Edit notebook metadata, there add something like:
"customDeps": [
"harsha2010 % magellan % 1.0.4-s_2.11"
]
Once done, you will need to restart the kernel, you can check in the browser console if the package is being downloaded properly.
The easy way, you should set or add the EXTRA_CLASSPATH environnent variable to point to your .jar file downloaded :
export EXTRA_CLASSPATH = </link/to/your.jar> or set EXTRA_CLASSPATH= </link/to/your.jar> in wondows OS. Here find the detailed solution.
In scala it is common practice to stack package statements to allow shorter imports, but when I load a file using stacked packages into the scala ide and I attempt to use an import starting with the same organization I get a compiler error from what appears to be the presentation compiler. The code compiles fine in sbt outside of the IDE.
An example code snippet is as follows:
package com.coltfred
package util
package time
import com.github.nscala_time.time.Imports._
On the import I get the error object github is not a member of package com.coltfred.util.com.
If I move the import to a single line the error will go away, but we've used this practice frequently in our code base so changing them all to be single line package statements would be a pain.
Why is this happening and is there anything I can do to fix it?
Edit:
I used the eclipse-sbt plugin to generate the eclipse project file for this. The directory structure is what it should be and all of the dependencies are in the classpath.
Edit 2:
It turns out there was a file in the test tree of the util package (which should have been in the same package), but had a duplicate package statement at the top. I didn't check the test tree because it shouldn't affect the compilation of the main tree, but apparently I was wrong.
Not sure why the Scala IDE is not liking this, but you can force the import to start at the top level using _root_:
import _root_.com.github.nscala_time.time.Imports._
See if that avoids irritating the IDE.
This is a common annoyance that annoyed paulp into an attempt to fix it. His idea was that a dir that doesn't contribute class files shouldn't be taken as a package. If you can take util as scala.util, you should do so in preference to foo.util where that util is empty.
The util dir is the usual suspect, because who doesn't have a util dir lying around, and in particular, ./util?
apm#mara:~/tmp/coltfred$ mkdir -p com/coltfred/util/time
apm#mara:~/tmp/coltfred$ mkdir -p com/coltfred/util/com
apm#mara:~/tmp/coltfred$ vi com/coltfred/util/time/test.scala
apm#mara:~/tmp/coltfred$ scalac com/coltfred/util/time/test.scala
./com/coltfred/util/time/test.scala:5: error: object github is not a member of package com.coltfred.util.com
import com.github.nscala_time.time._
^
one error found
apm#mara:~/tmp/coltfred$ cat com/coltfred/util/time/test.scala
package com.coltfred
package util
package time
import com.github.nscala_time.time._
class Test
apm#mara:~/tmp/coltfred$
To debug, find out where the offending package is getting loaded from.
Im using the following:
import java.util.Collections;
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import org.apache.commons.configuration.XMLConfiguration;
and i get:
The type org.apache.commons.lang.exception.NestableException cannot be resolved. It is indirectly referenced from required .class files
Im using eclipse...
how can i resolve this? he offers me to Configure build path but i dont really know how to solve this collision from there.....
Problem solved...
had to download the commons-lang-2.4.jar and include in project.
couldnt be more simple than that....
Sounds like what is really needed is an update to the PropertiesConfiguration lib so that it gets along with latest lang lib. If its a "free" lib then it might not be coming, considering that its been years since last reply on this thread and this is still happening.
I have been having this issue as well, and have not found a way of resolving it apart from the aforementioned inclusion of both lang libs ... which does not seem to present any problems, though strict repository framework implementations (like Maven) might have problems with both libs included.
Had to remove commons-lang3-3.4 from my Java Build Path and added 2.6 , it solved the problem!
I have a library which has root package "scala", and now I have a project using this library, and I have a sub package named "com.zjffdu.scala". And the class file in this package needs to import classes from the library. So I have the following import statement.
import scala._
But because this class is also in package "scala", the scala compiler will look for files in current directory rather than the library.
So how can I explicit to tell scala to import classes from the library.
Thanks
Use this:
import _root_.scala._
As you can see it's not very pretty — the best option is probably to avoid naming one of your packages scala.
And by the way — the root scala package is always preimported (though subpackages, of course, are not).