I have a data stack built around PredictionIO 0.14. I use ES 6.4.2, spark2.4.3, and scala 2.11.12(because spark 2.4.3 is back to scala 2.11 as default). I manage to do pio build, pio train, and pio deploy. I'm able to see the engine info if I open the page where the engine is deployed as web service, however I'm not able to query it with curl. I try to do:
curl -k -XPOST http://host:port/queries.json
(using the correct host and port)
but I get the following error:
Query:
Stack Trace:
java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/JsonInput;Z)Lorg/json4s/JsonAST$JValue;
at com.actionml.URAlgorithm.buildQueryShould(URAlgorithm.scala:668)
at com.actionml.URAlgorithm.buildQuery(URAlgorithm.scala:585)
at com.actionml.URAlgorithm.predict(URAlgorithm.scala:488)
at com.actionml.URAlgorithm.predict(URAlgorithm.scala:180)
I looked for info and I believe it's due to scala2.11 not having the proper libraries. I would upgrade to scala2.12 but that would again have other serious conflicts with ElasticSearch 6+.
Somehow a solution would be to upload the proper json4s libraries, but I don't know how to do that and where I should put my hands on. It.s strange that the curl command itself has this issue, but it looks like it's related to scala. Do you guys have any suggestion?
Downgrading ES from version 6 to version 5.6 worked for me. I believe ES6 still has incompatibilities with certain scala libraries.
Related
Spark json4s[java.lang.NoSuchMethodError:
org.json4s.jackson.JsonMethods$.parse(Lorg/json4s/Js]
Getting above error while parsing complex json when running spark scala structured streaming application on aws emr.
It looks like a binary compatibility error... Could you please check the dependency tree for incompatible versions of json4s artifacts?
If you will not able to upgrade them to use the same version then possible you can solve the problem by shading some of them with sbt-assembly plugin using rules like these.
In any case I would recommend to use more safe and efficient parser like jsoniter-scala (or dijon that is based on it).
Summary
gremlin server not getting started with orientDB
description
I am trying to connected gremlin server with orient db but not getting success. while i am trying to start the server it shows Graph [graph] was successfully configured via [conf/orientdb-empty.properties]. after that it gives some groovy.lang.GroovyRuntimeException: error.Below pictures will give you more clarity
https://groups.google.com/forum/#!topic/gremlin-users/YOmjLbl9ObY
followed this post
I'm not sure if those OrientDB "WARNING" messages regarding memory are significant or not. You may want to deal with those separately as OrientDB and Gremlin Server both need JVM memory allocation and you might not want OrientDB taking the full heap.
The error you asked about however is the issue with groovy exception. It looks like you have a simple jar conflict on your path. That can happen sometimes especially if you use a version of a TinkerPop-enabled graph whose TinkerPop version does not match version of Gremlin Server.
My guess is that OrientDB is bringing in an older version of groovy, the 2.4.7 and Gremlin Server already has 2.4.11. You can resolve that pretty easily by examining the ext/orientdb directory in Gremlin Server (assuming that's where the OrientDB plugin installed itself - it should be obvious when you look in the ext directory itself). Remove the groovy 2.4.7 jars from both the ext/orientdb/liband ext/orientdb/plugin directories. Do the same for other conflicts that you come across and things should work. If not, I think you'll have to ask a new more specific question about that particular issue.
Production system : HDP-2.5.0.0 using Ambari 2.4.0.1
Aplenty demands coming in for executing a range of code(Java MR etc., Scala, Spark, R) atop the HDP but from a desktop Windows machine IDE.
For Spark and R, we have R-Studio set-up.
The challenge lies with Java, Scala and so on, also, people use a range of IDEs from Eclipse to IntelliJ Idea.
I am aware that the Eclipse Hadoop plugin is NOT actively maintained and also has aplenty bugs when working with latest versions of Hadoop, IntelliJ Idea I couldn't find reliable inputs from the official website.
I believe the Hive and HBase client API is a reliable way to connect from Eclipse etc. but I am skeptical about executing MR or other custom Java/Scala code.
I referred several threads like this and this, however, I still have the question that is any IDE like Eclipse/Intellij Idea having an official support for Hadoop ? Even the Spring Data for Hadoop seems to lost traction, it anyways didn't work as expected 2 years ago ;)
As a realistic alternative, which tool/plugin/library should be used to test the MR and other Java/Scala code 'locally' i.e on the desktop machine using a standalone version of the cluster ?
Note : I do not wish to work against/in the sandbox, its about connecting to the prod. cluster directly.
I don't think that there is a genereal solution which would work for all Hadoop services equally. Each solution has it's own development, testing and deployment scenarios as they are different standalone products. For MR case you can use MRUnit to simulate your work locally from IDE. Another option is LocalJobRunner. They both allow you to check your MR logic directly from IDE. For Storm you can use backtype.storm.Testing library to simalate topology's workflow. But they all are used from IDE without direct cluster communications like in case wuth Spark and RStudio integration.
As for the MR recommendation your job should ideally pass the following lifecycle - writing the job and testing it locally, using MRUnit, then you should run it on some development cluster with some test data (see MiniCluster as an option) and then running in on real cluster with some custom counters which would help you to locate your malformed data and to properly maintaine the job.
I have a REST API that was working with Jersey 2.6 and Swagger 1.3.7. I read that Jersey 2.9 fixes a warning that I was getting so I upgraded to the latest Jersey 2.16 but then Swagger stopped working. I went back and upgraded one version at a time until I saw that Swagger was working with 2.15 so I settled on that. Now, the PUT API fails with Swagger with the following error:
The server refused this request because the request entity is in a format not supported by the requested resource for the requested method
The API works using FireFox RESTClient and specifying "application/json".
I do have "jersey-media-json-jackson" as a dependency and call "Client client = ClientBuilder.newClient().register(JacksonFeature.class);" in the program.
I tried upgrading Swagger but that did not help.
Has Swagger been verified to work with Jersey 2.15/2.16?
I've recently managed to get swagger-core to work with Jersey 2.16 with a similar issue. Keep in mind they are using the latest version (1.5.X) and not 1.3.X but the same solution will apply.
The problem is most likely with version resolution, specifically, the one of jackson-databind. For some reason, even jersey-media-json-jackson 2.16 depends on an older version of jackson-databind, even though it works fine with the latest version. Without having more details, it would be difficult to suggest a full solution, but you can follow the dependency tree and see the conflicts there.
If you do require further assistance, I'd suggest either using our mailing list, or even better, the IRC channel where we could interact online and resolve it.
I'm currently working on a project based on Spring 3.1.2 which needs to run over GAE.
After researching the logs, I think I know why I'm getting a 500 Internal Server Error whenever I need to interact with the App. The problem I'm facing is related with ASM (http://asm.ow2.org/).
It seems like GAE relies on ASM 4.0, while the project runs fine locally if I use ASM 3.1.
However, I found impossible to make it work when ASM 4.0 :(
I'm completely new to Google App Engine, so ... any tip to solve this issue is welcome! (I've already lost the whole morning finding what the problem was and trying too many different ways to solve it ... but no way).
Thank you very much in advance for your suggestions ;)
I found some information about that, try to upgrade your spring version to 3.2 M2. This version support cgilib 3.0 with asm 4.0!
Visit https://jira.springsource.org/browse/SPR-9669