I am trying to use UR with predictionio-0.11.0-incubating, with following configs:
elasticsearch.version=1.7.6
scala.version=2.11.8
It is giving an error message [ERROR] [Engine$] No engine found. Your build might have failed. Aborting.
It is not listing any detailed exception. Please find below the logs:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/neha/incubator-predictionio-0.11.0-incubating/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/neha/incubator-predictionio-0.11.0-incubating/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Engine$] Using command '/home/neha/incubator-predictionio-0.11.0-incubating/PredictionIO-0.11.0-incubating/sbt/sbt' at /home/neha/projects/universal-recommender to build.
[INFO] [Engine$] If the path above is incorrect, this process will fail.
[INFO] [Engine$] Uber JAR disabled. Making sure lib/pio-assembly-0.11.0-incubating.jar is absent.
[INFO] [Engine$] Going to run: /home/neha/incubator-predictionio-0.11.0-incubating/PredictionIO-0.11.0-incubating/sbt/sbt package assemblyPackageDependency in /home/neha/projects/universal-recommender
[INFO] [Engine$] Compilation finished successfully.
[INFO] [Engine$] Looking for an engine...
[ERROR] [Engine$] No engine found. Your build might have failed. Aborting.
Initial question posted at https://github.com/PredictionIO/template-scala-parallel-universal-recommendation/issues/63.
You must make sure the below mentioned points :
You are running pio build command in the same directory where your engine.json is placed
Mention scala version scalaVersion := "2.11.8" in your build.sbt
Related
I try to use the phoenix-spark jar to laod phoenix table to Spark 2.2.3 DataFrame
adding this dependency:
libraryDependencies += "org.apache.phoenix" % "phoenix-spark2" % "4.7.0.2.6.5.1102-5"
I tested this two resolvers one by one:
resolvers += "Hortonworks Repository" at "http://repo.hortonworks.com/content/repositories/releases/"
resolvers += "Hortonworks Releases" at "http://repo.hortonworks.com/content/groups/public/"
I had the folowing error:
[info] welcome to sbt 1.3.13 (Oracle Corporation Java 1.8.0_261)
[info] loading project definition from /home/ambac61n/IdeaProjects/phoenix_test/project
[info] loading settings for project phoenix_test from build.sbt ...
[info] set current project to phoenix_test (in build file:/home/my_user/IdeaProjects/phoenix_test/)
[info] sbt server started at local:///home/ambac61n/.sbt/1.0/server/0c2856c06fe3f2cf2706/sock
sbt:phoenix_test>
[info] Defining Global / sbtStructureOptions, Global / sbtStructureOutputFile and 1 others.
[info] The new values will be used by cleanKeepGlobs
[info] Run `last` for details.
[info] Reapplying settings...
[info] set current project to phoenix_test (in build file:/home/ambac61n/IdeaProjects/phoenix_test/)
[info] Applying State transformations org.jetbrains.sbt.CreateTasks from /home/my_user/.local/share/JetBrains/IdeaIC2020.2/Scala/repo/org.jetbrains/sbt-structure-extractor/scala_2.12/sbt_1.0/2018.2.1+4-88400d3f/jars/sbt-structure-extractor.jar
[info] Reapplying settings...
[info] set current project to phoenix_test (in build file:/home/my_user/IdeaProjects/phoenix_test/)
[warn]
[warn] Note: Unresolved dependencies path:
[error] stack trace is suppressed; run 'last update' for the full output
[error] stack trace is suppressed; run 'last ssExtractDependencies' for the full output
[error] (update) sbt.librarymanagement.ResolveException: Error downloading org.apache.phoenix:phoenix-spark2:4.7.0.2.6.5.1102-5
[error] Not found
[error] Not found
[error] not found: /home/ambac61n/.ivy2/local/org.apache.phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] not found: http://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.phoenix:phoenix-spark2:4.7.0.2.6.5.1102-5
[error] Not found
[error] Not found
[error] not found: /home/ambac61n/.ivy2/local/org.apache.phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] not found: http://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/4.7.0.2.6.5.1102-5/phoenix-spark2-4.7.0.2.6.5.1102-5.pom
[error] Total time: 3 s, completed 22 août 2020 05:56:14
[info] shutting down sbt server
Do you have any Idea?
After vising those repositories, I noticed that there is no package, indeed.
For the first repository,
https://repo1.maven.org/maven2/org/apache/phoenix/
No package for the phoenix-spark2
and the second repository,
https://repo.hortonworks.com/content/repositories/releases/org/apache/phoenix/phoenix-spark2/
No package for the 4.7.0.2.6.5.1102-5 version
Try with the other versions.
Facing issue while adding dependency from local file system.
Am Using systemPath tag to locate dependency.
But jar files is there in the location mentioned.
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.citi.166944:retail-basel-services:jar:1.0.0-SNAPSHOT
[WARNING] 'dependencies.dependency.systemPath' for com.citi.163431:ets-ra-tools-aggregate-api:jar should not point at files within the project directory, ${basedir}/external_lib/ets-ra-tools-aggregate-api-1.0.3-JAVA7.jar will be unresolvable by dependent projects # com.citi.166944:retail-basel-services:[unknown-version], C:\Users\jj48521\eclipse-workspace\retail-basel-services-trunk\pom.xml, line 38, column 16
[WARNING] 'dependencies.dependency.systemPath' for com.citi.163431:jrace-util:jar should not point at files within the project directory, ${basedir}/external_lib/jrace-util-1.17-basic-alpha-2.jar will be unresolvable by dependent projects # com.citi.166944:retail-basel-services:[unknown-version], C:\Users\jj48521\eclipse-workspace\retail-basel-services-trunk\pom.xml, line 232, column 16
[WARNING] 'dependencies.dependency.systemPath' for com.citi.163431:jrace-params:jar should not point at files within the project directory, ${basedir}/external_lib/jrace-params-1.17-basic-alpha-2.jar will be unresolvable by dependent projects # com.citi.166944:retail-basel-services:[unknown-version], C:\Users\jj48521\eclipse-workspace\retail-basel-services-trunk\pom.xml, line 239, column 16
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin org.apache.maven.plugins:maven-compiler-plugin # com.citi.166944:retail-basel-services:[unknown-version], C:\Users\jj\eclipse-workspace\retail-basel-services-trunk\pom.xml, line 266, column 12
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Retail Basel Services 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.apache.maven.plugins:maven-jar-plugin:jar:2.3.1 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.456 s
[INFO] Finished at: 2018-10-08T12:00:28+05:30
[INFO] Final Memory: 8M/309M
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-jar-plugin:2.3.1 or one of its dependencies could not be resolved: Cannot access citicore-plugins (http://sdfg.net:8081/nexus/content/repositories/citicore) in offline mode and the artifact org.apache.maven.plugins:maven-jar-plugin:jar:2.3.1 has not been downloaded from it before. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
I have pulled "zephrax/docker-prediction.io-universal-recommender" docker image and follow the instruction in https://github.com/PredictionIO/template-scala-parallel-universal-recommendation, when I do pio build it stops in this line and not going on :
root#0016c0672bc4:/universal-recommender-0.6.0# pio build --verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/PredictionIO-0.11.0-
incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-
incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/PredictionIO-0.11.0-
incubating/lib/pio-assembly-0.11.0-
incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Engine$] Using command '/PredictionIO-0.11.0-
incubating/sbt/sbt' at /universal-recommender-0.6.0 to build.
[INFO] [Engine$] If the path above is incorrect, this process will
fail.
[INFO] [Engine$] Uber JAR disabled, but current working directory does
not look like an engine project directory. Please delete lib/pio-
assembly-0.11.0-incubating.jar manually.
[INFO] [Engine$] Going to run: /PredictionIO-0.11.0-incubating/sbt/sbt
package assemblyPackageDependency in /universal-recommender-0.6.0
[INFO] [Engine$] [info] Loading project definition from /universal-
recommender-0.6.0/project
[INFO] [Engine$] [info] Updating {file:/universal-recommender-
0.6.0/project/}universal-recommender-0-6-0-build...
[INFO] [Engine$] [info] Resolving org.scalariform#sbt-
scalariform;1.6.0 ...
[info] Resolving org.scalariform#sbt-scalariform;1.6.0
...
what is the problem ? how can I find the log in this docker image?
thank you
When I try to build one of the predictionIO engines (https://github.com/peoplehum/BagOfWords_SentimentAnalysis_Template), I get error [INFO] [Engine$] Looking for an engine...
[ERROR] [Engine$] No engine found. Your build might have failed. Aborting.
I see that the build process created target/scala-2.10/santimentanalysis_bagofwords_model_2.10-1.0.jar
and target/scala-2.10/SantimentAnalysis_BagOfWords_Model-assembly-1.0-deps.jar but should have created them in target/scala-2.11.
Why is the engine building for scala-2.10 instead of scala-2.11?
I built PredictionIO with the following to handle my environment:
./make-distribution.sh -Dscala.version=2.11.6 -Dspark.version=2.1.0 -Delasticsearch.version=2.3.3
ubuntu1604vm:~/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template$ pio build --verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/b/mnt/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/b/mnt/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Engine$] Using command '/home/b/mnt/PredictionIO-0.11.0-incubating/sbt/sbt' at /home/b/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template to build.
[INFO] [Engine$] If the path above is incorrect, this process will fail.
[INFO] [Engine$] Uber JAR disabled. Making sure lib/pio-assembly-0.11.0-incubating.jar is absent.
[INFO] [Engine$] Going to run: /home/b/mnt/PredictionIO-0.11.0-incubating/sbt/sbt package assemblyPackageDependency in /home/b/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template
[INFO] [Engine$] [info] Loading project definition from /home/b/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template/project
[INFO] [Engine$] [info] Set current project to SantimentAnalysis_BagOfWords_Model (in build file:/home/b/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template/)
[INFO] [Engine$] [success] Total time: 0 s, completed Apr 30, 2017 7:53:12 PM
[INFO] [Engine$] [info] Including from cache: scala-library-2.10.5.jar
[INFO] [Engine$] [info] Checking every *.class/*.jar file's SHA-1.
[INFO] [Engine$] [info] Merging files...
[INFO] [Engine$] [warn] Merging 'META-INF/MANIFEST.MF' with strategy 'discard'
[INFO] [Engine$] [warn] Strategy 'discard' was applied to a file
[INFO] [Engine$] [info] Assembly up to date: /home/b/mnt/PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template/target/scala-2.10/SantimentAnalysis_BagOfWords_Model-assembly-1.0-deps.jar
[INFO] [Engine$] [success] Total time: 1 s, completed Apr 30, 2017 7:53:13 PM
[INFO] [Engine$] Compilation finished successfully.
[INFO] [Engine$] Looking for an engine...
[ERROR] [Engine$] No engine found. Your build might have failed. Aborting.
results of pio status
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/mnt/PredictionIO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-0.11.0-incubating.jar!/org/slf4j/i mpl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/mnt/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar!/org/slf4j/impl/StaticLogger Binder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
[INFO] [Management$] Inspecting PredictionIO...
[INFO] [Management$] PredictionIO 0.11.0-incubating is installed at /home/mnt/PredictionIO-0.11.0-incubating
[INFO] [Management$] Inspecting Apache Spark...
[INFO] [Management$] Apache Spark is installed at /home/mnt/PredictionIO-0.11.0-incubating/vendors/spark-2.1.0-bin-hadoop2.7
[INFO] [Management$] Apache Spark 2.1.0 detected (meets minimum requirement of 1.3.0)
[INFO] [Management$] Inspecting storage backend connections...
[INFO] [Storage$] Verifying Meta Data Backend (Source: PGSQL)...
[INFO] [Storage$] Verifying Model Data Backend (Source: PGSQL)...
[INFO] [Storage$] Verifying Event Data Backend (Source: PGSQL)...
[INFO] [Storage$] Test writing to Event Store (App Id 0)...
[INFO] [Management$] Your system is all ready to go.
env
PIO_SCALA_VERSION=2.11.6
PIO_FS_BASEDIR=/home/tmp/.pio_store
PIO_ELASTICSEARCH_VERSION=2.3.3
PIO_HADOOP_VERSION=2.7
PIO_SPARK_VERSION=2.1.0
PIO_HOME=/home/mnt/PredictionIO-0.11.0-incubating
To set a predictionIO engine template's Scala version for engine pio build, edit the engine's build.sbt
For the engine I was working with, I added
scalaVersion := "2.11.6" to build.sbt
to match my configuration.. PredictionIO-0.11.0-incubating scala.version=2.11.6 spark.version=2.1.0)
PredictionIO-0.11.0-incubating/vendors/BagOfWords_SentimentAnalysis_Template/build.sbt:
assemblySettings
name := "SantimentAnalysis_BagOfWords_Model"
version := "1.0"
scalaVersion := "2.11.6"
libraryDependencies ++= Seq(
"org.apache.predictionio" %% "apache-predictionio-core" % "0.10.0-incubating" % "provided",
"org.apache.spark" %% "spark-core" % "1.5.1" % "provided",
"org.apache.spark" %% "spark-mllib" % "1.5.1" % "provided"
)
I am trying to add log4j as a Maven Dependency. So I added it to the pom.xml
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.15</version>
</dependency>
Now I have a warning message that I am missing 6 artifacts (jars that log4j depends on). As described on the following site Maven Repository - log4j.
My goal is to get Maven to install the 6 dependencies for me. I have Eclipse Kepler with embedded Maven, so I cannot run "mvn install". I see from this stackoverflow answer Run Maven Install in Eclipse that I can right click on the project and Run As > Maven Install. But I get the following error:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building ############# 0.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.522s
[INFO] Finished at: Wed Nov 27 12:13:11 CST 2013
[INFO] Final Memory: 6M/122M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project #############: Could not resolve dependencies for project #############:#############:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: javax.jms:jms:jar:1.1, com.sun.jdmk:jmxtools:jar:1.2.1, com.sun.jmx:jmxri:jar:1.2.1: Could not transfer artifact javax.jms:jms:jar:1.1 from/to java.net (https://maven-repository.dev.java.net/nonav/repository): No connector available to access repository java.net (https://maven-repository.dev.java.net/nonav/repository) of type legacy using the available factories AsyncRepositoryConnectorFactory, WagonRepositoryConnectorFactory -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
Please let me know if I am on the right path and how best to proceed to get the log4j dependencies. Thank you in advance for any help.
UPDATE
After checking Preferences > Maven > User Settings, I discovered that the settings.xml file that is being referenced does not exist. I clicked "Update Settings" and I get the following error:
ArtifactTransferException: Could not transfer artifact com.sun.jdmk:jmxtools:jar:1.2.1 from/to java.net (https://maven-repository.dev.java.net/nonav/repository): No connector available to access repository java.net (https://maven-repository.dev.java.net/nonav/repository) of type legacy using the available factories AsyncRepositoryConnectorFactory, WagonRepositoryConnectorFactory
UPDATE 2
It looks like my issue is a result of Maven 3 (which comes with Kepler) is trying to access a Maven 2 repository and having compatibility issues. I have yet to figure out how to redirect to an appropriate repository.
I changed the version from 1.2.15 to 1.2.17 and it worked (1.2.16 works as well). I do not see any of the dependencies listed "here" listed in the Resolved Dependencies frame like I did with 1.2.15. I do not have a full understanding of why this worked, but it worked for me.
You can right-click on your project then Maven > Update Snapshots. (Make sure u check the "Force update on snapshots" checkbox. This should work.