I have a scala project which I've been running fine inside IntelliJ with a Run/Debug configuration set to Application, and now I'd like to create a jar to run it using 'java -jar .jar'. This is a maven project, not SBT.
I have three modules, and the top module which includes all other modules has an reference.conf in the resources folder. It's a minimal application.conf, not a lot of settings. Another one of the modules has its own reference.conf.
I created an artifact based on my top module, and I build it to create a jar. When I execute it with java
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.se.pd.Main.main(Main.scala)
Caused by: com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'akka.version'
at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:124)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:145)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:151)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:159)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:164)
at com.typesafe.config.impl.SimpleConfig.getString(SimpleConfig.java:206)
at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:168)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:504)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:141)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:118)
at com.se.pd.ServiceMain$class.$init$(ServiceMain.scala:25)
at com.se.pd.Main$.<init>(Main.scala:17)
at com.se.pd.Main$.<clinit>(Main.scala)
When I unzip the file, I see the merged reference.conf in the root dir.
What am I missing? Do I need to do something specific when I create my ActorSystem? I tried deleting all of my config files and running with all defaults, and it still didn't work, although in that case the error changed from complaining about "akka.version" to complaining about "akka". So it feels liek somehow I'm just not loading the akka defaults?
I fixed it for my purposes, but it doesn't really clarify to me what the issue is.
My fix (IntelliJ IDEA specific) was to change the artifact from:
"extract to target JAR"
to
"copy to output directory and link via manifest"
I suppose something gets lost when I create a big fat jar.
--
I just wanted to add a better solution to create a "fat jar" with maven. But I can't take credit, since it looks like the Akka page linked below by Zoltan now suggests the same solution, and in all honesty I cannot remember where I found this. I may have run into the updated page at some point in the past. Please consult that link for much better information.
So:
Use the maven shader plugin to create the fat jar
Then make sure you merge the configuration files:
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>myMainClass</mainClass>
</transformer>
Using that configuration, I can successfully build fat jars.
Related
I have Eclipse Maven GWT project and I added flowable-form-engine dependency that uses Liquibase. Liquibase is searching for changelog file org/flowable/form/db/liquibase/flowable-form-db-changelog.xml but finds two!
org.flowable.common.engine.api.FlowableException: Error initialising form data schema
at org.flowable.form.engine.impl.db.FormDbSchemaManager.initSchema(FormDbSchemaManager.java:58)
at org.flowable.form.engine.impl.cmd.SchemaOperationsFormEngineBuild.execute(SchemaOperationsFormEngineBuild.java:29)
at org.flowable.form.engine.impl.cmd.SchemaOperationsFormEngineBuild.execute(SchemaOperationsFormEngineBuild.java:24)
at org.flowable.common.engine.impl.interceptor.DefaultCommandInvoker.execute(DefaultCommandInvoker.java:10)
at org.flowable.common.engine.impl.interceptor.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53)
at org.flowable.common.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:71)
at org.flowable.common.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:30)
at org.flowable.common.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at org.flowable.form.engine.impl.FormEngineImpl.<init>(FormEngineImpl.java:45)
at org.flowable.form.engine.FormEngineConfiguration.buildFormEngine(FormEngineConfiguration.java:172)
at org.flowable.form.engine.configurator.FormEngineConfigurator.initFormEngine(FormEngineConfigurator.java:83)
at org.flowable.form.engine.configurator.FormEngineConfigurator.configure(FormEngineConfigurator.java:63)
at org.flowable.common.engine.impl.AbstractEngineConfiguration.configuratorsAfterInit(AbstractEngineConfiguration.java:859)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.init(ProcessEngineConfigurationImpl.java:985)
at org.flowable.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:887)
at sk.dominanz.coarui.server.services.WorkflowServiceAdditional.<clinit>(WorkflowServiceAdditional.java:64)
... 33 more
Caused by: liquibase.exception.ChangeLogParseException: Error Reading Migration File: Found 2 files that match org/flowable/form/db/liquibase/flowable-form-db-changelog.xml
at liquibase.parser.core.xml.XMLChangeLogSAXParser.parseToNode(XMLChangeLogSAXParser.java:118)
at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:15)
at liquibase.Liquibase.getDatabaseChangeLog(Liquibase.java:217)
at liquibase.Liquibase.update(Liquibase.java:190)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.Liquibase.update(Liquibase.java:175)
at liquibase.Liquibase.update(Liquibase.java:168)
at org.flowable.form.engine.impl.db.FormDbSchemaManager.initSchema(FormDbSchemaManager.java:52)
... 48 more
Caused by: java.io.IOException: Found 2 files that match org/flowable/form/db/liquibase/flowable-form-db-changelog.xml
at liquibase.util.StreamUtil.singleInputStream(StreamUtil.java:206)
at liquibase.parser.core.xml.XMLChangeLogSAXParser.parseToNode(XMLChangeLogSAXParser.java:71)
... 55 more
So I debugged it and it finds one resource from target directory:
jar:file:/C:/work/git/coarui/target/Main-1.0-SNAPSHOT/WEB-INF/lib/flowable-form-engine-6.4.0.jar!/org/flowable/form/db/liquibase/flowable-form-db-changelog.xml
and other resource from maven repository:
jar:file:/C:/Users/Piro/.m2/repository/org/flowable/flowable-form-engine/6.4.0/flowable-form-engine-6.4.0.jar!/org/flowable/form/db/liquibase/flowable-form-db-changelog.xml
Is there a way to ignore one of them, or is my build path or dependency definition wrong?
My build path contains:
src/main/java sources
src/test/java sources
JRE System library
Maven dependencies (in C:/Users/Piro/.m2/repository...)
JUnit4
Looking at source code resources are read classLoader.getResources(path); where class loader is jetty class loader JettyLauncher$WebAppContextWithReload$WebAppClassLoaderExtension.
In my run/debug configuration I have classpath similar to build path plus GWT library gwt-dev-2.8.1.jar - C:\Users\Piro\.m2\repository\com\google\gwt\gwt-dev\2.8.1
In pom.xml dependency is defined as:
<dependency>
<groupId>org.flowable</groupId>
<artifactId>flowable-form-engine-configurator</artifactId>
<version>${flowableVersion}</version>
</dependency>
Maven GWT plugin groupId=net.ltgt.gwt.maven, artifactId=gwt-maven-plugin has configuration parameter classpathScope but I tested <classpathScope>compile+runtime</classpathScope>, <classpathScope>compile</classpathScope> and <classpathScope>runtime</classpathScope> and the same error occurs.
Searching the internet I found similar posts on flowable forum (1, 2) but no solution is provided.
This generally happens when you mix client and server code in the same Maven module, and given how GWT works there's no (easy) around that, besides splitting your code in separate client and server (and possibly shared) Maven modules.
You can have a look at https://github.com/tbroyer/gwt-maven-archetypes for examples (disclaimer: I'm the author, but also a member of GWT's Steering Committee)
it seems that for some reason your .m2 is in the classpath. If it really makes sense to you then you can try to use <scope>provided</scope> in a pom where you declare dependency to flowable-form-engine-configurator. However you may decide to exclude .m2 from classpath which is better I think.
I've managed to get JUnit 4.12 + Hamcrest 1.3 + Mockito 2.8.47 to work in Eclipse so that when I add them as dependencies, my tests will run.
(The way I've done this is using the p2-maven-plugin to bundle the following
artifacts from Maven Central into plugins/a feature and provide them via P2:
junit 4.12
org.mockito.mockito-core 2.8.47
org.hamcrest.all 1.3.0
Adding the plugins to my test fragment as dependencies makes the tests
run in Eclipse.
However, the Tycho build of the same fragment will fail with the
following messages:
java.lang.LinkageError: loader constraint violation: loader (instance of org/eclipse/osgi/internal/loader/EquinoxClassLoader) previously initiated loading for a different type with name "org/hamcrest/Matcher"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at org.eclipse.osgi.internal.loader.ModuleClassLoader.defineClass(ModuleClassLoader.java:273)
at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.defineClass(ClasspathManager.java:632)
at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findClassImpl(ClasspathManager.java:586)
at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClassImpl(ClasspathManager.java:538)
at org.eclipse.osgi.internal.loader.classpath.ClasspathManager.findLocalClass(ClasspathManager.java:525)
at org.eclipse.osgi.internal.loader.ModuleClassLoader.findLocalClass(ModuleClassLoader.java:325)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:345)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:423)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:372)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:364)
at org.eclipse.osgi.internal.loader.ModuleClassLoader.loadClass(ModuleClassLoader.java:161)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:12)
at org.junit.Assert.assertThat(Assert.java:956)
at org.junit.Assert.assertThat(Assert.java:923)
So it seems that some other plugin is loading the package
org.hamcrest.Matcher before my fragment does. This is probably due
to the import/export/partial import/partial export chaos surrounding the
JUnit/Hamcrest/Mockito setup.
Does anyone have an idea -- or even better: a working example -- of how to
get the three components work together both within the IDE (for quick
checks on whether tests run) and Tycho (for checks during the build)?
Seems like that the loader want the dependencies in a bundle.
But I guess you haven't put your test lib in a bundle.
You could try to add them in the dependencies of your product to see how it reacts.
Background
The root of the problem is, that org.junit already has a dependency to org.hamcrest.core. So when your test-plugins has a dependency to org.hamcrest.all (which contains everything of hamcrest-core and all other hamcrest artifacts), all classes specified in hamcrest-core exist twice. Once in the bundle of hamcrest-core and once in hamcrest-all, which is why you get the linkage error.
If you open the Manifest of org.junit in the Manifest-Editor of Eclipse and go to the 'Dependencies' tab it should show you org.hamcreast.core in the "Required Plug-ins" section and org.hamcreast.core should be re-exported. Or in the raw-manifest it should look like this:
Require-Bundle: org.hamcrest.core;bundle-version="1.3.0";visibility:=reexport
Solution 1 - add hamcrest sub-modules
Instead of adding the all hamcrest-modul containing hamcrest.all as dependency to my Eclipse test-bundle/project (via 'Require-Bundle'), I add the hamcrest sub-modules that I require, except for hamcrest-core (because it is already re-exported in my case). For me hamcrest-library was sufficient.
The available hamcrest sub-modules are (according to the org.hamcrest:hamcrest-parent pom, which can be found here: https://repo1.maven.org/maven2/org/hamcrest/hamcrest-parent/1.3/hamcrest-parent-1.3.pom):
hamcrest-core
hamcrest-library
hamcrest-generator
hamcrest-integration
Creating the p2-Repo containing the required bundles
When using Maven and the 'org.reficio:p2-maven-plugin' to build the p2-repo that contains the mentioned test-bundles, the conversion of the maven-artifacts to OSGi-bundles does not produce fully working results by default.
Converting a maven-module to a full OSGi-bundle consists mainly of configuring the MANIFEST.MF to contain proper entries. For this the p2-maven-plugin utilizes "bnd tool".
By default the Java packages provided by all maven dependencies of a maven module are added as optional Imported-package when that module is converted into a OSGi-bundle.
In my case this had the consequence that the org.hamcrest.library bundle refereed to the packages from hamcrest-core only via Import-Package in its MANIFEST.MF.
But unfortunately with only this specified, the Equinox-ClassLoader did not find the classes from hamcrest-core in the test-runtime and threw a corresponding exception. Maybe this is also caused by the fact that hamcrest-core and hamcrest-library have a package "org.hamcrest" and bnd-tools adds the exported packages of a bundle to the imported packages again.
The solution in my case was to instruct the org.reficio:p2-maven-plugin respectively bnd-tools to add org.hamcrest.core as "Require-Bundle" to the Manifest of hamcrest-library. For this, the instructions-element shown below needs to be add to the artifact-element of org.hamcrest:hamcrest-library in the execution-configuration of the 'p2-maven-plugin' in the pom.xml used to build the p2-repo:
<artifact>
<id>org.hamcrest:hamcrest-library:1.3</id>
<instructions>
<Require-Bundle>org.hamcrest.core</Require-Bundle>
</instructions>
</artifact>
If hamcrest sub-modules other than hamcrest-library are are used, the instructions need to be analogous, corresponding to the dependencies listed in their pom.
Edit
Eclipse Orbit provides org.hamcrest.library, org.hamcrest.integrator and org.hamcrest.generator bundles that have have org.hamcrest.core as required bundle (if necessary):
https://download.eclipse.org/tools/orbit/downloads/
Appendix
In the end first solution caused a SecurityException:
java.lang.SecurityException: class "org.hamcrest.Matchers"'s signer information does not match signer information of other classes in the same package
Which is a known issue. The following two solutions avoid this issue and work properly during Tycho builds and from within Eclipse.
Solution 2 - bundle hamcrest sub-module jars with a plug-in
Another approach is to download the jar of the required hamcrest sub-module and bundle it with a Eclipse plugin directly, like it is described here:
https://www.vogella.com/tutorials/Hamcrest/article.html#hamcrest_eclipse
To bundle the jar with a plug-in, include it in the project and add it to the plug-ins classpath. Go to the Runtime-Tab of the Manifest-Editor and klick Add... in the Classpath section and select the jar. This should add the jar to the .classpath, MANIFEST.MF and build.properties file properly.
Make sure the jar is included before the other plug-in dependencies (which include hamcrest-core), as stated in the mentioned tutorial.
If hamcrest should be used in multiple test-projects/fragments, add the jar to a test plug-in all other test-projects depend on.
Solution 3 - use org.hamcrest 2.x
Since hamcrest-2 there is only one org.hamcrest jar/artifact that includes everything from hamcrest. Using hamcrest 2 avoids all the issues and is my preferred solution. Except for the changed packaing of hamcrest the API did not break, so it should be sufficient to just include org.hamcrest:
https://github.com/hamcrest/JavaHamcrest/releases/tag/v2.1
In order to create a p2-repo that includes org.hamcrest-2.2 the following sippet has to be included into the configuration-artifacts element of the p2-maven-plugin execution in the pom.xml:
<artifact>
<id>org.hamcrest:hamcrest-core:2.2</id>
<instructions>
<Require-Bundle>org.hamcrest;bundle-version="2.2.0";visibility:=reexport</Require-Bundle>
</instructions>
</artifact>
<artifact>
<id>org.hamcrest:hamcrest:2.2</id>
</artifact>
The IUs org.hamcrest.core 2.2 and org.hamcrest have to be included in the target-platform to make them available for plug-ins in Eclipse and during. All plug-ins which depend on org.junit now have org.hamcrest also available.
This aproach works because org.hamcrest.core still exists in version 2 stream, even tough it is deprected and empty. Its only purpose is to redirect build-systems to the new org.hamcrest-2.x jar/artifact. Therefore org.hamcrest.core-2.2 specifies a compile dependency to
org.hamcrest-2.2 in its pom.xml. Unfortunately the p2-maven-plugin dosn't translate it directly into a bundle-requirement for org.hamcrest in the manifest, but with the sippet above enforces that.
Because org.junit requires the bundle org.hamcrest.core with a minimal version of 1.3 (but without upper-bound) it uses the present org.hamcrest.core-2.2 . org.hamcrest.core-2.2 again requires org.hamcrest-2.2 and re-exports it. This makes org.junit use org.hamcrest-2.2 in the end and because org.junit re-exports hamcrest-core it also provides org.hamcrest-2.2 immediately to all depended plug-ins.
Note
If you want to play around with different variants of a jar, don't forget to clear (means delete on the drive) the bundle pools of Maven (in <your-home>/.m2/repository/p2/osgi/bundle/ and Eclipse PDE (in <your-workspace>/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/) in between. Otherwise you will always use the first one, because jar's with the same version are not updated.
When I'm trying to start changelog.groovy via liquibase command line it tells me that
Unexpected error running Liquibase: Cannot find parser that supports changelog.groovy
I'm doing the next: java -jar liquibase.jar update
My liquibase.properties are:
driver=org.postgresql.Driver
classpath=C:\Users\Andrii\org.postgresql.Driver.jar;C:\Users\Andrii\liquibase-3.5.1-bin\lib\liquibase-groovy-dsl-1.2.2-SNAPSHOT.jar
changeLogFile=D:\changelog.groovy
url=jdbc:postgresql://localhost:5432/test
username=postgres
password=rup
It finds those jars since if I change something in that path it will tell that jars cannot be found.
I downloaded the groovy-liquibase-dsl project, build it and added a jar into classpath. What am I doing wrong?
To make it work, you need to additionally include groovy and groovy-sql jars in Liquibase's classpath.
So say you store all the jars in C:\Users\Andrii\LiquibaseDependencies, update your the classpath property of your file as such:
classpath=C:\Users\Andrii\LiquibaseDependencies\org.postgresql.Driver.jar;
C:\Users\Andrii\LiquibaseDependencies\liquibase-groovy-dsl-1.2.1.jar;
C:\Users\Andrii\LiquibaseDependencies\groovy-2.4.6.jar;
C:\Users\Andrii\LiquibaseDependencies\groovy-sql-2.4.6.jar
I am trying to write a Play 2.3.8 application in Scala, managing it via sbt but editing it in Eclipse. I worked round one problem, but this seems to introduce another, and cannot work out how to solve it.
I set up the project using the exact instructions to Create a new application without Activator (except I also add
scalaVersion := "2.11.6"
to build.sbt), then I cd to my project directory, type sbt and once in sbt I type eclipse. Then I open Eclipse and happily import the project.
Now I create a simple template (app/views/Application/index.scala.html) and a controller which calls it (app/controllers/Application.scala). When I go into sbt and type run I can happily open my web browser at localhost:9000 and my populated template appears.
All is good apart from one problem (the first one). When I open up Application.scala in Eclipse I get a wiggly red error line saying "object Application is not a member of package views.html". I solved that using Nick Cooper's answer elsewhere on Stack Overflow. He said to go to Project > Properties > Java Build Path > Libraries > Add Class Folder... and then add target/scala-2.11/classes_managed. That makes the error go away. But I don't want to manage Eclipse's settings directly; I want to manage everything via sbt. So by trial and error I found that I can add this line to my build.sbt file...
unmanagedJars in Compile += ( baseDirectory.value / "target/scala-2.11/classes_managed" )
...and now I can type sbt followed by eclipse and Eclipse's config is generated correctly, with no wiggly red line errors.
But this creates a second problem. It turns out that by introducing that line into build.sbt the application no longer runs. Specifically when I go into sbt, type run and open localhost:9000 I get a NoClassDefFoundError exception in my sbt console:
java.lang.NoClassDefFoundError: controllers/Application$
at Routes$$anonfun$routes$1$$anonfun$applyOrElse$1$$anonfun$apply$1.apply(routes_routing.scala:51) ~[classes_managed/:na]
at Routes$$anonfun$routes$1$$anonfun$applyOrElse$1$$anonfun$apply$1.apply(routes_routing.scala:51) ~[classes_managed/:na]
at play.core.Router$HandlerInvokerFactory$$anon$13$$anon$14.call(Router.scala:217) ~[play_2.11-2.3.8.jar:2.3.8]
at play.core.Router$Routes$TaggingInvoker.call(Router.scala:464) ~[play_2.11-2.3.8.jar:2.3.8]
at Routes$$anonfun$routes$1$$anonfun$applyOrElse$1.apply(routes_routing.scala:51) ~[classes_managed/:na]
Caused by: java.lang.ClassNotFoundException: controllers.Application$
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_40]
at Routes$$anonfun$routes$1$$anonfun$applyOrElse$1$$anonfun$apply$1.apply(routes_routing.scala:51) ~[classes_managed/:na]
at Routes$$anonfun$routes$1$$anonfun$applyOrElse$1$$anonfun$apply$1.apply(routes_routing.scala:51) ~[classes_managed/:na]
[error] application - Error while rendering default error page
scala.MatchError: java.lang.NoClassDefFoundError: controllers/Application$ (of class java.lang.NoClassDefFoundError)
at play.api.GlobalSettings$class.onError(GlobalSettings.scala:148) ~[play_2.11-2.3.8.jar:2.3.8]
at play.api.DefaultGlobal$.onError(GlobalSettings.scala:206) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.Server$class.logExceptionAndGetResult$1(Server.scala:63) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.Server$$anonfun$getHandlerFor$4.apply(Server.scala:73) [play_2.11-2.3.8.jar:2.3.8]
at play.core.server.Server$$anonfun$getHandlerFor$4.apply(Server.scala:71) [play_2.11-2.3.8.jar:2.3.8]
You can see the entire (tiny) codebase on Github.
All the class files seem to be all there, and in exactly the same locations, regardless of whether or not I include the "unmanagedJars" line. It's not an Eclipse problem because it happens even when Eclipse is closed. It seems to be a classpath issue (but I can't understand why adding to a classpath should hide some classes). Regardless of that, I'd like to manage my project with sbt and use Eclipse just as the editor. What am I getting wrong?
I have found a solution through further trial and error. Instead of extending the classpath to "target/scala-2.11/classes_managed" it should be extended to "target/scala-2.11/classes". In other words the line in build.sbt should be
unmanagedJars in Compile += ( baseDirectory.value / "target/scala-2.11/classes" )
Now from sbt I can compile, test and run the application, and connect successfully to localhost:9000, and I can also open the files in Eclipse without seeing error lines.
I still don't know why the NoClassDefFoundError really occurred in the previous setup, but that's a problem for another day.
I am using activator but I guess it might be the same.
So, Eclipse does not really like structural changes in a Play project or adding/removing libraries (via libraryDependencies, etc) and displays red squiggly things everywhere. This is how I solve them:
activator clean compile
If I have added/removed libraries, I run
activator eclipse
so that Eclipse gets the changes.
It works everytime. Clean, clean, clean. I actually found this solution somewhere on the Net a while a go but cannot remember where, sorry.
EDIT
And refresh the project in Eclipse!
I'm not an "real" developper, but I have the right to at least write some code, and add some Jars to the Eclipse build path, without spending hours trying to figure out if the Jars are actually in the Build Path.
My problem (error here below) was resolved in question [NoClassDefFoundError, cannot run MapReduceColorCount (Avro 1.7.7) by adding the correct Jars.
[cloudera#localhost ~]$ hadoop jar avroColorCount.jar exos.MapReduceColorCount2 inavro01 outavro01
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/mapreduce/AvroKeyInputFormat
at exos.MapReduceColorCount2.run(MapReduceColorCount2.java:71)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at exos.MapReduceColorCount2.main(MapReduceColorCount2.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
The following are the different ways I've tried to add Jars to the build path:
1. Maven: adding Dependencies through the POM file, they appear afterwards under "Maven Dependencies".
2. "Configure Build Path": the Jars are actually located in my local file system, thus I add the (library) folders, and the folders appear under the "Referenced Libraries".
3. Create a "lib" folder in the project folder, copy/paste the Jars (located in my local file system), do a project Refresh (the lib folder appears in the Package Explorer), select all Jars and right-click "Add to Build Path"
I confirm that my code show no warnings/errors while performing either method. I usually to an "Export ..." of the Jar file in order to execute it.
Example: I've tried adding to Build Path external Jars from Cloudera's CDH5 (Hadoop 2.3.0-cdh5.1.2 and Avro 1.7.5-cdh5.1.2) which are localted locally in /opt/lib
The only method that really worked was method 3. Why it doesn't work with methods 1. or 2. ?
Thank you in advance for your support
I could not reproduce success with method 3., I received a "cannot cast to namespace.customClass" error instead of the "NoClassDefFoundError" error.
I've found an answer for the latter error with a workaround based on two variables export:
export LIBJARS=avrojar1,avrojar2,jar3
export HADOOP_CLASSPATH=avrojar1:avrojar2:jar3
and then running the hadoop jar command with -libjars ${LIBJARS}.
This was tested with method 1. and method 3. respectively.
I conclusion, my case was specific to avro-related Jars only
Thanks