I want to use thread pool in my scala project. It can runs well in my IntelliJ Idea, but it throws exception when I compile the project use maven command line: mvn compile. I have added the dependencies "scala-libray" and "scala-actors", but nothing ever effects. Can you help me?
My code:
import scala.actors.threadpool.Executors
val execService = Executors.newFixedThreadPool(10)
The error:
[ERROR] /project../server/EasyServer.scala:6: error: object actors is not a member of package scala
[ERROR] import scala.actors.threadpool.Executors
[ERROR] ^
[ERROR] /project../server/EasyServer.scala:16: error: not found: value Executors
[ERROR] val execService = Executors.newFixedThreadPool(10)
[ERROR] ^
[ERROR] two errors found
A part of my pom.xml:
<properties>
<scala.version>2.11.6</scala.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-actors</artifactId>
<version>${scala.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<resources>
<resource>
<directory>${basedir}/src/main/resources</directory>
<includes>
<include>**/*</include>
</includes>
<excludes>
<exclude>**/.svn/</exclude>
</excludes>
</resource>
</resources>
<plugins>
<!-- the Maven compiler plugin will compile Java source files -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.5.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
<configuration>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
<!-- the Maven Scala plugin will compile Scala source files -->
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<!--
Bind the maven-assembly-plugin to the package phase this will create
a jar file without the storm dependencies suitable for deployment to
a cluster.
-->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Starting from 2.11, scala actors are shipped as a separate library :
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-actors</artifactId>
<version>2.11.6</version>
</dependency>
Note that they are also deprecated in favor of Akka : http://docs.scala-lang.org/overviews/core/actors-migration-guide.html
Related
I have a scala-maven-spark project.
Here is my pom.xml file
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
<scala.version>2.11.8</scala.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.11</artifactId>
<version>3.0.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>28.1-jre</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.17.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.17.0</version>
</dependency>
</dependencies>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.7</version>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.3.3</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
</plugin>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<version>1.0</version>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>WDF TestSuite.txt</filereports>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<finalName>${project.artifactId}-${project.version}</finalName>
<relocations>
<relocation>
<pattern>com.google.protobuf</pattern>
<shadedPattern>shaded.com.google.protobuf</shadedPattern>
</relocation>
<relocation>
<pattern>com.google.common</pattern>
<shadedPattern>shaded.com.google.common</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
When I run mvn clean package I am getting the below error
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.3.3:compile (default) on project decisionengine-data-ingestion-maven: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
If I remove the net.alchim31.maven plugin the build completes successfully.
Error details
[ERROR] error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.
[ERROR] at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
[ERROR] at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
[ERROR] at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:53)
[E
If you have scala source code you need
to add the scala-maven-plugin (please use a more recent version see https://search.maven.org/search?q=a:scala-maven-plugin, today 4.5.6 is the latest, why using 3.3.3 from 2018-06 ?) to compile your code
to have the scala-library as an explicit dependency
And remove
<configuration>
<scalaVersion>${scala.version}</scalaVersion>
</configuration>
It's better to let the plugin detects the scala version from dependencies.
Updating an existing maven project from Java 8 to 11, and I'm struggling to find the right combination of dependency and plugin versions.
One of the modules uses the play framework, with its associated dependencies, here's what they are set to now:
java: 11.0.5
scala: 2.12.8
sbt: 0.13.17
sbt-compiler-maven-plugin: 1.0.0
sbtrun-maven-plugin: 1.0.1
play: 2.6.21
play2-maven-plugin: 1.0.0-rc5
play2-provider-play26: 1.0.0-rc5
akka: 2.5.27
Which is giving me the following build error:
[ERROR] Failed to execute goal com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance (default-play2-enhance) on project services: Execution default-play2-enhance of goal com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance failed: An API incompatibility was encountered while executing com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5:enhance: java.lang.NoSuchMethodError: 'scala.collection.mutable.ArrayOps scala.Predef$.byteArrayOps(byte[])'
[ERROR] -----------------------------------------------------
[ERROR] realm = extension>com.google.code.play2-maven-plugin:play2-maven-plugin:1.0.0-rc5
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
...
...
...
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import from realm ClassRealm[maven.api, parent: null]]
Note: I am hoping to keep akka to version 2.5.x, as 2.6.x requires a lot of code changes, but will go to a higher version if necessary
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>groupId</groupId>
<artifactId>artifactId</artifactId>
<version>version</version>
</parent>
<artifactId>artifactId</artifactId>
<packaging>play2</packaging>
<name>name</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<repositories>
<repository>
<id>typesafe</id>
<url>http://repo.typesafe.com/typesafe/releases/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play_2.12</artifactId>
</dependency>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play-test_2.12</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play-java_2.12</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-joda</artifactId>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-stream_2.12</artifactId>
</dependency>
<dependency>
<groupId>org.easytesting</groupId>
<artifactId>fest-assert-core</artifactId>
</dependency>
</dependencies>
<build>
<!-- Play source directory -->
<sourceDirectory>${basedir}/app</sourceDirectory>
<testSourceDirectory>${basedir}/test</testSourceDirectory>
<resources>
<resource>
<directory>${basedir}/conf</directory>
</resource>
<resource>
<directory>${basedir}/public</directory>
<targetPath>public</targetPath>
</resource>
<resource>
<directory>${basedir}/target/sbt/web/public/main</directory>
<targetPath>public</targetPath>
</resource>
</resources>
<plugins>
<plugin>
<groupId>com.google.code.play2-maven-plugin</groupId>
<artifactId>play2-maven-plugin</artifactId>
<version>1.0.0-rc5</version>
<extensions>true</extensions>
<configuration>
<mainLang>java</mainLang>
</configuration>
<dependencies>
<dependency>
<groupId>com.google.code.play2-maven-plugin</groupId>
<artifactId>play2-provider-play27</artifactId>
<version>1.0.0-rc5</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>default-play2-enhance</id>
<goals>
<goal>enhance</goal>
</goals>
</execution>
<execution>
<phase>package</phase>
<goals>
<goal>dist</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.google.code.sbt-compiler-maven-plugin</groupId>
<artifactId>sbt-compiler-maven-plugin</artifactId>
<version>1.0.0</version>
</plugin>
<plugin>
<groupId>com.google.code.sbtrun-maven-plugin</groupId>
<artifactId>sbtrun-maven-plugin</artifactId>
<version>1.0.1</version>
<executions>
<execution>
<id>compile-assets</id>
<phase>generate-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<args>web-assets:assets</args>
<jvmArgs>-Dscala.version=2.12.8</jvmArgs>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>default-jar</id>
<configuration>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<id>attach-artifacts</id>
<phase>package</phase>
<goals>
<goal>attach-artifact</goal>
</goals>
</execution>
</executions>
<configuration>
<!-- <!– These empty tags keep IntelliJ quiet –> -->
<name />
<regex />
<source />
<value />
<fileSet />
<artifacts>
<artifact>
<file>${project.build.directory}/service-${project.version}-dist.zip</file>
<type>zip</type>
<classifier>dist</classifier>
</artifact>
</artifacts>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>eclipse</id> <!-- for M2Eclipse only -->
<build>
<directory>${project.basedir}/target-eclipse</directory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<skipMain>true</skipMain>
<skip>true</skip>
<source>11</source>
<target>11</target>
</configuration>
<executions>
<execution>
<id>default-compile</id>
<goals><goal>compile</goal></goals>
</execution>
<execution>
<id>default-testCompile</id>
<goals><goal>testCompile</goal></goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
Edit:
Updated SBT and Scala version, added redacted pom
As per scala documentation
https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html
Your SBT version is not compatible with Java 11. I would recommend upgrading to sbt.version=1.2.3
Akka version you are using should be good if you don't want to upgrade to higher.
Update:
I can see you updated sbt version to 0.13.17 and you tried with higher, but you should forget that sbt 0.13 and stick with sbt 1.x since sbt 0.13 is using scala 2.10, and sbt 1.x scala 2.12.
I've also noticed that you don't have version tag for maven-compiler-plugin. If you are using maven 3, it should fail to build because of missing this tag, and since it's not, I guess you are using maven 2 which should be upgraded. Also you should add 3.8.0 version tag to maven-compiler-plugin.
Check this guide to see how to properly configure this plugin for java 11. You should also check other instructions to see if everything is properly configured.
I am using Scala IDE for creating a maven project with spark
1.I have created a maven project with skipped the archetype and added the following pom file.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.nivi</groupId>
<artifactId>testSpark</artifactId>
<version>0.0.1-SNAPSHOT</version>
<repositories>
<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
</repository>
<repository>
<id>general</id>
<url>https://repo1.maven.org/maven2</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.6</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.0-cdh5.8.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>1.6.0-cdh5.8.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.6.0-cdh5.8.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<!-- mixed scala/java compile -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<execution>
<id>compile</id>
<goals>
<goal>compile</goal>
</goals>
<phase>compile</phase>
</execution>
<execution>
<id>test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
<phase>test-compile</phase>
</execution>
<execution>
<phase>process-resources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>2.10.6</scalaVersion>
</configuration>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.4</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!-- for fatjar -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>fully.qualified.MainClass</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.scala-tools</groupId>
<artifactId>
maven-scala-plugin
</artifactId>
<versionRange>
[2.15.2,)
</versionRange>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</pluginExecutionFilter>
<action>
<execute></execute>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
I have added scala nature in Configure.
I have added src/main/scala folder in the Properties-->Build path-->Source
I have set java compiler 1.8 and scala compiler 2.10.6 with jvm 1.8 in my
project properties.
But i am getting the following error on maven clean install :
enter image description here
Go to this location
c:\users\Nivedita.m2\repository\org\apache\hadoop\hadoop-common\2.6.0-cdh5.8.2
and try deleting hadoop-common-2.6.0-cdh5.8.2.jar file and do maven build again.
Can you try with an recent version of scala plugin ? (it's the successor of the one you used, I'm the author of both)
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.3.2</version>
</plugin>
(see http://davidb.github.io/scala-maven-plugin/example_java.html)
if you use a jdk 9+ to compile (regardless of jvm 1.8 in the config) you can have this kind of issue. So check that the jdk is 1.8 .
package explore I tried to run the programme below. I am new to spark programming.i am getting following error in Scala IDE.I checked main class, no version problems in scala and spark. I mentioned output folder default. Please let me know what's wrong in my code.
package com.sparkarma.spark
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
object WordCount{
def main(args:Array[String]) {
val conf = new SparkConf().setAppName("Word Count").setMaster("local[2]")
val sc = new SparkContext(conf)
val bigFile = sc.textFile("hdfs://localhost:9000/home/chaitanya/files/bigText")
val words = bigFile.flatMap(line => line.split(" ")).count()
println(words)
}
}
ScreenShot3
Here is My pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.sparkarma.samp</groupId>
<artifactId>SparkMaro</artifactId>
<version>0.0.1-SNAPSHOT</version>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>http://scala-tools.org/repo-releases</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<artifactId>scala-library</artifactId>
<groupId>org.scala-lang</groupId>
<version>2.10.6</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- mixed scala/java compile -->
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<executions>
<execution>
<id>compile</id>
<goals>
<goal>compile</goal>
</goals>
<phase>compile</phase>
</execution>
<execution>
<id>test-compile</id>
<goals>
<goal>testCompile</goal>
</goals>
<phase>test-compile</phase>
</execution>
<execution>
<phase>process-resources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<!-- for fatjar -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>assemble-all</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>fully.qualified.MainClass</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.scala-tools</groupId>
<artifactId> maven-scala-plugin </artifactId>
<versionRange> [2.15.2,) </versionRange>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</pluginExecutionFilter>
<action>
<execute />
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
Your project folder don't have Scala library container. Seems you are missing Scala libraries or Scala Nature in your project. Review your pom.xml and add appropriate Scala library in your project. Check if below dependency is added.
<dependency>
<artifactId>scala-library</artifactId>
<groupId>org.scala-lang</groupId>
<version>${scala.version}</version>
</dependency>
If yes then follow below steps:
Right Click on project pom.xml ->Run As ->Maven Install
Right Click on Project ->Configure ->Add Scala Nature
Right Click on Project -> Scala -> Add scala library to build Path
Right Click on Scala Library container -> Properties -> Classpath container -> Select containers as per the version defined in pom.xml
Your code looks good. If you still see the issue please provide your pom.xml
Please upgrade your spark-core_2.10 version from 1.4.0 to 1.6.2. Use below maven dependency and follow the above steps.
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.2</version>
</dependency>
You are overriding the main method. Make sure that you don't do it
object WordCount{
def main(args:Array[String]) {
val conf = new SparkConf().setAppName("Word Count").setMaster("local[2]")
val sc = new SparkContext(conf)
val bigFile = sc.textFile("hdfs://localhost:9000/home/chaitanya/files/bigText")
val words = bigFile.flatMap(line => line.split(" ")).count()
println(words)
}
}
After looking in to your project explorer you need to right click scala folder and select build path -> use as source folder.
I'm trying configure my Eclipse (Kepler), when I execute Run as -> Maven Clean, that works. After I execute Run as -> Maven Install, this error below is showed in console.
There's a solution? Thanks for attention.
[ERROR] Failed to execute goal org.codehaus.mojo:gwt-maven-plugin:2.6.0:compile (default) on project ProjectSetup: Command [[
[ERROR] C:\Program Files\Java\jdk1.7.0_51\jre\bin\java -Xmx512m -classpath C:\Dev\ProjectSetup\target\ProjectSetup-0.0.1-SNAPSHOT\WEB-INF\classes;C:\Dev\ProjectSetup\src\main\java;C:\Dev\ProjectSetup\target\generated-sources\gwt;C:\Users\Paulo\.m2\repository\com\google\gwt\gwt-user\2.6.0\gwt-user-2.6.0.jar;C:\Users\Paulo\.m2\repository\org\json\json\20090211\json-20090211.jar;C:\Users\Paulo\.m2\repository\com\github\gwtbootstrap\gwt-bootstrap\2.3.2.0\gwt-bootstrap-2.3.2.0.jar;C:\Users\Paulo\.m2\repository\com\google\gwt\gwt-user\2.6.0\gwt-user-2.6.0.jar;C:\Users\Paulo\.m2\repository\com\google\gwt\gwt-dev\2.6.0\gwt-dev-2.6.0.jar com.google.gwt.dev.Compiler -logLevel INFO -style OBF -war C:\Dev\ProjectSetup\target\ProjectSetup-0.0.1-SNAPSHOT -localWorkers 4 -XfragmentCount -1 -sourceLevel auto -gen C:\Dev\ProjectSetup\target\.generated com.gft.start.projectsetup.ProjectSetup
[ERROR] ]] failed with status 1
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[EDIT] Here my pom.xml, I'm need configure Maven-Eclipse with GWT and GWTBootstrap. Thanks for attention and help again.
<?xml version="1.0" encoding="UTF-8"?>
<project
xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<!-- POM file generated with GWT webAppCreator -->
<modelVersion>4.0.0</modelVersion>
<groupId>com.gft.start</groupId>
<artifactId>ProjectSetup</artifactId>
<packaging>war</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>GWT Maven Archetype</name>
<properties>
<!-- Convenience property to set the GWT version -->
<gwtVersion>2.6.0</gwtVersion>
<!-- GWT needs at least java 1.5 -->
<webappDirectory>${project.build.directory}/${project.build.finalName}</webappDirectory>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-servlet</artifactId>
<version>${gwtVersion}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.google.gwt</groupId>
<artifactId>gwt-user</artifactId>
<version>${gwtVersion}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
<version>1.0.0.GA</version>
<classifier>sources</classifier>
<scope>test</scope>
</dependency>
<!-- Source: http://maven-repository.com/artifact/com.github.gwtbootstrap/gwt-bootstrap/2.3.2.0 -->
<dependency>
<groupId>com.github.gwtbootstrap</groupId>
<artifactId>gwt-bootstrap</artifactId>
<version>2.3.2.0</version>
</dependency>
</dependencies>
<build>
<!-- Generate compiled stuff in the folder used for developing mode -->
<outputDirectory>${webappDirectory}/WEB-INF/classes</outputDirectory>
<plugins>
<!-- GWT Maven Plugin -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.6.0</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>test</goal>
<goal>i18n</goal>
<!-- <goal>generateAsync</goal> -->
</goals>
</execution>
</executions>
<!-- Plugin configuration. There are many available options, see
gwt-maven-plugin documentation at codehaus.org -->
<configuration>
<runTarget>ProjectSetup.html</runTarget>
<hostedWebapp>${webappDirectory}</hostedWebapp>
<i18nMessagesBundle>com.gft.start.projectsetup.client.Messages</i18nMessagesBundle>
</configuration>
</plugin>
<!-- Copy static web files before executing gwt:run -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.1.1</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>exploded</goal>
</goals>
</execution>
</executions>
<configuration>
<webappDirectory>${webappDirectory}</webappDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
<!-- As per: http://stackoverflow.com/a/16275036 -->
<pluginManagement>
<plugins>
<!-- This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself. -->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<versionRange>[2.6.0,)</versionRange>
<goals><goal>i18n</goal></goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<versionRange>[2.1.1,)</versionRange>
<goals><goal>exploded</goal></goals>
</pluginExecutionFilter>
<action>
<ignore></ignore>
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
Try to match your pom.xml with below one that is used for GWT compile.
There may be some issue in source or class path or war directory location.
Note: Replace versions and EntryPointClassName in below configuration and change the JAVA version to 1.7 also
<build>
<sourceDirectory>src/main/java</sourceDirectory>
<testSourceDirectory>src/main/test</testSourceDirectory>
<resources>
<resource>
<directory>src/main/resources</directory>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<!-- http://mojo.codehaus.org/gwt-maven-plugin/compile-mojo.html -->
<groupId>org.codehaus.mojo</groupId>
<artifactId>gwt-maven-plugin</artifactId>
<version>2.5.0</version>
<executions>
<execution>
<configuration>
<extraJvmArgs>-Xmx640M -Xss1024k</extraJvmArgs>
</configuration>
<goals>
<goal>compile</goal>
<goal>eclipse</goal>
</goals>
</execution>
</executions>
<configuration>
<optimizationLevel>9</optimizationLevel>
<style>DETAILED</style>
<warSourceDirectory>src/main/webapp</warSourceDirectory>
<modules>
<module>com.x.y.z.EntryPointClassName</module>
</modules>
</configuration>
</plugin>
....