How to add Spark to Maven project in Eclipse? - eclipse

Question

I would like to start Spark project in Eclipse using Maven.
I've installed m2eclipse and I have a working HelloWorld Java application in my Maven project.
I would like to use Spark framework and I'm following directions from the official site.
I've added Spark repository to my pom.xml:
<repository>
<id>Spark repository</id>
<url>http://www.sparkjava.com/nexus/content/repositories/spark/</url>
</repository>
And then the dependency:
<dependency>
<groupId>spark</groupId>
<artifactId>spark</artifactId>
<version>0.9.9.4-SNAPSHOT</version>
</dependency>
But I'm getting an error in Eclipse:
Missing artifact spark:spark:jar:0.9.9.4-SNAPSHOT
How can I resolve this issue? I don't want to download Spark's jar file and place in the local repository.
This is my pom.xml file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.myproject</groupId>
<artifactId>Spark1</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Spark1</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<repository>
<id>Spark repository</id>
<url>http://www.sparkjava.com/nexus/content/repositories/spark/</url>
</repository>
<dependencies>
<!-- (...) -->
<dependency>
<groupId>spark</groupId>
<artifactId>spark</artifactId>
<version>0.9.9.4-SNAPSHOT</version>
</dependency>
</dependencies>
</project>

Answer 1

Currently no repository is required to add for Spark library loading
You just need to add
<dependency>
<groupId>com.sparkjava</groupId>
<artifactId>spark-core</artifactId>
<version>2.6.0</version>
</dependency>
And that's it.
Useful tutorials to play with is here

Answer 2

The repository block needs to be wrapped in a repositories block:
<repositories>
<repository>
<id>Spark repository</id>
<url>http://www.sparkjava.com/nexus/content/repositories/spark/</url>
</repository>
</repositories>

Answer 3

Reason for failure is 0.9.9.4-SNAPSHOT is not available.Below is the list of the snapshots available. Use one among them based on your requirement.
0.9.8-SNAPSHOT/ Sat May 21 21:54:23 UTC 2011
0.9.9-SNAPSHOT/ Mon May 23 10:57:38 UTC 2011
0.9.9.1-SNAPSHOT/ Thu May 26 09:47:03 UTC 2011
0.9.9.3-SNAPSHOT/ Thu Sep 01 07:53:59 UTC 2011
Thanks,
Sankara Reddy

Answer 4

I had run into same issue because initially I started with different repository url for spark and then to use earlier version I changed the repository url. Some how it didn't seem to come into effect until I changed the repository id. Try changing the repository id.
Could be bug in maven because running maven from console also couldn't resolve the dependency without updating the id.

Answer 5

please add the repository , tag inside the repositories tag like below
<repositories>
<repository>
<id>Spark repository</id>
<url>http://www.sparkjava.com/nexus/content/repositories/spark/</url>
</repository>
</repositories>

Answer 6

The last versions (2.1 and later) of Spark only need the dependency defined inside the pom.xml file
<dependency>
<groupId>com.sparkjava</groupId>
<artifactId>spark-core</artifactId>
<version>2.1</version>
</dependency>
the repository definition is not longer required

Answer 7

use this latest repository.
http://mvnrepository.com/artifact/org.apache.spark/spark-core_2.10/1.6.0
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.6.0</version>
</dependency>

Answer 8

use this, and also make sure you change spark library to version 2.11.x in eclipse project build path
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.spark-scala</groupId>
<artifactId>spark-scala</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>${project.artifactId}</name>
<description>Spark in Scala</description>
<inceptionYear>2010</inceptionYear>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<encoding>UTF-8</encoding>
<scala.tools.version>2.10</scala.tools.version>
<!-- Put the Scala version of the cluster -->
<scala.version>2.10.4</scala.version>
</properties>
<!-- repository to add org.apache.spark -->
<repositories>
<repository>
<id>cloudera-repo-releases</id>
<url>https://repository.cloudera.com/artifactory/repo/</url>
</repository>
</repositories>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<testSourceDirectory>src/test/scala</testSourceDirectory>
<plugins>
<plugin>
<!-- see http://davidb.github.com/scala-maven-plugin -->
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.13</version>
<configuration>
<useFile>false</useFile>
<disableXmlReport>true</disableXmlReport>
<includes>
<include>**/*Test.*</include>
<include>**/*Suite.*</include>
</includes>
</configuration>
</plugin>
<!-- "package" command plugin -->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>1.2.1</version>
</dependency>
</dependencies>
</project>