SparkContext: error initializing SparkContext. java.net.UnknownHostException: master - scala

Question

I am new Spark and Intellij. I have my build.sbt file below:
name := "TestSpark"
version := "1.0"
scalaVersion := "2.11.8"
// https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11
libraryDependencies ++= Seq("org.apache.spark" % "spark-core_2.11" % "1.6.2","org.apache.hadoop" % "hadoop-client" % "2.6.2")
I also have TestMain.scala in src/main/scala-2.11/TestMain.scala
import org.apache.spark.{SparkContext,SparkConf}
/**
* Created by tuannv5 on 24/08/2016.
*/
object TestMain {
def main(args: Array[String]) {
val conf = new SparkConf().setMaster("local[*]").setAppName("Test Spark").set("spark.executor.memory","2g")
val sc = new SparkContext(conf)
val data = sc.parallelize(1 to 1000000).filter(_<10000)
data.foreach(println)
}
}
For some reason when I do Run app in Intellij. I get the error:
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
16/08/24 10:29:28 INFO SparkContext: Running Spark version 1.6.2
16/08/24 10:29:29 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/08/24 10:29:29 ERROR SparkContext: Error initializing SparkContext.
java.net.UnknownHostException: master: master: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:789)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:782)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:782)
at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:839)
at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:839)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.util.Utils$.localHostName(Utils.scala:839)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:420)
at TestMain$.main(TestMain.scala:8)
at TestMain.main(TestMain.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.net.UnknownHostException: master: unknown error
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
... 15 more
16/08/24 10:29:29 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.net.UnknownHostException: master: master: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:789)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:782)
at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:782)
at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:839)
at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:839)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.util.Utils$.localHostName(Utils.scala:839)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:420)
at TestMain$.main(TestMain.scala:8)
at TestMain.main(TestMain.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.net.UnknownHostException: master: unknown error
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
... 15 more
Process finished with exit code 1
Can someone please explain to me the problem stated in this error? Is this because my dependencies were not installed correctly or is it because of another reason?

Answer 1

Your hostname is not proper.
Type hostname and check whether it's the same in /etc/hosts:
For example:
root#google:/home/makbul# hostname
Google has to same in /etc/hosts:
127.0.1.1 google