Scala: Adding elements to Set inside 'foreach' doesn't persist - scala

Question

I create a mutable set and iterate over a list using a 'foreach' to populate the set. When I print the set inside the foreach, it prints the contents of the set correctly. However, the set is empty after the end of 'foreach'. I am not able to figure out what I am missing.
import org.apache.spark._
import org.apache.spark.graphx._
import org.apache.spark.SparkConf
import org.apache.spark.rdd.RDD
object SparkTest {
def main(args: Array[String]) {
val conf = new SparkConf().setAppName("Spark Test")
val sc = new SparkContext(conf)
val graph = GraphLoader.edgeListFile(sc, "followers.txt")
val edgeList = graph.edges
var mapperResults = iterateMapper(edgeList)
sc.stop()
}
def iterateMapper(edges: EdgeRDD[Int, Int]) : scala.collection.mutable.Set[(VertexId, VertexId)] = {
var mapperResults = scala.collection.mutable.Set[(VertexId, VertexId)]()
val mappedValues = edges.mapValues(edge => (edge.srcId, edge.dstId)) ++ edges.mapValues(edge => (edge.dstId, edge.srcId))
mappedValues.foreach {
edge => {
var src = edge.attr._1
var dst = edge.attr._2
mapperResults += ((src, dst))
}
}
println(mapperResults)
return mapperResults
}
}
This is the code I'm working with. It is a modified example from Spark.
The
println(mapperResults)
prints out an empty set.

Answer 1

Actually it works, but in the worker.
the foreach is a function that exists for the side effects, but it work on the worker, so you wont see the updated Set.
Other issue is that it design to be Immutable! so do not use mutable collection there. Also there is no need for that. The following code should do what do you meant to do:
var mapperResults = mappedValues.map(_.attr).distinct.collect
It shorter, cleaner and do the map work on the workers.