Unable to convert Spark RDD to Schema RDD - scala

Question

I am trying to execute the example provided in Spark programming guide.
https://spark.apache.org/docs/1.1.0/sql-programming-guide.html
But I am facing the compilation error.
(I am a Scala newbie)
Below is my code:
import org.apache.spark.{SparkContext,SparkConf}
import org.apache.spark.sql._
import org.apache.spark.sql
object Temp {
def main(args: Array[String]) {
val sparkConf = new SparkConf().setMaster("local").setAppName("SPARK SQL example")
val sc= new SparkContext(sparkConf)
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
import sqlContext.createSchemaRDD
case class Person(name: String, age: Int)
val people = sc.textFile("examples/src/main/resources/people.txt").map(_.split(",")).map(p => Person(p(0), p(1).trim.toInt))
people.registerTempTable("people")
val teenagers = sqlContext.sql("SELECT name FROM people WHERE age >= 13 AND age <= 19")
teenagers.map(t => "Name: " + t(0)).collect().foreach(println)
}
}
I am facing the compilation error No TypeTag available for Person at the line people.registerTempTable("people").
How to resolve this error?

Answer 1

It is failing because the Person class is defined inside of the function and as such the Scala compiler will not create a TypeTag for the class. As Paul suggested you can move it out of the function to the top level.
I'll add that there is a JIRA to relax this restriction: https://issues.apache.org/jira/browse/SPARK-4842