scala forward reference extends over definition of value dataframe - scala

Question

I'm trying to typecast the columns in the data frame df_trial which has all the columns as string, based on an XML file I'm trying to type cast each column.
val columnList = sXml \\ "COLUMNS" \ "COLUMN"
val df_trial = sqlContext.createDataFrame(rowRDD, schema_allString)
columnList.foreach(i => {
var columnName = (i \\ "#ID").text.toLowerCase()
var dataType = (i \\ "#DATA_TYPE").text.toLowerCase()
if (dataType == "number") {
print("number")
var DATA_PRECISION: Int = (i \\ "#DATA_PRECISION").text.toLowerCase().toInt
var DATA_SCALE: Int = (i \\ "#DATA_SCALE").text.toLowerCase().toInt;
var decimalvalue = "decimal(" + DATA_PRECISION + "," + DATA_SCALE + ")"
val df_intermediate: DataFrame =
df_trial.withColumn(s"$columnName",
col(s"$columnName").cast(s"$decimalvalue"))
val df_trial: DataFrame = df_intermediate
} else if (dataType == "varchar2") {
print("varchar")
var DATA_LENGTH = (i \\ "#DATA_LENGTH").text.toLowerCase().toInt;
var varcharvalue = "varchar(" + DATA_LENGTH + ")"
val df_intermediate =
df_trial.withColumn(s"$columnName",
col(s"$columnName").cast(s"$varcharvalue"))
val df_trial: DataFrame = df_intermediate
} else if (dataType == "timestamp") {
print("time")
val df_intermediate =
df_trial.withColumn(s"$columnName", col(s"$columnName").cast("timestamp"))
val df_trial: DataFrame = df_intermediate
}
});

Answer 1

In each branch of the if-else you're using the values called df_trial before you've defined them. You'll need to rearrange the code to define them first.
Note: the way you have it, the df_trial at the very top is not being used. Depending on what you are trying to do, you may want to change the first df_trial to a var and remove the val from the other usages. (This is probably still wrong since you will be overwriting the same variable multiple times as you loop over columnList).