SCALA: Read YAML file using Scala with class constructor - scala

Question

I was trying to read YAML file using scala with a constructor call as below
import org.yaml.snakeyaml.constructor.Constructor
import java.io.{File, FileInputStream}
import scala.collection.mutable.ArrayBuffer
import org.yaml.snakeyaml.Yaml
object aggregation {
def main(args:Array[String]) : Unit = {
//val conf = new SparkConf().setAppName("yaml test").setMaster("local[*]")
//val sc = new SparkContext(conf)
val yamlfile = "C:\\Users\\***\\Desktop\\mongoDB\\sparkTest\\project\\properties.yaml"
val input1 = new FileInputStream(new File(yamlfile))
val yaml = new Yaml(new Constructor(classOf[ReadProperties]))
val e = yaml.load(input1).asInstanceOf[ReadProperties]
println(e.file1)
}
}
And I have a separate class so that I can have the YAML items as beans as below,
class ReadProperties(#BeanProperty var file1:String,#BeanProperty var file2:String) {
//constructor
}
And the content of my yaml file(properties.yaml) is as below,
file1: C:\\data\\names.txt
file2: C:\\data\\names2.txt
but the error is that
Can't construct a java object for tag:yaml.org,2002:ReadProperties; exception=java.lang.NoSuchMethodException: ReadProperties.<init>()
in 'reader', line 1, column 1:
file1: C:\\data\\names.txt
^
at org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:350)
at org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:182)
But if I use the below code it works(without constructor class),
val yaml = new Yaml
val obj = yaml.load(input)
val e = obj.asInstanceOf[java.util.HashMap[String,String]]
println(e)
result :
{file1=C:\\data\\names.txt, file2=C:\\data\\names2.txt}
16/10/02 01:24:28 INFO SparkContext: Invoking stop() from shutdown hook
I want my constructor to work and wanted to directly refer the values of parameters in yaml properties file. (for example, there are two parameters in yaml file "file1" and "file2" so I wanted to refer them directly)
Any help would be appreciated. thanks in advance!

Answer 1

This post suggests to use
val yaml = new Yaml
val obj = yaml.load(input1)
val e = obj.asInstanceOf[ReadProperties]
Because SnakeYAML requires a non-argument constructor if directly deserializing to a custom class object.
It is also possible to provide a custom constructor for the ReadProperties class, but since I do not really know Scala, it is beyond my abilities to show code for that. The official documentation may help.

Answer 2

It worked, actual issue seems to be something wrong environment. Anyways, here is how I did, yes i know its too late to comment here,
def main(args:Array[String]) : Unit = {
val conf = new SparkConf().setAppName("yaml test").setMaster("local[*]")
val sc = new SparkContext(conf)
val yamlfile = "C:\\Users\\Chaitu-Padi\\Desktop\\mongoDB\\sparkTest\\project\\properties.yaml"
val input1 = new FileInputStream(new File(yamlfile))
val yaml = new Yaml(new Constructor(classOf[yamlProperties]))
val e = yaml.load(input1)
val d = e.asInstanceOf[yamlProperties]
println(d)
}
class yamlProperties {
#BeanProperty var file1: String = null
#BeanProperty var file2: String = null
#BeanProperty var file3: String = null
override def toString: String = {
//return "%s,%s".format(file1, file2)
return ( "%s,%s,%s").format (file1, file2, file3)
}
}