I'm new to the concept of using the Option type but I've tried to use it multiple places in this class to avoid these errors.
The following class is used to store data.
class InTags(val tag35: Option[String], val tag11: Option[String], val tag_109: Option[String], val tag_58: Option[String])
This following code takes a string and converts it into a Int -> String map by seperating on an equals sign.
val message= FIXMessage("8=FIX.4.29=25435=D49=REDACTED56=REDACTED115=REDACTED::::::::::CENTRAL34=296952=20151112-17:11:1111=Order7203109=CENTRAL1=TestAccount63=021=155=CSCO48=CSCO.O22=5207=OQ54=160=20151112-17:11:1338=5000040=244=2815=USD59=047=A13201=CSCO.O13202=510=127
")
val tag58 = message.fields(Some(58)).getOrElse("???")
val in_messages= new InTags(message.fields(Some(35)), message.fields(Some(11)), message.fields(Some(109)), Some(tag58))
println(in_messages.tag_109.getOrElse("???"))
where the FIXMessage object is defined as follows:
class FIXMessage (flds: Map[Option[Int], Option[String]]) {
val fields = flds
def this(fixString: String) = this(FIXMessage.parseFixString(Some(fixString)))
override def toString: String = {
fields.toString
}
}
object FIXMessage{
def apply(flds: Map[Option[Int], Option[String]]) = {
new FIXMessage(flds)
}
def apply(flds: String) = {
new FIXMessage(flds)
}
def parseFixString(fixString: Option[String]): Map[Option[Int], Option[String]] = {
val str = fixString.getOrElse("str=???")
val parts = str.split(1.toChar)
(for {
part <- parts
p = part.split('=')
} yield Some(p(0).toInt) -> Some(p(1))).toMap
}
}
The error I'm getting is ERROR key not found: Some(58) but doesnt the option class handle this? Which basically means that the string passed into the FIXMessage object doesnt contain a substring of the format 58=something(which is true) What is the best way to proceed?
You are using the apply method in Map, which returns the value or throw NoSuchElementException if key is not present.
Instead you could use getOrElse like
message.fields.getOrElse(Some(58), Some("str"))
Related
I'm trying to come up with a CSV Parser that can be called like this:
parser parse "/path/to/csv/file" using parserConfiguration
Where the parser will be a class that contains the target case class into which the CSV file will be parsed into:
class CSVParser[A] {
def parse(path: String) = Source.fromFile(fromFilePath).getLines().mkString("\n")
def using(cfg: ParserConfig) = ??? How do I chain this optionally???
}
val parser = CSVParser[SomeCaseClass]
I managed to get up to the point where I can call:
parser parse "/the/path/to/the/csv/file/"
But I do not want to run the parse method yet as I want to apply the configuration using the using like DSL as mentioned above! So there are two rules here. If the caller does not supply a parserConfig, I should be able to run with the default, but if the user supplies a parserConfig, I want to apply the config and then run the parse method. I tried it with a combination of implicits, but could not get them to work properly!
Any suggestions?
EDIT: So the solution looks like this as per comments from "Cyrille Corpet":
class CSVReader[A] {
def parse(path: String) = ReaderWithFile[A](path)
case class ReaderWithFile[A](path: String) {
def using(cfg: CSVParserConfig): Seq[A] = {
val lines = Source.fromFile(path).getLines().mkString("\n")
println(lines)
println(cfg)
null
}
}
object ReaderWithFile {
implicit def parser2parsed[A](parser: ReaderWithFile[A]): Seq[A] = parser.using(defaultParserCfg)
}
}
object CSVReader extends App {
def parser[A] = new CSVReader[A]
val sss: Seq[A] = parser parse "/csv-parser/test.csv" // assign this to a val so that the implicit conversion gets applied!! Very important to note!
}
I guess I need to get the implicit in scope at the location where I call the parser parse, but at the same time I do not want to mess up the structure that I have above!
If you replace using with an operator with a higher precedence than parse you can get it to work without needing extra type annotations. Take for instance <<:
object parsedsl {
class ParserConfig
object ParserConfig {
val default = new ParserConfig
}
case class ParseUnit(path: String, config: ParserConfig)
object ParseUnit {
implicit def path2PU(path: String) = ParseUnit(path, ParserConfig.default)
}
implicit class ConfigSyntax(path: String) {
def <<(config: ParserConfig) = ParseUnit(path, config)
}
class CSVParser {
def parse(pu: ParseUnit) = "parsing"
}
}
import parsedsl._
val parser = new CSVParser
parser parse "path" << ParserConfig.default
parser parse "path"
Your parse method should just give a partial result, without doing anything at all. To deal with default implem, you can use implicit conversion to output type:
class CSVParser[A] {
def parse(path: String) = ParserWithFile[A](path)
}
case class ParserWithFile[A](path: String) {
def using(cfg: ParserConfig): A = ???
}
object ParserWithFile {
implicit def parser2parsed[A](parser: ParserWithFile[A]): A = parser.using(ParserConfig.default)
}
val parser = CSVParser[SomeCaseClass]
This is somewhat of a theoretical question but something I might want to do. Is it possible to return multiple data data types from a Scala function but limit the types that are allowed? I know I can return one type by specifying it, or I can essentially allow any data type by not specifying the return type, but I would like to return 1 of 3 particular data types to preserve a little bit of type safety. Is there a way to write an 'or' in the return type like:
def myFunc(input:String): [Int || String] = { ...}
The main context for this is trying to write universal data loading script. Some of my users use Spark, some Scalding, and who knows what will be next. I want my users to be able to use a generic loading script that might return a RichPipe, RDD, or some other data format depending on the framework they are using, but I don't want to throw type safety completely out the window.
You can use the Either type provided by the Scala Library.
def myFunc(input:String): Either[Int, String] = {
if (...)
Left(42) // return an Int
else
Right("Hello, world") // return a String
}
You can use more than two types by nesting, for instance Either[A,Either[B,C]].
As already noted in comments you'd better use Either for this task, but if you really want it, you can use implicits
object IntOrString {
implicit def fromInt(i: Int): IntOrString = new IntOrString(None, Some(i))
implicit def fromString(s: String): IntOrString = new IntOrString(Some(s), None)
}
case class IntOrString(str: Option[String], int: Option[Int])
implicit def IntOrStringToInt(v: IntOrString): Int = v.int.get
implicit def IntOrStringToStr(v: IntOrString): String = v.str.get
def myFunc(input:String): IntOrString = {
if(input.isEmpty) {
1
} else {
"test"
}
}
val i: Int = myFunc("")
val s: String = myFunc("123")
//exception
val ex: Int = myFunc("123")
I'd make the typing by the user less implicit and more explicit. Here are three examples:
def loadInt(input: String): Int = { ... }
def loadString(input: String): String = { ... }
That's nice and simple. Alternatively, we can have a function that returns the appropriate curried function using an implicit context:
def loader[T]()(implicit context: String): String => T = {
context match {
case "RDD" => loadInt _ // or loadString _
}
}
Then the user would:
implicit val context: String = "RDD" // simple example
val loader: String => Int = loader()
loader(input)
Alternatively, can turn it into an explicit parameter:
val loader: String => Int = loader("RDD")
Is it possible to add functionality before calling constructor in extra constructor in scala ?
Lets say, I have class User, and want to get one string - and to split it into attributes - to send them to the constructor:
class User(val name: String, val age: Int){
def this(line: String) = {
val attrs = line.split(",") //This line is leading an error - what can I do instead
this(attrs(0), attrs(1).toInt)
}
}
So I know I'm not able to add a line before sending to this, because all constructors need to call another constructor as the first statement of the constructor.
Then what can I do instead?
Edit:
I have a long list of attributes, so I don't want to repeat line.split(",")
I think this is a place where companion object and apply() method come nicely into play:
object User {
def apply(line: String): User = {
val attrs = line.split(",")
new User(attrs(0), attrs(1).toInt)
}
}
class User(val name: String, val age: Int)
Then you just create your object the following way:
val u1 = User("Zorro,33")
Also since you're exposing name and age anyway, you might consider using case class instead of standard class and have consistent way of constructing User objects (without new keyword):
object User {
def apply(line: String): User = {
val attrs = line.split(",")
new User(attrs(0), attrs(1).toInt)
}
}
case class User(name: String, age: Int)
val u1 = User("Zorro,33")
val u2 = User("Zorro", "33")
Ugly, but working solution#1:
class User(val name: String, val age: Int){
def this(line: String) = {
this(line.split(",")(0), line.split(",")(1).toInt)
}
}
Ugly, but working solution#2:
class User(val name: String, val age: Int)
object User {
def fromString(line: String) = {
val attrs = line.split(",")
new User(attrs(0), attrs(1).toInt)
}
}
Which can be used as:
val johny = User.fromString("johny,35")
You could use apply in place of fromString, but this will lead to a confusion (in one case you have to use new, in the other you have to drop it) so I prefer to use different name
Another ugly solution:
class User(line: String) {
def this(name: String, age: Int) = this(s"$name,$age")
val (name, age) = {
val Array(nameStr,ageStr) = line.split(",")
(nameStr,ageStr.toInt)
}
}
But using a method of the companion object is probably better.
let say I have 2 classes:
class A {
def sayHello(name: String) {
println("Hi " + name)
}
}
class B {
var methodMaps = Map[String, String => Unit]()
def registerMethod(methodName: String, method: String => Unit) {
methodMaps += (methodName -> method)
}
}
Okay, normally, I will call something like:
val b = new B
val a = new A
b.registerMethod("sayHello", a.sayHello)
But now I want to put the information into a config file, for example:
<method class="A" name="sayHello" />
Now, in the code it need to be something like this:
val b = new B
val className = readFromConfig()
val methodName = readFromConfig()
val aInstance = createInstanceFromReflection(className)
b.registerMethod(methodName, ...)
The problem is I don't know how to get the a.sayHello to pass to registerMethod, I can get the MethodMirror for sayHello, but how can I pass it to the registerMethod?
Thanks.
I guess that in your case, you need plain old reflection. From your example, in the expression b.registerMethod("sayHello", a.sayHello), the Scala compiler will lift the a.sayHello into a function.
When you are using dynamic class data from a file, the compiler can't help you; meaning that you need to do the work yourself. Following your own example, we should have:
val className = readFromConfig()
val methodName = readFromConfig()
val clazz = Class.forName(className)
val method = clazz.getMethods.find(x=>x.getName == "methodName" && x.getParameterTypes().length==1)
val aInstance = clazz.newInstance()
def invoke1[T,U](obj:Any, method:Method)(param:T):U = method.invoke(obj,Seq(param.asInstanceOf[java.lang.Object]):_*).asInstanceOf[U]
Now you should be able to register such construct in your map:
registerMethod(methodName, invoke1(aInstance,method) _ )
(*) This is not tested, but should be in the right direction, we use a similar construct in some part of our system.
I am quite new to the scala programming language, and I currently need to do the following. I have a signleton object like the following:
object MyObject extends Serializable {
val map: HashMap[String, Int] = null
val x: int = -1;
val foo: String = ""
}
Now i want to avoid to have to serialize each field of this object separately, thus I was considering writing the whole object to a file, and then, in the next execution of the program, read the file and initialize the singleton object from there. Is there any way to do this?
Basically what I want is when the serialization file doesn't exist, those variables to be initialized to new structures, while when it exists, the fields to be initialized from the ones on the file. But I want to avoid having to serialize/deserialize every field manually...
UPDATE:
I had to use a custom deserializer as presented here: https://issues.scala-lang.org/browse/SI-2403, since i had issues with a custom class I use inside the HashMap as values.
UPDATE2:
Here is the code I use to serialize:
val store = new ObjectOutputStream(new FileOutputStream(new File("foo")))
store.writeObject(MyData)
store.close
And the code to deserialize (in a different file):
#transient private lazy val loadedData: MyTrait = {
if(new File("foo").exists()) {
val in = new ObjectInputStream(new FileInputStream("foo")) {
override def resolveClass(desc: java.io.ObjectStreamClass): Class[_] = {
try { Class.forName(desc.getName, false, getClass.getClassLoader) }
catch { case ex: ClassNotFoundException => super.resolveClass(desc) }
}
}
val obj = in.readObject().asInstanceOf[MyTrait]
in.close
obj
}
else null
}
Thanks,
No needs to serialize an object with only immutable fields (because the compiler will do it for you...) I will assume that the object provides default values. Here is a way to do this:
Start by writing an trait with all the required fields:
trait MyTrait {
def map: HashMap[String, Int]
def x: Int
def foo: String
}
Then write an object with the defaults:
object MyDefaults extends MyTrait {
val map = Map()
val x = -1
val foo =
}
Finally write an implementation unserializing data if it exists:
object MyData extends MyTrait {
private lazy val loadedData: Option[MyTrait] = {
if( /* filename exists */ ) Some( /*unserialize filename as MyTrait*/)
else None
}
lazy val map = loadedData.getOrElse( MyDefault ).map
lazy val x = loadedData.getOrElse( MyDefault ).x
lazy val foo = loadedData.getOrElse( MyDefault ).foo
}