I have a strange thing in my code and cannot figure the cause. Maybe is it a Scala feature that I'm not aware of.
case class Page(index:Int) {
private var lines = Map[Int, Line]()
def set(index:Int, line:Line):Page = {
val copy = this.copy()
copy.lines = this.lines + (index -> line) // <-- My problem
copy
}
}
My problem is that the line copy.lines = this.lines + (index -> line) raise a NullPointerException because this.lines is null. However copy.lines is not null.
Do you know why the lines map is initialized when I copy a Page but not when I create a new one ?
Thanks
Try replacing the lines variable with a immutable variable
case class Page(index:Int, lines: Map[Int, Line] = Map()) {
def set(index:Int, line:Line): Page = {
copy(lines = lines ++ Map(index -> line))
}
}
You can try it out on http://www.scalakata.com/
So,
The problem was due to my persitence layer. The system I'm using use java beans conventions and so it ignore the private var lines. Then when I restore a Page it set it to null..
One of the solutions, because I have forgotten that we can have private val in a case class was to create another class who expose the lines and map/unmap it in my persistence layer.
Hopefully, #Rovak remind me that I can also do case classs Page(index: Int, private val lines:Map[Int, Line]=Map()). So that I can remove all those ugly mapping lines and benefit on a cleaner approach. Another advantage is that the set method is more clean and don't have to modify a case class.
Related
Consider the following case (this is a simplified version of what I have. The numberOfScenarios is the most important variable here: I usually use a hardcoded number instead of it, but I'm trying to see if it's possible to calculate the value):
object ScenarioHelpers {
val identifierList = (1 to Scenarios.numberOfScenarios).toArray
val concurrentIdentifierQueue = new ConcurrentLinkedQueue[Int](identifierList.toSeq)
}
abstract class AbstractScenario {
val identifier = ScenarioHelpers.concurrentIdentifierQueue.poll()
}
object Test1 extends AbstractScenario {
val scenario1 = scenario("test scenario 1").exec(/..steps../)
}
object Test2 extends AbstractScenario {
val scenario2 = scenario("test scenario 2").exec(/..steps../)
}
object Scenarios {
val scenarios = List(Test1.scenario1, Test2.scenario2)
val numberOfScenarios = scenarios.length
}
object TestPreparation {
val feeder = ScenarioHelpers.identifierList.map(n => Map("counter" -> n))
val prepScenario = scenario("test preparation")
.feed(feeder)
.exec(/..steps../)
}
Not sure if it matters, but the simulation starts with executing the TestPreparation.prepScenario.
I see that this code contains a circular dependency which makes this case impossible in and of itself. But I get a NullPointerException on the line in AbstractScenario where identifier is being initialized.
I don't fully understand all this, but I think it has something to do with the vals being simply declared at first and the initialization does not happen until later. So when identifier is being initialized, the concurrentIdentifierQueue is not yet initialized and is therefore null.
I'm just trying to understand the reasons behind the NullPointerException and also if there's any way to get this working with minimal changes? Thank you.
NPEs during trait initialization is a very common problem.
The most robust way to resolve it is avoiding implementation inheritance at all.
if it is not possible for some reasons you can mark problematic fields lazy val or def instead of val.
You answered that yourself:
I see that this code contains a circular dependency which makes this case impossible in and of itself. But I get a NullPointerException on the line in AbstractScenario where identifier is being initialized.
val feeder = ScenarioHelpers.identifierList... calls ScenarioHelpers initialization
val identifierList = (1 to Scenarios.numberOfScenarios).toArray calls Scenarios initialization
val scenarios = List(Test1.scenario1, Test2.scenario2) calls Test1 inicialization including AbstractScenario
Here val identifier = ScenarioHelpers.concurrentIdentifierQueue.poll() ScenarioHelpers is still initializing and identifierList is null.
You have to get numberOfScenarios in noncyclic way. Personally I would remove identifierList and assign identifier other way - incrementing counter or so.
For scala case class with number of parameters (21!!)
e.g. case class Car(type: String, brand: String, door: Int ....)
where type = jeep, brand = toyota, door = 4 ....etc
And there is a copy method which allow override with named parameter: Car.copy(brand = Kia)
where would become type = jeep, brand = Kia, door = 2...etc
My question is, is there anyway I can provide the named parameter dynamically?
def copyCar(key: String, name: String) = {
Car.copy("key" = "name") // this is something I make up and want to see if would work
}
Is scala reflection library could provide a help here?
The reason I am using copy method is that I don't want to repeat the 21 parameters assignment every time when I create a case class which only have 1 or 2 parameter changed.
Many Thanks!
FWIW, I've just implemented a Java reflection version: CaseClassCopy.scala. I tried a TypeTag version but it wasn't that useful; TypeTag was too restrictive for this purpose.
def copy(o: AnyRef, vals: (String, Any)*) = {
val copier = new Copier(o.getClass)
copier(o, vals: _*)
}
/**
* Utility class for providing copying of a designated case class with minimal overhead.
*/
class Copier(cls: Class[_]) {
private val ctor = cls.getConstructors.apply(0)
private val getters = cls.getDeclaredFields
.filter {
f =>
val m = f.getModifiers
Modifier.isPrivate(m) && Modifier.isFinal(m) && !Modifier.isStatic(m)
}
.take(ctor.getParameterTypes.size)
.map(f => cls.getMethod(f.getName))
/**
* A reflective, non-generic version of case class copying.
*/
def apply[T](o: T, vals: (String, Any)*): T = {
val byIx = vals.map {
case (name, value) =>
val ix = getters.indexWhere(_.getName == name)
if (ix < 0) throw new IllegalArgumentException("Unknown field: " + name)
(ix, value.asInstanceOf[Object])
}.toMap
val args = (0 until getters.size).map {
i =>
byIx.get(i)
.getOrElse(getters(i).invoke(o))
}
ctor.newInstance(args: _*).asInstanceOf[T]
}
}
It is not possible using case classes.
Copy method generated at compile time and named parameters handled on compile time to. There is no possibility to do it ar runtime.
Dynamic may help to solve your issue: http://hacking-scala.tumblr.com/post/49051516694/introduction-to-type-dynamic
Yes, you would need to use reflection to do that.
It is a bit involved, because copy is a synthetic method and you'll have to invoke the getters for all fields except the one you want to replace.
To give you an idea, the copy method in this class does exactly that, except using an argument index instead of name. It calls the companion object's apply method, but the effect is the same.
I'm a bit confused - how is the following not what you need?
car: Car = ... // Retrieve an instance of Car somehow.
car.copy(type = "jeep") // Copied instance, only the type has been changed.
car.copy(door = 4) // Copied instance, only the number of doors has changed.
// ...
Is it because you have a lot of parameters for the initial instance creation? In that case, can you not use default values?
case class Car(type: String = "Jeep", door: Int = 4, ...)
You seem to know about both these features and feel that they don't fit your need - could you explain why?
I would like to access csv files in scala in a strongly typed manner. For example, as I read each line of the csv, it is automatically parsed and represented as a tuple with the appropriate types. I could specify the types beforehand in some sort of schema that is passed to the parser. Are there any libraries that exist for doing this? If not, how could I go about implementing this functionality on my own?
product-collections appears to be a good fit for your requirements:
scala> val data = CsvParser[String,Int,Double].parseFile("sample.csv")
data: com.github.marklister.collections.immutable.CollSeq3[String,Int,Double] =
CollSeq((Jan,10,22.33),
(Feb,20,44.2),
(Mar,25,55.1))
product-collections uses opencsv under the hood.
A CollSeq3 is an IndexedSeq[Product3[T1,T2,T3]] and also a Product3[Seq[T1],Seq[T2],Seq[T3]] with a little sugar. I am the author of product-collections.
Here's a link to the io page of the scaladoc
Product3 is essentially a tuple of arity 3.
If your content has double-quotes to enclose other double quotes, commas and newlines, I would definitely use a library like opencsv that deals properly with special characters. Typically you end up with Iterator[Array[String]]. Then you use Iterator.map or collect to transform each Array[String] into your tuples dealing with type conversions errors there. If you need to do process the input without loading all in memory, you then keep working with the iterator, otherwise you can convert to a Vector or List and close the input stream.
So it may look like this:
val reader = new CSVReader(new FileReader(filename))
val iter = reader.iterator()
val typed = iter collect {
case Array(double, int, string) => (double.toDouble, int.toInt, string)
}
// do more work with typed
// close reader in a finally block
Depending on how you need to deal with errors, you can return Left for errors and Right for success tuples to separate the errors from the correct rows. Also, I sometimes wrap of all this using scala-arm for closing resources. So my data maybe wrapped into the resource.ManagedResource monad so that I can use input coming from multiple files.
Finally, although you want to work with tuples, I have found that it is usually clearer to have a case class that is appropriate for the problem and then write a method that creates that case class object from an Array[String].
You can use kantan.csv, which is designed with precisely that purpose in mind.
Imagine you have the following input:
1,Foo,2.0
2,Bar,false
Using kantan.csv, you could write the following code to parse it:
import kantan.csv.ops._
new File("path/to/csv").asUnsafeCsvRows[(Int, String, Either[Float, Boolean])](',', false)
And you'd get an iterator where each entry is of type (Int, String, Either[Float, Boolean]). Note the bit where the last column in your CSV can be of more than one type, but this is conveniently handled with Either.
This is all done in an entirely type safe way, no reflection involved, validated at compile time.
Depending on how far down the rabbit hole you're willing to go, there's also a shapeless module for automated case class and sum type derivation, as well as support for scalaz and cats types and type classes.
Full disclosure: I'm the author of kantan.csv.
I've created a strongly-typed CSV helper for Scala, called object-csv. It is not a fully fledged framework, but it can be adjusted easily. With it you can do this:
val peopleFromCSV = readCSV[Person](fileName)
Where Person is case class, defined like this:
case class Person (name: String, age: Int, salary: Double, isNice:Boolean = false)
Read more about it in GitHub, or in my blog post about it.
Edit: as pointed out in a comment, kantan.csv (see other answer) is probably the best as of the time I made this edit (2020-09-03).
This is made more complicated than it ought to because of the nontrivial quoting rules for CSV. You probably should start with an existing CSV parser, e.g. OpenCSV or one of the projects called scala-csv. (There are at least three.)
Then you end up with some sort of collection of collections of strings. If you don't need to read massive CSV files quickly, you can just try to parse each line into each of your types and take the first one that doesn't throw an exception. For example,
import scala.util._
case class Person(first: String, last: String, age: Int) {}
object Person {
def fromCSV(xs: Seq[String]) = Try(xs match {
case s0 +: s1 +: s2 +: more => new Person(s0, s1, s2.toInt)
})
}
If you do need to parse them fairly quickly and you don't know what might be there, you should probably use some sort of matching (e.g. regexes) on the individual items. Either way, if there's any chance of error you probably want to use Try or Option or somesuch to package errors.
I built my own idea to strongly typecast the final product, more than the reading stage itself..which as pointed out might be better handled as stage one with something like Apache CSV, and stage 2 could be what i've done. Here's the code you are welcome to it. The idea is to typecast the CSVReader[T] with type T .. upon construction, you must supply the reader with a Factor object of Type[T] as well. The idea here is that the class itself (or in my example a helper object) decides the construction detail and thus decouples this from the actual reading. You could use Implicit objects to pass the helper around but I've not done that here. The only downside is that each row of the CSV must be of the same class type, but you could expand this concept as needed.
class CsvReader/**
* #param fname
* #param hasHeader : ignore header row
* #param delim : "\t" , etc
*/
[T] ( factory:CsvFactory[T], fname:String, delim:String) {
private val f = Source.fromFile(fname)
private var lines = f.getLines //iterator
private var fileClosed = false
if (lines.hasNext) lines = lines.dropWhile(_.trim.isEmpty) //skip white space
def hasNext = (if (fileClosed) false else lines.hasNext)
lines = lines.drop(1) //drop header , assumed to exist
/**
* also closes the file
* #return the line
*/
def nextRow ():String = { //public version
val ans = lines.next
if (ans.isEmpty) throw new Exception("Error in CSV, reading past end "+fname)
if (lines.hasNext) lines = lines.dropWhile(_.trim.isEmpty) else close()
ans
}
//def nextObj[T](factory:CsvFactory[T]): T = past version
def nextObj(): T = { //public version
val s = nextRow()
val a = s.split(delim)
factory makeObj a
}
def allObj() : Seq[T] = {
val ans = scala.collection.mutable.Buffer[T]()
while (hasNext) ans+=nextObj()
ans.toList
}
def close() = {
f.close;
fileClosed = true
}
} //class
next the example Helper Factory and example "Main"
trait CsvFactory[T] { //handles all serial controls (in and out)
def makeObj(a:Seq[String]):T //for reading
def makeRow(obj:T):Seq[String]//the factory basically just passes this duty
def header:Seq[String] //must define headers for writing
}
/**
* Each class implements this as needed, so the object can be serialized by the writer
*/
case class TestRecord(val name:String, val addr:String, val zip:Int) {
def toRow():Seq[String] = List(name,addr,zip.toString) //handle conversion to CSV
}
object TestFactory extends CsvFactory[TestRecord] {
def makeObj (a:Seq[String]):TestRecord = new TestRecord(a(0),a(1),a(2).toDouble.toInt)
def header = List("name","addr","zip")
def makeRow(o:TestRecord):Seq[String] = {
o.toRow.map(_.toUpperCase())
}
}
object CsvSerial {
def main(args: Array[String]): Unit = {
val whereami = System.getProperty("user.dir")
println("Begin CSV test in "+whereami)
val reader = new CsvReader(TestFactory,"TestCsv.txt","\t")
val all = reader.allObj() //read the CSV info a file
sd.p(all)
reader.close
val writer = new CsvWriter(TestFactory,"TestOut.txt", "\t")
for (x<-all) writer.printObj(x)
writer.close
} //main
}
Example CSV (tab seperated.. might need to repair if you copy from an editor)
Name Addr Zip "Sanders, Dante R." 4823 Nibh Av. 60797.00 "Decker, Caryn G." 994-2552 Ac Rd. 70755.00 "Wilkerson, Jolene Z." 3613 Ultrices. St. 62168.00 "Gonzales, Elizabeth W." "P.O. Box 409, 2319 Cursus. Rd." 72909.00 "Rodriguez, Abbot O." Ap #541-9695 Fusce Street 23495.00 "Larson, Martin L." 113-3963 Cras Av. 36008.00 "Cannon, Zia U." 549-2083 Libero Avenue 91524.00 "Cook, Amena B." Ap
#668-5982 Massa Ave 69205.00
And finally the writer (notice the factory methods require this as well with "makerow"
import java.io._
class CsvWriter[T] (factory:CsvFactory[T], fname:String, delim:String, append:Boolean = false) {
private val out = new PrintWriter(new BufferedWriter(new FileWriter(fname,append)));
if (!append) out.println(factory.header mkString delim )
def flush() = out.flush()
def println(s:String) = out.println(s)
def printObj(obj:T) = println( factory makeRow(obj) mkString(delim) )
def printAll(objects:Seq[T]) = objects.foreach(printObj(_))
def close() = out.close
}
If you know the the # and types of fields, maybe like this?:
case class Friend(id: Int, name: String) // 1, Fred
val friends = scala.io.Source.fromFile("friends.csv").getLines.map { line =>
val fields = line.split(',')
Friend(fields(0).toInt, fields(1))
}
I know there are multiple questions addressing related problems, but I'm not sure it does attack exactly what I'm looking for. I'm still new to Scala, after several years of Java development. I'm looking for the best way to test if an object has been initialized, and if not, initialize it then. For example, in Java:
private MyObject myObj = null;
and at some point in the future:
public void initMyObj(){
if (myObj == null){
myObj = new MyObj();
}
// do something with myObj
}
After this, I might reassign myObj to a different object, but it is unlikely. In Scala, I have this:
class Test {
var myObj: MyObj = _
}
I've read that I could use Option instead, something like:
var myObj = None : Option[MyObj]
and then my check:
myObj match {
case None => ...
case Some(value) => ...
}
but it feels ackward to use this pattern when I might not make this kind of check anywhere else at any other time - though being so new to Scala, I might be wrong. Is this the best way to achieve what I want or is there any other option not involving Option?
It is not generally ideal practice in Scala to leave partially-constructed objects lying around. You would normally rethink how your objects were getting instantiated to see if you can't use a different pattern that is less fragile. For example, instead of setting uninitialized variables in methods:
class Foo { var a: String = null; var b: String = null }
def initFooA(s: String, f: Foo) { if (f.a == null) f.a = s }
def initFooB(s: String, f: Foo) { if (f.b == null) f.b = s }
f
initFooA("salmon", f)
// Do stuff
initFooB("herring", f)
you would attempt to restructure your code to generate the values you need on demand, and delay the instantiation of foo until then:
case class Bar(a: String, b: String) {}
def initBarA(s: String) = s
def initBarB(s: String) = s
val iba = initBarA("halibut")
// Do stuff
val ibb = initBarB("cod")
Bar(iba, ibb)
Because Scala has easy access to tuples (and type inference), this can be a lot less painful than in Java.
Another thing you can do is defer the late initialization to someone else.
case class Baz(a: String)(bMaker: => String) {
lazy val b = bMaker
}
Now you pass in something that will make parameter b, and arrange for it to handle any late initialization stuff that needs to be handled. This doesn't always avoid needing to set vars, but it can help push it out of your class code into your initialization logic (which is usually a better place for it).
Doing this with vars is a little less straightforward. Realistically, you're probably best off just devoting a class to it e.g. by:
class LazyVar[A](initial: => A) {
private[this] var loaded = false
private[this] var variable: A = _
def apply() = { if (!loaded) { loaded = true; variable = initial }; variable }
def update(a: A) { loaded = true; variable = a }
}
where you then (sadly) have to use () on every read and write.
scala> val lv = new LazyVar({ println("Hi!"); 5 })
lv: LazyVar[Int] = LazyVar#2626ea08
scala> lv()
Hi!
res2: Int = 5
scala> lv() = 7
scala> lv()
res4: Int = 7
Then you use an instance of this class instead of the actual var and pass through the lazy initializer. (lazy val is very much like this under the hood; the compiler just protects you from noticing.)
Finally, if you want to have a fully-functional object that is occasionally missing a value, var x: Option[X] is the construct you want to use; if you can't find a way around the standard Java creation patterns (and you don't want to try something more exotic like objects that create each other with more and more information, either because performance is critical and you can't afford it, or you dislike writing that much boilerplate to allow type-checking to verify that your object is properly created) but you otherwise want to use it, var x: X = null is what I'd choose, not _. If X is a primitive, you probably need to choose the correct value wisely anyway (for example, Double.NaN instead of 0.0, -1 rather than 0 for Int) to indicate I-am-not-initialized. If it's generic code and you want Any instead of AnyRef, asInstanceOf-ing back and forth between Any and AnyRef is probably the best way out of poorly typechecked situation (assuming you really, really can't use Option, which at that point is much clearer).
Maybe a lazy variable is what you need.
lazy val myObj: MyObj = //here you put the object creation code
In this way the object creation is postponed to the first time the code tries to access it.
From an example in book "Begining in Scala", the script is:
import scala.collection.mutable.Map
object ChecksumAccumulator {
private val cache=Map[String,Int]()
def calculate(s: String):Int =
if(cache.contains(s))
cache(s)
else{
val acc = new ChecksumAccumulator
for(c <- s)
acc.add(c.toByte)
val cs=acc.checksum
cache+= (s -> cs)
cs
}
}
but, when I tried to compile this file
$scalac ChecksumAccumulator.scala, then generate an error, "not found: type ChecksumAccumulator val acc = new ChecksumAccumulator", any suggest?
Thanks,
'object' keyword defines a singleton object, not a class. So you can't new an object, the 'new' keyword requires a class.
check this Difference between object and class in Scala
You probably left some code out that looks like
class ChecksumAccumulator {
//...
}
The other answers are correct in saying what the problem is, but not really helping you understand why the example from the book is apparently not correct.
However, if you look at the Artima site, you will find the example is in a file here
Your code is an incomplete fragment. The file also includes these lines
// In file ChecksumAccumulator.scala
class ChecksumAccumulator {
private var sum = 0
def add(b: Byte) { sum += b }
def checksum(): Int = ~(sum & 0xFF) + 1
}
... without which you will get the error you had.
your issue is here
val acc = new ChecksumAccumulator
you cannot use new keyword with the object.
objects cannot be re-instantiated. You always have single instance of an object in scala. This is similar to static members in java.
Your code, probably meant as a companion object. That's kinda factories in imperative languages.
Basicaly, you have object and class pair. Object (singleton in imperative langs) can't be instantiated multiple times, as people here already noted, and usually used to define some static logic. In fact, there is only one instantiation -- when you call him for the first time. But object can have compaion -- regular class, and, as I think you've missed definition of that regular class, so object can't see anyone else, but itself.
The solution is to define that class, or to omit new (but i think that would be logicaly wrong).