NPE in using scala Array.fill - scala

I am reading Programming in Scala, Third Edition (Also present in Fourth Edition), by Lex Spoon; Bill Venners; Martin Odersky, and trying out examples along the way.
Following example form the book | Run in ScalaFiddle
abstract class Element {
def contents: Array[String]
val height = contents.length
val width = if (height == 0) 0 else contents(0).length
}
class UniformElement(
ch: Char,
override val width: Int,
override val height: Int
) extends Element {
private val line = ch.toString * width
def contents = Array.fill(height)(line)
}
val e: Element = new UniformElement('x', 2, 3)
gives java.lang.NullPointerException, when tried in REPL, or in Eclipse worksheet.
If I change
private val line = ch.toString * width
to
private def line = ch.toString * width
no error occurs.
Can someone explain, please?
I am using scala 2.11.8

The problem here is that contents is still not defined in the constructor, when you define line. If line is a val, it does not pick the overridden width, instead it uses the abstract one which in turn uses contents, which is still undefined and you get the NPE. You can see this by looking at the stacktrace and noting that the NPE is thrown by the definition of width in the abstract class.
When line is defined as a method, it does not execute until you call it and by that time contents will be fully defined, because it can call line (another method) which will be fully defined.
Run on ScalaFiddle
abstract class Element {
def contents: Array[String]
val height = contents.length
val width = if (height == 0) 0 else contents(0).length
}
class UniformElement(
ch: Char,
override val width: Int,
override val height: Int
) extends Element {
private def line: String = ch.toString * width
def contents = Array.fill(height)(line)
}
val e3: Element = new UniformElement('x', 2, 3)
Bottom line: you have a kind of "circular dependency" between line and contents.

Related

Modifying var type of data set in Spark using its own method

The following is a minimal example of the problem I am facing. I have an array that I want to modify in-place as it has about a million elements. the following code works except for the very last statement.
import spark.implicits._
case class Frame(x: Double, var y: Array[Double]) {
def total(): Double = {
return y.sum
}
def modifier(): Unit = {
for (i <- 0 until y.length) {
y(i) += 10
}
return
}
}
val df = Seq(
(1.0, Array(0, 2, 1)),
(8.0, Array(1, 2, 3)),
(9.0, Array(11, 21, 23))
).toDF("x", "y")
val ds = df.as[Frame]
ds.show
ds.map(_.total()).show // works
ds.map(_.modifier()).show // does not work
The error is as follows:
scala> ds.map(_.modifier()).show
<console>:50: error: Unable to find encoder for type Unit. An implicit Encoder[Unit] is needed to store Unit instances in a Dataset. Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._ Support for serializing other types will be added in future releases.
ds.map(_.modifier()).show
I cannot see the origin of the problem. I would be grateful for any help in fixing the bug.
Actually, this has nothing to do with 'var' or 'val', its about mutable data structures. The problem is that modifier returns Unit (e.g. nothing), so you cannot map on this results. You can run it using :
case class Frame(x: Double, var y: Array[Double]) {
def total(): Double = {
return y.sum
}
def modifier(): Frame = {
for (i <- 0 until y.length) {
y(i) += 10
}
return this
}
}
But I does not make much sense in my opinion, you should avoid mutable state. In addition, I would keep case classes simple (i.e. without logic) in spark, use them as data containers only. If you must increase every element by then, you can do it also like this:
case class Frame(x: Double, val y: Array[Double])
ds.map(fr => fr.copy(y = fr.y.map(_+10.0))).show

Why "Error:...diverging implicit expansion..." when I use SortedSet?

I'm working my way through Scala for the Impatient and am struggling to write tests for my solution to Chapter 10's 2nd exercise: Define a class OrderedPoint by mixing scala.math.Ordered[Point] into java.awt.Point.
Use lexicographic ordering, i.e. (x, y) < (x’, y’) if x < x’ or x = x’ and y < y’.
My class does what the book asked for, but I can't seem to get it to work with SortedSet.
The class definition is like...
package sfti.ch10
import java.awt.Point
class OrderedPoint(x: Int, y: Int) extends java.awt.Point(x, y) with scala.math.Ordered[Point] {
override def compare(that: Point): Int = {
if (getX == that.getX) {
getY.compare(that.getY)
} else {
getX.compare(that.getX)
}
}
}
The Spec definition is simple enough, but fails to compile once I put the SortedSet in there.
package stfi.ch10
import org.scalatest.FlatSpec
import sfti.ch10.OrderedPoint
class Ex02_OrderedPointSpec extends FlatSpec {
"An OrderedPoint" must "make java.awt.Points comparable" in {
val p_0_1 = new OrderedPoint(0, 1)
val p_1_0 = new OrderedPoint(1, 0)
val p_1_1 = new OrderedPoint(1, 1)
val p_0_0 = new OrderedPoint(0, 0)
assert(p_0_0.compare(p_0_0) == 0)
assert(p_0_0.compare(p_0_1) < 0)
assert(p_1_1.compare(p_1_0) > 0)
// this tips over the compiler
val sortedSet = scala.collection.SortedSet(p_1_1, p_0_0, p_1_0, p_0_1)
}
}
Error:(19, 53) diverging implicit expansion for type sfti.ch10.OrderedPoint => Comparable[sfti.ch10.OrderedPoint]
starting with method $conforms in object Predef
val s = scala.collection.SortedSet[OrderedPoint](p_1_1, p_0_0, p_1_0, p_0_1)
Why won't SortedSet respect my Ordered compare?
I already looked at What is a diverging implicit expansion error? and its like they are speaking in tongues. What do I have to do to my OrderedPoint or SortedSet invocation to get the SortedSet to use the compare method from OrderedPoint?
Change it to
with scala.math.Ordered[OrderedPoint]
Ordered is invariant so the type parameter needs to match exactly.

what's the difference between def and var/val for an anonymous function

I'm confused about anonymous function definitions as following:
var plusOne = (x:Int)=>x+1
// or val plusOne=(x:Int)=>x+1
println(plusOne(2))
Or
def plusOne = (x:Int)=>x+1
println(plusOne(2))
What's the difference please in var/val and def for a function name.
val declares an "immutable variable or rather symbol" that doesn't allow reassignment, right hand side of the assignment is evaluated immediately
var declares a "mutable variable" that allows reassignments later to the symbol, right hand side of the assignment is evaluated immediately just like val
def declares an "immutable symbol" that doesn't allow reassignment, right hand side is evaluated lazily, i.e. whenever that symbol is referenced later in the code
Example -
var plusOneVar = (x:Int)=>x+1
val plusOneVal = (x:Int)=>x+1
def plusOneDef = (x:Int)=>x+1
plusOneVar = (x:Int)=>x+2 // Reassignment to var is valid
plusOneVal = (x:Int)=>x+2 // Compile time error, reassignment to val
plusOneDef = (x:Int)=>x+2 // Compile time error, reassignment to val
Because you are looking at an example with functions, it is hard to understand. Let's try to understand it with simple variables.
var symbolVar = 100 // line 1
val symbolVal = symbolVar // line 2
def symbolDef = symbolVar // line 3
println(symbolVar) // prints 100
println(symbolVal) // prints 100
println(symbolDef) // prints 100 - no surprise yet
symbolVar = symbolVar + 1
println(symbolVal) // still prints 100 which was evaluated and assigned on line 2
println(symbolDef) // prints 101 as symbolDef is a def and it depends on symbolVar, line 3 is evaluated again
var plusOne can be reassigned. val plusOne cannot be reassigned. Both are evaluated once. def plusOne is evaluated each time it is called
Note also with val (or var) one instance of the function is created and is used for any number of invocations to that function, whereas with def a new instance of the function is created for each invocation. Yet, for
def f (i:Int) = i+1
f: (i: Int)Int
note
val g = f _
g: Int => Int = <function1>
It might be clear from the Class File Disassembler results using javap. Save the following code as Test.scala
class Test {
val fVal: Int => Int = x => x + 1
var fVar: Int => Int = x => x + 1
def fDef(x: Int): Int = { x + 1 }
}
and do scalac Test.scala; javap Test will show
Compiled from "Test.scala"
public class Test {
public scala.Function1<java.lang.Object, java.lang.Object> fVal();
public scala.Function1<java.lang.Object, java.lang.Object> fVar();
public void fVar_$eq(scala.Function1<java.lang.Object, java.lang.Object>);
public int fDef(int);
public Test();
}
As is shown in the results above, val and fVar are represented as methods that return a Function1 object. The difference is fVar has an additional "setter". While fDef is like normal Java method.

Problem with Scala's getter/setters

I'm currently learning Scala, and just discovered the way to create custom field getters/setters. I have a simple example working:
class Thing(private val a:Int){
override def toString = "Thing[" + a + "]"
private var _value = a
def value = _value
def value_= (newVal:Int) = _value = newVal
}
On the console I can do:
scala> var t = new Thing(2)
t: dylan.code.Thing = Thing[2]
scala> t.value
res1: Int = 2
scala> t.value = 3
scala> t.value
res2: Int = 3
Now I'm trying to bring this concept to a slightly more complicated example; I'll try to whittle the code down to what's relevant:
abstract class CellExpression[Type] extends Publisher[CellUpdateEvent[Type]] with Subscriber[CellUpdateEvent[Type], CellExpression[Type]]{
protected var cachedValue: Type = recalculateValue()
protected def recalculateValue(): Type
protected def changeValue(newValue: Type):Unit = {
val oldValue = value()
if(newValue != oldValue){
cachedValue = newValue
publish(new CellUpdateEvent(this, oldValue, newValue))
}
}
def value() = cachedValue
def notify(pub: CellExpression[Type], event: CellUpdateEvent[Type]) = changeValue(recalculateValue())
}
//....
class CellVariable[Type](private val initialValue:Type) extends CellExpression[Type]{
cachedValue = initialValue
protected def recalculateValue() = { cachedValue }
override def toString = "CellVariable[" + value + "]"
def value_= (newValue:Type) = {changeValue(newValue)}
}
As far as I can tell, I've done what I need to in order to be able to treate value as a field via its getter and setter. But when I try it out in the console, I get:
scala> var i = new CellVariable(2)
i: dylan.code.CellVariable[Int] = CellVariable[2]
scala> i.value = 3
<console>:11: error: reassignment to val
i.value = 3
^
What have I done wrong, and how can I fix it?
I actually stumbled onto the solution.
The line where I declare my value function: def value() = cachedValue is the culprit.
If I remove the parentheses to make the line def value = cachedValue everything seems to work as I expected.
You cannot change values in Scala. A value is assigned once and only once. If you want to do this then you need to use variables instead of values. In other words, change the declaration from val to var.
The problem is inside one of your class definitions and may be on a line without val because I believe that if you neglect to declare a name, then Scala assumes that it is a value and therefore immutable.
Not sure what you want getters and setters for though. Scala enables you to ignore all of that Java overhead.
It is probably the line that says cachedValue = initialValue because it is not declared with var anywhere in that class. The definition in the other class is a different name because it is in a different scope. You would have to say something like class.varname to change a variable defined in another class.

Interesting Scala typing solution, doesn't work in 2.7.7?

I'm trying to build some image algebra code that can work with images (basically a linear pixel buffer + dimensions) that have different types for the pixel. To get this to work, I've defined a parametrized Pixel trait with a few methods that should be able to get used with any Pixel subclass. (For now, I'm only interested in operations that work on the same Pixel type.) Here it is:
trait Pixel[T <: Pixel[T]] {
def div(v: Double): T
def div(v: T): T
}
Now I define a single Pixel type that has storage based on three doubles (basically RGB 0.0-1.0), I've called it TripleDoublePixel:
class TripleDoublePixel(v: Array[Double]) extends Pixel[TripleDoublePixel] {
var data: Array[Double] = v
def this() = this(Array(0.0, 0.0, 0.0))
def div(v: Double): TripleDoublePixel = {
new TripleDoublePixel(data.map(x => x / v))
}
def div(v: TripleDoublePixel): TripleDoublePixel = {
var tmp = new Array[Double](3)
tmp(0) = data(0) / v.data(0)
tmp(1) = data(1) / v.data(1)
tmp(2) = data(2) / v.data(2)
new TripleDoublePixel(tmp)
}
}
Then we define an Image using Pixels:
class Image[T](nsize: Array[Int], ndata: Array[T]) {
val size: Array[Int] = nsize
val data: Array[T] = ndata
def this(isize: Array[Int]) {
this(isize, new Array[T](isize(0) * isize(1)))
}
def this(img: Image[T]) {
this(img.size, new Array[T](img.size(0) * img.size(1)))
for (i <- 0 until img.data.size) {
data(i) = img.data(i)
}
}
}
(I think I should be able to do without the explicit declaration of size and data, and use just what has been named in the default constructor, but I haven't gotten that to work.)
Now I want to write code to use this, that doesn't have to know what type the pixels are. For example:
def idiv[T](a: Image[T], b: Image[T]) {
for (i <- 0 until a.data.size) {
a.data(i) = a.data(i).div(b.data(i))
}
}
Unfortunately, this doesn't compile:
(fragment of lindet-gen.scala):145:
error: value div is not a member of T
a.data(i) = a.data(i).div(b.data(i))
I was told in #scala that this worked for someone else, but that was on 2.8. I've tried to get 2.8-rc1 going, but the RC1 doesn't compile for me. Is there any way to get this to work in 2.7.7?
Your idiv function has to know it'll actually be working with pixels.
def idiv[T <: Pixel[T]](a: Image[T], b: Image[T]) {
for (i <- 0 until a.data.size) {
a.data(i) = a.data(i).div(b.data(i))
}
}
A plain type parameter T would define the function for all possible types T, which of course don't all support a div operation. So you'll have to put a generic constraint limiting the possible types to Pixels.
(Note that you could put this constraint on the Image class as well, assuming that an image out of something different than pixels doesn't make sense)