I have two case classes: addSmall and addBig.
addSmall contains only one field.
addBig contains several fields.
case class AddSmall(set: Set[Int] = Set.empty[Int]) {
def add(e: Int) = copy(set + e)
}
case class AddBig(set: Set[Int] = Set.empty[Int]) extends Foo {
def add(e: Int) = copy(set + e)
}
trait Foo {
val a = "a"; val b = "b"; val c = "c"; val d = "d"; val e = "e"
val f = "f"; val g = "g"; val h = "h"; val i = "i"; val j = "j"
val k = "k"; val l = "l"; val m = "m"; val n = "n"; val o = "o"
val p = "p"; val q = "q"; val r = "r"; val s = "s"; val t = "t"
}
A quick benchmark using JMH shows that copying addBig objects is way more exprensive even if i change only one field..
import java.util.concurrent.TimeUnit
import org.openjdk.jmh.annotations._
#State(Scope.Benchmark)
class AddState {
var elem: Int = _
var addSmall: AddSmall = _
var addBig: AddBig = _
#Setup(Level.Trial)
def setup(): Unit = {
addSmall = AddSmall()
addBig = AddBig()
elem = 1
}
}
#OutputTimeUnit(TimeUnit.MILLISECONDS)
#BenchmarkMode(Array(Mode.Throughput))
class SetBenchmark {
#Benchmark
def addSmall(state: AddState): AddSmall = {
state.addSmall.add(state.elem)
}
#Benchmark
def addBig(state: AddState): AddBig = {
state.addBig.add(state.elem)
}
}
And the results show that copying addBig is more than 10 times slower than copying addSmall!
> jmh:run -i 5 -wi 5 -f1 -t1
[info] Benchmark Mode Cnt Score Error Units
[info] LocalBenchmarks.Set.SetBenchmark.addBig thrpt 5 10732.569 ± 349.577 ops/ms
[info] LocalBenchmarks.Set.SetBenchmark.addSmall thrpt 5 126711.722 ± 10538.611 ops/ms
How come copying the object is much slower for addBig?
As far as i understand structural sharing, since all fields are immutable copying the object should be very efficient as it only needs to store the changes ("delta") which in this case is only the set s, and should thus give the same performance as addSmall.
EDIT: The same performance issue arises when the state is part of the case class.
case class AddBig(set: Set[Int] = Set.empty[Int], a: String = "a", b: String = "b", ...) {
def add(e: Int) = copy(set + e)
}
I guess, that this is because AddBig class extends Foo trait, which has all this String fields - a to t. It seems like, in result object they will be declared as regular fields, not the static fields if compare to Java, hence allocating memory for the object, might be the root cause of slower copy performance.
UPDATE:
In order to verify this theory you can try to use JOL (Java Object Layout) tool - openjdk.java.net/projects/code-tools/jol
Here is the simple code example:
import org.openjdk.jol.info.{ClassLayout, GraphLayout}
println(ClassLayout.parseClass(classOf[AddSmall]).toPrintable())
println(ClassLayout.parseClass(classOf[AddBig]).toPrintable())
println(GraphLayout.parseInstance(AddSmall()).toPrintable)
println(GraphLayout.parseInstance(AddBig()).toPrintable)
Which in my case produced next output (short version for answer readability):
xample.AddSmall object internals:
OFFSET SIZE TYPE DESCRIPTION VALUE
0 12 (object header) N/A
12 4 scala.collection.immutable.Set AddSmall.set N/A
Instance size: 16 bytes
Space losses: 0 bytes internal + 0 bytes external = 0 bytes total
example.AddBig object internals:
OFFSET SIZE TYPE DESCRIPTION VALUE
0 12 (object header) N/A
12 4 scala.collection.immutable.Set AddBig.set N/A
16 4 java.lang.String AddBig.a N/A
20 4 java.lang.String AddBig.b N/A
24 4 java.lang.String AddBig.c N/A
Instance size: 96 bytes
Space losses: 0 bytes internal + 0 bytes external = 0 bytes total
example.AddSmall#ea1a8d5d object externals:
ADDRESS SIZE TYPE PATH VALUE
770940b28 16 example.AddSmall (object)
770940b38 470456 (something else) (somewhere else) (something else)
7709b38f0 16 scala.collection.immutable.Set$EmptySet$ .set (object)
example.AddBig#480bdb19d object externals:
ADDRESS SIZE TYPE PATH VALUE
770143658 24 java.lang.String .h (object)
770143670 24 [C .h.value [h]
770143688 15536 (something else) (somewhere else) (something else)
770147338 24 java.lang.String .m (object)
770147350 24 [C .m.value [m]
770147368 1104264 (something else) (somewhere else) (something else)
770254cf0 24 java.lang.String .r (object)
770254d08 24 [C .r.value [r]
770254d20 7140768 (something else) (somewhere else) (something else)
7709242c0 24 java.lang.String .a (object)
So as you can see fields from parent trait become class fields as well, so will be copied along with the object.
Hope this helps!
Have you checked this question?
scala case class copy implementation
You can check compiler generated things to elaborate this. There's a probability that these vals became regular fields of case class and being copied each time class copied.
Your Foo trait adds 20 members to every subclass even though they are constants. This is going to use more memory and make copying the class slower.
Consider
1) Making them def rather than val so they are no longer data members
OR
2) Moving them into the companion class for the trait and accessing as Foo.a etc.
Related
I wrote this simple program in my attempt to learn how Cats Writer works
import cats.data.Writer
import cats.syntax.applicative._
import cats.syntax.writer._
import cats.instances.vector._
object WriterTest extends App {
type Logged2[A] = Writer[Vector[String], A]
Vector("started the program").tell
val output1 = calculate1(10)
val foo = new Foo()
val output2 = foo.calculate2(20)
val (log, sum) = (output1 + output2).pure[Logged2].run
println(log)
println(sum)
def calculate1(x : Int) : Int = {
Vector("came inside calculate1").tell
val output = 10 + x
Vector(s"Calculated value ${output}").tell
output
}
}
class Foo {
def calculate2(x: Int) : Int = {
Vector("came inside calculate 2").tell
val output = 10 + x
Vector(s"calculated ${output}").tell
output
}
}
The program works and the output is
> run-main WriterTest
[info] Compiling 1 Scala source to /Users/Cats/target/scala-2.11/classes...
[info] Running WriterTest
Vector()
50
[success] Total time: 1 s, completed Jan 21, 2017 8:14:19 AM
But why is the vector empty? Shouldn't it contain all the strings on which I used the "tell" method?
When you call tell on your Vectors, each time you create a Writer[Vector[String], Unit]. However, you never actually do anything with your Writers, you just discard them. Further, you call pure to create your final Writer, which simply creates a Writer with an empty Vector. You have to combine the writers together in a chain that carries your value and message around.
type Logged[A] = Writer[Vector[String], A]
val (log, sum) = (for {
_ <- Vector("started the program").tell
output1 <- calculate1(10)
foo = new Foo()
output2 <- foo.calculate2(20)
} yield output1 + output2).run
def calculate1(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate1").tell
output = 10 + x
_ <- Vector(s"Calculated value ${output}").tell
} yield output
class Foo {
def calculate2(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate2").tell
output = 10 + x
_ <- Vector(s"calculated ${output}").tell
} yield output
}
Note the use of for notation. The definition of calculate1 is really
def calculate1(x: Int): Logged[Int] = Vector("came inside calculate1").tell.flatMap { _ =>
val output = 10 + x
Vector(s"calculated ${output}").tell.map { _ => output }
}
flatMap is the monadic bind operation, which means it understands how to take two monadic values (in this case Writer) and join them together to get a new one. In this case, it makes a Writer containing the concatenation of the logs and the value of the one on the right.
Note how there are no side effects. There is no global state by which Writer can remember all your calls to tell. You instead make many Writers and join them together with flatMap to get one big one at the end.
The problem with your example code is that you're not using the result of the tell method.
If you take a look at its signature, you'll see this:
final class WriterIdSyntax[A](val a: A) extends AnyVal {
def tell: Writer[A, Unit] = Writer(a, ())
}
it is clear that tell returns a Writer[A, Unit] result which is immediately discarded because you didn't assign it to a value.
The proper way to use a Writer (and any monad in Scala) is through its flatMap method. It would look similar to this:
println(
Vector("started the program").tell.flatMap { _ =>
15.pure[Logged2].flatMap { i =>
Writer(Vector("ended program"), i)
}
}
)
The code above, when executed will give you this:
WriterT((Vector(started the program, ended program),15))
As you can see, both messages and the int are stored in the result.
Now this is a bit ugly, and Scala actually provides a better way to do this: for-comprehensions. For-comprehension are a bit of syntactic sugar that allows us to write the same code in this way:
println(
for {
_ <- Vector("started the program").tell
i <- 15.pure[Logged2]
_ <- Vector("ended program").tell
} yield i
)
Now going back to your example, what I would recommend is for you to change the return type of compute1 and compute2 to be Writer[Vector[String], Int] and then try to make your application compile using what I wrote above.
I'm confused about anonymous function definitions as following:
var plusOne = (x:Int)=>x+1
// or val plusOne=(x:Int)=>x+1
println(plusOne(2))
Or
def plusOne = (x:Int)=>x+1
println(plusOne(2))
What's the difference please in var/val and def for a function name.
val declares an "immutable variable or rather symbol" that doesn't allow reassignment, right hand side of the assignment is evaluated immediately
var declares a "mutable variable" that allows reassignments later to the symbol, right hand side of the assignment is evaluated immediately just like val
def declares an "immutable symbol" that doesn't allow reassignment, right hand side is evaluated lazily, i.e. whenever that symbol is referenced later in the code
Example -
var plusOneVar = (x:Int)=>x+1
val plusOneVal = (x:Int)=>x+1
def plusOneDef = (x:Int)=>x+1
plusOneVar = (x:Int)=>x+2 // Reassignment to var is valid
plusOneVal = (x:Int)=>x+2 // Compile time error, reassignment to val
plusOneDef = (x:Int)=>x+2 // Compile time error, reassignment to val
Because you are looking at an example with functions, it is hard to understand. Let's try to understand it with simple variables.
var symbolVar = 100 // line 1
val symbolVal = symbolVar // line 2
def symbolDef = symbolVar // line 3
println(symbolVar) // prints 100
println(symbolVal) // prints 100
println(symbolDef) // prints 100 - no surprise yet
symbolVar = symbolVar + 1
println(symbolVal) // still prints 100 which was evaluated and assigned on line 2
println(symbolDef) // prints 101 as symbolDef is a def and it depends on symbolVar, line 3 is evaluated again
var plusOne can be reassigned. val plusOne cannot be reassigned. Both are evaluated once. def plusOne is evaluated each time it is called
Note also with val (or var) one instance of the function is created and is used for any number of invocations to that function, whereas with def a new instance of the function is created for each invocation. Yet, for
def f (i:Int) = i+1
f: (i: Int)Int
note
val g = f _
g: Int => Int = <function1>
It might be clear from the Class File Disassembler results using javap. Save the following code as Test.scala
class Test {
val fVal: Int => Int = x => x + 1
var fVar: Int => Int = x => x + 1
def fDef(x: Int): Int = { x + 1 }
}
and do scalac Test.scala; javap Test will show
Compiled from "Test.scala"
public class Test {
public scala.Function1<java.lang.Object, java.lang.Object> fVal();
public scala.Function1<java.lang.Object, java.lang.Object> fVar();
public void fVar_$eq(scala.Function1<java.lang.Object, java.lang.Object>);
public int fDef(int);
public Test();
}
As is shown in the results above, val and fVar are represented as methods that return a Function1 object. The difference is fVar has an additional "setter". While fDef is like normal Java method.
Minimal Working Example (Scala 2.9.2):
object Main extends App {
class A {
var a=0
}
val b = Array.fill(2)(new A)
b(1).a = 9
println(b(0).a) //prints 0
println(b(1).a) //prints 9
val a = new A
val c = Array.fill(2)(a)
c(1).a = 9
println(c(0).a) //prints 9
println(c(1).a) //prints 9
}
A related question is "Is it the same with imported Java classes?"
How can I workaround, if I need to fill an Array inside a function with copies of an instance passed as argument?
[Regarding identical copies, it was worth to me to check out the easy cloning library.
Just adding the workaround to a function call, based on answers:
class A {
var a=0
}
def f(a: => A) { // "=>" added
val b = Array.fill(2)(a)
b(1).a=9
println(b(0).a) //prints 0
println(b(1).a) //prints 9
}
f(new A)
Another way is to declare a function, not a value def a = new A:
object Main extends App {
class A {
var a=0
}
val b = Array.fill(2)(new A)
b(1).a = 9
println(b(0).a) //prints 0
println(b(1).a) //prints 9
def a = new A
val c = Array.fill(2)(a)
c(1).a = 9
println(c(0).a) //prints 0
println(c(1).a) //prints 9
}
The fill method is defined with it's second parameter as "call-by-name". This means that the passed-in block is re-evaluated for every cell in the Array. See in the definition that the type of elem is => T, not simply T:
def fill[T: ClassManifest](n: Int)(elem: => T): Array[T]
So in your first version, the block new A is re-evaluated for each cell, meaning that each cell gets a fresh A object. In the second version, new A is called only once, and that object is placed into every cell.
You can actually see this if you run on the REPL:
scala> val b = Array.fill(2)(new A)
b: Array[A] = Array(A#2049bed2, A#498edd8d) // two different objects
scala> val c = Array.fill(2)(a)
c: Array[A] = Array(A#31e0c0b6, A#31e0c0b6) // the same object repeated
Having a look at the signature of fill
def fill[T: ClassManifest](n: Int)(elem: => T)
So it gets a call-by-name argument, which means new A will be executed every time you process a cell in the array.
One fills the array with a single instance of A, the other fills the array with a new instance of A.
I would like to be able to grow an Array-like structure up to a maximum size, after which the oldest (1st) element would be dropped off the structure every time a new element is added. I don't know what the best way to do this is, but one way would be to extend the ArrayBuffer class, and override the += operator so that if the maximum size has been reached, the first element is dropped every time a new one is added. I haven't figured out how to properly extend collections yet. What I have so far is:
class FiniteGrowableArray[A](maxLength:Int) extends scala.collection.mutable.ArrayBuffer {
override def +=(elem:A): <insert some return type here> = {
// append element
if(length > maxLength) remove(0)
<returned collection>
}
}
Can someone suggest a better path and/or help me along this one? NOTE: I will need to arbitrarily access elements within the structure multiple times in between the += operations.
Thanks
As others have discussed, you want a ring buffer. However, you also have to decide if you actually want all of the collections methods or not, and if so, what happens when you filter a ring buffer of maximum size N--does it keep its maximum size, or what?
If you're okay with merely being able to view your ring buffer as part of the collections hierarchy (but don't want to use collections efficiently to generate new ring buffers) then you can just:
class RingBuffer[T: ClassManifest](maxsize: Int) {
private[this] val buffer = new Array[T](maxsize+1)
private[this] var i0,i1 = 0
private[this] def i0up = { i0 += 1; if (i0>=buffer.length) i0 -= buffer.length }
private[this] def i0dn = { i0 -= 1; if (i0<0) i0 += buffer.length }
private[this] def i1up = { i1 += 1; if (i1>=buffer.length) i1 -= buffer.length }
private[this] def i1dn = { i1 -= 1; if (i1<0) i1 += buffer.length }
private[this] def me = this
def apply(i: Int) = {
val j = i+i0
if (j >= buffer.length) buffer(j-buffer.length) else buffer(j)
}
def size = if (i1<i0) buffer.length+i1-i0 else i1-i0
def :+(t: T) = {
buffer(i1) = t
i1up; if (i1==i0) i0up
this
}
def +:(t: T) = {
i0dn; if (i0==i1) i1dn
buffer(i0) = t
this
}
def popt = {
if (i1==i0) throw new java.util.NoSuchElementException
i1dn; buffer(i1)
}
def poph = {
if (i1==i0) throw new java.util.NoSuchElementException
val ans = buffer(i0); i0up; ans
}
def seqView = new IndexedSeq[T] {
def apply(i: Int) = me(i)
def length = me.size
}
}
Now you can use this easily directly, and you can jump out to IndexedSeq when needed:
val r = new RingBuffer[Int](4)
r :+ 7 :+ 9 :+ 2
r.seqView.mkString(" ") // Prints 7 9 2
r.popt // Returns 2
r.poph // Returns 7
r :+ 6 :+ 5 :+ 4 :+ 3
r.seqView.mkString(" ") // Prints 6 5 4 3 -- 7 fell off the end
0 +: 1 +: 2 +: r
r.seqView.mkString(" ") // Prints 0 1 2 6 -- added to front; 3,4,5 fell off
r.seqView.filter(_>1) // Vector(2,6)
and if you want to put things back into a ring buffer, you can
class RingBufferImplicit[T: ClassManifest](ts: Traversable[T]) {
def ring(maxsize: Int) = {
val rb = new RingBuffer[T](maxsize)
ts.foreach(rb :+ _)
rb
}
}
implicit def traversable2ringbuffer[T: ClassManifest](ts: Traversable[T]) = {
new RingBufferImplicit(ts)
}
and then you can do things like
val rr = List(1,2,3,4,5).ring(4)
rr.seqView.mkString(" ") // Prints 2,3,4,5
Scala doesn't have type-safe enums like Java has. Given a set of related constants, what would be the best way in Scala to represent those constants?
I must say that the example copied out of the Scala documentation by skaffman above is of limited utility in practice (you might as well use case objects).
In order to get something most closely resembling a Java Enum (i.e. with sensible toString and valueOf methods -- perhaps you are persisting the enum values to a database) you need to modify it a bit. If you had used skaffman's code:
WeekDay.valueOf("Sun") //returns None
WeekDay.Tue.toString //returns Weekday(2)
Whereas using the following declaration:
object WeekDay extends Enumeration {
type WeekDay = Value
val Mon = Value("Mon")
val Tue = Value("Tue")
... etc
}
You get more sensible results:
WeekDay.valueOf("Sun") //returns Some(Sun)
WeekDay.Tue.toString //returns Tue
http://www.scala-lang.org/docu/files/api/scala/Enumeration.html
Example use
object Main extends App {
object WeekDay extends Enumeration {
type WeekDay = Value
val Mon, Tue, Wed, Thu, Fri, Sat, Sun = Value
}
import WeekDay._
def isWorkingDay(d: WeekDay) = ! (d == Sat || d == Sun)
WeekDay.values filter isWorkingDay foreach println
}
There are many ways of doing.
1) Use symbols. It won't give you any type safety, though, aside from not accepting non-symbols where a symbol is expected. I'm only mentioning it here for completeness. Here's an example of usage:
def update(what: Symbol, where: Int, newValue: Array[Int]): MatrixInt =
what match {
case 'row => replaceRow(where, newValue)
case 'col | 'column => replaceCol(where, newValue)
case _ => throw new IllegalArgumentException
}
// At REPL:
scala> val a = unitMatrixInt(3)
a: teste7.MatrixInt =
/ 1 0 0 \
| 0 1 0 |
\ 0 0 1 /
scala> a('row, 1) = a.row(0)
res41: teste7.MatrixInt =
/ 1 0 0 \
| 1 0 0 |
\ 0 0 1 /
scala> a('column, 2) = a.row(0)
res42: teste7.MatrixInt =
/ 1 0 1 \
| 0 1 0 |
\ 0 0 0 /
2) Using class Enumeration:
object Dimension extends Enumeration {
type Dimension = Value
val Row, Column = Value
}
or, if you need to serialize or display it:
object Dimension extends Enumeration("Row", "Column") {
type Dimension = Value
val Row, Column = Value
}
This can be used like this:
def update(what: Dimension, where: Int, newValue: Array[Int]): MatrixInt =
what match {
case Row => replaceRow(where, newValue)
case Column => replaceCol(where, newValue)
}
// At REPL:
scala> a(Row, 2) = a.row(1)
<console>:13: error: not found: value Row
a(Row, 2) = a.row(1)
^
scala> a(Dimension.Row, 2) = a.row(1)
res1: teste.MatrixInt =
/ 1 0 0 \
| 0 1 0 |
\ 0 1 0 /
scala> import Dimension._
import Dimension._
scala> a(Row, 2) = a.row(1)
res2: teste.MatrixInt =
/ 1 0 0 \
| 0 1 0 |
\ 0 1 0 /
Unfortunately, it doesn't ensure that all matches are accounted for. If I forgot to put Row or Column in the match, the Scala compiler wouldn't have warned me. So it gives me some type safety, but not as much as can be gained.
3) Case objects:
sealed abstract class Dimension
case object Row extends Dimension
case object Column extends Dimension
Now, if I leave out a case on a match, the compiler will warn me:
MatrixInt.scala:70: warning: match is not exhaustive!
missing combination Column
what match {
^
one warning found
It's used pretty much the same way, and doesn't even need an import:
scala> val a = unitMatrixInt(3)
a: teste3.MatrixInt =
/ 1 0 0 \
| 0 1 0 |
\ 0 0 1 /
scala> a(Row,2) = a.row(0)
res15: teste3.MatrixInt =
/ 1 0 0 \
| 0 1 0 |
\ 1 0 0 /
You might wonder, then, why ever use an Enumeration instead of case objects. As a matter of fact, case objects do have advantages many times, such as here. The Enumeration class, though, has many Collection methods, such as elements (iterator on Scala 2.8), which returns an Iterator, map, flatMap, filter, etc.
This answer is essentially a selected parts from this article in my blog.
A slightly less verbose way of declaring named enumerations:
object WeekDay extends Enumeration("Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat") {
type WeekDay = Value
val Sun, Mon, Tue, Wed, Thu, Fri, Sat = Value
}
WeekDay.valueOf("Wed") // returns Some(Wed)
WeekDay.Fri.toString // returns Fri
Of course the problem here is that you will need to keep the ordering of the names and vals in sync which is easier to do if name and val are declared on the same line.
You can use a sealed abstract class instead of the enumeration, for example:
sealed abstract class Constraint(val name: String, val verifier: Int => Boolean)
case object NotTooBig extends Constraint("NotTooBig", (_ < 1000))
case object NonZero extends Constraint("NonZero", (_ != 0))
case class NotEquals(x: Int) extends Constraint("NotEquals " + x, (_ != x))
object Main {
def eval(ctrs: Seq[Constraint])(x: Int): Boolean =
(true /: ctrs){ case (accum, ctr) => accum && ctr.verifier(x) }
def main(args: Array[String]) {
val ctrs = NotTooBig :: NotEquals(5) :: Nil
val evaluate = eval(ctrs) _
println(evaluate(3000))
println(evaluate(3))
println(evaluate(5))
}
}
Starting from Scala 3, there is now enum keyword which can represent a set of constants (and other use cases)
enum Color:
case Red, Green, Blue
scala> val red = Color.Red
val red: Color = Red
scala> red.ordinal
val res0: Int = 0
In Scala it is very comfortable with https://github.com/lloydmeta/enumeratum
Project is really good with examples and documentation
Just this example from their docs should makes you interested in
import enumeratum._
sealed trait Greeting extends EnumEntry
object Greeting extends Enum[Greeting] {
/*
`findValues` is a protected method that invokes a macro to find all `Greeting` object declarations inside an `Enum`
You use it to implement the `val values` member
*/
val values = findValues
case object Hello extends Greeting
case object GoodBye extends Greeting
case object Hi extends Greeting
case object Bye extends Greeting
}
// Object Greeting has a `withName(name: String)` method
Greeting.withName("Hello")
// => res0: Greeting = Hello
Greeting.withName("Haro")
// => java.lang.IllegalArgumentException: Haro is not a member of Enum (Hello, GoodBye, Hi, Bye)
// A safer alternative would be to use `withNameOption(name: String)` method which returns an Option[Greeting]
Greeting.withNameOption("Hello")
// => res1: Option[Greeting] = Some(Hello)
Greeting.withNameOption("Haro")
// => res2: Option[Greeting] = None
// It is also possible to use strings case insensitively
Greeting.withNameInsensitive("HeLLo")
// => res3: Greeting = Hello
Greeting.withNameInsensitiveOption("HeLLo")
// => res4: Option[Greeting] = Some(Hello)
// Uppercase-only strings may also be used
Greeting.withNameUppercaseOnly("HELLO")
// => res5: Greeting = Hello
Greeting.withNameUppercaseOnlyOption("HeLLo")
// => res6: Option[Greeting] = None
// Similarly, lowercase-only strings may also be used
Greeting.withNameLowercaseOnly("hello")
// => res7: Greeting = Hello
Greeting.withNameLowercaseOnlyOption("hello")
// => res8: Option[Greeting] = Some(Hello)