Suppose we have the following case classes:
abstract sealed class Tree
case class Leaf(i: Int) extends Tree
case class Node(left: Tree, right: Tree) extends Tree
Every time we call a case class constructor, a new object is created in memory. For instance, in the code below:
val a = Leaf(0)
val b = Leaf(0)
a and b point to distinct objects in memory:
a == b // true
a eq b // false
I would like to override the "apply" method of the case classes, to make them return a cached object, in case it already exists, so that, in the minimal example above, "a eq b" would return true.
I found these two related answers in Stackoverflow:
How to override apply in a case class companion (shows how to override "apply" method)
Why do each new instance of case classes evaluate lazy vals again in Scala? (shows a simple way to cache case class instances)
I am planning to implement my overriding "apply" method with caching in a way that combines the two approaches linked above. But I am wondering if there are alternative ways that I should consider. If you know any, could you please share your solution here?
Caching instances of case classes seems to be a very useful and natural thing to do to reduce memory consumption. And yet, the solution I am planning to implement (based on the two answers linked above) seems quite convoluted, requiring a lot of boilerplate code that will compromise the elegance and succinctness of case classes. Does anyone know if future versions of the Scala language might allow us to achieve case class instance caching by writing something simple like this:
abstract sealed class Tree
cached case class Leaf(i: Int) extends Tree
cached case class Node(left: Tree, right: Tree) extends Tree
??
Caching instances of case classes seems to be a very useful and natural thing to do to reduce memory consumption.
Note that this isn't even remotely an automatic improvement, and very much depends on usage pattern of the case class (not just yours, but anybody who uses your library):
You need to take into account the memory cache needs and inability to garbage collect instances referenced from the cache (note that using a WeakHashMap won't help: it requires "that value objects do not strongly refer to their own keys, either directly or indirectly").
If the keys are primitives (as in Leaf), they need to be boxed before lookup which will often already be a constructor call.
Lookup in a map is significantly slower than a trivial constructor call.
Escape analysis will often ensure the objects aren't actually constructed, while making sure your program works as if they were. Of course, caching will ensure that objects do escape.
But neglecting all that, you can write a macro annotation which will allow you #cached case class Leaf(i: Int) extends Tree and generate the code you want (or at least #cachedcase class; I am not sure if you'll be able to override apply otherwise). Because of the above I just wouldn't expect it to be a part of the language any time soon.
Related
I have been using sealed traits and case objects to define enumerated types in Scala and I recently came across another approach to extend the Enumeration class in Scala like this below:
object CertificateStatusEnum extends Enumeration {
val Accepted, SignatureError, CertificateExpired, CertificateRevoked, NoCertificateAvailable, CertChainError, ContractCancelled = Value
}
against doing something like this:
sealed trait CertificateStatus
object CertificateStatus extends {
case object Accepted extends CertificateStatus
case object SignatureError extends CertificateStatus
case object CertificateExpired extends CertificateStatus
case object CertificateRevoked extends CertificateStatus
case object NoCertificateAvailable extends CertificateStatus
case object CertChainError extends CertificateStatus
case object ContractCancelled extends CertificateStatus
}
What is considered a good approach?
They both get the job done for simple purposes, but in terms of best practice, the use of sealed traits + case objects is more flexible.
The story behind is that since Scala came with everything Java had, so Java had enumerations and Scala had to put them there for interoperability reasons. But Scala does not need them, because it supports ADTs (algebraic data types) so it can generate enumeration in a functional way like the one you just saw.
You'll encounter certain limitations with the normal Enumeration class:
the inability of the compiler to detect pattern matches exhaustively
it's actually harder to extend the elements to hold more data besides the String name and the Int id, because Value is final.
at runtime, all enums have the same type because of type erasure, so limited type level programming - for example, you can't have overloaded methods.
when you did object CertificateStatusEnum extends Enumeration your enumerations will not be defined as CertificateStatusEnum type, but as CertificateStatusEnum.Value - so you have to use some type aliases to fix that. The problem with this is the type of your companion will still be CertificateStatusEnum.Value.type so you'll end up doing multiple aliases to fix that, and have a rather confusing enumeration.
On the other hand, the algebraic data type comes as a type-safe alternative where you specify the shape of each element and to encode the enumeration you just need sum types which are expressed exactly using sealed traits (or abstract classes) and case objects.
These solve the limitations of the Enumeration class, but you'll encounter some other (minor) drawbacks, though these are not that limiting:
case objects won't have a default order - so if you need one, you'll have to add your id as an attribute in the sealed trait and provide an ordering method.
a somewhat problematic issue is that even though case objects are serializable, if you need to deserialize your enumeration, there is no easy way to deserialize a case object from its enumeration name. You will most probably need to write a custom deserializer.
you can't iterate over them by default as you could using Enumeration. But it's not a very common use case. Nevertheless, it can be easily achieved, e.g. :
object CertificateStatus extends {
val values: Seq[CertificateStatus] = Seq(
Accepted,
SignatureError,
CertificateExpired,
CertificateRevoked,
NoCertificateAvailable,
CertChainError,
ContractCancelled
)
// rest of the code
}
In practice, there's nothing that you can do with Enumeration that you can't do with sealed trait + case objects. So the former went out of people's preferences, in favor of the latter.
This comparison only concerns Scala 2.
In Scala 3, they unified ADTs and their generalized versions (GADTs) with enums under a new powerful syntax, effectively giving you everything you need. So you'll have every reason to use them. As Gael mentioned, they became first-class entities.
It depends on what you want from enum.
In the first case, you implicitly have an order on items (accessed by id property). Reordering has consequences.
I'd prefer 'case object', in some cases enum item could have extra info in the constructor (like, Color with RGB, not just name).
Also, I'd recommend https://index.scala-lang.org/mrvisser/sealerate or similar libraries. That allows iterating over all elements.
In the Scala spec, it's said that in a class template sc extends mt1, mt2, ..., mtn
Each trait reference mti must denote a trait. By contrast, the
superclass constructor sc normally refers to a class which is not a
trait. It is possible to write a list of parents that starts with a
trait reference, e.g. mt1 with …… with mtn. In that case the
list of parents is implicitly extended to include the supertype of
mt1 as first parent type. The new supertype must have at least one
constructor that does not take parameters. In the following, we will
always assume that this implicit extension has been performed, so that
the first parent class of a template is a regular superclass
constructor, not a trait reference.
If I understand it correctly, I think it means:
trait Base1 {}
trait Base2 {}
class Sub extends Base1 with Base2 {}
Will be implicitly extended to:
trait Base1 {}
trait Base2 {}
class Sub extends Object with Base1 with Base2 {}
My questions are:
Is my understanding correct?
Does this requirement (the first subclass in the parent list must be non-trait class) and the implicit extension only applies to class template (e.g. class Sub extends Mt1, Mt2) or also trait template (e.g. trait Sub extends Mt1, Mt2)?
Why this requirement and the implicit extension is necessary?
Disclaimer: I'm not and never was a member of the "Scala design committee" or anything like that, so the answer on the "why?" question is mostly speculation but I think a useful one.
Disclaimer #2: I've written this post over several hours and in several takes so it is probably not very consistent
Disclaimer #3 (a shameful self-promotion for the future readers): If you find this quite long answer useful, you might also take a look at my another long answer to another question by Lifu Huang on a similar topic.
Short answers
This is one of those complicated things for which I don't think there is a good short answer unless you already know what the answer is. Although my real answer will be long, here are my best short answers:
Why the first base class in parent list must be non-trait class?
Because there has to be only one non-trait base class and it makes thing easier if it is always the first
Is my understanding correct?
Yes, your implicit example is what will happen. However I'm not sure that it shows full understanding of the topic.
Does this requirement (the first subclass in the parent list must be non-trait class) and the implicit extension only applies to class template (e.g. class Sub extends Mt1, Mt2) or also trait template (e.g. trait Sub extends Mt1, Mt2)?
No, implicit extensions happens for traits as well. Actually how else you could expect Mt1 to have its own "supertype" to be promoted down to the class that extends it?
Actually here are two IMHO non-obvious examples proving this is true:
Example #1
trait TAny extends Any
trait TNo
// works
class CGood(val value: Int) extends AnyVal with TAny
// fails
// illegal inheritance; superclass AnyVal is not a subclass of the superclass Object
class CBad(val value: Int) extends AnyVal with TNo
This example fails because the spec says
The extends clause extends scsc with mt1mt1 with …… with mtnmtn can be omitted, in which case extends scala.AnyRef is assumed.
so TNo actually extends AnyRef which is incompatible with AnyVal.
Example #2
class CFirst
class CSecond extends CFirst
// did you know that traits can extend classes as well?
trait TFirst extends CFirst
trait TSecond extends CSecond
// works
class ChildGood extends TSecond with TFirst
// fails
// illegal inheritance; superclass CFirst is not a subclass of the superclass CSecond of the mixin trait TSecond
class ChildBad extends TFirst with TSecond
Again ChildBad fails because TSecond requires CSecond but TFirst only provides CFirst as the base class.
Why this requirement and the implicit extension is necessary?
There are three major reasons:
Compatibility with the main target platform (JVM)
Traits have "mixin" semantics: you have a class and you mix additional behavior in
Completeness, consistency and simplicity of the rest of the spec (e.g. of linearization rules). This might be restated as following: each class must declare 0 or 1 base non-trait classes and after compilation the target platform enforces that there will be exactly 1 non-trait base class. So it makes the rest of the spec easier if you just assume there is always exactly one base class. In such way you have to write this implicit extension rules only once rather than each time when the behavior depends on the base class.
Scala spec goals/intentions
I believe that when one reads a spec there are two different sets of questions:
What exactly is written? What is the meaning of the spec?
Why it is written so? What was the intention?
Actually I think in many cases #2 is more important than #1 but unfortunately specs rarely explicitly contain insights into that area. Anyway I will start with my speculations over #2: what were the intentions/goals/limitations of the classes system in Scala? The main high-level goal was to create a type system richer than the one in Java or .Net (which are quite similar) but that can be:
compiled back to an efficient code in those target platforms
allow reasonable two-way interaction between the Scala code and the "native" code in the target platforms
Side note: Support of the .Net was dropped years ago but it was one of the target platforms for years and this affected the design.
Single base class
Short summary: this section describes some reasons why Scala designers had a strong motivation to have the "exactly one base class" rule in the language.
A major problem with OO design and particularly inheritance is that AFAIK the question: "where exactly is the border between the "good and useful" practices and the "bad" ones?" is open. It means that each language must find out its own trade off between making impossible what is wrong and making possible (and easy) what is useful. Many believe that in C++, which obviously was a major inspiration for Java and .Net, that trade off is shifted too much into "allow everything even if it is potentially harmful" zone. It made many designers of newer languages to seek for more restricting trade off. Particularly both JVM and .Net platform enforce the rule that all types are split into "value types" (aka primitive types), "classes" and "interfaces" and each class, except the root class (java.lang.Object/System.Object), has exactly one "base class" and zero or more "base interfaces". This decision was a reaction to many issues of multiple inheritance including infamous "diamond problem" but actually many others as well.
Sidenote (about memory layout): Another major problem with multiple inheritance is objects layout in memory. Consider following ridiculous (and impossible in current Scala) example inspired by Achilles and the tortoise:
trait Achilles {
def getAchillesPos: Int
def stepAchilles(): Unit
}
class AchillesImpl(var achillesPos: Int) extends Achilles {
def getAchillesPos: Int = achillesPos
def stepAchilles(): Unit = {
achillesPos += 2
}
}
class TortoiseImpl(var tortoisePos: Int) {
def getTortoisePos: Int = tortoisePos
def stepTortoise(): Unit = {
tortoisePos += 1
}
}
class AchillesAndTortoise(handicap: Int) extends AchillesImpl(0) with TortoiseImpl(handicap) {
def catchTortoise(): Int = {
var time = 0
while (getAchillesPos < getTortoisePos) {
time += 1
stepAchilles()
stepTortoise()
}
time
}
}
The tricky part here is how to actually lay achillesPos and tortoisePos fields out in the memory (of the object). The issue is that you probably want to have only one compiled copy of all the methods in the memory and you want the code to be efficient. This means that getAchillesPos and stepAchilles should have know some fixed offset of the achillesPos regarding to the this pointer. Similarly getTortoisePos and stepTortoise should have know some fixed offset of the tortoisePos regarding to the this pointer. And all choices you have to achieve this goal don't look nice. For example:
You might decide that achillesPos is always first and tortoisePos is always second. But this means that in the instances of TortoiseImpl tortoisePos should also be the second field but there is nothing to fill the first field with so you waste some memory. Moreover if both AchillesImpl and TortoiseImpl come from pre-compiled libraries, you should have some way to move access to the fields in them as well.
You might try to "fix" this pointer on-the-fly when you call into TortoiseImpl (AFAIK this is the way C++ really works). This becomes especially funny when TortoiseImpl is an abstract class that is aware of the trait Achilles (but not the specific class AchillesImpl) via extends and tries to call back some methods from there via this or pass this to some method that takes Achilles as an argument so this has to be "fixed back". Note that this is not the same as the "diamond problem" because there is only one copy of all fields and implementations.
You might agree to have a unique copy of the methods compiled for each specific class that are aware of the specific layout. This is bad for memory usage and performance because it blows CPU caches and forces JIT to make independent optimizations for each.
You might say that no method except for getter and setter can have direct access to the fields and should use getters and setters instead. Or store all the fields in some kind of a dictionary which is effectively the same. This might be bad for performance (but this is the closest to what Scala does with mixin-traits).
In the actual Scala this issue does not exist because trait can't really declare any fields. When you declare val or var in a trait, you actually declare a getter (and a setter) method(s) that will be implemented by particular class that extends the trait and each class has full control over layout of the fields. And actually in terms of performance this most probably would work OK because JVM (JIT) can inline such a virtual call in many real-world scenarios.
End of the Sidenote
Another major point is interoperability with the target platform. Even if Scala somehow supported true multiple-inheritance so you can have a type that inherits from String with Date and that can be passed to both methods that expect String and that expect Date, how this would look like from the Java point of view? Also if the target platform enforces the rule that every class has to be an (indirect) sub-type of the same root class (Object), you can't work this around in your higher level language.
Traits and Mix-ins
Many think that "one class and many interfaces" trade-off that was made in Java and .Net is too restrictive. For example it makes it hard to share common default implementation of some of the interface methods between different classes. Actually over the time Java and .Net designers seem to come to the same conclusion and rolled out they own fixes for this kind of issues: Extension methods in .Net and then Default methods in Java. Scala designers added a feature called Mixins that was known to fare well in many practical cases. However unlike many other dynamic languages that has similar feature, Scala still had to meet the "exactly one base class" rule and other limitations of the target platform.
It is important to note that there are important scenarios when mixins are used in practice is to implement a variation of the Decorator or Adapter patterns both of which relies on the fact that you can restrict your base type to something more specific than Any or AnyRef. Prime example of such usage is the scala.collection package.
Scala syntax
So now you have following goals/restrictions:
Exactly one base class for each class
Ability to add logic to classes from mixins
Support of mixins with restricted base type
Classes from the target platform (Java) when seen from Scala are mapped to the Scala classes (because what else they can be mapped to?) and they come pre-compiled and we don't want to mess with their implementation
Other good qualities such as simplicity, type safety, determinism, etc.
If you want some kind of multiple inheritance support in your language, you need to develop conflict resolution rules: what happens when several base types provide some logic that would fit the same "slot" in your class. After prohibition of fields in traits we are left with the following "slots":
Base class in terms of the target platform
Constructors
Methods with the same name and signature
And possible conflict resolution strategies are:
Prohibit (fail compilation)
Decide which one wins and wipes others
Somehow chain them
Somehow preserve all with renaming. This is not really possible in JVM. For example in .Net see Explicit Interface Implementation
In a sense Scala uses all available (i.e. first 3) strategies but the high-level goal is: let's try to preserve as many logic as we can.
The most important part for this discussion is conflicts resolution for constructors and methods.
We want the rules to be the same for different slots because otherwise it is not clear how to achieve safety (if traits A and B both override methods foo and bar but resolution rules for foo and bar are different, invariants for A and B might easily be broken). Scala's approach is based on the class linearization. In short these is the way to "flatten" hierarchy of the base classes into a simple linear structure in some predictive way that is based on the idea that the lefter type in the with chain - the more "base" (higher in the inheritance) it is. After you do this, conflict resolution rule for methods becomes simple: you go through the list of the base types and chain behavior via super calls; if super is not called, you stop chaining. This produce quite predictable semantics that people can reason about.
Now assume you allow non-trait class to be not first. Consider following example:
class CBase {
def getValue = 2
}
trait TFirst extends CBase {
override def getValue = super.getValue + 1
}
trait TSecond extends CFirst {
override def getValue = super.getValue * 2
}
class CThird extends CBase with TSecond {
override def getValue = 100 - super.getValue
}
class Child extends TFirst with TSecond with CThird
In which order TFirst.getValue and TSecond.getValue should be called? Obviously CThird is already compiled and you can't change what the super for it is, so it has to be moved to the first position and there is already TSecond.getValue call inside it. But on the other hand this breaks the rule that everything on the left is base and everything on the right is child. The simplest way to not introduce such confusion is to enforce the rule that non-trait classes must go first.
The same logic applies if you just extend the previous example by substituting class CThird with a trait that extends it:
trait TFourth extends CThird
class AnotherChild extends TFirst with TSecond with TFourth
Again, the only non-trait class AnotherChild can extend is CThird and this again makes conflict resolution rules quite hard to reason about.
That's why Scala makes a rule much simpler: whatever provides the base class must come from the first position. And then it makes sense to extend the same rule upon the traits as well so if the first position is occupied by some trait - it also defines the base class.
1) Basically yes, your understanding is correct. Like in Java, every class inherits from java.lang.Object (AnyRef in Scala). So, since you are defining a concrete class, you will implicitly inherits from Object. If you check with the REPL, you got:
scala> trait Base1 {}
defined trait Base1
scala> trait Base2 {}
defined trait Base2
scala> class Sub extends Base1 with Base2 {}
defined class Sub
scala> classOf[Sub].getSuperclass
res0: Class[_ >: Sub] = class java.lang.Object
2) Yes, from the "Traits" paragraph in the specs, this applies also to them. In "Templates" paragraph we have:
The new supertype must have at least one constructor that does not take parameters
And then in "Traits" paragraph:
Unlike normal classes, traits cannot have constructor parameters. Furthermore, no constructor arguments are passed to the superclass of the trait. This is not necessary as traits are initialized after the superclass is initialized.
Assume a trait D defines some aspect of an instance x of type C (i.e. D is a base class of C). Then the actual supertype of D in x is the compound type consisting of all the base classes in L(C) that succeed D.
This is needed to define the base constructor with no-parameters.
3) As per answer (2), it's needed to define the base constructor
case classes have some nice percs, like copy, hashCode, toString, Pattern Matching. Why not make every Scala class a case class?
A case class is extremely good to hold complex values, like entity objects. They are thought precisely for that case, so they provide you methods that make sense precisely for this use case by synthesizing the methods you mentioned and furthermore making your class Serializable and creating a companion object with a "factory" method (other than the extractor for pattern matching).
The drawbacks are the following:
some of the properties that a case class has may not be interesting for the class you're creating: would you want an equals method on an object holding a database connection? Would it make sense for it to be Serializable? And if it did, would it be secure?
all these features are not free: they require the compiler to do some extra work and add to your final artifact size; why having these if you don't need the extra features a case class provides?
you cannot inherit from case class to another case class, which may go against how you are modeling your domain. Why? Short answer: equality. You can find a longer answer here.
Case classes have clear semantics -- data container (much better POJOs or ADT blocks, depends on your background).
Sometimes methods like copy or unapply can have confusing meaning -- e.g. if fields are mutable. Case classes are designed to be used in "idiomatic scala style", that might not be applicable everywhere.
Last but not the least -- technical disadvantages (more code in .class, more code to serialize, issues with inheritance).
I'm new to Scala and trying to understand how I should be modeling these objects.
The goal here is to have an object that will be stored into a database. The data to store will come from a POST. The post does not contain all of the data that will be persisted.
The OO side of me says to make a base class with the common fields. Extend it represent the data that is posted, and extend that to represent the object that is persisted. However, it seems that case classes are used for this sort of thing, and case class inheritance is discouraged/deprecated/buggy, so I'm not quite sure what I should be doing.
Also, the repetition feels very... wrong.
I am hoping someone a bit more experienced can offer some insight on how to approach this.
abstract class TestBase(val someField: String)
case class TestPost(override val someField: String) extends TestBase(someField)
case class Test(testId: String, override val someField: String) extends TestBase(someField)
Also, if I did continue with this approach, how would you copy fields from a TestPost instance to a Test instance?
although you are doing OOP and you can in Scala you should also be pressing on the functional way of doing things, functional programming states your objects should be immutable and case classes that's what they are for, they represent the value object pattern built right in the language. My advice would be to use composition instead.
case class A(field : Field)
case class B(a : A, moreFields : Fields)
I am not sure how you are trying to persist things to a database.
Ok, I'll explain why I ask this question. I begin to read Lift 2.2 source code these days.
It's good if you happened to read lift source code before.
In Lift, I found that, define inner class and inner trait are very heavily used.
object Menu has 2 inner traits and 4 inner classes. object Loc has 18 inner classes, 5 inner traits, 7 inner objects.
There're tons of codes write like this. I wanna to know why the author write like this.
Is it because it's the author's
personal taste or a powerful use of
language feature?
Is there any trade-off for this kind
of usage?
Before 2.8, you had to choose between packages and objects. The problem with packages is that they cannot contain methods or vals on their own. So you have to put all those inside another object, which can get awkward. Observe:
object Encrypt {
private val magicConstant = 0x12345678
def encryptInt(i: Int) = i ^ magicConstant
class EncryptIterator(ii: Iterator[Int]) extends Iterator[Int] {
def hasNext = ii.hasNext
def next = encryptInt(ii.next)
}
}
Now you can import Encrypt._ and gain access to the method encryptInt as well as the class EncryptIterator. Handy!
In contrast,
package encrypt {
object Encrypt {
private[encrypt] val magicConstant = 0x12345678
def encryptInt(i: Int) = i ^ magicConstant
}
class EncryptIterator(ii: Iterator[Int]) extends Iterator[Int] {
def hasNext = ii.hasNext
def next = Encrypt.encryptInt(ii.next)
}
}
It's not a huge difference, but it makes the user import both encrypt._ and encrypt.Encrypt._ or have to keep writing Encrypt.encryptInt over and over. Why not just use an object instead, as in the first pattern? (There's really no performance penalty, since nested classes aren't actually Java inner classes under the hood; they're just regular classes as far as the JVM knows, but with fancy names that tell you that they're nested.)
In 2.8, you can have your cake and eat it too: call the thing a package object, and the compiler will rewrite the code for you so it actually looks like the second example under the hood (except the object Encrypt is actually called package internally), but behaves like the first example in terms of namespace--the vals and defs are right there without needing an extra import.
Thus, projects that were started pre-2.8 often use objects to enclose lots of stuff as if they were a package. Post-2.8, one of the main motivations has been removed. (But just to be clear, using an object still doesn't hurt; it's more that it's conceptually misleading than that it has a negative impact on performance or whatnot.)
(P.S. Please, please don't try to actually encrypt anything that way except as an example or a joke!)
Putting classes, traits and objects in an object is sometimes required when you want to use abstract type variables, see e.g. http://programming-scala.labs.oreilly.com/ch12.html#_parameterized_types_vs_abstract_types
It can be both. Among other things, an instance of an inner class/trait has access to the variables of its parent. Inner classes have to be created with a parent instance, which is an instance of the outer type.
In other cases, it's probably just a way of grouping closely related things, as in your object example. Note that the trait LocParam is sealed, which means that all subclasses have to be in the same compile unit/file.
sblundy has a decent answer. One thing to add is that only with Scala 2.8 do you have package objects which let you group similar things in a package namespace without making a completely separate object. For that reason I will be updating my Lift Modules proposal to use a package object instead of a simple object.