I am trying to initialize a derived class from text file input. A simple example of what I am trying to do:
file.txt:
1
2
main.py:
class Base:
def __init__(self, val1):
self.val1 = val1
def input_from_text(cls, init_deque):
#return cls(init_deque.popleft())
class Derived(Base):
def __init__(self, val1, val2):
Base.__init__(self, val1)
self.val2 = val2
def input_from_text(cls, init_deque):
#initialize base and derived here and return derived
def main(argv=None):
initialized_derived = Derived.input_from_text(deque(open("file.txt")))
assert initialized_derived.val1 is 1
assert initialized_derived.val2 is 2
Is there a good way to do this? Basically looking for something similar to what you would find in C++ with:
//calls operator>>(Base) then operator>>(Derived)
cin >> initialized_derived;
This way each class is nicely encapsulated and the base/derived classes don't need to know anything about each other (excepting __init__ which knows the number of args base takes).
Just realized that I was going about this the wrong way. Simple fix is to do something like:
class Base:
def __init__(self):
pass
def input_from_text(self, init_deque):
self.val1 = init_deque.popleft()
class Derived(Base):
def __init__(self):
Base.__init__(self)
def input_from_text(self, init_deque):
Base.input_from_text(self, init_deque)
self.val2 = init_deque.popleft()
Related
Consider the following case:
trait A {
protected val mydata = ???
def f(args) = ??? //uses mydata
}
class B
class C
class D(arg1: String) extends B with A {
override val mydata = ??? /// some calculation based on arg1
}
class E(arg1: String) extends C with A{
override val mydata = ??? /// some calculation based on arg1
}
A must be a trait as it is used by different unrelated classes. The problem is how to implement the definition of mydata.
The standard way (suggested in many places would be to define mydata as def and override it in the children. However, if f assumes mydata never changes then it can cause issues when some child extends with a function which changes between calls instead of with a val.
Another way would be to do:
trait A {
protected val mydata = g
protected def g()
}
The problem with this (beyond adding another function) is that if g depends on construction variables in the child then these must become members of the child (which can be a problem for example if the data is large and given in the construction):
class D(arg1: Seq[String]) {
def g() = ??? // some operation on arg1
}
If I leave the val in the trait as abstract I can reach issues such as those found here).
What I am looking for is a way to define the value of the val in the children, ensuring it would be a val and without having to save data for late calculations. Something similar as to how in java I can define a final val and fill it in the constructor
The standard way (suggested in many places would be to define mydata as def and override it in the children... If I leave the val in the trait as abstract I can reach issues such as those found here).
This is a common misunderstanding, shown in the accepted answer to the linked question as well. The issue is implementing as a val, which you require anyway. Having a concrete val which is overridden only makes it worse: abstract one can at least be implemented by a lazy val. The only way to avoid the issue for you is to ensure mydata is not accessed in a constructor of A or its subtypes, directly or indirectly, until it's initialized. Using it in f is safe (provided f is not called in a constructor, again, which would be an indirect access to mydata).
If you can ensure this requirement, then
trait A {
protected val mydata
def f(args) = ??? //uses mydata
}
class D(arg1: String) extends B with A {
override val mydata = ??? /// some calculation based on arg1
}
class E(arg1: String) extends C with A{
override val mydata = ??? /// some calculation based on arg1
}
is exactly the correct definition.
If you can't, then you have to live with your last solution despite the drawback, but mydata needs to be lazy to avoid similar initialization order issues, which would already give the same drawback on its own.
Sometimes I create a class and its only used to store some values or static methods and I never want to create an instance.
Is there some way to express this in Python3?
Eg:
class MyClass:
#staticmethod
def hello():
print("world")
# is there a better way to do this?
def __init__(self):
raise Exception("instantiation isnt supported for this class!")
# OK
MyClass.hello()
# NOT OK
c = MyClass()
You can use the objectless baseclass found in this answer (which is actually an answer to a different question).
class objectless:
def __new__(cls, *args, **kwargs):
raise RuntimeError('%s should not be instantiated' % cls)
class UninstantiateableClass(objectless):
#classmethod
def foo(cls):
return 'bar'
...
Note that in python2, objectless should explicitly subclass object, in order to be a new-style class.
I have the following where I set information and extractors for different schemes of data:
trait DataScheme {
type Type <: List[Any]
class ExtractorMethods(ticker: String, dataList: List[Type]) {
def getDatetime(datum: Type): Date = new Date(datum(columnIndex(Names.datetime)).toString)
def upperDatum(date: Date): Type = dataList.minBy(datum => getDatetime(datum) >= date)
def lowerDatum(date: Date): Type = dataList.maxBy(datum => getDatetime(datum) <= date)
}
}
trait IndexScheme extends DataScheme {
type Type = (Date, Double, Double, Double, Double, Long)
class ExtractorMethods(ticker: String, dataList: List[Type]) extends super.ExtractorMethods(ticker: String, dataList: List[Type]){
def testing12(int: Int):Int = 12
val test123 = 123
}
}
I want anything extending DataScheme to use its ExtractorMethods methods (e.g. lowerDatum) but also have its own methods (e.g. testing12).
There is a class definition for lists of data elements:
class Data[+T <: DataScheme](val ticker: String, val dataList: List[T#Type], val isSorted: Boolean)
(implicit m: Manifest[T], mm: Manifest[T#Type]) extends Symbols {
def this(ticker: String, dataList: List[T#Type])(implicit m: Manifest[T], mm: Manifest[T#Type]) = this(ticker, dataList, false)(m: Manifest[T], mm: Manifest[T#Type])
val dataScheme: T
val extractorMethods = new dataScheme.ExtractorMethods(ticker, dataList.asInstanceOf[List[dataScheme.Type]])
}
A Data class should make accessible the methods in ExtractorMethods of the scheme so they can be used in the main program through the instance of Data that has been defined. For example if sortedData is an instance of Data[IndexScheme], the following works:
val lowerDatum = sortedData.extractorMethods.lowerDatum(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse("2010-03-31 00:00:00"))
but this does not:
val testing = sortedData.extractorMethods.testing12(123)
because 'testing 123 is not a member of sortedData.dataScheme.extractorMethods'. So my question is how can the subclasses of ExtractorMethods in the subtraits of DataScheme like IndexScheme be made accessible? How is it possible using Manifests and TypeTags? Thanks.
So you want the generic class Data[DataScheme] or Data[IndexScheme] to have access to the methods of whichever type Data has been parameterised with. You've tried to do this several different ways, from the evidence in your code.
To answer your last question - manifests can't help in this particular case and TypeTags are only part of the answer. If you really want to do this, you do it with mirrors.
However, you will have to make some changes to your code. Scala only has instance methods; there are no such things as static methods in Scala. This means that you can only use reflection to invoke a method on an instance of a class, trait or object. Your traits are abstract and can't be instantiated.
I can't really tell you how to clean up your code, because what you have pasted up here is a bit of a mess and is full of different things you have tried. What I can show you is how to do it with a simpler set of classes:
import scala.reflect.runtime.universe._
class t1 {
class Methods {
def a = "a"
def b = "b"
}
def methods = new Methods
}
class t2 extends t1 {
class Methods extends super.Methods {
def one = 1
def two = 2
}
override def methods = new Methods
}
class c[+T <: t1](implicit tag: TypeTag[T]) {
def generateT = {
val mirror = runtimeMirror(getClass.getClassLoader)
val cMirror = mirror.reflectClass(typeOf[T].typeSymbol.asClass)
cMirror.reflectConstructor(typeOf[T].declaration(nme.CONSTRUCTOR).asMethod)
}
val t = generateT().asInstanceOf[T]
}
val v1 = new c[t1]
val v2 = new c[t2]
If you run that, you'll find that v1.t.methods gives you a class with only methods a and b, but v2.t.methods gives a class with methods one and two as well.
This really is not how to do this - reaching for reflection for this kind of job shows a very broken model. But I guess that's your business.
I stick by what I said below, though. You should be using implicit conversions (and possibly implicit parameters) with companion objects. Use Scala's type system the way it's designed - you are fighting it all the way.
ORIGINAL ANSWER
Well, I'm going to start by saying that I would never do things the way you are doing this; it seems horribly over-complicated. But you can do what you want to do, roughly the way you are doing it, by
Using mixins
Moving the extractorMethods creation code into the traits.
Here's a greatly simplified example:
trait t1 {
class Methods {
def a = "a"
def b = "b"
}
def methods = new Methods
}
trait t2 extends t1 {
class Methods extends super.Methods {
def one = 1
def two = 2
}
override def methods = new Methods
}
class c1 extends t1
val v1 = new c1
// v1.methods.a will return "a", but v1.methods.one does not exist
class c2 extends c1 with t2
val v2 = new c2
// v2.methods.a returns "a" and v2.methods.one returns 1
I could replicate your modus operandi more closely by defining c1 like this:
class c1 extends t1 {
val myMethods = methods
}
in which case v1.myMethods would only have methods a and b but v2.myMethods would have a, b, one and two.
You should be able to see how you can adapt this to your own class and trait structure. I know my example doesn't have any of your complex type logic in it, but you know better than I what you are trying to achieve there. I'm just trying to demonstrate a simple mechanism.
But dude, way to make your life difficult...
EDIT
There are so many things I could say about what is wrong with your approach here, both on the small and large scale. I'm going to restrict myself to saying two things:
You can't do what you are trying to do in the Data class because it is abstract. You cannot force Scala to magically replace an uninitialised, abstract method of a non-specific type with the specific type, just by littering everything with Type annotations. You can only solve this with a concrete class which provides the specific type.
You should be doing this with implicit conversions. Implicits would help you do it the wrong way you seem fixated on, but would also help you do it the right way. Oh, and use a companion object, either for the implicits or to hold a factory (or bot).
I want to do something like this:
sealed abstract class Base(val myparam:String)
case class Foo(override val myparam:String) extends Base(myparam)
case class Bar(override val myparam:String) extends Base(myparam)
def getIt( a:Base ) = a.copy(myparam="changed")
I can't, because in the context of getIt, I haven't told the compiler that every Base has a 'copy' method, but copy isn't really a method either so I don't think there's a trait or abstract method I can put in Base to make this work properly. Or, is there?
If I try to define Base as abstract class Base{ def copy(myparam:String):Base }, then case class Foo(myparam:String) extends Base results in class Foo needs to be abstract, since method copy in class Base of type (myparam: String)Base is not defined
Is there some other way to tell the compiler that all Base classes will be case classes in their implementation? Some trait that means "has the properties of a case class"?
I could make Base be a case class, but then I get compiler warnings saying that inheritance from case classes is deprecated?
I know I can also:
def getIt(f:Base)={
(f.getClass.getConstructors.head).newInstance("yeah").asInstanceOf[Base]
}
but... that seems very ugly.
Thoughts? Is my whole approach just "wrong" ?
UPDATE I changed the base class to contain the attribute, and made the case classes use the "override" keyword. This better reflects the actual problem and makes the problem more realistic in consideration of Edmondo1984's response.
This is old answer, before the question was changed.
Strongly typed programming languages prevent what you are trying to do. Let's see why.
The idea of a method with the following signature:
def getIt( a:Base ) : Unit
Is that the body of the method will be able to access a properties visible through Base class or interface, i.e. the properties and methods defined only on the Base class/interface or its parents. During code execution, each specific instance passed to the getIt method might have a different subclass but the compile type of a will always be Base
One can reason in this way:
Ok I have a class Base, I inherit it in two case classes and I add a
property with the same name, and then I try to access the property on
the instance of Base.
A simple example shows why this is unsafe:
sealed abstract class Base
case class Foo(myparam:String) extends Base
case class Bar(myparam:String) extends Base
case class Evil(myEvilParam:String) extends Base
def getIt( a:Base ) = a.copy(myparam="changed")
In the following case, if the compiler didn't throw an error at compile time, it means the code would try to access a property that does not exist at runtime. This is not possible in strictly typed programming languages: you have traded restrictions on the code you can write for a much stronger verification of your code by the compiler, knowing that this reduces dramatically the number of bugs your code can contain
This is the new answer. It is a little long because few points are needed before getting to the conclusion
Unluckily, you can't rely on the mechanism of case classes copy to implement what you propose. The way the copy method works is simply a copy constructor which you can implement yourself in a non-case class. Let's create a case class and disassemble it in the REPL:
scala> case class MyClass(name:String, surname:String, myJob:String)
defined class MyClass
scala> :javap MyClass
Compiled from "<console>"
public class MyClass extends java.lang.Object implements scala.ScalaObject,scala.Product,scala.Serializable{
public scala.collection.Iterator productIterator();
public scala.collection.Iterator productElements();
public java.lang.String name();
public java.lang.String surname();
public java.lang.String myJob();
public MyClass copy(java.lang.String, java.lang.String, java.lang.String);
public java.lang.String copy$default$3();
public java.lang.String copy$default$2();
public java.lang.String copy$default$1();
public int hashCode();
public java.lang.String toString();
public boolean equals(java.lang.Object);
public java.lang.String productPrefix();
public int productArity();
public java.lang.Object productElement(int);
public boolean canEqual(java.lang.Object);
public MyClass(java.lang.String, java.lang.String, java.lang.String);
}
In Scala, the copy method takes three parameter and can eventually use the one from the current instance for the one you haven't specified ( the Scala language provides among its features default values for parameters in method calls)
Let's go down in our analysis and take again the code as updated:
sealed abstract class Base(val myparam:String)
case class Foo(override val myparam:String) extends Base(myparam)
case class Bar(override val myparam:String) extends Base(myparam)
def getIt( a:Base ) = a.copy(myparam="changed")
Now in order to make this compile, we would need to use in the signature of getIt(a:MyType) a MyType that respect the following contract:
Anything that has a parameter myparam and maybe other parameters which
have default value
All these methods would be suitable:
def copy(myParam:String) = null
def copy(myParam:String, myParam2:String="hello") = null
def copy(myParam:String,myParam2:Option[Option[Option[Double]]]=None) = null
There is no way to express this contract in Scala, however there are advanced techniques that can be helpful.
The first observation that we can do is that there is a strict relation between case classes and tuples in Scala. In fact case classes are somehow tuples with additional behaviour and named properties.
The second observation is that, since the number of properties of your classes hierarchy is not guaranteed to be the same, the copy method signature is not guaranteed to be the same.
In practice, supposing AnyTuple[Int] describes any Tuple of any size where the first value is of type Int, we are looking to do something like that:
def copyTupleChangingFirstElement(myParam:AnyTuple[Int], newValue:Int) = myParam.copy(_1=newValue)
This would not be to difficult if all the elements were Int. A tuple with all element of the same type is a List, and we know how to replace the first element of a List. We would need to convert any TupleX to List, replace the first element, and convert the List back to TupleX. Yes we will need to write all the converters for all the values that X might assume. Annoying but not difficult.
In our case though, not all the elements are Int. We want to treat Tuple where the elements are of different type as if they were all the same if the first element is an Int. This is called
"Abstracting over arity"
i.e. treating tuples of different size in a generic way, independently of their size. To do it, we need to convert them into a special list which supports heterogenous types, named HList
Conclusion
Case classes inheritance is deprecated for very good reason, as you can find out from multiple posts in the mailing list: http://www.scala-lang.org/node/3289
You have two strategies to deal with your problem:
If you have a limited number of fields you require to change, use an approach such as the one suggested by #Ron, which is having a copy method. If you want to do it without losing type information, I would go for generifying the base class
sealed abstract class Base[T](val param:String){
def copy(param:String):T
}
class Foo(param:String) extends Base[Foo](param){
def copy(param: String) = new Foo(param)
}
def getIt[T](a:Base[T]) : T = a.copy("hello")
scala> new Foo("Pippo")
res0: Foo = Foo#4ab8fba5
scala> getIt(res0)
res1: Foo = Foo#5b927504
scala> res1.param
res2: String = hello
If you really want to abstract over arity, a solution is to use a library developed by Miles Sabin called Shapeless. There is a question here which has been asked after a discussion : Are HLists nothing more than a convoluted way of writing tuples? but I tell you this is going to give you some headache
If the two case classes would diverge over time so that they have different fields, then the shared copy approach would cease to work.
It is better to define an abstract def withMyParam(newParam: X): Base. Even better, you can introduce an abstract type to retain the case class type upon return:
scala> trait T {
| type Sub <: T
| def myParam: String
| def withMyParam(newParam: String): Sub
| }
defined trait T
scala> case class Foo(myParam: String) extends T {
| type Sub = Foo
| override def withMyParam(newParam: String) = this.copy(myParam = newParam)
| }
defined class Foo
scala>
scala> case class Bar(myParam: String) extends T {
| type Sub = Bar
| override def withMyParam(newParam: String) = this.copy(myParam = newParam)
| }
defined class Bar
scala> Bar("hello").withMyParam("dolly")
res0: Bar = Bar(dolly)
TL;DR: I managed to declare the copy method on Base while still letting the compiler auto generate its implementations in the derived case classes. This involves a little trick (and actually I'd myself just redesign the type hierarchy) but at least it goes to show that you can indeed make it work without writing boiler plate code in any of the derived case classes.
First, and as already mentioned by ron and Edmondo1984, you'll get into troubles if your case classes have different fields.
I'll strictly stick to your example though, and assume that all your case classes have the same fields (looking at your github link, this seems to be the case of your actual code too).
Given that all your case classes have the same fields, the auto-generated copy methods will have the same signature which is a good start. It seems reasonable then to just add the common definition in Base, as you did:
abstract class Base{ def copy(myparam: String):Base }
The problem is now that scala won't generate the copy methods, because there is already one in the base class.
It turns out that there is another way to statically ensure that Base has the right copy method, and it is through structural typing and self-type annotation:
type Copyable = { def copy(myParam: String): Base }
sealed abstract class Base(val myParam: String) { this : Copyable => }
And unlike in our earlier attempt, this will not prevent scala to auto-generate the copy methods.
There is one last problem: the self-type annotation makes sure that sub-classes of Base have a copy method, but it does not make it publicly availabe on Base:
val foo: Base = Foo("hello")
foo.copy()
scala> error: value copy is not a member of Base
To work around this we can add an implicit conversion from Base to Copyable. A simple cast will do, as a Base is guaranteed to be a Copyable:
implicit def toCopyable( base: Base ): Base with Copyable = base.asInstanceOf[Base with Copyable]
Wrapping up, this gives us:
object Base {
type Copyable = { def copy(myParam: String): Base }
implicit def toCopyable( base: Base ): Base with Copyable = base.asInstanceOf[Base with Copyable]
}
sealed abstract class Base(val myParam: String) { this : Base. Copyable => }
case class Foo(override val myParam: String) extends Base( myParam )
case class Bar(override val myParam: String) extends Base( myParam )
def getIt( a:Base ) = a.copy(myParam="changed")
Bonus effect: if we try to define a case class with a different signature, we get a compile error:
case class Baz(override val myParam: String, truc: Int) extends Base( myParam )
scala> error: illegal inheritance; self-type Baz does not conform to Base's selftype Base with Base.Copyable
To finish, one warning: you should probably just revise your design to avoid having to resort to the above trick.
In your case, ron's suggestion to use a single case class with an additional etype field seems more than reasonable.
I think this is what extension methods are for. Take your pick of implementation strategies for the copy method itself.
I like here that the problem is solved in one place.
It's interesting to ask why there is no trait for caseness: it wouldn't say much about how to invoke copy, except that it can always be invoked without args, copy().
sealed trait Base { def p1: String }
case class Foo(val p1: String) extends Base
case class Bar(val p1: String, p2: String) extends Base
case class Rab(val p2: String, p1: String) extends Base
case class Baz(val p1: String)(val p3: String = p1.reverse) extends Base
object CopyCase extends App {
implicit class Copy(val b: Base) extends AnyVal {
def copy(p1: String): Base = b match {
case foo: Foo => foo.copy(p1 = p1)
case bar: Bar => bar.copy(p1 = p1)
case rab: Rab => rab.copy(p1 = p1)
case baz: Baz => baz.copy(p1 = p1)(p1.reverse)
}
//def copy(p1: String): Base = reflect invoke
//def copy(p1: String): Base = macro xcopy
}
val f = Foo("param1")
val g = f.copy(p1="param2") // normal
val h: Base = Bar("A", "B")
val j = h.copy("basic") // enhanced
println(List(f,g,h,j) mkString ", ")
val bs = List(Foo("param1"), Bar("A","B"), Rab("A","B"), Baz("param3")())
val vs = bs map (b => b copy (p1 = b.p1 * 2))
println(vs)
}
Just for fun, reflective copy:
// finger exercise in the api
def copy(p1: String): Base = {
import scala.reflect.runtime.{ currentMirror => cm }
import scala.reflect.runtime.universe._
val im = cm.reflect(b)
val ts = im.symbol.typeSignature
val copySym = ts.member(newTermName("copy")).asMethod
def element(p: Symbol): Any = (im reflectMethod ts.member(p.name).asMethod)()
val args = for (ps <- copySym.params; p <- ps) yield {
if (p.name.toString == "p1") p1 else element(p)
}
(im reflectMethod copySym)(args: _*).asInstanceOf[Base]
}
This works fine for me:
sealed abstract class Base { def copy(myparam: String): Base }
case class Foo(myparam:String) extends Base {
override def copy(x: String = myparam) = Foo(x)
}
def copyBase(x: Base) = x.copy("changed")
copyBase(Foo("abc")) //Foo(changed)
There is a very comprehensive explanation of how to do this using shapeless at http://www.cakesolutions.net/teamblogs/copying-sealed-trait-instances-a-journey-through-generic-programming-and-shapeless ; in case the link breaks, the approach uses the copySyntax utilities from shapeless, which should be sufficient to find more details.
Its an old problem, with an old solution,
https://code.google.com/p/scala-scales/wiki/VirtualConstructorPreSIP
made before the case class copy method existed.
So in reference to this problem each case class MUST be a leaf node anyway, so define the copy and a MyType / thisType plus the newThis function and you are set, each case class fixes the type. If you want to widen the tree/newThis function and use default parameters you'll have to change the name.
as an aside - I've been waiting for compiler plugin magic to improve before implementing this but type macros may be the magic juice. Search in the lists for Kevin's AutoProxy for a more detailed explanation of why my code never went anywhere
I've been cleaning up some code from a module I'm extending and I can't seem to find a way to Pythonify this code:
global_next_id = 1
class Obj:
def __init__(self):
global global_next_id
self.id = global_next_id
global_next_id += 1
This code uses a global id to keep track of instances of a class (I need the variable self.id internally as well, and it needs to be a number).
Can anyone suggest a way to Pythonify this code?
You could consider using a class attribute to provide a counter. Each instance needs only to ask for the next value. They each get something unique. Eg:
from itertools import count
class Obj(object):
_ids = count(0)
def __init__(self):
self.id = next(self._ids)
This should do the job:
class Obj:
_counter = 0
def __init__(self):
Obj._counter += 1
self.id = Obj._counter
Here is a way to count instances without descendant classes sharing the same id/count. A metaclass is used to create a separate id counter for each class.
Uses Python 3 syntax for Metaclasses.
import itertools
class InstanceCounterMeta(type):
""" Metaclass to make instance counter not share count with descendants
"""
def __init__(cls, name, bases, attrs):
super().__init__(name, bases, attrs)
cls._ids = itertools.count(1)
class InstanceCounter(object, metaclass=InstanceCounterMeta):
""" Mixin to add automatic ID generation
"""
def __init__(self):
self.id = next(self.__class__._ids)
I found the following solution:
class Obj:
counter = 0
def __init__(self):
type(self).counter += 1
def __del__(self):
type(self).counter -= 1
It's better to use type(self).counter instead of Obj.counter
Generator?
def get_next_id():
curr_id = 1
while True:
yield curr_id
curr_id += 1
class InstanceCounter(object):
# the instance counter
counter = 0
def __init__(self, val):
self.val = all
# incrementing every time an instance is created
InstanceCounter.counter += 1
def set_val(self, val):
self.val = val
def get_val(self, val):
return self.val
# accessing the instance counter should be done through a class method
#classmethod
def get_counter(cls):
return cls.counter
# See the instance counter as it increments as new instances are created
a=InstanceCounter(5)
print(a.get_counter())
b=InstanceCounter(7)
print(a.get_counter(), b.get_counter())
c=InstanceCounter(9)
print(a.get_counter(), b.get_counter(), c.get_counter())
You can use dir() function, which returns all properties and functions in the current script, to count the numbers of instances of a certain class.
len([i for i in dir() if isinstance(eval(i), ClassName)])