I'm implementing a web service using Slick (3.3) for the DB layer. I am trying to make the Slick implementation as generic as possible, hoping to achieve DB-agnosticism as well as generic table, table query, and DAO classes that abstract over the services models as much as possible. I'm trying to do this combining several techniques:
A model hierarchy extending from a common base trait
A table hierarchy, extracting common columns for model traits (inspired by http://gavinschulz.com/posts/2016-01-30-common-model-fields-with-slick-3-part-i.html)
DB-agnosticism, relying on the JdbcProfile trait rather than any DB-specific profile implementation ( as described here: https://stackoverflow.com/a/31128239/4234254 and in the slick multi-db docs)
Cake pattern for dependency injection
I'm having trouble layering some of the schema elements, however, and not being a Scala type expert, I've been unable to figure out the solution on my own. I've created a reproduction of the issue, trying to minimalize it as much as possible, and using a mock slick library. The full code can be found here: https://github.com/anqit/slick_cake_minimal_error_repro/blob/master/src/main/scala/com/anqit/repro/Repro.scala but I'll go through it below.
My "slick" library and model classes:
abstract class Table[E]
type TableQuery[W <: Table[_]] = List[W] // not actually a list, but need a concrete type constructor to demonstrate the issue
object TableQuery {
def apply[W <: Table[_]]: TableQuery[W] = List[W]()
}
trait BaseEntity
case class SubEntityA() extends BaseEntity
case class SubEntityB() extends BaseEntity
Combining technique 2 in the list above with the cake pattern, I'm creating traits that wrap schema elements for each model. The base schema includes columns common to entity tables (e.g. id), and entity tables inherit from that:
trait BaseSchema[E <: BaseEntity] {
// provides common functionality
abstract class BaseTableImpl[E] extends Table[E]
def wrapper: TableQuery[_ <: BaseTableImpl[E]]
}
// functionality specific to SubEntityA
trait SchemaA extends BaseSchema[SubEntityA] {
class TableA extends BaseTableImpl[SubEntityA]
// this definition compiles fine without a type annotation
val queryA = TableQuery[TableA]
def wrapper = queryA
}
// functionality specific to SubEntityB that depends on SchemaA
trait SchemaB extends BaseSchema[SubEntityB] { self: SchemaA =>
class TableB extends BaseTableImpl[SubEntityB] {
// uses SchemaA's queryA to make a FK
}
/*
attempting to define wrapper here without a type annotation results in the following compilation error (unlike defining wrapper for SchemaA above):
def wrapper = Wrapper[WrappedB]
type mismatch;
[error] found : Repro.this.Wrapper[SubB.this.WrappedB]
[error] (which expands to) List[SubB.this.WrappedB]
[error] required: Repro.this.Wrapper[_ <: SubB.this.BaseWrapMeImpl[_1]]
[error] (which expands to) List[_ <: SubB.this.BaseWrapMeImpl[_1]]
[error] def wrapper = Wrapper[WrappedB]
[error] ^
it does, however, compile if defined with an explicit type annotation as below
*/
val queryB = TableQuery[TableB]
def wrapper: TableQuery[TableB] = queryB
}
This is where I get my first error, a type mismatch, that I have currently worked around using an explicit type annotation, but I suspect it is related to the main error, stay tuned.
A base DAO, that will provide common query methods:
trait BaseDao[E <: BaseEntity] { self: BaseSchema[E] => }
And finally, putting all the cake layers together:
// now, the actual injection of the traits
class DaoA extends SchemaA
with BaseDao[SubEntityA]
// so far so good...
class DaoB extends SchemaA
with SchemaB
with BaseDao[SubEntityB] // blargh! failure! :
/*
illegal inheritance;
[error] self-type Repro.this.DaoB does not conform to Repro.this.BaseDao[Repro.this.SubEntityB]'s selftype Repro.this.BaseDao[Repro.this.SubEntityB] with Repro.this.BaseSchema[Repro.this.SubEntityB]
[error] with BaseDao[SubEntityB]
[error] ^
*/
The first error (the type mismatch in SchemaB), I'm completely at a loss. One of the few tricks in my bag is to add explicit type annotations when I run in to type-related errors in Scala, which is the only reason I tried that, and got it to compile. I would love an explanation as to why that is happening, and I suspect fixing my code such that I can write that without the type would probably help me with the second error. Which brings me to... the second error. To me, it looks like I've included all of the necessary traits to satisfy the self-type tree, but I guess not. My guess is that SchemaB, while extending BaseSchema[SubEntityB], is somehow not being recognized as a BaseSchema[SubEntityB]? Have I not set up my hierarchy properly? Or maybe I need to use bounds instead of strict type references?
Related
How do I define implicit class that can pimp both base and derived class ?
I want to implement class that would pimp both Flow and Source. I tried this
implicit class FlowOpsMatExt[T, Mat](val flow: FlowOpsMat[T, Mat]) {
def groupSortedByKey[K](keyForItem: T ⇒ K, maxBufferSize: Int): flow.Repr[Vector[T]]
}
But called on instance of Source[] I get FlowOpsMat which I can no longer use as Source. So I tried to trick it
implicit class FlowOpsMatExt[T, Mat, C <: FlowOpsMat[T, Mat]](val flow2: C)
but it fails to detect types for some reason. In explicit call
new FlowOpsMatExt(source_instance)
I get following error
Error:(106, 5) inferred type arguments [Nothing,Nothing,akka.stream.scaladsl.Source[akka.util.ByteString,akka.NotUsed]] do not conform to class FlowOpsMatExt's type parameter bounds [T,Mat,C <: akka.stream.scaladsl.FlowOpsMat[T,Mat]]
new FlowOpsMatExt(src)
Minimal example to reproduce: https://gist.github.com/931a313546f14e809b705e86743dcdb0
It compiles if I specify all types explicitly new FlowOpsMatExt[ByteString, NotUsed, Source[ByteString, NotUsed]](src) but it kills benefit of using implicit class.
What am I missing ?
The best I could come up with is to keep implementation of extension function in abstract class and implement two implicit classes from it
abstract class FlowOpsMatExt[T, Mat, C <: FlowOpsMat[T, Mat]] {
protected val flow: C
}
implicit class FlowExt2[In, Out, Mat](val flow: Flow[In, Out, Mat]) extends FlowOpsMatExt[Out, Mat, Flow[In, Out, Mat]] {
}
implicit class SourceExt[T, Mat](val flow: Source[T, Mat]) extends FlowOpsMatExt[T, Mat, Source[T, Mat]] {
}
From the documentation:
Advanced Scala users may wonder whether it is possible to write extension methods that enrich FlowOps to allow nicer syntax. The short answer is that Scala 2 does not support this in a fully generic fashion, the problem is that it is impossible to abstract over the kind of stream that is being extended because Source, Flow and SubFlow differ in the number and kind of their type parameters. While it would be possible to write an implicit class that enriches them generically, this class would require explicit instantiation with all type parameters due to SI-2712. For a partial workaround that unifies extensions to Source and Flow see this sketch by R. Kuhn.
...
It is interesting to note that a simplified form of this problem has found its way into the dotty test suite. Dotty is the development version of Scala on its way to Scala 3.
An easy thing to do in many languages but not in Scala is:
Define archetype 'Super', such that all implementations of 'Super' has to define a constructor 'create()'.
I found this constraint very important and is able to identify a lot of problems before runtime. However this feature is only partially enforced in Java (by defining an 'abstract' static method that always throws an error) and completely missing in Scala (companion object is completely detached from class and cannot be enforced in archetype).
is there a macro or tool that allows me to do this?
UPDATE Sorry my question was missing context and examples. Here is a formal use case in scala:
In project A, we define an interface that can be extended by all subprojects:
trait AbstractFoo {}
This interface should always have a default 0-parameter builder/constructor, so project A can initialize it on-demand, however, the implementation of each constructor is unknown to project A:
object AbstractFoo {
def default[T <: AbstractFoo: ClassTag](): T
}
So the problem becomes: How to rigorously define AbstractFoo, such that for all subprojects of A, any implementation(s) of AbstractFoo:
case class Foo(...) extends AbstractFoo
must satisfy:
'Foo' must have a 0-parameter builder/constructor defined (presumably in its companion object)
calling AbstractFoo.defaultFoo can invoke this 0-parameter builder/constructor
It should be noted that in an alternative conditions, a solution exists which is to define every companion object as an implicit type class:
trait FooBuilder[T <: AbstractFoo] {
def default(): T
}
object AbstractFoo {
implicit object Foo extends FooBuilder[Foo] {
def default() = {...}
}
def default[T <: AbstractFoo: FooBuilder](): T = {
implicitly[FooBuilder[T]].default
}
}
Such that if the implicit object is undefined the compiler will give an implicit not found error (my code snippet may have some syntax error, the idea is from http://www.cakesolutions.net/teamblogs/demystifying-implicits-and-typeclasses-in-scala)
Unfortunately it's not always convenient, because this subproject of A is usually unknown to project A. Yet the default implicit builder cannot be redefined, this makes every invocation of default() more covoluted.
I believe scala is a very extendable language, so there should be at least 1 way to enforce it whether if using macro, annotation or other metaprogramming techniques. Is my question clear enough now?
UPDATE2: I believe I found the solution after carefully study Scaladoc, there is a comment hidden in a corner:
if there are several eligible arguments which match the implicit parameter’s type, a most specific one will be chosen using the rules of static overloading resolution (see Scala Specification §6.26.4):
...
Implicit scope of type arguments (2.8.0)
...
So all I need is to write an implicit function in FooBuilder:
trait FooBuilder[T <: AbstractFoo] {
def default(): T
implicit def self = this
}
object Foo extends FooBuilder[Foo]
So everytime someone call:
default[Foo]
scala will refer to the scope of class Foo, which include object Foo, which contains the implicit value Foo, and eventually find the 0-parameter constructor.
I think this definition is better than defining it under object FooBuilder, since you can only define FooBuilder once, thus its not quite extendable. Would you agree with me? If so, could you please revise your answer so I can award you point?
I don't understand why an abstract class or even a Trait won't allow this to be done?
abstract class DefineCreate{
def create(): Unit
}
case class Foo(one: Int)
object Foo extends DefineCreate{
def create(): Unit = { Console.out.println("side-effect") }
}
Thus I force a user to make a create method on the object in question because all implementations of DefineCreate must do so in order to compile.
Update Following Comments
Well, without having to resort to macros and the like, you could achieve the same sort of thing with type classes:
trait Constructor[A]{
def create(): A
}
object Construct{
def create[A](implicit cr: Constructor[A]): A = cr.create()
}
Which doesn't explicitly force the companion object to sprout methods but it does force a user to make the type class if they want to use the Constructor.create[Foo] pattern.
I have an abstract class Model from which I create case classes:
abstract class Model
case class User(.) extends Model
an abstract class Table taking such a Model as type parameter, used in one of its default concrete methods:
abstract class Table[M <: Model] {
def parser = SomeExternalBuilder[M]
}
The meaning is rather simple: "Give every instance of Table a default parser based on its own class".
The problem is that SomeExternalBuilder will only accept a case class as argument ("case class expected: M"), so it does not compile.
Can I make Table take only case classes as type parameter?
I have seen a few answers providing a missing copy method (ref1, ref2), so I tried this:
trait Model[T] {
def copy: T
}
abstract class Table[M <: Model[M]]
but now case class User extends Model[User] and must overwrite copy too, every function creating a Model takes a type parameter, and honestly the code quickly starts being atrocious, all that for that single line in Table.
Is there no better way than copying that def parser line in every child's body?
Edit: N.B. The real function is def parser: anorm.Macro.namedParser[M] from the "anorm" library for Play.
Edit: Source of the type check by this macro: https://github.com/playframework/anorm/blob/0a1b19055ba3e3749044ad8a54a6b2326235f7c8/core/src/main/scala/anorm/Macro.scala#L117
The problem is that SomeExternalBuilder will only accept a case class as argument ("case class expected: M"), so it does not compile.
I don't think you can ever get such a message from Scala compiler itself, which means that SomeExternalBuilder.apply is a macro. It requires a specific case class in order to know its fields, so that it doesn't matter if you could limit M to be a case class (which you can't): it still wouldn't accept a type parameter.
What you can do is create a macro annotation, so that when you write e.g.
#HasModel
class SomeTable extends Table[SomeModel] {
...
}
the val parser = namedParser[SomeModel] is generated automatically.
Alternately, write #HasModel[SomeModel] class SomeTable { ... } and generate extends Table[SomeModel] as well.
It wouldn't be hard (as macros go), but you still need to annotate each class extending Table.
Not fool proof solution but worth a try
case classes extend Product and Serialisable. Constraint Product with Serialisable will help you get some type safety. M can be any class which extends Product with Serialisable. But Product is extended by case class mostly
abstract class Table[M <: (Product with Serializable)] {
def parser = SomeExternalBuilder[M]
}
I am trying to create a trait that implements a tree with bidrectional links such that when a node adds a parent, that node is added to the parent's children. The error I get below is:
type mismatch; found :PolyTree.this.type (with underlying type PolyTree[T]) required: T
Any idea why this code is getting an error and what is needed in order to make this code work:
trait PolyTree[T <: PolyTree[T]] {
private val _parents: ListBuffer[T] = ListBuffer()
private val _children: ListBuffer[T] = ListBuffer()
def addParent(parent: T): PolyTree[T] = {
if (parent == this)
throw new IllegalArgumentException();
_parents += parent
parent._children += this // Error
this
}
}
The error occurs because the type of 'parent._children' is 'T', while the type of 'this' is 'PolyTree[T]', which are different types in Scala.
You can fix the error by inserting the the following self-type annotation at the top of the trait:
self: T =>
This is necessary because without it, the following would be valid code:
class TreeOne extends PolyTree[TreeOne]
class TreeTwo extends PolyTree[TreeOne]
TreeTwo is allowed to use TreeOne as the type parameter, because TreeOne satisfies the condition that T <: PolyTree[T]. However, once you add the self-type annotation, Scala essentially tries to cast self/'this' in TreeTwo to 'T' (TreeOne) at compile-time, finds that this isn't type-safe, and rejects the declaration of TreeTwo with the error:
error: illegal inheritance
self-type TreeB does not conform to PolyTree[TreeA]'s selftype TreeA'
I'm not the best at understanding or explaining this stuff, but you can garner a bit more knowledge from Chapter 12. The Scala Type System in O'Reilly's 'Programming Scala'.
There seems to be a lot of enthusiasm among Scala bloggers lately for the type classes pattern, in which a simple class has functionality added to it by an additional class conforming to some trait or pattern. As a vastly oversimplified example, the simple class:
case class Wotsit (value: Int)
can be adapted to the Foo trait:
trait Foo[T] {
def write (t: T): Unit
}
with the help of this type class:
implicit object WotsitIsFoo extends Foo[Wotsit] {
def write (wotsit: Wotsit) = println(wotsit.value)
}
The type class is typically captured at compile time with implicts, allowing both the Wotsit and its type class to be passed together into a higher order function:
def writeAll[T] (items: List[T])(implicit tc: Foo[T]) =
items.foreach(w => tc.write(w))
writeAll(wotsits)
(before you correct me, I said it was an oversimplified example)
However, the use of implicits assumes that the precise type of the items is known at compile time. I find in my code this often isn't the case: I will have a list of some type of item List[T], and need to discover the correct type class to work on them.
The suggested approach of Scala would appear to be to add the typeclass argument at all points in the call hierarchy. This can get annoying as an the code scales and these dependencies need to be passed down increasingly long chains, through methods to which they are increasingly irrelevant. This makes the code cluttered and harder to maintain, the opposite of what Scala is for.
Typically this is where dependency injection would step in, using a library to supply the desired object at the point it's needed. Details vary with the library chosen for DI - I've written my own in Java in the past - but typically the point of injection needs to define precisely the object desired.
Trouble is, in the case of a type class the precise value isn't known at compile time. It must be selected based on a polymorphic description. And crucially, the type information has been erased by the compiler. Manifests are Scala's solution to type erasure, but it's far from clear to me how to use them to address this issue.
What techniques and dependency injection libraries for Scala would people suggest as a way of tackling this? Am I missing a trick? The perfect DI library? Or is this really the sticking point it seems?
Clarification
I think there are really two aspects to this. In the first case, the point where the type class is needed is reached by direct function calls from the point where the exact type of its operand is known, and so sufficient type wrangling and syntactic sugar can allow the type class to be passed to the point it's needed.
In the second case, the two points are separated by a barrier - such as an API that can't be altered, or being stored in a database or object store, or serialised and send to another computer - that means the type class can't be passed along with its operand. In this case, given an object whose type and value are known only at runtime, the type class needs somehow to be discovered.
I think functional programmers have a habit of assuming the first case - that with a sufficiently advanced language, the type of the operand will always be knowable. David and mkniessl provided good answers for this, and I certainly don't want to criticise those. But the second case definitely does exist, and that's why I brought dependency injection into the question.
A fair amount of the tediousness of passing down those implicit dependencies can be alleviated by using the new context bound syntax. Your example becomes
def writeAll[T:Foo] (items: List[T]) =
items.foreach(w => implicitly[Foo[T]].write(w))
which compiles identically but makes for nice and clear signatures and has fewer "noise" variables floating around.
Not a great answer, but the alternatives probably involve reflection, and I don't know of any library that will just make this automatically work.
(I have substituted the names in the question, they did not help me think about the problem)
I'll attack the problem in two steps. First I show how nested scopes avoid having to declare the type class parameter all the way down its usage. Then I'll show a variant, where the type class instance is "dependency injected".
Type class instance as class parameter
To avoid having to declare the type class instance as implicit parameter in all intermediate calls, you can declare the type class instance in a class defining a scope where the specific type class instance should be available. I'm using the shortcut syntax ("context bound") for the definition of the class parameter.
object TypeClassDI1 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance as implicit
implicit object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A class defining a scope with a type class instance known to be available
class ATypeClassUser[T:ATypeClass] {
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the evidence class parameter defined
// with the context bound is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassUser = new ATypeClassUser[Something]
aTypeClassUser.bar(List(Something(42), Something(4711)))
}
}
Type class instance as writable field (setter injection)
A variant of the above which would be usable using setter injection. This time the type class instance is passed via a setter call to the bean using the type class.
object TypeClassDI2 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance (not implicit here)
object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A "service bean" class defining a scope with a type class instance.
// Setter based injection style for simplicity.
class ATypeClassBean[T] {
implicit var aTypeClassInstance: ATypeClass[T] = _
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the implicit var is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassBean = new ATypeClassBean[Something]()
// "inject" the type class instance
aTypeClassBean.aTypeClassInstance = SomethingInstance
aTypeClassBean.bar(List(Something(42), Something(4711)))
}
}
Note that the second solution has the common flaw of setter based injection that you can forget to set the dependency and get a nice NullPointerException upon use...
The argument against type classes as dependency injection here is that with type classes the "precise type of the items is known at compile time" whereas with dependency injection, they are not. You might be interested in this Scala project rewrite effort where I moved from the cake pattern to type classes for dependency injection. Take a look at this file where the implicit declarations are made. Notice how the use of environment variables determines the precise type? That is how you can reconcile the compile time requirements of type classes with the run time needs of dependency injection.