Scala implicit conversion to monadic value within effectful macro - scala

Edit: I updated the question to be more descriptive.
Note: I use the Scala 2.11 compiler, because that is the compiler version used by the LMS tutorial project.
I am porting a DSL written in Haskell to Scala. The DSL is an imperative language, so I used monads with do-notation, namely WriterT [Stmt] (State Label) a. I had trouble porting this to Scala, but worked around it by using the ReaderWriterState monad and just using Unit for the Reader component. I then went about searching for an alternative for the do-notation found in Haskell. For-comprehensions are supposed to be this alternative in Scala, but they are tailored towards sequences, I e.g. could not pattern match a tuple, it would insert a call to filter. So I started looking for alternatives and found multiple libraries: effectful, monadless, and each. I tried effectful first which does exactly what I wanted to achieve, I even prefer it over Haskell's do-notation, and it worked well with the ReaderWriterState monad from ScalaZ I had been using. In my DSL I have actions like Drop() (a case class) that I want to be able to use directly as a statement. I hoped to use implicits for this, but because the ! method of effectful (or monadless equivalent for that matter) is too general, I cannot get Scala to automatically convert my Action case classes to something of type Stmt (the ReaderWriterState that returns Unit).
So if not implicits, would there be a different way to achieve it?
As shown in Main.passes2, I did figure out a workaround that I would not mind using, but I am curious whether I stumbled upon is some limitation of the language, or is simply my lack of experience with Scala.
With Main.fails1 I will get the following error message: Implicit not found: scalaz.Unapply[scalaz.Monad, question.Label]. Unable to unapply type question.Label into atype constructor of kind M[_] that is classified by the type class scalaz.Monad. Check that the type class is defined by compiling implicitly[scalaz.Monad[type constructor]] and review the implicits in object Unapply, which only cover common type 'shapes.'
Which comes from ScalaZ its Unapply: https://github.com/scalaz/scalaz/blob/d2aba553e444f951adc847582226d617abae24da/core/src/main/scala/scalaz/Unapply.scala#L50
And with Main.fails2 I will just get: value ! is not a member of question.Label
I assume it is just a matter of writing the missing implicit definition, but I am not quite sure which one Scala wants me to write.
The most important bits of my build.sbt are the version:
scalaVersion := "2.11.2",
And the dependencies:
libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.11.2",
libraryDependencies += "org.scala-lang" % "scala-library" % "2.11.2",
libraryDependencies += "org.scala-lang" % "scala-reflect" % "2.11.2",
libraryDependencies += "org.pelotom" %% "effectful" % "1.0.1",
Here is the relevant code, containing the things I tried and the code necessary to run those things:
package question
import scalaz._
import Scalaz._
import effectful._
object DSL {
type GotoLabel = Int
type Expr[A] = ReaderWriterState[Unit, List[Action], GotoLabel, A]
type Stmt = Expr[Unit]
def runStmt(stmt: Stmt, startLabel: GotoLabel): (Action, GotoLabel) = {
val (actions, _, updatedLabel) = stmt.run(Unit, startLabel)
val action = actions match {
case List(action) => action
case _ => Seq(actions)
}
(action, updatedLabel)
}
def runStmt(stmt: Stmt): Action = runStmt(stmt, 0)._1
def getLabel(): Expr[GotoLabel] =
ReaderWriterState((_, label) => (Nil, label, label))
def setLabel(label: GotoLabel): Stmt =
ReaderWriterState((_, _) => (Nil, Unit, label))
implicit def actionStmt(action: Action): Stmt =
ReaderWriterState((_, label) => (List(action), Unit, label))
}
import DSL._
final case class Label(label: String) extends Action
final case class Goto(label: String) extends Action
final case class Seq(seq: List[Action]) extends Action
sealed trait Action {
def stmt(): Stmt = this
}
object Main {
def freshLabel(): Expr[String] = effectfully {
val label = getLabel.! + 1
setLabel(label).!
s"ants-$label"
}
def passes0() =
freshLabel()
.flatMap(before => Label(before))
.flatMap(_ => freshLabel())
.flatMap(after => Label(after));
def passes1() = effectfully {
unwrap(actionStmt(Label(unwrap(freshLabel()))))
unwrap(actionStmt(Label(unwrap(freshLabel()))))
}
def fails1() = effectfully {
unwrap(Label(unwrap(freshLabel())))
unwrap(Label(unwrap(freshLabel())))
}
def pasess2() = effectfully {
Label(freshLabel.!).stmt.!
Label(freshLabel.!).stmt.!
}
def fails2() = effectfully {
Label(freshLabel.!).!
Label(freshLabel.!).!
}
def main(args: Array[String]): Unit = {
runStmt(passes0())
}
}

Question quality
I want to start with complaining about the question quality. You provide almost no textual description of what you are trying to achieve and then just show us a wall of code but don't explicitly reference your dependencies. This is far from what could count as a Minimal, Complete, and Verifiable example. And typically you get much more chances to get some answers if you provide a clear problem that is easy to understand and reproduce.
Back to the business
When you write something like
unwrap(Label(unwrap(freshLabel())))
you ask too much from the Scala compiler. Particularly unwrap can unwrap only something that is wrapped into some Monad, but Label is not a monad. It is not just the fact that there is no Monad[Label] instance, it is the fact that it structurally doesn't fit that kills you. Simply speaking an instance of ScalaZ Unapply is an object that allows you to split an applied generic type MonadType[SpecificType] (or other FunctorType[SpecificType]) into an "unapplied"/"partially-applied" MonadType[_] and SpecificType even when (as in your case) MonadType[_] is actually something complicated like ReaderWriterState[Unit, List[Action], GotoLabel, _]. And so the error says that there is no known way to split Label into some MonadType[_] and SpecifictType. You probably expected that your implicit actionStmt would do the trick to auto-convert Label into a Statement but this step is too much for the Scala compiler because for it to work, it also implies splitting the composite type. Note that this conversion is far from obvious for the compiler as the unwrap is itself a generic method that can handle any Monad. Actually in some sense ScalaZ needs Unapply exactly because the compiler can't do such things automatically. Still if you help the compiler just a little bit by specifying generic type, it can do the rest of the work:
def fails1() = effectfully {
// fails
// unwrap(Label(unwrap(freshLabel())))
// unwrap(Label(unwrap(freshLabel())))
// works
unwrap[Stmt](Label(unwrap(freshLabel())))
unwrap[Stmt](Label(unwrap(freshLabel())))
}
There is also another possible solution but it is a quite dirty hack: you can roll out your custom Unapply to persuade the compiler that Label is actually the same as Expr[Unit] which can be split as Expr[_] and Unit:
implicit def unapplyAction[AC <: Action](implicit TC0: Monad[Expr]): Unapply[Monad, AC] {
type M[X] = Expr[X]
type A = Unit
} = new Unapply[Monad, AC] {
override type M[X] = Expr[X]
override type A = Unit
override def TC: Monad[Expr] = TC0
// This can't be implemented because Leibniz really witness only exactly the same types rather than some kind of isomorphism
// Luckily effectful doesn't use leibniz implementation
override def leibniz: AC === Expr[Unit] = ???
}
The obvious reason why this is a dirty hack is that Label is actually not the same as Expr[Unit] and you can see it by the fact that you can't implement leibniz in your Unapply. Anyway, if you import unapplyAction, even your original fails1 will compile and work because effectful doesn't use leibniz inside.
As for your fails2, I don't think you can make it work in any simple way. Probably the only way that you may try is to create another macro that would convert in-place call to ! on your action (or its implicit wrapper) into the effectful's ! on the action.stmt. This might work but I didn't try it.

Related

How to normalise a Union Type (T | Option[T])?

I have the following case class:
case class Example[T](
obj: Option[T] | T = None,
)
This allows me to construct it like Example(myObject) instead of Example(Some(myObject)).
To work with obj I need to normalise it to Option[T]:
lazy val maybeIn = obj match
case o: Option[T] => o
case o: T => Some(o)
the type test for Option[T] cannot be checked at runtime
I tried with TypeTest but I got also warnings - or the solutions I found look really complicated - see https://stackoverflow.com/a/69608091/2750966
Is there a better way to achieve this pattern in Scala 3?
I don't know about Scala3. But you could simply do this:
case class Example[T](v: Option[T] = None)
object Example {
def apply[T](t: T): Example[T] = Example(Some(t))
}
One could also go for implicit conversion, regarding the specific use case of the OP:
import scala.language.implicitConversions
case class Optable[Out](value: Option[Out])
object Optable {
implicit def fromOpt[T](o: Option[T]): Optable[T] = Optable(o)
implicit def fromValue[T](v: T): Optable[T] = Optable(Some(v))
}
case class SomeOpts(i: Option[Int], s: Option[String])
object SomeOpts {
def apply(i: Optable[Int], s: Optable[String]): SomeOpts = SomeOpts(i.value, s.value)
}
println(SomeOpts(15, Some("foo")))
We have a specialized Option-like type for this purpose: OptArg (in Scala 2 but should be easily portable to 3)
import com.avsystem.commons._
def gimmeLotsOfParams(
intParam: OptArg[Int] = OptArg.Empty,
strParam: OptArg[String] = OptArg.Empty
): Unit = ???
gimmeLotsOfParams(42)
gimmeLotsOfParams(strParam = "foo")
It relies on an implicit conversion so you have to be a little careful with it, i.e. don't use it as a drop-in replacement for Option.
The implementation of OptArg is simple enough that if you don't want external dependencies then you can probably just copy it into your project or some kind of "commons" library.
EDIT: the following answer is incorrect. As of Scala 3.1, flow analysis is only able to check for nullability. More information is available on the Scala book.
I think that the already given answer is probably better suited for the use case you proposed (exposing an API can can take a simple value and normalize it to an Option).
However, the question in the title is still interesting and I think it makes sense to address it.
What you are observing is a consequence of type parameters being erased at runtime, i.e. they only exist during compilation, while matching happens at runtime, once those have been erased.
However, the Scala compiler is able to perform flow analysis for union types. Intuitively I'd say there's probably a way to make it work in pattern matching (as you did), but you can make it work for sure using an if and isInstanceOf (not as clean, I agree):
case class Example[T](
obj: Option[T] | T = None
) {
lazy val maybeIn =
if (obj.isInstanceOf[Option[_]]) {
obj
} else {
Some(obj)
}
}
You can play around with this code here on Scastie.
Here is the announcement from 2019 when flow analysis was added to the compiler.

Sequencing a StateMonad in Cats

I just had a look into the cats library in scala, more specifically the State Monad.
As a toy example I wanted to create some logic that splits a potentially large string (the StringBuilder) and returns the split up String and the remaining StringBuilder:
object Splitter {
def apply(maxSize: Int, overlap: Int): State[StringBuilder, String] = State(
builder => {
val splitPoint = math.min(builder.size, maxSize + overlap)
(builder.drop(maxSize), builder.substring(0, splitPoint))
}
)
}
Running one step of the State monad works fine but I wanted to chain all the steps until the StringBuilder eventually is empty:
val stop = State.inspect((s: StringBuilder) => s.isEmpty)
Splitter(3, 2).untilM[Vector](stop).run(new StringBuilder("tarsntiars"))
However, this doesn't work as untilM is a member of the Monad trait and there are no implicit conversions in scope. What works is:
val monad = StateT.catsDataMonadForStateT[Eval, StringBuilder]
monad.untilM[List, String](Splitter(3, 2))(stop).run(new StringBuilder("tarsntiars"))
However, I think the shorter is much more readable so I am wondering why it doesn't work? Why does the usual MonadOps mechanism doesn't work here?
After SI-2712 was fixed, the Unapply workaround was removed from Cats: https://github.com/typelevel/cats/pull/1583. Now you need the -Ypartial-unification compiler flag (assuming you are using Scala 2.11 or 2.12) in order for State to be treated as a Monad.
Scalaz still has the Unapply machinery so your code should work with Scalaz without the compiler flag.

How to test type conformance of higher-kinded types in Scala

I am trying to test whether two "containers" use the same higher-kinded type. Look at the following code:
import scala.reflect.runtime.universe._
class Funct[A[_],B]
class Foo[A : TypeTag](x: A) {
def test[B[_]](implicit wt: WeakTypeTag[B[_]]) =
println(typeOf[A] <:< weakTypeOf[Funct[B,_]])
def print[B[_]](implicit wt: WeakTypeTag[B[_]]) = {
println(typeOf[A])
println(weakTypeOf[B[_]])
}
}
val x = new Foo(new Funct[Option,Int])
x.test[Option]
x.print[Option]
The output is:
false
Test.Funct[Option,Int]
scala.Option[_]
However, I expect the conformance test to succeed. What am I doing wrong? How can I test for higher-kinded types?
Clarification
In my case, the values I am testing (the x: A in the example) come in a List[c.Expr[Any]] in a Macro. So any solution relying on static resolution (as the one I have given), will not solve my problem.
It's the mixup between underscores used in type parameter definitions and elsewhere. The underscore in TypeTag[B[_]] means an existential type, hence you get a tag not for B, but for an existential wrapper over it, which is pretty much useless without manual postprocessing.
Consequently typeOf[Funct[B, _]] that needs a tag for raw B can't make use of the tag for the wrapper and gets upset. By getting upset I mean it refuses to splice the tag in scope and fails with a compilation error. If you use weakTypeOf instead, then that one will succeed, but it will generate stubs for everything it couldn't splice, making the result useless for subtyping checks.
Looks like in this case we really hit the limits of Scala in the sense that there's no way for us to refer to raw B in WeakTypeTag[B], because we don't have kind polymorphism in Scala. Hopefully something like DOT will save us from this inconvenience, but in the meanwhile you can use this workaround (it's not pretty, but I haven't been able to come up with a simpler approach).
import scala.reflect.runtime.universe._
object Test extends App {
class Foo[B[_], T]
// NOTE: ideally we'd be able to write this, but since it's not valid Scala
// we have to work around by using an existential type
// def test[B[_]](implicit tt: WeakTypeTag[B]) = weakTypeOf[Foo[B, _]]
def test[B[_]](implicit tt: WeakTypeTag[B[_]]) = {
val ExistentialType(_, TypeRef(pre, sym, _)) = tt.tpe
// attempt #1: just compose the type manually
// but what do we put there instead of question marks?!
// appliedType(typeOf[Foo], List(TypeRef(pre, sym, Nil), ???))
// attempt #2: reify a template and then manually replace the stubs
val template = typeOf[Foo[Hack, _]]
val result = template.substituteSymbols(List(typeOf[Hack[_]].typeSymbol), List(sym))
println(result)
}
test[Option]
}
// has to be top-level, otherwise the substituion magic won't work
class Hack[T]
An astute reader will notice that I used WeakTypeTag in the signature of foo, even though I should be able to use TypeTag. After all, we call foo on an Option which is a well-behaved type, in the sense that it doesn't involve unresolved type parameters or local classes that pose problems for TypeTags. Unfortunately, it's not that simple because of https://issues.scala-lang.org/browse/SI-7686, so we're forced to use a weak tag even though we shouldn't need to.
The following is an answer that works for the example I have given (and might help others), but does not apply to my (non-simplified) case.
Stealing from #pedrofurla's hint, and using type-classes:
trait ConfTest[A,B] {
def conform: Boolean
}
trait LowPrioConfTest {
implicit def ctF[A,B] = new ConfTest[A,B] { val conform = false }
}
object ConfTest extends LowPrioConfTest {
implicit def ctT[A,B](implicit ev: A <:< B) =
new ConfTest[A,B] { val conform = true }
}
And add this to Foo:
def imp[B[_]](implicit ct: ConfTest[A,Funct[B,_]]) =
println(ct.conform)
Now:
x.imp[Option] // --> true
x.imp[List] // --> false

Chaining logging with a simple expression in Scala

I usually use Scala with SLF4J through the Loggable wrapper in LiftWeb. This works decently well with the exception of the quite common method made up only from 1 chain of expressions.
So if you want to add logging to such a method, the simply beautiful, no curly brackets
def method1():Z = a.doX(x).doY(y).doZ()
must become:
def method1():Z = {
val v = a.doX(x).doY(y).doZ()
logger.info("the value is %s".format(v))
v
}
Not quite the same, is it? I gave it a try to solve it with this:
class ChainableLoggable[T](val v:T){
def logInfo(logger:Logger, msg:String, other:Any*):T = {
logger.info(msg.format(v, other))
v
}
}
implicit def anyToChainableLogger[T](v:T):ChainableLoggable[T] = new ChainableLoggable(v)
Now I can use a simpler form
def method1():Z = a.doX(x).doY(y).doZ() logInfo(logger, "the value is %s")
However 1 extra object instantiation and an implicit from Any starts to look like a code stink.
Does anyone know of any better solution? Or maybe I shouldn't even bother with this?
Scala 2.10 has just a solution for you - that's a new feature Value Class which allows you to gain the same effect as the implicit wrappers provide but with no overhead coming from instantiation of those wrapper classes. To apply it you'll have to rewrite your code like so:
implicit class ChainableLoggable[T](val v : T) extends AnyVal {
def logInfo(logger:Logger, msg:String, other:Any*) : T = {
logger.info(msg.format(v, other))
v
}
}
Under the hood the compiler will transform the logInfo into an analogue of Java's common "util" static method by prepending your v : T to it's argument list and updating its usages accordingly - see, nothing gets instantiated.
That looks like the right way to do it, especially if you don't have the tap implicit around (not in the standard library, but something like this is fairly widely used--and tap is standard in Ruby):
class TapAnything[A](a: A) {
def tap(f: A => Any): A = { f(a); a }
}
implicit def anything_can_be_tapped[A](a: A) = new TapAnything(a)
With this, it's less essential to have the info implicit on its own, but if you use it it's an improvement over
.tap(v => logger.info("the value is %s".format(v)))
If you want to avoid using implicits, you can define functions like this one in your own logging trait. Maybe not as pretty as the solution with implicits though.
def info[A](a:A)(message:A=>String) = {
logger.info(message(a))
a
}
info(a.doX(x).doY(y).doZ())("the value is " + _)

Is there a SoftHashMap in Scala?

I'm aware of this question for java, but none of those implementations seem to play well with scala.collection.JavaConversions.
I'm looking for something simple (e.g. single file, not a whole library) that implements SoftHashMap such that it plays well with Scala Map (i.e. supports getOrElseUpdate, unzip, and the remaining Scala Map methods).
Implementation inspired by this java WeakHashMap:
import scala.collection.mutable.{Map, HashMap}
import scala.ref._
class SoftMap[K, V <: AnyRef] extends Map[K, V]
{
class SoftValue[K, +V <: AnyRef](val key:K, value:V, queue:ReferenceQueue[V]) extends SoftReference(value, queue)
private val map = new HashMap[K, SoftValue[K, V]]
private val queue = new ReferenceQueue[V]
override def += (kv: (K, V)): this.type =
{
processQueue
val sv = new SoftValue(kv._1, kv._2, queue)
map(kv._1) = sv
this
}
private def processQueue
{
while (true)
{
queue.poll match
{
case Some(sv:SoftValue[K, _]) => map.remove(sv.key)
case _ => return
}
}
}
override def get(key: K): Option[V] = map.get(key) match
{
case Some(sv) => sv.get match
{ case v:Some[_] => v
case None => {map.remove(key); None} }
case None => None
}
override def -=(key: K):this.type =
{
processQueue
map.remove(key)
this
}
override def iterator: Iterator[(K, V)] =
{
processQueue
map.iterator.collect{ case (key, sv) if sv.get.isDefined => (key, sv.get.get) }
}
override def empty:SoftMap[K, V] = new SoftMap[K, V]
override def size = {processQueue; map.size}
}
I found one in liftweb.
I do not use it yet, so check it yourself please.
http://scala-tools.org/mvnsites/liftweb-2.4-M5/net/liftweb/util/SoftReferenceCache.html
Occasionally you get asked a question like, "what's the best way to poke your eye out with a stick" that you can go to great lengths answering how you should harden and sterilise the stick after carving a 1 inch hook into the end, and follow up with where the stick should be inserted and so on. Really though, the best answer is probably not exactly what was asked – but the question as to why on earth you'd want to do this in the first place!
This is one of those questions.
SoftReferences are something that initially sound like something you might want. A reference that does not get garbage collected until there is GC pressure. Presumably, you'd use this to cache something that was worth caching, usually because it was expensive to create in the first place.
The problem is, SoftRefs clear almost exactly when you don't want them to, when the GC is under pressure! This means that they will need to be recreated (expensive op) right when the VM is already busy and under GC pressure.
Furthermore, there is no way to hint the VM as to the priority of objects that are soft-referenced. The particular algorithm used for selecting which objects to clear is unspecified and VM dependent.
Essentially, SoftReferences are a misguided attempt to off-load an application level concern (caching) to the garbage-collector. You really should never* actually use them.
*never, modulo some very small and very specialised use-cases
As other people have observed, SoftReferences are usually not The Right Thing to build a cache. However, some libraries provide better replacements. While the OP requires not using a library, I still think this is the best answer possible. Plus, with SBT downloading the library is quite simple.
In build.sbt, assuming you're building your project with SBT >= 0.10 (tested with 0.12), add:
libraryDependencies += "com.google.guava" % "guava" % "13.0"
libraryDependencies += "com.google.code.findbugs" % "jsr305" % "1.3.9" //Needed by guava, but marked as optional; at least Scalac 2.10 however does require it to parse annotations.
In client code, you can build your map as follows (look into CacheBuilder's option for the meaning of the various parameters; the ones below are the ones I chose for my use case):
For Scala 2.10:
import com.google.common.cache.CacheBuilder
import collection.JavaConversions._
def buildCache[K <: AnyRef, V <: AnyRef]: collection.concurrent.Map[K, V] =
CacheBuilder.newBuilder()
.maximumSize(10000).expireAfterAccess(10, TimeUnit.MINUTES)
.concurrencyLevel(2)
.build[K, V]().asMap()
For Scala 2.9 (deprecated/not compiling in 2.10):
import com.google.common.cache.CacheBuilder
import collection.{JavaConversions, mutable}
import JavaConversions._
def buildCache2_9[K <: AnyRef, V <: AnyRef]: mutable.ConcurrentMap[K, V] =
CacheBuilder.newBuilder()
.maximumSize(10000).expireAfterAccess(10, TimeUnit.MINUTES)
.concurrencyLevel(2)
.build[K, V]().asMap()
To make it work for both versions, call the implicit conversion explicitly - it's JavaConversions.asScalaConcurrentMap. I've reported the problem on the scala-language mailing list (and will also report it on the bug tracker), so I hope that the 2.9 code will at least compile in 2.10 (while still causing deprecation warning):
https://groups.google.com/d/topic/scala-language/uXKRiGXb-44/discussion.