How to use scalaz's `Tagged Type` to replace my type alias? - scala

I want to define objects as functions and depend it in other functions:
type FetchPage = String => String
type FindImages = String => List[String]
object WillFetchPage extends FetchPage {
def apply(url:String):String = /* get content of the url */
}
class WillFindImages(fetchPage: FetchPage) extends FindImages {
def apply(url:String):List[String] = {
val html = fetchPage(url)
// get image srcs from the <img> tags
}
}
Then I can inject WillFetchPage to WillFindImages:
val findImages = new WillFindImages(WillFetchPage)
Also test WillFindImages easily by injecting a mock function:
val testFindImages = new WillFindImages(_ => "html-have-3-images")
val images = testFindImages("any-url")
// assertion
You can see the type alias FetchPage is just a type alias, so I can pass other String => String functions to WillFindImages by mistake, so I'm looking for a type-safe solution.
Then I heard of Tagged type from scalaz: http://eed3si9n.com/learning-scalaz/Tagged+type.html
The example is exciting:
sealed trait KiloGram
def KiloGram[A](a: A): A ## KiloGram = Tag[A, KiloGram](a)
val mass = KiloGram(20.0)
2 * mass
You can see the mass here is actually a double 20.0, but it has some unique type.
I want to use it to improve my code, but sadly I can't find a way to do it. I tried:
object FetchPage extends ((String => Try[String]) ## FetchType)
But it provides an compilation error:
Error:(18, 51) class type required but String => scala.util.Try[String] with
scalaz.Tagged[object_as_func.FetchType] found
object FetchPage extends ((String => Try[String]) ## FetchType) {
^
How to fix it?

What scalaz version do you use? It's a bit different in 7.0.6 and 7.1.0. Here is example for 7.1.0
import scalaz.##
object PageFetcher
type FetchPage = String => String
type FindImages = String => List[String]
val WillFetchPage: FetchPage ## PageFetcher.type =
scalaz.Tag[FetchPage, PageFetcher.type](url => "Content")
class WillFindImages(fetchPage: FetchPage ## PageFetcher.type) extends FindImages {
def apply(url: String): List[String] = scalaz.Tag.unwrap(fetchPage)(url).grouped(1).toList
}
val images = new WillFindImages(WillFetchPage)("url")
println(images)
val testFetcher: FetchPage ## PageFetcher.type =
scalaz.Tag[FetchPage, PageFetcher.type](url => "TestContent")
val testImages = new WillFindImages(testFetcher)("url")
println(testImages)

Related

Scala 2 macro type class derivation for `Coder[P <: Product]` ends with error `P does not take parameters`

I am a starter with Scala 2 Macros (before I switch to Dotty) who after trying out the shapeless type class derivation wanted to go one step beyond and write a macro that can generate a type class instances for any scala.Product without it.
(for the sake of example let's ignore nested recursive types, so my goal is flat case classes.)
My type class is an abstract class Coder[T] (e.g. trait with encode() / decode()).
So the generated code for:
case class Pojo(
s: String,
i: Int,
l: List[Int]
)
should be something like:
import com.github.fpopic.scalamacros.Pojo
import org.apache.beam.sdk.coders.Coder
import java.io.{ByteArrayInputStream, ByteArrayOutputStream}
import java.util
class PojoCoder extends Coder[Pojo] {
import com.github.fpopic.scalamacros.beam.DefMacroCoder.{
stringCoder,
intCoder,
listCoder
}
override def encode(value: Pojo, os: OutputStream): Unit = {
stringCoder.encode(value.s, os)
intCoder.encode(value.i, os)
listCoder(intCoder).encode(value.l, os)
}
override def decode(is: InputStream): Pojo = {
Pojo(
s = stringCoder.decode(is),
i = intCoder.decode(is),
l = listCoder(intCoder).decode(is)
)
}
override def getCoderArguments: util.List[_ <: Coder[_]] = {
Collections.emptyList()
}
override def verifyDeterministic(): Unit = ()
}
(removed fully specified class names to improve readability)
In the macro I try to:
inspect the weakTypeOf[P]
iterate over each case class constructor field (of type F)
implicitly find their typeclass Coder[F] instance and add it to the tree
and append their encode() and decode() expression to the tree that contributes to the final Coder[P] methods.
def materializeProductCoder[P: c.WeakTypeTag](c: blackbox.Context): c.Expr[Coder[P]] = {
import c.universe._
val tpe = c.weakTypeOf[P]
val helper = new MacrosHelper[c.type](c)
val expressions =
helper.getPrimaryConstructorMembers(tpe).map { field =>
val fieldTerm = field.asTerm.name // e.g. value.s (for now just s)
val fieldType = field.typeSignature.finalResultType // e.g. String
val fieldCoderName = c.freshName(TermName("coder")) // e.g. give friendly name coder$...
val fieldCoderInstance = // e.g. finds instance of Coder[String]
c.typecheck(
tree = q"""_root_.scala.Predef.implicitly[org.apache.beam.sdk.coders.Coder[${fieldType}]]""",
silent = false
)
val fieldCoderExpression =
q"private val ${fieldCoderName}: org.apache.beam.sdk.coders.Coder[${fieldType}] = ${fieldCoderInstance}"
val fieldEncodeExpression =
q"${fieldCoderName}.encode(value.${fieldTerm}, os)" // replace with full relative name (with dots) instead of value
val fieldDecodeExpression =
q"${field.asTerm} = ${fieldCoderName}.decode(is)"
(fieldCoderExpression, fieldEncodeExpression, fieldDecodeExpression)
}
val fieldCodersExpression = expressions.map(_._1).distinct
val coderEncodeExpresions = expressions.map(_._2)
val coderDecodeExpresions = expressions.map(_._3)
val coderExpression =
q"""{
new org.apache.beam.sdk.coders.Coder[${tpe}] {
{import ${c.prefix}._}
..${fieldCodersExpression}
override def encode(value: ${tpe}, os: java.io.OutputStream): _root_.scala.Unit = {
..${coderEncodeExpresions}
}
override def decode(is: java.io.InputStream): ${tpe} = {
${tpe.typeConstructor}(
..${coderDecodeExpresions}
)
}
override def getCoderArguments: java.util.List[_ <: org.apache.beam.sdk.coders.Coder[_]] = {
java.util.Collections.emptyList
}
override def verifyDeterministic(): _root_.scala.Unit = ()
}
}
"""
val ret = coderExpression
c.Expr[Coder[P]](ret)
}
But get an error after invoking sbt Test / compile:
(a bit struggling with imports and implicit search so for now having intermediate private vals, and distinct is useless)
{
final class $anon extends org.apache.beam.sdk.coders.Coder[com.github.fpopic.scalamacros.beam.Pojo] {
def <init>() = {
super.<init>();
()
};
{
import DefMacroCoder._;
()
};
private val coder$macro$1: org.apache.beam.sdk.coders.Coder[String] = scala.Predef.implicitly[org.apache.beam.sdk.coders.Coder[String]](DefMacroCoder.stringCoder);
private val coder$macro$2: org.apache.beam.sdk.coders.Coder[Int] = scala.Predef.implicitly[org.apache.beam.sdk.coders.Coder[Int]](DefMacroCoder.intCoder);
private val coder$macro$3: org.apache.beam.sdk.coders.Coder[List[Int]] = scala.Predef.implicitly[org.apache.beam.sdk.coders.Coder[List[Int]]](DefMacroCoder.listCoder[Int](DefMacroCoder.intCoder));
override def encode(value: com.github.fpopic.scalamacros.beam.Pojo, os: java.io.OutputStream): _root_.scala.Unit = {
coder$macro$1.encode(value.s, os);
coder$macro$2.encode(value.i, os);
coder$macro$3.encode(value.l, os)
};
override def decode(is: java.io.InputStream): com.github.fpopic.scalamacros.beam.Pojo = com.github.fpopic.scalamacros.beam.Pojo(s = coder$macro$1.decode(is), i = coder$macro$2.decode(is), l = coder$macro$3.decode(is));
override def getCoderArguments: java.util.List[_$1] forSome {
<synthetic> type _$1 <: org.apache.beam.sdk.coders.Coder[_$2] forSome {
<synthetic> type _$2
}
} = java.util.Collections.emptyList;
override def verifyDeterministic(): _root_.scala.Unit = ()
};
new $anon()
}
[error] .../DefMacroCoderSpec.scala:17:56: com.github.fpopic.scalamacros.beam.Pojo does not take parameters
[error] val coder: Coder[Pojo] = DefMacroCoder.productCoder[Pojo]
[error] ^
[error] one error found
Which I believe comes from here but don't fully understand what the compiler is trying to tell me?
Link to the full code sample can be found here
Link to CI error can be found here.
The way you're instantiating your class is wrong:
${tpe.typeConstructor}(...)
It should be
new $tpe(...)
or if you want to do it with case class companion object's apply instead of plain constructor:
${tpe.typeSymbol.companion}(...)
NOTE: type constructor (also known as higher kinded type) has nothing to do with class constructor

Is there some way in scala that I can return a type?

I have a lot of classes such as DataFrameFlow, TextFlow, RDDFlow. They all derive from base class Flow.
Now I want to write a function judgeFlow which can read from a path: String and return something representing exact Flow type from which I can create corresponding instance. The whole code seems like the following
def judgeFlow(path:String) = /*1*/ {
Flow.getStoreType(path) match {
case StoreType.tdw =>
DataFrameFlow
case StoreType.hdfs =>
TextFlow
}
}
def createFlow(typeInfo:/*2*/) = /*3*/{
new typeInfo()
}
However, I don't know how to write in place 1, 2 and 3.
EDIT
Knowing how to construct them is not enough here, because I also want the following:
pattern matching through typeInfo
some ways to do asInstanceOf
EDIT 2
Definition of Flow
abstract class Flow(var outputName: String) extends Serializable{
def this() = this("")
...
}
Definition of DataFrameFlow
class DataFrameFlow(d: DataFrame, path: String) extends Flow {
var data: DataFrame = d
def this(data: DataFrame) = this(data, "")
def this(path: String) = this(null, path)
def this() = this(null, "")
...
}
Pattern matching can't return different types from different cases. The type returned by pattern matching is the least upper bound of types returned in cases.
When someone wants to return different types, most probably he/she wants a type class.
sealed abstract class Flow
class DataFrameFlow extends Flow
class TextFlow extends Flow
class RDDFlow extends Flow
trait JudgeFlow[In] {
type Out <: Flow
def judgeFlow(in: In): Out
}
object JudgeFlow {
implicit val `case1`: JudgeFlow[???] { type Out = DataFrameFlow } = ???
implicit val `case2`: JudgeFlow[???] { type Out = TextFlow } = ???
implicit val `case3`: JudgeFlow[???] { type Out = RDDFlow } = ???
}
def judgeFlow[In](in: In)(implicit jf: JudgeFlow[In]): jf.Out = jf.judgeFlow(in)
But the trouble is that types are resolved at compile time. You seem to want to choose a case based on a value of string i.e. at runtime. So you can't return more specific types than just Flow at compile time.
flatMap with Shapeless yield FlatMapper not found
It's hard to guess your use case completely.
But using Scala reflection you can try
import scala.reflect.runtime.universe._
import scala.reflect.runtime.currentMirror
def judgeFlow(path:String): Type = {
Flow.getStoreType(path) match {
case StoreType.tdw =>
typeOf[DataFrameFlow]
case StoreType.hdfs =>
typeOf[TextFlow]
}
}
def createFlow(typeInfo: Type): Flow = {
val constructorSymbol = typeInfo.decl(termNames.CONSTRUCTOR).asMethod
val classSymbol = typeInfo.typeSymbol.asClass
val classMirror = currentMirror.reflectClass(classSymbol)
val constructorMirror = classMirror.reflectConstructor(constructorSymbol)
constructorMirror().asInstanceOf[Flow]
}

How do I pass an argument to the macro annotation?

I want get StaticAnnotation's parameters defined as:
class Log(logTag: List[LogTag] = Info() :: Nil )
(implicit logger: String => Unit = a => {println(a)})
extends scala.annotation.StaticAnnotation {
inline def apply(defn: Any): Any = meta {
val logTag = ??? //todo
val logger = ??? //todo
}
}
How to get the logTag and logger Function values in meta block?
I have seem meta tutorials about this point. But the Lit just refers Int/Double,etc, rather then custom class type.
You can deconstruct this as a scala.meta.Tree to get the class parameters. See http://scalameta.org/tutorial/#HowdoIpassanargumenttothemacroannotation%3F for an example

How to use Shapeless in a Quasiquote?

I'm trying to call a Shapeless macro from inside a quasiquote with Scala and I'm not getting what I would like to get.
My macro doesn't return any errors but it doesn't expand Witness(fieldName) into Witness.Lt[String]
val implicits = schema.fields.map { field =>
val fieldName:String = field.name
val fieldType = TypeName(field.valueType.fullName)
val in = TermName("implicitField"+fieldName)
val tn = TermName(fieldName)
val cc = TermName("cc")
q"""implicit val $in = Field.apply[$className,$fieldType](Witness($fieldName), ($cc: $className) => $cc.$tn)"""
}
Here is my Field definition:
sealed abstract class Field[CC, FieldName] {
val fieldName: String
type fieldType
// How to extract this field
def get(cc : CC) : fieldType
}
object Field {
// fieldType is existencial in Field but parametric in Fied.Aux
// used to explict constraints on fieldType
type Aux[CC, FieldName, fieldType_] = Field[CC, FieldName] {
type fieldType = fieldType_
}
def apply[CC, fieldType_](fieldWitness : Witness.Lt[String], ext : CC => fieldType_) : Field.Aux[CC, fieldWitness.T, fieldType_] =
new Field[CC, fieldWitness.T] {
val fieldName : String = fieldWitness.value
type fieldType = fieldType_
def get(cc : CC) : fieldType = ext(cc)
}
}
In this case the implicit I generate looks like:
implicit val implicitFieldname : Field[MyCaseClass, fieldWitness.`type`#T]{
override type fieldType = java.lang.String
}
If it had been defined outside a quasiquote it would generate something like:
implicit val implicitFieldname : Field.Aux[MyCaseClass, Witness.Lt[String]#T, String] = ...
Is there something that can be done?
This is my working solution using old-style macro annotations.
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
import scala.annotation.StaticAnnotation
class fieldable extends StaticAnnotation {
def macroTransform(annottees: Any*): Any = macro fieldableMacro.impl
}
object fieldableMacro {
def impl(c: Context)(annottees: c.Expr[Any]*): c.Tree = {
import c.universe._
annottees.map(_.tree) match {
case (param # q"case class $className(..$fields)") :: Nil => {
val implicits = fields.collect {
case field # q"$mods val $tname: $tpt" => q"""
implicit val $tname = Field.apply[$className,$tpt](
Witness(${tname.decodedName.toString}), _.$tname
)"""
}; q"$param; object ${className.toTermName} {..$implicits}"
}
}
}
}
It can be, for sure, improved using better quasiquotes, but my goal was to show something as cleaner as possible.
It can be used as:
#fieldable
case class MyCaseClass(foo: String, bar: Int)
This produces a MyCaseClass companion object having required Fields implicits:
implicit val foo = Field.apply[MyCaseClass, String](Witness("foo"), ((x$1) => x$1.foo));
implicit val bar = Field.apply[MyCaseClass, Int](Witness("bar"), ((x$2) => x$2.bar));
As it was already pointed out, without a complete working example, it's quite difficult to write an exhaustive answer.

How do I create an enum in scala that has an extra field

In Java I have something like this
public enum FlatFileHeaderMapping {
HEADER_EL(1),
HEADER_RESERVED1(5),
HEADER_RESERVED2(2),
HEADER_MESSAGE_TYPE(4)
public final int fieldSize;
private FlatFileHeaderMapping(int fieldSize) {
this.fieldSize = fieldSize;
}
}
which I can then use it place each line into a map and later access the keys in the map via this enum (like symbols)
Enumeration does not have this quality as far as I can see, and case classes are not ordered like the enum declarations - so cannot be used to match a record layout as shown above. At least not without the support of an ordered collection.
I could be missing something obvious, hence the question!
Thanks
Ray
overthink is right, but there's a less verbose way of declaring the case objects:
sealed abstract class FlatFileHeaderMapping(val fieldSize: Int)
case object HEADER_EL extends FlatFileHeaderMapping(1)
case object HEADER_RESERVED1 extends FlatFileHeaderMapping(5)
case object HEADER_RESERVED2 extends FlatFileHeaderMapping(2)
case object HEADER_MESSAGE_TYPE extends FlatFileHeaderMapping(4)
You could try using case objects:
sealed trait FlatFileHeaderMapping { val fieldSize: Int }
case object HEADER_EL extends FlatFileHeaderMapping { val fieldSize = 1 }
case object HEADER_RESERVED1 extends FlatFileHeaderMapping { val fieldSize = 5 }
case object HEADER_RESERVED2 extends FlatFileHeaderMapping { val fieldSize = 2 }
case object HEADER_MESSAGE_TYPE extends FlatFileHeaderMapping { val fieldSize = 4 }
You can then use the enum like so:
object Test {
def foo(x: FlatFileHeaderMapping) {
val result =
x match {
case HEADER_EL => "it's a HEADER_EL!"
case other => "its field size is: " + other.fieldSize
}
println(result)
}
def main(args: Array[String]) {
foo(HEADER_EL)
foo(HEADER_MESSAGE_TYPE)
}
}
The main nicety you get here is compile-time checking that all enum values are handled. i.e in the x match { ... } code above you'd get a compile error if you didn't have the 'case other => ...` clause in there.
I'm pretty much just restating this answer, which lists pros and cons of this approach.
object Direction extends Enumeration {
val North = Value("North")
val East = Value("East")
val South = Value("South")
val West = Value("West")
}
scala> import Direction._
scala> values foreach println
scala> val map = HashMap(North -> 1, South -> 2)
This is answered in Enumeration with constructor and lookup table
A simpler solution exist for integer value:
object FlatFileHeaderMapping extends Enumeration {
type FlatFileHeaderMapping = Value
val HEADER_EL = Value(1, "HEADER_EL")
val HEADER_RESERVED1 = Value(5, "HEADER_RESERVED1")
val HEADER_RESERVED2 = Value(2, "HEADER_RESERVED2")
val HEADER_MESSAGE_TYPE = Value(4, "HEADER_MESSAGE_TYPE")
}
Reproducing the contents of the accepted answer, as it's hidden behind a broken Tumblr link (that I accessed via Archive.org), which in turn points to this page.
trait Enum { //DIY enum type
import java.util.concurrent.atomic.AtomicReference //Concurrency paranoia
type EnumVal <: Value //This is a type that needs to be found in the implementing class
private val _values = new AtomicReference(Vector[EnumVal]()) //Stores our enum values
//Adds an EnumVal to our storage, uses CCAS to make sure it's thread safe, returns the ordinal
private final def addEnumVal(newVal: EnumVal): Int = { import _values.{get, compareAndSet => CAS}
val oldVec = get
val newVec = oldVec :+ newVal
if((get eq oldVec) && CAS(oldVec, newVec)) newVec.indexWhere(_ eq newVal) else addEnumVal(newVal)
}
def values: Vector[EnumVal] = _values.get //Here you can get all the enums that exist for this type
//This is the trait that we need to extend our EnumVal type with, it does the book-keeping for us
protected trait Value { self: EnumVal => //Enforce that no one mixes in Value in a non-EnumVal type
final val ordinal = addEnumVal(this) //Adds the EnumVal and returns the ordinal
def name: String //All enum values should have a name
override def toString = name //And that name is used for the toString operation
override def equals(other: Any) = this eq other.asInstanceOf[AnyRef]
override def hashCode = 31 * (this.getClass.## + name.## + ordinal)
}
}
//And here's how to use it, if you want compiler exhaustiveness checking
object Foos extends Enum {
sealed trait EnumVal extends Value /*{ you can define your own methods etc here }*/
val F = new EnumVal { val name = "F" }
val X = new EnumVal { val name = "X" }
}
/**
scala> Foos.values.find(_.name == "F")
res3: Option[Foos.EnumVal] = Some(F)
scala> Foos.X.ordinal
res4: Int = 1
scala> def doSmth(foo: Foos.EnumVal) = foo match {
case Foos.X => println("pigdog")
}
<console>:10: warning: match is not exhaustive!
missing combination $anon$1
missing combination $anon$2
scala> def doSmth(foo: Foos.EnumVal) = foo match {
case Foos.X => println("pigdog")
case Foos.F => println("dogpig")
}
doSmth: (foo: Foos.EnumVal)Unit
**/
//But if you don't care about getting exhaustiveness warnings, you can do:
object Foos extends Enum {
case class EnumVal private[Foos](name: String) extends Value /* { you can define your own methods and stuff here } */
val F = EnumVal("F")
val X = EnumVal("X")
}
/**
Which is a bit less boilerplatey.
Cheers,
√
**/