Related
I'm reading the scala-with-cats book and follow it's exercise. When I come to the case study: data validation, I encounter some problems.
Here is my entire code (just the same with the book):
package org.scala.ch10.final_recap
import cats.Semigroup
import cats.data.Validated
import cats.data.Validated._
import cats.data.Kleisli
import cats.data.NonEmptyList
import cats.instances.either._
import cats.syntax.apply._
import cats.syntax.semigroup._
import cats.syntax.validated._
sealed trait Predicate[E, A] {
import Predicate._
def and(that: Predicate[E, A]): Predicate[E, A] =
And(this, that)
def or(that: Predicate[E, A]): Predicate[E, A] =
Or(this, that)
/**
* This part is for Kleislis
* #return
*/
def run(implicit s: Semigroup[E]): A => Either[E, A] =
(a: A) => this(a).toEither
def apply(a: A)(implicit s: Semigroup[E]): Validated[E, A] =
this match {
case Pure(func) =>
func(a)
case And(left, right) => (left(a), right(a)).mapN((_, _) => a)
case Or(left, right) =>
left(a) match {
case Valid(_) => Valid(a)
case Invalid(e1) =>
right(a) match {
case Valid(_) => Invalid(e1)
case Invalid(e2) => Invalid(e1 |+| e2)
}
}
}
}
object Predicate {
final case class And[E, A](left: Predicate[E, A], right: Predicate[E, A]) extends Predicate[E, A]
final case class Or[E, A](left: Predicate[E, A], right: Predicate[E, A]) extends Predicate[E, A]
final case class Pure[E, A](func: A => Validated[E, A]) extends Predicate[E, A]
def apply[E, A](f: A => Validated[E, A]): Predicate[E, A] = Pure(f)
def lift[E, A](err: E, fn: A => Boolean): Predicate[E, A] = Pure(a => if(fn(a)) a.valid else err.invalid)
}
object FinalRecapPredicate {
type Errors = NonEmptyList[String]
def error(s: String): NonEmptyList[String] = NonEmptyList(s, Nil)
type Result[A] = Either[Errors, A]
type Check[A, B] = Kleisli[Result, A, B]
def check[A, B](func: A => Result[B]): Check[A, B] = Kleisli(func)
def checkPred[A](pred: Predicate[Errors, A]): Check[A, A] =
Kleisli[Result, A, A](pred.run)
def longerThan(n: Int): Predicate[Errors, String] =
Predicate.lift(
error(s"Must be longer than $n characters"),
str => str.length > n
)
val alphanumeric: Predicate[Errors, String] =
Predicate.lift(
error(s"Must be all alphanumeric characters"),
str => str.forall(_.isLetterOrDigit)
)
def contains(char: Char): Predicate[Errors, String] =
Predicate.lift(
error(s"Must contain the character $char"),
str => str.contains(char)
)
def containsOnce(char: Char): Predicate[Errors, String] =
Predicate.lift(
error(s"Must contain the character $char only once"),
str => str.count(_ == char) == 1
)
val checkUsername: Check[String, String] = checkPred(longerThan(3) and alphanumeric)
val splitEmail: Check[String, (String, String)] = check(_.split('#') match {
case Array(name, domain) =>
Right((name, domain))
case _ =>
Left(error("Must contain a single # character"))
})
val checkLeft: Check[String, String] = checkPred(longerThan(0))
val checkRight: Check[String, String] = checkPred(longerThan(3) and contains('.'))
val joinEmail: Check[(String, String), String] =
check {
case (l, r) => (checkLeft(l), checkRight(r)).mapN(_ + "#" + _)
}
val checkEmail: Check[String, String] = splitEmail andThen joinEmail
final case class User(username: String, email: String)
def createUser(username: String, email: String): Either[Errors, User] =
(checkUsername.run(username),
checkEmail.run(email)).mapN(User)
def main(args: Array[String]): Unit = {
println(createUser("", "noel#underscore.io#io"))
}
}
It supposes the code should end up with the error message Left(NonEmptyList(Must be longer than 3 characters), Must contain a single # character) But what I actually is Left(NonEmptyList(Must be longer than 3 characters))
Obviously, it does not work as expected. It fails fast instead of accumulating errors... How to fix that plz? (I've spent hours now and can't get a workaround)
This is the "problematic" part:
def createUser(username: String, email: String): Either[Errors, User] =
(checkUsername.run(username),
checkEmail.run(email)).mapN(User)
You are combining a tuple of Results, where
type Result[A] = Either[Errors, A]
This means you are really doing a mapN on a pair of Eithers, an operation provided by the Semigroupal type class. This operation will not accumulate results.
There are several reasons for this, but one that I find particularly important is the preserving of behaviour if we find ourselves using a Semigroupal / Applicative which also happens to be a Monad. Why is that a problem? Because Monads are sequencing operations, making each step depend on the previous one, and having "fail early" semantics. When using some Monad, one might expect those semantics to be preserved when using constructs from the underlying Applicative (every Monad is also an Applicative). In that case, if the implementation of Applicative used "accumulation" semantics instead of "fail early" semantics, we would ruin some important laws like referential transparency.
You can use a parallel version of mapN, called parMapN, whose contract guarantees that the implementation will be evaluating all results in parallel. This means that it definitely cannot be expected to have the "fail early" semantics, and accumulating results is fine in that case.
Note that Validated accumulates results as well, usually in a NonEmptyList or a NonEmptyChain. This is probably why you expected to see your accumulated results; the only problem is, you were not using Validated values in the "problematic" part of your code, but raw Eithers instead.
Here's some simple code that demonstrates the above concepts:
import cats.data._
import cats.implicits._
val l1: Either[String, Int] = Left("foo")
val l2: Either[String, Int] = Left("bar")
(l1, l2).mapN(_ + _)
// Left(foo)
(l1, l2).parMapN(_ + _)
// Left(foobar)
val v1: ValidatedNel[String, Int] = l1.toValidatedNel
val v2: ValidatedNel[String, Int] = l2.toValidatedNel
(v1, v2).mapN(_ + _)
// Invalid(NonEmptyList(foo, bar))
The following code succeeds, but is there a better way of doing the same thing? Perhaps something specific to case classes? In the following code, for each field of type String in my simple case class, the code goes through my list of instances of that case class and finds the length of the longest string of that field.
case class CrmContractorRow(
id: Long,
bankCharges: String,
overTime: String,
name$id: Long,
mgmtFee: String,
contractDetails$id: Long,
email: String,
copyOfVisa: String)
object Go {
def main(args: Array[String]) {
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
c.getClass.getDeclaredFields.filter(p => p.getType == classOf[String]).foreach{f =>
f.setAccessible(true)
println(f.getName + ": " + rows.map(row => f.get(row).asInstanceOf[String]).maxBy(_.length))
}
}
}
Result:
bankCharges: 3
overTime: 3
mgmtFee: 5
email: 9
copyOfVisa: 3
If you want to do this kind of thing with Shapeless, I'd strongly suggest defining a custom type class that handles the complicated part and allows you to keep that stuff separate from the rest of your logic.
In this case it sounds like the tricky part of what you're specifically trying to do is getting the mapping from field names to string lengths for all of the String members of a case class. Here's a type class that does that:
import shapeless._, shapeless.labelled.FieldType
trait StringFieldLengths[A] { def apply(a: A): Map[String, Int] }
object StringFieldLengths extends LowPriorityStringFieldLengths {
implicit val hnilInstance: StringFieldLengths[HNil] =
new StringFieldLengths[HNil] {
def apply(a: HNil): Map[String, Int] = Map.empty
}
implicit def caseClassInstance[A, R <: HList](implicit
gen: LabelledGeneric.Aux[A, R],
sfl: StringFieldLengths[R]
): StringFieldLengths[A] = new StringFieldLengths[A] {
def apply(a: A): Map[String, Int] = sfl(gen.to(a))
}
implicit def hconsStringInstance[K <: Symbol, T <: HList](implicit
sfl: StringFieldLengths[T],
key: Witness.Aux[K]
): StringFieldLengths[FieldType[K, String] :: T] =
new StringFieldLengths[FieldType[K, String] :: T] {
def apply(a: FieldType[K, String] :: T): Map[String, Int] =
sfl(a.tail).updated(key.value.name, a.head.length)
}
}
sealed class LowPriorityStringFieldLengths {
implicit def hconsInstance[K, V, T <: HList](implicit
sfl: StringFieldLengths[T]
): StringFieldLengths[FieldType[K, V] :: T] =
new StringFieldLengths[FieldType[K, V] :: T] {
def apply(a: FieldType[K, V] :: T): Map[String, Int] = sfl(a.tail)
}
}
This looks complex, but once you start working with Shapeless a bit you learn to write this kind of thing in your sleep.
Now you can write the logic of your operation in a relatively straightforward way:
def maxStringLengths[A: StringFieldLengths](as: List[A]): Map[String, Int] =
as.map(implicitly[StringFieldLengths[A]].apply).foldLeft(
Map.empty[String, Int]
) {
case (x, y) => x.foldLeft(y) {
case (acc, (k, v)) =>
acc.updated(k, acc.get(k).fold(v)(accV => math.max(accV, v)))
}
}
And then (given rows as defined in the question):
scala> maxStringLengths(rows).foreach(println)
(bankCharges,3)
(overTime,3)
(mgmtFee,5)
(email,9)
(copyOfVisa,3)
This will work for absolutely any case class.
If this is a one-off thing, you might as well use runtime reflection, or you could use the Poly1 approach in Giovanni Caporaletti's answer—it's less generic and it mixes up the different parts of the solution in a way I don't prefer, but it should work just fine. If this is something you're doing a lot of, though, I'd suggest the approach I've given here.
If you want to use shapeless to get the string fields of a case class and avoid reflection you can do something like this:
import shapeless._
import labelled._
trait lowerPriorityfilterStrings extends Poly2 {
implicit def default[A] = at[Vector[(String, String)], A] { case (acc, _) => acc }
}
object filterStrings extends lowerPriorityfilterStrings {
implicit def caseString[K <: Symbol](implicit w: Witness.Aux[K]) = at[Vector[(String, String)], FieldType[K, String]] {
case (acc, x) => acc :+ (w.value.name -> x)
}
}
val gen = LabelledGeneric[CrmContractorRow]
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
val result = rows
// get for each element a Vector of (fieldName -> stringField) pairs for the string fields
.map(r => gen.to(r).foldLeft(Vector[(String, String)]())(filterStrings))
// get the maximum for each "column"
.reduceLeft((best, row) => best.zip(row).map {
case (kv1#(_, v1), (_, v2)) if v1.length > v2.length => kv1
case (_, kv2) => kv2
})
result foreach { case (k, v) => println(s"$k: $v") }
You probably want to use Scala reflection:
import scala.reflect.runtime.universe._
val rm = runtimeMirror(getClass.getClassLoader)
val instanceMirrors = rows map rm.reflect
typeOf[CrmContractorRow].members collect {
case m: MethodSymbol if m.isCaseAccessor && m.returnType =:= typeOf[String] =>
val maxValue = instanceMirrors map (_.reflectField(m).get.asInstanceOf[String]) maxBy (_.length)
println(s"${m.name}: $maxValue")
}
So that you can avoid issues with cases like:
case class CrmContractorRow(id: Long, bankCharges: String, overTime: String, name$id: Long, mgmtFee: String, contractDetails$id: Long, email: String, copyOfVisa: String) {
val unwantedVal = "jdjd"
}
Cheers
I have refactored your code to something more reuseable:
import scala.reflect.ClassTag
case class CrmContractorRow(
id: Long,
bankCharges: String,
overTime: String,
name$id: Long,
mgmtFee: String,
contractDetails$id: Long,
email: String,
copyOfVisa: String)
object Go{
def main(args: Array[String]) {
val a = CrmContractorRow(1,"1","1",4444,"1",1,"1","1")
val b = CrmContractorRow(22,"22","22",22,"55555",22,"nine long","22")
val c = CrmContractorRow(333,"333","333",333,"333",333,"333","333")
val rows = List(a,b,c)
val initEmptyColumns = List.fill(a.productArity)(List())
def aggregateColumns[Tin:ClassTag,Tagg](rows: Iterable[Product], aggregate: Iterable[Tin] => Tagg) = {
val columnsWithMatchingType = (0 until rows.head.productArity).filter {
index => rows.head.productElement(index) match {case t: Tin => true; case _ => false}
}
def columnIterable(col: Int) = rows.map(_.productElement(col)).asInstanceOf[Iterable[Tin]]
columnsWithMatchingType.map(index => (index,aggregate(columnIterable(index))))
}
def extractCaseClassFieldNames[T: scala.reflect.ClassTag] = {
scala.reflect.classTag[T].runtimeClass.getDeclaredFields.filter(!_.isSynthetic).map(_.getName)
}
val agg = aggregateColumns[String,String] (rows,_.maxBy(_.length))
val fieldNames = extractCaseClassFieldNames[CrmContractorRow]
agg.map{case (index,value) => fieldNames(index) + ": "+ value}.foreach(println)
}
}
Using shapeless would get rid of the .asInstanceOf, but the essence would be the same. The main problem with the given code was that it was not re-usable since the aggregation logic was mixed with the reflection logic to get the field names.
When I learn State Monad, I'm not sure how to compose two functions with different State return types.
State Monad definition:
case class State[S, A](runState: S => (S, A)) {
def flatMap[B](f: A => State[S, B]): State[S, B] = {
State(s => {
val (s1, a) = runState(s)
val (s2, b) = f(a).runState(s1)
(s2, b)
})
}
def map[B](f: A => B): State[S, B] = {
flatMap(a => {
State(s => (s, f(a)))
})
}
}
Two different State types:
type AppendBang[A] = State[Int, A]
type AddOne[A] = State[String, A]
Two methods with differnt State return types:
def addOne(n: Int): AddOne[Int] = State(s => (s + ".", n + 1))
def appendBang(str: String): AppendBang[String] = State(s => (s + 1, str + " !!!"))
Define a function to use the two functions above:
def myAction(n: Int) = for {
a <- addOne(n)
b <- appendBang(a.toString)
} yield (a, b)
And I hope to use it like this:
println(myAction(1))
The problem is myAction is not compilable, it reports some error like this:
Error:(14, 7) type mismatch;
found : state_monad.State[Int,(Int, String)]
required: state_monad.State[String,?]
b <- appendBang(a.toString)
^
How can I fix it? Do I have to define some Monad transformers?
Update: The question may be not clear, let me give an example
Say I want to define another function, which uses addOne and appendBang internally. Since they all need existing states, I have to pass some to it:
def myAction(n: Int)(addOneState: String, appendBangState: Int): ((String, Int), String) = {
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState(appendBangState)
((addOneState2, appendBangState2), n3)
}
I have to run addOne and appendBang one by one, passing and getting the states and result manually.
Although I found it can return another State, the code is not improved much:
def myAction(n: Int): State[(String, Int), String] = State {
case (addOneState: String, appendBangState: Int) =>
val (addOneState2, n2) = addOne(n).runState(addOneState)
val (appendBangState2, n3) = appendBang(n2.toString).runState( appendBangState)
((addOneState2, appendBangState2), n3)
}
Since I'm not quite familiar with them, just wondering is there any way to improve it. The best hope is that I can use for comprehension, but not sure if that's possible
Like I mentioned in my first comment, it will be impossible to use a for comprehension to do what you want, because it can not change the type of the state (S).
Remember that a for comprehension can be translated to a combination of flatMaps, withFilter and one map. If we look at your State.flatMap, it takes a function f to change a State[S,A] into State[S, B]. We can use flatMap and map (and thus a for comprehension) to chain together operations on the same state, but we can't change the type of the state in this chain.
We could generalize your last definition of myAction to combine, compose, ... two functions using state of a different type. We can try to implement this generalized compose method directly in our State class (although this is probably so specific, it probably doesn't belong in State). If we look at State.flatMap and myAction we can see some similarities:
We first call runState on our existing State instance.
We then call runState again
In myAction we first use the result n2 to create a State[Int, String] (AppendBang[String] or State[S2, B]) using the second function (appendBang or f) on which we then call runState. But our result n2 is of type String (A) and our function appendBang needs an Int (B) so we need a function to convert A into B.
case class State[S, A](runState: S => (S, A)) {
// flatMap and map
def compose[B, S2](f: B => State[S2, B], convert: A => B) : State[(S, S2), B] =
State( ((s: S, s2: S2) => {
val (sNext, a) = runState(s)
val (s2Next, b) = f(convert(a)).runState(s2)
((sNext, s2Next), b)
}).tupled)
}
You then could define myAction as :
def myAction(i: Int) = addOne(i).compose(appendBang, _.toString)
val twoStates = myAction(1)
// State[(String, Int),String] = State(<function1>)
twoStates.runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)
If you don't want this function in your State class you can create it as an external function :
def combineStateFunctions[S1, S2, A, B](
a: A => State[S1, A],
b: B => State[S2, B],
convert: A => B
)(input: A): State[(S1, S2), B] = State(
((s1: S1, s2: S2) => {
val (s1Next, temp) = a(input).runState(s1)
val (s2Next, result) = b(convert(temp)).runState(s2)
((s1Next, s2Next), result)
}).tupled
)
def myAction(i: Int) =
combineStateFunctions(addOne, appendBang, (_: Int).toString)(i)
Edit : Bergi's idea to create two functions to lift a State[A, X] or a State[B, X] into a State[(A, B), X].
object State {
def onFirst[A, B, X](s: State[A, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextA, x) = s.runState(a)
((nextA, b), x)
}
State(runState.tupled)
}
def onSecond[A, B, X](s: State[B, X]): State[(A, B), X] = {
val runState = (a: A, b: B) => {
val (nextB, x) = s.runState(b)
((a, nextB), x)
}
State(runState.tupled)
}
}
This way you can use a for comprehension, since the type of the state stays the same ((A, B)).
def myAction(i: Int) = for {
x <- State.onFirst(addOne(i))
y <- State.onSecond(appendBang(x.toString))
} yield y
myAction(1).runState(("", 1))
// ((String, Int), String) = ((.,2),2 !!!)
I thought PartialFunction can be Monoid. Is my thought process correct ?
For example,
import scalaz._
import scala.{PartialFunction => -->}
implicit def partialFunctionSemigroup[A,B]:Semigroup[A-->B] = new Semigroup[A-->B]{
def append(s1: A-->B, s2: => A-->B): A-->B = s1.orElse(s2)
}
implicit def partialFunctionZero[A,B]:Zero[A-->B] = new Zero[A-->B]{
val zero = new (A-->B){
def isDefinedAt(a:A) = false
def apply(a:A) = sys.error("error")
}
}
But current version Scalaz(6.0.4) is not included it. Is there a reason for something not included ?
Let's shine a different light on this.
PartialFunction[A, B] is isomorphic to A => Option[B]. (Actually, to be able to check if it is defined for a given A without triggering evaluation of the B, you would need A => LazyOption[B])
So if we can find a Monoid[A => Option[B]] we've proved your assertion.
Given Monoid[Z], we can form Monoid[A => Z] as follows:
implicit def readerMonoid[Z: Monoid] = new Monoid[A => Z] {
def zero = (a: A) => Monoid[Z].zero
def append(f1: A => Z, f2: => A => Z) = (a: A) => Monoid[Z].append(f1(a), f2(a))
}
So, what Monoid(s) do we have if we use Option[B] as our Z? Scalaz provides three. The primary instance requires a Semigroup[B].
implicit def optionMonoid[B: Semigroup] = new Monoid[Option[B]] {
def zero = None
def append(o1: Option[B], o2: => Option[B]) = o1 match {
case Some(b1) => o2 match {
case Some(b2) => Some(Semigroup[B].append(b1, b2)))
case None => Some(b1)
case None => o2 match {
case Some(b2) => Some(b2)
case None => None
}
}
}
Using this:
scala> Monoid[Option[Int]].append(Some(1), Some(2))
res9: Option[Int] = Some(3)
But that's not the only way to combine two Options. Rather than appending the contents of the two options in the case they are both Some, we could simply pick the first or the last of the two. Two trigger this, we create a distinct type with trick called Tagged Types. This is similar in spirit to Haskell's newtype.
scala> import Tags._
import Tags._
scala> Monoid[Option[Int] ## First].append(Tag(Some(1)), Tag(Some(2)))
res10: scalaz.package.##[Option[Int],scalaz.Tags.First] = Some(1)
scala> Monoid[Option[Int] ## Last].append(Tag(Some(1)), Tag(Some(2)))
res11: scalaz.package.##[Option[Int],scalaz.Tags.Last] = Some(2)
Option[A] ## First, appended through it's Monoid, uses the same orElse semantics as your example.
So, putting this all together:
scala> Monoid[A => Option[B] ## First]
res12: scalaz.Monoid[A => scalaz.package.##[Option[B],scalaz.Tags.First]] =
scalaz.std.FunctionInstances0$$anon$13#7e71732c
No, this looks good, satisfying both the requirements for (non-commutative) Monoid. Interesting idea. What use case are you trying to support?
Your zero certainly violates the axiom for the identity element, but I think the identity (partial) function would be OK.
Your append also doesn't fulfill the Monoid laws, but instead of orElse you could call andThen (composition). But this would only work for A == B:
implicit def partialFunctionSemigroup[A]: Semigroup[A --> A] = new Semigroup[A --> A] {
def append(s1: A --> A, s2: => A --> A): A-->A = s1 andThen s2
}
implicit def partialFunctionZero[A]: Zero[A --> A] = new Zero[A --> A] {
val zero = new (A --> A) {
def isDefinedAt(a:A) = true
def apply(a:A) = a
}
}
I have two functions.
def process(date: DateTime, invoice: Invoice, user: User, reference: Reference) : (Action, Iterable[Billable])
def applyDiscount(billable: Billable) : Billable
How can I compose these so that I have a single function of (DateTime, Invoice, User, Reference) => (Action, Iterable[Billable])
Here is the poor mans way of what I want
def buildFromInvoice(user: User, order: Invoice, placementDate: DateTime, reference: Reference) = {
val ab = billableBuilder.fromInvoice(user, order, placementDate, reference)
(ab._1, ab._2.map(applyDiscount(_))
}
What you have (simplifying) is:
val f: A => (B, M[C]) //M is a Functor
val g: C => C
I can think of a few ways of doing this. I think my preference is:
(a: A) => g.lift[M].second apply f(a)
Or also:
(a: A) => f(a) :-> g.lift[M]
However, there is possibly a pointfree way - although not necessarily so, of course
lift is a method on Function1W which lifts the function into the realm of the functor M
second is a method on MAB which applies the function down the right-hand-side of a Bifunctor
:-> is a method available to Bifunctors denoting the application of a function on the rhs.
EDIT - missingfaktor appears to be correct in saying f andThen g.lift[M].second works:
scala> import scalaz._; import Scalaz._
import scalaz._
import Scalaz._
scala> case class A(); case class B(); case class C()
defined class A
defined class B
defined class C
scala> lazy val f: A => (B, List[C]) = sys.error("")
f: A => (B, List[C]) = <lazy>
scala> lazy val g: C => C = sys.error("")
g: C => C = <lazy>
Pointfree:
scala> lazy val h = f andThen g.lift[List].second
h: A => (B, List[C]) = <lazy>