I'm learning the whole Cats-Effect FP framework, and am wondering about how to best implement logging. I'm struggling with finding an elegant way to use logging in methods that return non-side-effect types, particularly class constructors. For example:
case class Limit(name: String, value: Int)
object Limit {
def max(name: String): Limit = Limit(name, Int.MaxValue)
}
Now, if I want to add logging to the construction, which is obviously a side-effect, I would have to change the method signature to something like:
object Limit {
def max[F[_] : Sync](name: String): F[Limit] = {
for {
_ <- Slf4jLogger.getLogger[F].debug("Creating max limit.")
} yield {
Limit(name, Int.MaxValue)
}
}
}
Such a change breaks code compatibility, and now every place that uses Limit.max needs to be refactored to deal with the type signatures, mapping and/or call .unsafeRun. Quite a radical change for something as small as adding a log line.
Is there some way to do this more elegantly? Do I just drop log4cats and use the regular, unsafe SLF4J? Or do I just need to accept that in PF/Cats-Effect world it's safer to just return F[..] from everything everywhere, no matter how small or trivial the method is?
I think it would be better to use log4cats library. If you use cats-effect, probably your Main class returns IO[Exit] and almost whole your logic wouldn't require any unsafe calls.
But at the same time I saw a lot of projects that uses just a plain logging without wrapping logs inside of IO. It depends on you and/or your team/project. I personally prefer FP way.
I left an example of using log4cats here:
implicit def logger[F[_]: Sync]: Logger[F] = Slf4jLogger.getLogger[F]
case class Limit(name: String, value: Int)
object Limit {
def max[F[_] : Sync: Logger](name: String): F[Limit] = {
for {
_ <- Logger[F].info("Creating max limit.")
} yield {
Limit(name, Int.MaxValue)
}
}
}
build.sbt file:
libraryDependencies ++= Seq(
"org.typelevel" %% "log4cats-core" % "2.5.0", // Only if you want to Support Any Backend
"org.typelevel" %% "log4cats-slf4j" % "2.5.0", // Direct Slf4j Support - Recommended
// and some backend such as logback or smth
)
Related
I'm building a data binding library, which has 3 fundamental classes
trait DValue[+T] {
def get:T
}
class DField[T] extends DValue[T] {
// allow writes + notifying observers
}
class DFunction[T](deps:DValue[_]*)(compute :=> T) extends DValue[T] {
def get = compute // internally compute will use values in deps
}
However, in this approach, the definition of DFunction is not quite robust - it requires the user of DFunction to make sure all DValues used in compute are put into the 'deps' list. So I want the user to be able to do something like this:
val dvCount:DValue[Int] = DField(3)
val dvElement:DValue[String] = DField("Hello")
val myFunction = DFunction(dvCount, dvElement) { (count, element) => // compiler knows their type
Range(count).map(_ => element).toSeq
}
As you can see when I'm constructing 'myFunction', the referenced fields and the usage is clearly mapped.
I feel maybe HList would allow me to provide something at library level that'd allow this, but I cannot figure out how, would this be possible with HList? or there's something else that'd help achieve this?
shapeless.ops.hlist.Mapper allows you to do this with a Poly function.
Unfortunately the documentation on it isn't great; you might need to do some source diving to see how to use it
I understand that generally speaking there is a lot to say about deciding what one wants to model as effect This discussion is introduce in Functional programming in Scala on the chapter on IO.
Nonethless, I have not finished the chapter, i was just browsing it end to end before takling it together with Cats IO.
In the mean time, I have a bit of a situation for some code I need to deliver soon at work.
It relies on a Java Library that is just all about mutation. That library was started a long time ago and for legacy reason i don't see them changing.
Anyway, long story short. Is actually modeling any mutating function as IO a viable way to encapsulate a mutating java library ?
Edit1 (at request I add a snippet)
Readying into a model, mutate the model rather than creating a new one. I would contrast jena to gremlin for instance, a functional library over graph data.
def loadModel(paths: String*): Model =
paths.foldLeft(ModelFactory.createOntologyModel(new OntModelSpec(OntModelSpec.OWL_MEM)).asInstanceOf[Model]) {
case (model, path) ⇒
val input = getClass.getClassLoader.getResourceAsStream(path)
val lang = RDFLanguages.filenameToLang(path).getName
model.read(input, "", lang)
}
That was my scala code, but the java api as documented in the website look like this.
// create the resource
Resource r = model.createResource();
// add the property
r.addProperty(RDFS.label, model.createLiteral("chat", "en"))
.addProperty(RDFS.label, model.createLiteral("chat", "fr"))
.addProperty(RDFS.label, model.createLiteral("<em>chat</em>", true));
// write out the Model
model.write(system.out);
// create a bag
Bag smiths = model.createBag();
// select all the resources with a VCARD.FN property
// whose value ends with "Smith"
StmtIterator iter = model.listStatements(
new SimpleSelector(null, VCARD.FN, (RDFNode) null) {
public boolean selects(Statement s) {
return s.getString().endsWith("Smith");
}
});
// add the Smith's to the bag
while (iter.hasNext()) {
smiths.add(iter.nextStatement().getSubject());
}
So, there are three solutions to this problem.
1. Simple and dirty
If all the usage of the impure API is contained in single / small part of the code base, you may just "cheat" and do something like:
def useBadJavaAPI(args): IO[Foo] = IO {
// Everything inside this block can be imperative and mutable.
}
I said "cheat" because the idea of IO is composition, and a big IO chunk is not really composition. But, sometimes you only want to encapsulate that legacy part and do not care about it.
2. Towards composition.
Basically, the same as above but dropping some flatMaps in the middle:
// Instead of:
def useBadJavaAPI(args): IO[Foo] = IO {
val a = createMutableThing()
mutableThing.add(args)
val b = a.bar()
b.computeFoo()
}
// You do something like this:
def useBadJavaAPI(args): IO[Foo] =
for {
a <- IO(createMutableThing())
_ <- IO(mutableThing.add(args))
b <- IO(a.bar())
result <- IO(b.computeFoo())
} yield result
There are a couple of reasons for doing this:
Because the imperative / mutable API is not contained in a single method / class but in a couple of them. And the encapsulation of small steps in IO is helping you to reason about it.
Because you want to slowly migrate the code to something better.
Because you want to feel better with yourself :p
3. Wrap it in a pure interface
This is basically the same that many third party libraries (e.g. Doobie, fs2-blobstore, neotypes) do. Wrapping a Java library on a pure interface.
Note that as such, the amount of work that has to be done is way more than the previous two solutions. As such, this is worth it if the mutable API is "infecting" many places of your codebase, or worse in multiple projects; if so then it makes sense to do this and publish is as an independent module.
(it may also be worth to publish that module as an open-source library, you may end up helping other people and receive help from other people as well)
Since this is a bigger task is not easy to just provide a complete answer of all you would have to do, it may help to see how those libraries are implemented and ask more questions either here or in the gitter channels.
But, I can give you a quick snippet of how it would look like:
// First define a pure interface of the operations you want to provide
trait PureModel[F[_]] { // You may forget about the abstract F and just use IO instead.
def op1: F[Int]
def op2(data: List[String]): F[Unit]
}
// Then in the companion object you define factories.
object PureModel {
// If the underlying java object has a close or release action,
// use a Resource[F, PureModel[F]] instead.
def apply[F[_]](args)(implicit F: Sync[F]): F[PureModel[F]] = ???
}
Now, how to create the implementation is the tricky part.
Maybe you can use something like Sync to initialize the mutable state.
def apply[F[_]](args)(implicit F: Sync[F]): F[PureModel[F]] =
F.delay(createMutableState()).map { mutableThing =>
new PureModel[F] {
override def op1: F[Int] = F.delay(mutableThing.foo())
override def op2(data: List[String]): F[Unit] = F.delay(mutableThing.bar(data))
}
}
By trial and probably error, as a plugin author I've fallen into using the following style, which seems to work:
object AmazingPlugin extends AutoPlugin {
object autoImport {
val amaze = TaskKey[Amazement]("Do something totally effing amazing")
}
import autoImport._
lazy val ethDefaults : Seq[sbt.Def.Setting[_]] = Seq(
amaze in Compile := { amazeTask( Compile ).value },
amaze in Test := { amazeTask( Test ).value }
)
def amazeTask( config : Configuration ) : Initialize[Task[Amazement]] = Def.task {
???
}
}
Unfortunately, I do not actually understand how all of these constructs work, why it is an Initialize[Task[T]] I generate rather than a Task[T], for example. I assume that this idiom "does the right thing", which I take to mean that the amazeTask function generates some wrapper or seed or generator of an immutable Task for each Configuration and binds it just once to the appropriate task key. But it is entirely opaque to me how this might work. For example, when I look up Initialize, I see a value method that requires an argument, which I do not supply in the idiom above. I assume in designing the SBT configuration DSL tricks with implicits and/or macros were used, and shrug it off.
However, recently I've wanted to factor out some logic from my tasks into what are logically private functions, but which also require access to the value of tasks and settings. If just used private functions, gathering all the arguments would become repetitive boilerplate of the form
val setting1 = (setting1 in config).value
val setting2 = (setting2 in config).value
...
val settingN = (settingN in config).value
val derivedValue = somePrivateFunction( setting1, setting2 ... settingN )
everywhere I need to use get the value derived from settings. So it's better to factor all this into a derivedValue task, and I can replace all of the above with
derivedValue.value
Kewl.
But I do want derivedValue to be private, so I don't bind it to a task key. I just do this:
private def findDerivedValueTask( config : Configuration ) : Initialize[Task[DerivedValue]] = Def.task {
val setting1 = (setting1 in config).value
val setting2 = (setting2 in config).value
...
val settingN = (settingN in config).value
somePrivateFunction( setting1, setting2 ... settingN )
}
Now it works fine to implement my real, public task as...
def amazeTask( config : Configuration ) : Initialize[Task[Amazement]] = Def.task {
val derivedValue = findDerivedValueTask( config ).value
// do stuff with derivedValue that computes and Amazement object
???
}
Great! It really does work just fine.
But I have a hard time presuming that there is some magic by which this idiom does the right thing, that is generate an immutable Task object just once per config and reuse it. So, I think to myself, I should memoize the findDerivedValueTask function, so that after it generates a task, the task gets stored in a Map, whose keys are Configurations but whose values are logically Tasks.
But now my nonunderstanding of what happens behind the scenes bites. Should I store Initialize[Task[DerivedValue]] or just Task[DerivedValue], or what? Do I have to bother, does sbt have some clever magic that is already taking care of this for me? I really just don't know.
If you have read this far, I am very grateful. If you can clear this up, or point me to documentation that explains how this stuff works, I'll be even more grateful. Thank you!
Edit: I updated the question to be more descriptive.
Note: I use the Scala 2.11 compiler, because that is the compiler version used by the LMS tutorial project.
I am porting a DSL written in Haskell to Scala. The DSL is an imperative language, so I used monads with do-notation, namely WriterT [Stmt] (State Label) a. I had trouble porting this to Scala, but worked around it by using the ReaderWriterState monad and just using Unit for the Reader component. I then went about searching for an alternative for the do-notation found in Haskell. For-comprehensions are supposed to be this alternative in Scala, but they are tailored towards sequences, I e.g. could not pattern match a tuple, it would insert a call to filter. So I started looking for alternatives and found multiple libraries: effectful, monadless, and each. I tried effectful first which does exactly what I wanted to achieve, I even prefer it over Haskell's do-notation, and it worked well with the ReaderWriterState monad from ScalaZ I had been using. In my DSL I have actions like Drop() (a case class) that I want to be able to use directly as a statement. I hoped to use implicits for this, but because the ! method of effectful (or monadless equivalent for that matter) is too general, I cannot get Scala to automatically convert my Action case classes to something of type Stmt (the ReaderWriterState that returns Unit).
So if not implicits, would there be a different way to achieve it?
As shown in Main.passes2, I did figure out a workaround that I would not mind using, but I am curious whether I stumbled upon is some limitation of the language, or is simply my lack of experience with Scala.
With Main.fails1 I will get the following error message: Implicit not found: scalaz.Unapply[scalaz.Monad, question.Label]. Unable to unapply type question.Label into atype constructor of kind M[_] that is classified by the type class scalaz.Monad. Check that the type class is defined by compiling implicitly[scalaz.Monad[type constructor]] and review the implicits in object Unapply, which only cover common type 'shapes.'
Which comes from ScalaZ its Unapply: https://github.com/scalaz/scalaz/blob/d2aba553e444f951adc847582226d617abae24da/core/src/main/scala/scalaz/Unapply.scala#L50
And with Main.fails2 I will just get: value ! is not a member of question.Label
I assume it is just a matter of writing the missing implicit definition, but I am not quite sure which one Scala wants me to write.
The most important bits of my build.sbt are the version:
scalaVersion := "2.11.2",
And the dependencies:
libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.11.2",
libraryDependencies += "org.scala-lang" % "scala-library" % "2.11.2",
libraryDependencies += "org.scala-lang" % "scala-reflect" % "2.11.2",
libraryDependencies += "org.pelotom" %% "effectful" % "1.0.1",
Here is the relevant code, containing the things I tried and the code necessary to run those things:
package question
import scalaz._
import Scalaz._
import effectful._
object DSL {
type GotoLabel = Int
type Expr[A] = ReaderWriterState[Unit, List[Action], GotoLabel, A]
type Stmt = Expr[Unit]
def runStmt(stmt: Stmt, startLabel: GotoLabel): (Action, GotoLabel) = {
val (actions, _, updatedLabel) = stmt.run(Unit, startLabel)
val action = actions match {
case List(action) => action
case _ => Seq(actions)
}
(action, updatedLabel)
}
def runStmt(stmt: Stmt): Action = runStmt(stmt, 0)._1
def getLabel(): Expr[GotoLabel] =
ReaderWriterState((_, label) => (Nil, label, label))
def setLabel(label: GotoLabel): Stmt =
ReaderWriterState((_, _) => (Nil, Unit, label))
implicit def actionStmt(action: Action): Stmt =
ReaderWriterState((_, label) => (List(action), Unit, label))
}
import DSL._
final case class Label(label: String) extends Action
final case class Goto(label: String) extends Action
final case class Seq(seq: List[Action]) extends Action
sealed trait Action {
def stmt(): Stmt = this
}
object Main {
def freshLabel(): Expr[String] = effectfully {
val label = getLabel.! + 1
setLabel(label).!
s"ants-$label"
}
def passes0() =
freshLabel()
.flatMap(before => Label(before))
.flatMap(_ => freshLabel())
.flatMap(after => Label(after));
def passes1() = effectfully {
unwrap(actionStmt(Label(unwrap(freshLabel()))))
unwrap(actionStmt(Label(unwrap(freshLabel()))))
}
def fails1() = effectfully {
unwrap(Label(unwrap(freshLabel())))
unwrap(Label(unwrap(freshLabel())))
}
def pasess2() = effectfully {
Label(freshLabel.!).stmt.!
Label(freshLabel.!).stmt.!
}
def fails2() = effectfully {
Label(freshLabel.!).!
Label(freshLabel.!).!
}
def main(args: Array[String]): Unit = {
runStmt(passes0())
}
}
Question quality
I want to start with complaining about the question quality. You provide almost no textual description of what you are trying to achieve and then just show us a wall of code but don't explicitly reference your dependencies. This is far from what could count as a Minimal, Complete, and Verifiable example. And typically you get much more chances to get some answers if you provide a clear problem that is easy to understand and reproduce.
Back to the business
When you write something like
unwrap(Label(unwrap(freshLabel())))
you ask too much from the Scala compiler. Particularly unwrap can unwrap only something that is wrapped into some Monad, but Label is not a monad. It is not just the fact that there is no Monad[Label] instance, it is the fact that it structurally doesn't fit that kills you. Simply speaking an instance of ScalaZ Unapply is an object that allows you to split an applied generic type MonadType[SpecificType] (or other FunctorType[SpecificType]) into an "unapplied"/"partially-applied" MonadType[_] and SpecificType even when (as in your case) MonadType[_] is actually something complicated like ReaderWriterState[Unit, List[Action], GotoLabel, _]. And so the error says that there is no known way to split Label into some MonadType[_] and SpecifictType. You probably expected that your implicit actionStmt would do the trick to auto-convert Label into a Statement but this step is too much for the Scala compiler because for it to work, it also implies splitting the composite type. Note that this conversion is far from obvious for the compiler as the unwrap is itself a generic method that can handle any Monad. Actually in some sense ScalaZ needs Unapply exactly because the compiler can't do such things automatically. Still if you help the compiler just a little bit by specifying generic type, it can do the rest of the work:
def fails1() = effectfully {
// fails
// unwrap(Label(unwrap(freshLabel())))
// unwrap(Label(unwrap(freshLabel())))
// works
unwrap[Stmt](Label(unwrap(freshLabel())))
unwrap[Stmt](Label(unwrap(freshLabel())))
}
There is also another possible solution but it is a quite dirty hack: you can roll out your custom Unapply to persuade the compiler that Label is actually the same as Expr[Unit] which can be split as Expr[_] and Unit:
implicit def unapplyAction[AC <: Action](implicit TC0: Monad[Expr]): Unapply[Monad, AC] {
type M[X] = Expr[X]
type A = Unit
} = new Unapply[Monad, AC] {
override type M[X] = Expr[X]
override type A = Unit
override def TC: Monad[Expr] = TC0
// This can't be implemented because Leibniz really witness only exactly the same types rather than some kind of isomorphism
// Luckily effectful doesn't use leibniz implementation
override def leibniz: AC === Expr[Unit] = ???
}
The obvious reason why this is a dirty hack is that Label is actually not the same as Expr[Unit] and you can see it by the fact that you can't implement leibniz in your Unapply. Anyway, if you import unapplyAction, even your original fails1 will compile and work because effectful doesn't use leibniz inside.
As for your fails2, I don't think you can make it work in any simple way. Probably the only way that you may try is to create another macro that would convert in-place call to ! on your action (or its implicit wrapper) into the effectful's ! on the action.stmt. This might work but I didn't try it.
I asked myself this question a couple of times and came up with a solution for that feels very dirty. Maybe you can give me any advice since I think this is a basic problem for every DSL written in Scala.
I want to have a hierarchical structure of nested objects without adding any extra syntax. Specs is a good example for this:
MySpec extends Specification {
"system" should {
"example0" in { ... }
"example1" in { ... }
"example2" in { ... }
}
"system" can {
"example0" in { ... }
}
}
For instance I do not have to write "example0" in { ... } :: "example1" in { ... } :: "example2" in { ... } :: Nil.
This is exactly the same behaviour I would like to have. I think this is achieved by an implicit definition in the Specification class in Specs like (please do not be offended if you are the Specs author and I missunderstood something :))
implicit def sus2spec(sus: Sus): Specification = {
suslist += sus
this
}
My main problem arises now when I want to nest such objects. Imagine I have this grammar:
root: statement*;
statement:
IDENT '{' statement* '}'
| declaration*
;
declaration: IDENT ':=' INT+;
I would like to translate this into a DSL that looks like this:
MyRoot extends Root {
"statement0" is {
"nested_statement0" is {
"nested_nested_statement0" is {
"declaration0" := 0
}
"declaration1" := 1
"declaration2" := 2
}
"declaration3" := 3
}
"statement1" is {
"declaration4" := 4
}
}
The problem that arises here is for me that the implicit solution does not work. The implicit definition would be executed in the scope of the root object which means I would add all objects to the root and the hierarchy is lost.
Then I thought I can use something like a Stack[Statement]. I could push an object to it for every call to is but that feels very dirty.
To put the question in one sentence: How do I create a recursive DSL wich respect to its hierarchy without adding any extra syntax and is there a solution to do this with immutable objects only?
I've seen a nice trick in XScalaWT to achieve the nesting in DSL. I didn't check if specs uses the same, or something different.
I think the following example shows the main idea. The heart of it is the setups function: it accepts some functions (more precisely closures, if I'm not mistaken) that needs only a Nestable and will call them on the current one.
printName happens to be such a method, just like addChild, with parameters filled for the first list of params.
For me understanding this was the revealing part. After that you can relatively simply add many other fancy features (like implicit magic, dsl methods based on structural typing, etc.).
Of course you can have any "context like" class instead of Nestable, especially if you go for pure immutable stuff. If parents need references to children you can collect the children during the setups() and create the parent only at the end.
In this case you would probably have something like
private def setupChildren[A, B](a : A, setups:(A => B)*) : Seq[B] = {
for (setup <- setups) yield setup(a)
}
You would pass in the "context", and create the parent using the returned children.
BTW I think this setup thing was needed in XScalaWT because it's for SWT where child objects need a reference to their parent control. If you don't need it (or anything from the current "context") then everything becomes a bit easier.
Using companion objects with proper apply methods should mostly solve the problem. Most likely they should also accept other functions, (having the same number of params, or a tuple if you need more).
One disadvantage of this trick is that you have to have a separate dsl method (even if a simple one) for each method that you want to call on your classes. Alternatively you can use lines like
x => x.printName
which will do the job, but not so nice (especially if you have to do it often).
object NestedDsl {
object Nestable {
def apply(name: String, setups:(Nestable => Unit)*): Nestable = {
val n = new Nestable(None, name)
setup(n, setups: _*)
n
}
}
class Nestable(parent: Option[Nestable], name: String) {
def printName() { println(name) }
}
// DSL part
def addChild(name: String, setups:(Nestable => Unit)*)(parent: Nestable) = {
val n = new Nestable(Some(parent), name)
setup(n, setups: _*)
n
}
def printName(n: Nestable) = n.printName
private def setup[T](t : T, setups:(T => Unit)*) : T = {
setups.foreach(setup => setup(t))
t
}
def main(args: Array[String]) {
Nestable("root",
addChild(
"first",
addChild("second",
printName
)
)
)
}
}
I have had a look at specs and they do not do it any differnet. Basically all you need is a mutable stack. You can have a look at the result here: cssx-dsl
The code is quite simple. Basically I have a mutable builder and convert it to an immutable representation afterwards.