Using tagless final (without using IO, but rather a generic F) how can I abstract over something like this:
def doSomething(s: String): IO[Unit] = ???
List("authMethods", "secretEngines", "plugins", "CAs", "common").parTraverse(doSomething)
The closest I can get is using parTraverseN from the Concurrent object, but I assume this will run concurrently instead of in parallel (as in parallelism). It also forces me to choose an n where as parTraverse does not.
The size of the list is just an example, it could be way bigger. doSomething is a pure function, multiple executions of it can run in parallel without problems.
Ideally, given that doSomething returns IO[Unit] I would like to abstract over parTraverse_ to an F with the correct typeclass instance.
Here's a similar complete working example:
import cats.Applicative
import cats.instances.list._
import cats.syntax.foldable._
trait Service[F[_]] {
val items = List("authMethods", "secretEngines", "plugins", "CAs", "common")
def doSomething(s: String): F[Unit] = ???
def result(implicit F: Applicative[F]): F[Unit] =
items.traverse_(doSomething)
}
If you want to use parTraverse_ here, the minimal changes necessary would look something like this:
import cats.{Applicative, Parallel}
import cats.instances.list._
import cats.syntax.parallel._
trait Service[F[_]] {
val items = List("authMethods", "secretEngines", "plugins", "CAs", "common")
def doSomething(s: String): F[Unit] = ???
def result(implicit F: Applicative[F], P: Parallel[F]): F[Unit] =
items.parTraverse_(doSomething)
}
Alternatively you could use Parallel.parTraverse_(items)(doSomething) and skip the syntax import. Both approaches require a Foldable instance for List (provided here by the cats.instances.list._ import, which will no longer be necessary in Cats 2.2.0), and a Parallel instance for F, which you get via the P constraint.
(Note that the Applicative constraint on result is no longer necessary in the second version, but that's only because this is a very simple example—I'm assuming your real code relies on something like Sync instead, and will need both that and Parallel.)
This answer needs a couple of footnotes, though. The first is that it might not actually be a good thing that parTraverse_ doesn't make you specify a bound in the way that parTraverseN does, and may result in excessive memory use, etc. (but this will depend on e.g. the expected size of your lists and the kind of work doSomething is doing, and is probably outside the scope of the question).
The second footnote is that "parallel" in the sense of the Parallel type class is more general than the "parallel" in the parallel-vs.-concurrent distinction in the Cats "Concurrency Basics" document. The Parallel type class models a very generic kind of logical parallelism that also encompasses error accumulation, for example. So when you write:
I assume this will run concurrently instead of in parallel (as in parallelism).
…your assumption is correct, but not exactly because the parTraverseN method is on Concurrent instead of Parallel; note that Concurrent.parTraverseN still requires a Parallel instance. When you see par or the Parallel type class in the context of cats.effect.Concurrent, you should think of concurrency, not "parallelism" in the "Concurrency Basics" sense.
So I'm trying to work through Norvig & Russell's "Artificial Intelligence, A Modern Approach" as a way to learn Scala. I have a pretty good grasp on the language basics at this point, but I still find myself often "fighting" the type system.
Long story short, breadth-first and depth-first search algorithms are the same aside from the mechanics of pushing/popping to their underlying collection. Depth-first would prepend new possibilities and use a Stack, while Breadth-first would append and use a Queue.
To keep my algorithm the same, I created a typeclass called "GiveGrab" (I know, horrible name) with the intention of pimping ... err ... enriching collections with these "default" push (give) and pop-like (grab) operations.For example, grab would result in a call to .dequeue() for queues, and .pop() for stacks.
Here's (a somewhat abbreviated version of) the code:
object Example extends App {
trait GiveGrab[A, M[A]] {
def give(x: A*): M[A]
def grab(): A
}
implicit class GiveGrabQueue[T](q: Queue[T]) extends GiveGrab[T,Queue[T]] {
override def give(x: T*) = q ++= x
override def grab() = q.dequeue()
}
class TestClass[T, X <% GiveGrab[T, Queue[T]]](var storage: X) {}
val test = new TestClass[Int, Queue[Int]](new Queue[Int]())
}
When trying to compile this, I get the following errors:
Error:(18, 39) scala.collection.mutable.Queue[T] takes no type parameters, expected: one
class TestClass[T, X <% GiveGrab[T, Queue[T]]](var storage: X) {}
^
Error:(13, 67) scala.collection.mutable.Queue[T] takes no type parameters, expected: one
implicit class GiveGrabQueue[T](q: Queue[T]) extends GiveGrab[T,Queue[T]] {
^
That said, it took me a lot of trial and error to even get to this point. I'm not sure if my trait is really supposed to be typed
trait GiveGrab[A, M[A]]
or
trait GiveGrab[A, M[_]]
or
trait GiveGrab[A, M]
The error "takes no type parameters, expected: one" doesn't make a whole lot of sense to me at this point, and there's only a handful of other posts about that message (some related to dependent types, and some related to the Play framework).
Somewhat related: is there a good article for understanding Scala type signatures? I read through Programming in Scala 2nd Ed, but it didn't really touch on this sort of type gymnastics (either that, or I just missed it.)
Edit: Typos
What #PatrykĆwiek proposed is not a workaround but actually what you are meant to be doing: M[A] in trait GiveGrab defines a type function. Roughly speaking this means: M is a type where you can apply a single type parameter to yield a concrete type. That the parameter is called A is pure coincidence. The following means the same:
trait GiveGrab[A,M[MyRandomName]] { ... }
In the definition of give, you actually use this type function to create a type, when saying M[A]. Therefore, as #PatrykĆwiek said, you should write Queue instead of Queue[T]. While Queue is precisely one of these type functions, Queue[T] is a concrete type and therefore doesn't apply to the definition of M.
The error message you get says exactly that: In the place of M, you are supposed to put a type that takes a parameter (like Queue), but you have put one which takes none (Queue[T] in your case, another example would be String or Int).
I'm currently working on a project that includes Gaussian Processes for Machine Learning. Considering the examples and explanations in the book, I'm trying to create a generic function for the various parameters that are part of a trained GP-object - thus, the following declaration is the most general one for a (simple) training function.
def train[T, M <: MatrixInverter[T], S <: Kernel[T]](): GP_Spawn[T] = null
(I've removed the parameter list and the implementation, just if you're wondering.)
Tdescribes the numeric type, e.g. it may be Double or Int. MatrixInverter[T] is a trait that enforces a calculateInverse function. Kernel[T] is the corresponding trait for a kernel-function.
As some of you may already know, training in a gaussian process can be changed (somehow simplified) when using a Cholesky-Decomposition as the matrix-inverter - thus, I've considered to specialize the function mentioned above. Due to the documentation of the #specialized tag, it should be something like this:
def train[T, #specialized(CholeskyDecomposition[T]) M <: MatrixInverter[T], S <: Kernel[T]](): GP_Spawn[T]
It's obvious that all parameters are more or less depending on T, since they need to use some variables (types T,Vector[T],Matrix[T]) that depend on it. If I try to compile the code mentioned above, the scala-compiler (2.9.2) complains about
error: not found: value CholeskyDecomposition
I'm not sure what this means, since the import import algorithms.{CholeskyDecomposition, MatrixInverter} is correct. Besides, it's curious to see that the import of CholeskyDecomposition is marked as Unused import statement. CholeskyDecomposition has a companion that includes some constants that are related to the algorithm itself, but I don't assume this aspect to be the reason for this error.
Any ideas what may cause this error ? And, furthermore, how to solve it without cutting of the generic approach ?
Edit:
After reading the answers are considering some re-ordering of my code, I ended up with a solution at runtime that uses type-matching.
val testMat = new Matrix[T](3, 3)
val testInv = fac(testMat)
testInv match {
case chol : CholeskyDecomposition[T] => println("Found Cholesky!")
case _ => println("Found something different.")
}
And it works now :) Thanks to all!
You can only specialize a generic parameter with a primitive type: Int, Double, etc. So you can specialize T but not Foo[T] even if T is a primitive.
If you have a class C that is specialized on type T, then
class D[#specialized T, C[T]](c: C[T]) { ... }
will use the T-specialized verson of C.
This is all you need anyway. There is no point to specialization on object; generic code works just as well since everything non-primitive is an object anyway.
As per the API, it says:
Type T can be specialized on a subset of the primitive types by
specifying a list of primitive types to specialize at:
So it is basically only for primitive types.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
So far implicit parameters in Scala do not look good for me -- it is too close to global variables, however since Scala seems like rather strict language I start doubting in my own opinion :-).
Question: could you show a real-life (or close) good example when implicit parameters really work. IOW: something more serious than showPrompt, that would justify such language design.
Or contrary -- could you show reliable language design (can be imaginary) that would make implicit not neccessary. I think that even no mechanism is better than implicits because code is clearer and there is no guessing.
Please note, I am asking about parameters, not implicit functions (conversions)!
Updates
Global variables
Thank you for all great answers. Maybe I clarify my "global variables" objection. Consider such function:
max(x : Int,y : Int) : Int
you call it
max(5,6);
you could (!) do it like this:
max(x:5,y:6);
but in my eyes implicits works like this:
x = 5;
y = 6;
max()
it is not very different from such construct (PHP-like)
max() : Int
{
global x : Int;
global y : Int;
...
}
Derek's answer
This is great example, however if you can think of as flexible usage of sending message not using implicit please post an counter-example. I am really curious about purity in language design ;-).
In a sense, yes, implicits represent global state. However, they are not mutable, which is the true problem with global variables -- you don't see people complaining about global constants, do you? In fact, coding standards usually dictate that you transform any constants in your code into constants or enums, which are usually global.
Note also that implicits are not in a flat namespace, which is also a common problem with globals. They are explicitly tied to types and, therefore, to the package hierarchy of those types.
So, take your globals, make them immutable and initialized at the declaration site, and put them on namespaces. Do they still look like globals? Do they still look problematic?
But let's not stop there. Implicits are tied to types, and they are just as much "global" as types are. Does the fact that types are global bother you?
As for use cases, they are many, but we can do a brief review based on their history. Originally, afaik, Scala did not have implicits. What Scala had were view types, a feature many other languages had. We can still see that today whenever you write something like T <% Ordered[T], which means the type T can be viewed as a type Ordered[T]. View types are a way of making automatic casts available on type parameters (generics).
Scala then generalized that feature with implicits. Automatic casts no longer exist, and, instead, you have implicit conversions -- which are just Function1 values and, therefore, can be passed as parameters. From then on, T <% Ordered[T] meant a value for an implicit conversion would be passed as parameter. Since the cast is automatic, the caller of the function is not required to explicitly pass the parameter -- so those parameters became implicit parameters.
Note that there are two concepts -- implicit conversions and implicit parameters -- that are very close, but do not completely overlap.
Anyway, view types became syntactic sugar for implicit conversions being passed implicitly. They would be rewritten like this:
def max[T <% Ordered[T]](a: T, b: T): T = if (a < b) b else a
def max[T](a: T, b: T)(implicit $ev1: Function1[T, Ordered[T]]): T = if ($ev1(a) < b) b else a
The implicit parameters are simply a generalization of that pattern, making it possible to pass any kind of implicit parameters, instead of just Function1. Actual use for them then followed, and syntactic sugar for those uses came latter.
One of them is Context Bounds, used to implement the type class pattern (pattern because it is not a built-in feature, just a way of using the language that provides similar functionality to Haskell's type class). A context bound is used to provide an adapter that implements functionality that is inherent in a class, but not declared by it. It offers the benefits of inheritance and interfaces without their drawbacks. For example:
def max[T](a: T, b: T)(implicit $ev1: Ordering[T]): T = if ($ev1.lt(a, b)) b else a
// latter followed by the syntactic sugar
def max[T: Ordering](a: T, b: T): T = if (implicitly[Ordering[T]].lt(a, b)) b else a
You have probably used that already -- there's one common use case that people usually don't notice. It is this:
new Array[Int](size)
That uses a context bound of a class manifests, to enable such array initialization. We can see that with this example:
def f[T](size: Int) = new Array[T](size) // won't compile!
You can write it like this:
def f[T: ClassManifest](size: Int) = new Array[T](size)
On the standard library, the context bounds most used are:
Manifest // Provides reflection on a type
ClassManifest // Provides reflection on a type after erasure
Ordering // Total ordering of elements
Numeric // Basic arithmetic of elements
CanBuildFrom // Collection creation
The latter three are mostly used with collections, with methods such as max, sum and map. One library that makes extensive use of context bounds is Scalaz.
Another common usage is to decrease boiler-plate on operations that must share a common parameter. For example, transactions:
def withTransaction(f: Transaction => Unit) = {
val txn = new Transaction
try { f(txn); txn.commit() }
catch { case ex => txn.rollback(); throw ex }
}
withTransaction { txn =>
op1(data)(txn)
op2(data)(txn)
op3(data)(txn)
}
Which is then simplified like this:
withTransaction { implicit txn =>
op1(data)
op2(data)
op3(data)
}
This pattern is used with transactional memory, and I think (but I'm not sure) that the Scala I/O library uses it as well.
The third common usage I can think of is making proofs about the types that are being passed, which makes it possible to detect at compile time things that would, otherwise, result in run time exceptions. For example, see this definition on Option:
def flatten[B](implicit ev: A <:< Option[B]): Option[B]
That makes this possible:
scala> Option(Option(2)).flatten // compiles
res0: Option[Int] = Some(2)
scala> Option(2).flatten // does not compile!
<console>:8: error: Cannot prove that Int <:< Option[B].
Option(2).flatten // does not compile!
^
One library that makes extensive use of that feature is Shapeless.
I don't think the example of the Akka library fits in any of these four categories, but that's the whole point of generic features: people can use it in all sorts of way, instead of ways prescribed by the language designer.
If you like being prescribed to (like, say, Python does), then Scala is just not for you.
Sure. Akka's got a great example of it with respect to its Actors. When you're inside an Actor's receive method, you might want to send a message to another Actor. When you do this, Akka will bundle (by default) the current Actor as the sender of the message, like this:
trait ScalaActorRef { this: ActorRef =>
...
def !(message: Any)(implicit sender: ActorRef = null): Unit
...
}
The sender is implicit. In the Actor there is a definition that looks like:
trait Actor {
...
implicit val self = context.self
...
}
This creates the implicit value within the scope of your own code, and it allows you to do easy things like this:
someOtherActor ! SomeMessage
Now, you can do this as well, if you like:
someOtherActor.!(SomeMessage)(self)
or
someOtherActor.!(SomeMessage)(null)
or
someOtherActor.!(SomeMessage)(anotherActorAltogether)
But normally you don't. You just keep the natural usage that's made possible by the implicit value definition in the Actor trait. There are about a million other examples. The collection classes are a huge one. Try wandering around any non-trivial Scala library and you'll find a truckload.
One example would be the comparison operations on Traversable[A]. E.g. max or sort:
def max[B >: A](implicit cmp: Ordering[B]) : A
These can only be sensibly defined when there is an operation < on A. So, without implicits we’d have to supply the context Ordering[B] every time we’d like to use this function. (Or give up type static checking inside max and risk a runtime cast error.)
If however, an implicit comparison type class is in scope, e.g. some Ordering[Int], we can just use it right away or simply change the comparison method by supplying some other value for the implicit parameter.
Of course, implicits may be shadowed and thus there may be situations in which the actual implicit which is in scope is not clear enough. For simple uses of max or sort it might indeed be sufficient to have a fixed ordering trait on Int and use some syntax to check whether this trait is available. But this would mean that there could be no add-on traits and every piece of code would have to use the traits which were originally defined.
Addition:
Response to the global variable comparison.
I think you’re correct that in a code snipped like
implicit val num = 2
implicit val item = "Orange"
def shopping(implicit num: Int, item: String) = {
"I’m buying "+num+" "+item+(if(num==1) "." else "s.")
}
scala> shopping
res: java.lang.String = I’m buying 2 Oranges.
it may smell of rotten and evil global variables. The crucial point, however, is that there may be only one implicit variable per type in scope. Your example with two Ints is not going to work.
Also, this means that practically, implicit variables are employed only when there is a not necessarily unique yet distinct primary instance for a type. The self reference of an actor is a good example for such a thing. The type class example is another example. There may be dozens of algebraic comparisons for any type but there is one which is special.
(On another level, the actual line number in the code itself might also make for a good implicit variable as long as it uses a very distinctive type.)
You normally don’t use implicits for everyday types. And with specialised types (like Ordering[Int]) there is not too much risk in shadowing them.
Based on my experience there is no real good example for use of implicits parameters or implicits conversion.
The small benefit of using implicits (not needing to explicitly write a parameter or a type) is redundant in compare to the problems they create.
I am a developer for 15 years, and have been working with scala for the last 1.5 years.
I have seen many times bugs that were caused by the developer not aware of the fact that implicits are used, and that a specific function actually return a different type that the one specified. Due to implicit conversion.
I also heard statements saying that if you don't like implicits, don't use them.
This is not practical in the real world since many times external libraries are used, and a lot of them are using implicits, so your code using implicits, and you might not be aware of that.
You can write a code that has either:
import org.some.common.library.{TypeA, TypeB}
or:
import org.some.common.library._
Both codes will compile and run.
But they will not always produce the same results since the second version imports implicits conversion that will make the code behave differently.
The 'bug' that is caused by this can occur a very long time after the code was written, in case some values that are affected by this conversion were not used originally.
Once you encounter the bug, its not an easy task finding the cause.
You have to do some deep investigation.
Even though you feel like an expert in scala once you have found the bug, and fixed it by changing an import statement, you actually wasted a lot of precious time.
Additional reasons why I generally against implicits are:
They make the code hard to understand (there is less code, but you don't know what he is doing)
Compilation time. scala code compiles much slower when implicits are used.
In practice, it changes the language from statically typed, to dynamically typed. Its true that once following very strict coding guidelines you can avoid such situations, but in real world, its not always the case. Even using the IDE 'remove unused imports', can cause your code to still compile and run, but not the same as before you removed 'unused' imports.
There is no option to compile scala without implicits (if there is please correct me), and if there was an option, none of the common community scala libraries would have compile.
For all the above reasons, I think that implicits are one of the worst practices that scala language is using.
Scala has many great features, and many not so great.
When choosing a language for a new project, implicits are one of the reasons against scala, not in favour of it. In my opinion.
Another good general usage of implicit parameters is to make the return type of a method depend on the type of some of the parameters passed to it. A good example, mentioned by Jens, is the collections framework, and methods like map, whose full signature usually is:
def map[B, That](f: (A) ⇒ B)(implicit bf: CanBuildFrom[GenSeq[A], B, That]): That
Note that the return type That is determined by the best fitting CanBuildFrom that the compiler can find.
For another example of this, see that answer. There, the return type of the method Arithmetic.apply is determined according to a certain implicit parameter type (BiConverter).
It's easy, just remember:
to declare the variable to be passed in as implicit too
to declare all the implicit params after the non-implicit params in a separate ()
e.g.
def myFunction(): Int = {
implicit val y: Int = 33
implicit val z: Double = 3.3
functionWithImplicit("foo") // calls functionWithImplicit("foo")(y, z)
}
def functionWithImplicit(foo: String)(implicit x: Int, d: Double) = // blar blar
Implicit parameters are heavily used in the collection API. Many functions get an implicit CanBuildFrom, which ensures that you get the 'best' result collection implementation.
Without implicits you would either pass such a thing all the time, which would make normal usage cumbersome. Or use less specialized collections which would be annoying because it would mean you loose performance/power.
I am commenting on this post a bit late, but I have started learning scala lately.
Daniel and others have given nice background about implicit keyword.
I would provide me two cents on implicit variable from practical usage perspective.
Scala is best suited if used for writing Apache Spark codes. In Spark, we do have spark context and most likely the configuration class that may fetch the configuration keys/values from a configuration file.
Now, If I have an abstract class and if I declare an object of configuration and spark context as follows :-
abstract class myImplicitClass {
implicit val config = new myConfigClass()
val conf = new SparkConf().setMaster().setAppName()
implicit val sc = new SparkContext(conf)
def overrideThisMethod(implicit sc: SparkContext, config: Config) : Unit
}
class MyClass extends myImplicitClass {
override def overrideThisMethod(implicit sc: SparkContext, config: Config){
/*I can provide here n number of methods where I can pass the sc and config
objects, what are implicit*/
def firstFn(firstParam: Int) (implicit sc: SparkContext, config: Config){
/*I can use "sc" and "config" as I wish: making rdd or getting data from cassandra, for e.g.*/
val myRdd = sc.parallelize(List("abc","123"))
}
def secondFn(firstParam: Int) (implicit sc: SparkContext, config: Config){
/*following are the ways we can use "sc" and "config" */
val keyspace = config.getString("keyspace")
val tableName = config.getString("table")
val hostName = config.getString("host")
val userName = config.getString("username")
val pswd = config.getString("password")
implicit val cassandraConnectorObj = CassandraConnector(....)
val cassandraRdd = sc.cassandraTable(keyspace, tableName)
}
}
}
As we can see the code above, I have two implicit objects in my abstract class, and I have passed those two implicit variables as function/method/definition implicit parameters.
I think this is the best use case that we can depict in terms of usage of implicit variables.
Having written a few scala tools, I'm trying to come to grips with the best way to arrange my code - particularly implicits. I have 2 goals:
Sometimes, I want to be able to import just the implicits I ask for.
Othertimes, I want to just import everything.
To avoid duplicating the implicits, I've come up with this structure (similar to the way scalaz is arranged):
case class StringW(s : String) {
def contrived = s + "?"
}
trait StringWImplicits {
implicit def To(s : String) = StringW(s)
implicit def From(sw : StringW) = sw.s
}
object StringW extends StringWImplicits
// Elsewhere on Monkey Island
object World extends StringWImplicits with ListWImplicits with MoreImplicits
This allows me to just
import StringW._ // Selective import
or (in most cases)
import World._. // Import everything
How does everyone else do it?
I think that implicit conversions are dangerous if you don't know where they are coming from. In my case, I put my implicits in a Conversions class and import it as close to the use as possible
def someMethod(d: Date) ; Unit {
import mydate.Conversions._
val tz = TimeZone.getDefault
val timeOfDay = d.getTimeOfDay(tz) //implicit used here
...
}
I'm not sure I like "inheriting" implicits from various traits for the same reason it was considered bad Java practice to implement an interface so you could use its constants directly (static imports are preferred instead).
I usually had implicit conversions in an object which clearly signals that what it is imported is an implicit conversion.
For example, if I have a class com.foo.bar.FilthyRichString, the implicit conversions would go into com.foo.bar.implicit.FilthyRichStringImplicit. I know the names are a bit long, but that's why we have IDEs (and Scala IDE support is getting better). The way I do this is that I feel it is important that all the implicit conversions can be clearly viewed in a 10 second code review. I could look at the following code:
// other imports
import com.foo.bar.FilthyRichString
import com.foo.bar.util.Logger
import com.foo.bar.util.FileIO
import com.foo.bar.implicits.FilthyRichStringImplicit._
import com.foo.bar.implicits.MyListImplicit._
// other implicits
and at a glance see all the implicit conversions that are active in this source file. They would also be all gathered together, if you use the convention that imports are grouped by packages, with a new line between different packages.
Along the lines of the same argument, I wouldn't like a catch-all object that holds all of the implicit conversions. In a big project, would you really use all of the implicit conversions in all your source files? I think that doing that means very tight coupling between different parts of your code.
Also, a catch-all object is not very good for documentation. In the case of explicitly writing all the implicit conversions used in a file, one can just look at your import statements and straight away jump to the documentation of the implicit class. In the case of a catch-all object, one would have to look at that object (which in a big project might be huge) and then search for the implicit conversion they are after.
I agree with oxbow_lakes that having implicit conversion in traits is bad because of the temptation of inheriting from it, which is, as he said, bad practice. Along those lines, I would make the objects holding the implicit conversions final just to avoid the temptation altogether. His idea of importing them as close to the use as possible is very nice as well, if implicit conversions are just used sparingly in the code.
-- Flaviu Cipcigan