I'm trying to implement something like clever parameters converter function with Scala.
Basically in my program I need to read parameters from a properties file, so obviously they are all strings and I would like then to convert each parameter in a specific type that I pass as parameter.
This is the implementation that I start coding:
def getParam[T](key : String , value : String, paramClass : T): Any = {
value match {
paramClass match {
case i if i == Int => value.trim.toInt
case b if b == Boolean => value.trim.toBoolean
case _ => value.trim
}
}
/* Exception handling is missing at the moment */
}
Usage:
val convertedInt = getParam("some.int.property.key", "10", Int)
val convertedBoolean = getParam("some.boolean.property.key", "true", Boolean)
val plainString = getParam("some.string.property.key", "value",String)
Points to note:
For my program now I need just 3 main type of type: String ,Int and Boolean,
if is possible I would like to extends to more object type
This is not clever, cause I need to explicit the matching against every possibile type to convert, I would like an more reflectional like approach
This code doesn't work, it give me compile error: "object java.lang.String is not a value" when I try to convert( actually no conversion happen because property values came as String).
Can anyone help me? I'm quite newbie in Scala and maybe I missing something
The Scala approach for a problem that you are trying to solve is context bounds. Given a type T you can require an object like ParamMeta[T], which will do all conversions for you. So you can rewrite your code to something like this:
trait ParamMeta[T] {
def apply(v: String): T
}
def getParam[T](key: String, value: String)(implicit meta: ParamMeta[T]): T =
meta(value.trim)
implicit case object IntMeta extends ParamMeta[Int] {
def apply(v: String): Int = v.toInt
}
// and so on
getParam[Int](/* ... */, "127") // = 127
There is even no need to throw exceptions! If you supply an unsupported type as getParam type argument, code will even not compile. You can rewrite signature of getParam using a syntax sugar for context bounds, T: Bound, which will require implicit value Bound[T], and you will need to use implicitly[Bound[T]] to access that values (because there will be no parameter name for it).
Also this code does not use reflection at all, because compiler searches for an implicit value ParamMeta[Int], founds it in object IntMeta and rewrites function call like getParam[Int](..., "127")(IntMeta), so it will get all required values at compile time.
If you feel that writing those case objects is too boilerplate, and you are sure that you will not need another method in these objects in future (for example, to convert T back to String), you can simplify declarations like this:
case class ParamMeta[T](f: String => T) {
def apply(s: String): T = f(s)
}
implicit val stringMeta = ParamMeta(identity)
implicit val intMeta = ParamMeta(_.toInt)
To avoid importing them every time you use getParam you can declare these implicits in a companion object of ParamMeta trait/case class, and Scala will pick them automatically.
As for original match approach, you can pass a implicit ClassTag[T] to your function, so you will be able to match classes. You do not need to create any values for ClassTag, as the compiler will pass it automatically. Here is a simple example how to do class matching:
import scala.reflect.ClassTag
import scala.reflect._
def test[T: ClassTag] = classTag[T].runtimeClass match {
case x if x == classOf[Int] => "I'm an int!"
case x if x == classOf[String] => "I'm a string!"
}
println(test[Int])
println(test[String])
However, this approach is less flexible than ParamMeta one, and ParamMeta should be preferred.
Related
I hope to process various generic data sources accessed by Kafka, so I developed the following code:
def accessKafkaSource[T: ClassTag](sEnv: StreamExecutionEnvironment): DataStream[T] = {
val kafkaSource: KafkaSource[T] = KafkaSource.builder()
.setBootstrapServers("")
.setGroupId("")
.setTopics("test")
.setStartingOffsets(OffsetsInitializer.committedOffsets(OffsetResetStrategy.LATEST))
.setValueOnlyDeserializer(new AbstractDeserializationSchema[T]() {
override def deserialize(msg: Array[Byte]): T = {
// JSONUtil.toBean(StrUtil.str(msg, StandardCharsets.UTF_8), classOf[T])
JSONUtil.toBean(StrUtil.str(msg, StandardCharsets.UTF_8), classTag[T].runtimeClass)
}
})
.build()
Since the commented out code will get an error: class type required but t found, I modified the code, but caused a new problem: type mismatch; found : _$ 1 where type _$ 1 required: Tć How should my needs be realized?
As AminMal notes, runtimeClass is not guaranteed to return the class object of T, just what it erases to in the runtime. AnyVals in particular will break this.
If everything you wish to deserialize is an AnyRef (this is likely the case), you can often safely cast the result of runtimeClass:
def kindaSafeClass[T <: AnyRef : ClassTag]: Class[T] = classTag[T].runtimeClass.asInstanceOf[Class[T]]
The situation this would be unsafe is when generics are involved (erasure...), as can be seen by
val clazz = kindaSafeClass[List[String]]
val lst = List(1)
val cast =
if (clazz.isInstance(lst)) {
println(s"$lst is an instance of $clazz")
clazz.cast(lst)
} else ???
println(cast)
println(cast.head.isEmpty)
which will print List(1) is an instance of class scala.collection.immutable.List, then List(1), and then blow up with a ClassCastException when we try to cast 1 to a String.
But if your T will always be an AnyRef and you can be sure that it's not generic, you can
// Note: T must not be generic (e.g. List[String])
def accessKafkaSource[T <: AnyRef : ClassTag](sEnv: StreamExecutionEnvironment): DataStream[T] =
// as before until...
JSONUtils.toBean(StrUtil.str(msg, StandardCharsets.UTF_8), kindaSafeClass[T])
// as before...
That's because runtimeClass returns a Class[_], not a Class[T]. This kind of approach would make perfect sense in Java, like:
JSONUtils.toBean(whateverString, MyClass.class); // and so on
In Scala, (of course there are unsafe approaches to make this work) but if you're using some JSON library (like Play Json, circe, etc,.), you can do this:
// method signature would look something like this
def accessKafkaSource[T : Reads](sEnv: StreamExecutionEnvironment): DataStream[T] = {
// just going to write the deserialization part:
override def deserialize(msg: Array[Byte]): T = {
Json.parse(msg).as[T] // asOpt is safer, not going to get into exception handling and other stuff
}
}
This behavior is also applicable to other json libraries in scala. Or if you have other kind of documents like xml, expect an implicit function from Array[Byte] => DocumentType (Like JsValue, String, Xml, anything), and another one DocumentType => T. Because this function accessKafkaSource should not be responsible to figure out how data should be deserialized/serialized.
I'm fairly new to Scala in general, and Scala 3 in particular, and I'm trying to write some code that deals with transparently encoding + decoding values before they are passed to another library.
Basically, I need to map a set of types like Ints to a counterpart in the underlying library. The code I've written is too verbose to replicate here in full, but here's a minimal example demonstrating the kind of thing, using a higher-kinded Encoder type that encapsulates encoding values into types which depend on the values' original types:
trait Encoder[T] {
type U
def encode(v: T): U
}
object Encoder {
given Encoder[Int] with {
override type U = String
override def encode(v: Int): String = v.toString
}
}
case class Value[T : Encoder](v: T) {
val encoder: Encoder[T] = summon[Encoder[T]]
}
I also need to be able to write functions that deal with specific types of Value and which have 'concrete' return types. Like this:
def doStuff(v1: Value[Int]): String = {
v1.encoder.encode(v1.v)
}
However, even though in this case v1.codec.encode does indeed return a String, I get an error:
-- [E007] Type Mismatch Error: -------------------------------------------------
2 | v1.encoder.encode(v1.v)
| ^^^^^^^^^^^^^^^^^^^^^^^
| Found: v1.encoder.U
| Required: String
What can I do differently to solve this error? Really appreciate any pointers to help a newbie out š
Answering the question in the comments
Is there any sensible way I tell the compiler that Iām only interested in Values with Encoders that encode to String?
You can force Value to remember its encoder's result type with an extra type argument.
case class Value[T, R](val v: T)(
using val encoder: Encoder[T],
val eqv: encoder.U =:= R,
)
The encoder is the same as your encoder, just moved to the using list so we can use it in implicit resolution.
eqv is a proof that R (our type parameter) is equivalent to the encoder's U type.
Then doStuff can take a Value[Int, String]
def doStuff(v1: Value[Int, String]): String = {
v1.eqv(v1.encoder.encode(v1.v))
}
Let's be clear about what's happening here. v1.encoder.encode(v1.v) returns an encoder.U. Scala isn't smart enough to know what that is. However, we also have a proof that encoder.U is equal to String, and that proof can be used to convert an encoder.U to a String. And that's exactly what =:=.apply does.
We have to do this back in the case class because you've already lost the type information by the time we hit doStuff. Only the case class (which instantiates the implicit encoder) knows what the result type is, so we need to expose it there.
If you have other places in your codebase where you don't care about the result type, you can fill in a type parameter R for it, or use a wildcard Value[Int, ?].
I would also suggest giving Match Types a try if we are only talking about Scala 3 here.
import scala.util.Try
type Encoder[T] = T match
case Int => String
case String => Either[Throwable, Int]
case class Value[T](v: T):
def encode: Encoder[T] = v match
case u: Int => u.toString
case u: String => Try(u.toInt).toEither
object Main extends App:
val (v1, v2) = (Value(1), Value(2))
def doStuff(v: Value[Int]): String =
v.encode
println(doStuff(v1) + doStuff(v2)) //12
println(Value(v1.encode).encode) //Right(1)
I'm trying to write a method which uses the isEmpty method on types String, Option and List. These classes don't share a common base trait with that method, so I've tried to pass an implicit EmptyChecker in with them:
trait EmptyChecker[Field] {
def isEmpty(data: Field): Boolean
}
implicit val StringEmptyChecker: EmptyChecker[String] = new EmptyChecker[String] {
def isEmpty(string: String): Boolean = string.isEmpty
}
def printEmptiness[Field](field: Field)(implicit emptyChecker: EmptyChecker[Field]): Unit = {
if (emptyChecker.isEmpty(field))
println("Empty")
else
println("Not empty")
}
printEmptiness("abc") // Works fine
The String empty checker works fine, but I've hit problems with making empty checkers for type constructors like Option and List.
For example, Option doesn't work:
implicit val OptionChecker: EmptyChecker[Option[_]] = new EmptyChecker[Option[_]] {
def isEmpty(option: Option[_]): Boolean = option.isEmpty
}
// Both fail compilation: "could not find implicit value for parameter emptyChecker: EmptyChecker[Some[Int]]
printEmptiness(Some(3))
printEmptiness[Option[Int]](Some(3))
If I use a specific Option[Int] checker, it works a little better, but is a bit ugly:
implicit val OptionIntChecker: EmptyChecker[Option[Int]] = new EmptyChecker[Option[Int]] {
def isEmpty(optionInt: Option[Int]): Boolean = optionInt.isEmpty
}
// Fails like above:
printEmptiness(Some(3))
// Passes compilation:
printEmptiness[Option[Int]](Some(3))
So my question is: is it possible to make a single EmptyChecker for each Option and List type and have them work with my method without needing to explicitly declare the type whenever I call it? I'm trying to get a type safe duck typing effect.
I'm using scala 2.11.6.
Thanks in advance!
The source of your problem is that the type of Some(1) is Some[Int], not Option[Int]. There are a couple of ways around this; you can explicitly upcast the expression with a type ascription: printEmptiness(Some(3): Option[Int]). Alternatively, you can define a helper method to do this for you automatically, and if you're using Scalaz, there's one of these provided:
import scalaz.syntax.std.option._
printEmptiness(3.some)
Furthermore if you do use Scalaz, you may find looking at the PlusEmpty/ApplicativePlus/MonadPlus type classes useful.
I just realized that my generic method:
def method[A](list: List[A]): A = { ... }
will result in a non-generic function type
val methodFun = method _
-> methodFun : (scala.List[Nothing]) => Nothing
when currying it, instead of keeping its generic type. Is there a possibility to keep the generic type information? I found out that I can define some explicit type as for example String by setting
val methodFun = method[String] _
-> methodFun : (scala.List[String]) => String
but this is not really what I want. I currently tend to use raw types to avoid this problems (as soon as I find out how) or is there a better solution?
Thanks for help!
PS: For why I want to do it:
def method1[A](list: List[A]): A = { ... }
def method2[A](element: A): Int = { ... }
// This will not cause a compiler error as stated before
// but this will result in (List[Nothing]) => Int
// but I want a (List[A]) => Int
val composedFun = method1 _ andThen method2
// The next line is possible
// but it gives me a (List[String]) => Int
val composedFunNonGeneric = method1[String] _ andThen method2[String]
Let's look at your example:
def method1[A](list: List[A]): A = { ... }
def method2[A](element: A): String = { ... }
// The next line will cause a compiler error
val composed = method1 _ andThen method2
First, that doesn't give me a compiler error, but rather has the too-specific type (List[Nothing]=>String) that you mentioned.
If you want to understand why this doesn't work, think about it this way: what is the type you're expecting for composed? I think you want something like this List[A]=>String. However, composed is a val, not a def (i.e. it's an instance of a function object, not a method). Object instances must have specific types. If you wanted to use a generic type here, then you'd have to wrap this val in a class definition with a generic type, but even then the generic type would be restricted to the type specified/inferred for each specific instance of that class.
In short, if you want to compose methods and keep the type parameter, you need to compose them manually and declare it with def instead:
def composed[A](list: List[A]): String = method2(method1(list))
I was making my way through the Scala playframework tutorial and I came across this snippet of code which had me puzzled:
def newTask = Action { implicit request =>
taskForm.bindFromRequest.fold(
errors => BadRequest(views.html.index(Task.all(), errors)),
label => {
Task.create(label)
Redirect(routes.Application.tasks())
}
)
}
So I decided to investigate and came across this post.
I still don't get it.
What is the difference between this:
implicit def double2Int(d : Double) : Int = d.toInt
and
def double2IntNonImplicit(d : Double) : Int = d.toInt
other than the obvious fact they have different method names.
When should I use implicit and why?
I'll explain the main use cases of implicits below, but for more detail see the relevant chapter of Programming in Scala.
Implicit parameters
The final parameter list on a method can be marked implicit, which means the values will be taken from the context in which they are called. If there is no implicit value of the right type in scope, it will not compile. Since the implicit value must resolve to a single value and to avoid clashes, it's a good idea to make the type specific to its purpose, e.g. don't require your methods to find an implicit Int!
example:
// probably in a library
class Prefixer(val prefix: String)
def addPrefix(s: String)(implicit p: Prefixer) = p.prefix + s
// then probably in your application
implicit val myImplicitPrefixer = new Prefixer("***")
addPrefix("abc") // returns "***abc"
Implicit conversions
When the compiler finds an expression of the wrong type for the context, it will look for an implicit Function value of a type that will allow it to typecheck. So if an A is required and it finds a B, it will look for an implicit value of type B => A in scope (it also checks some other places like in the B and A companion objects, if they exist). Since defs can be "eta-expanded" into Function objects, an implicit def xyz(arg: B): A will do as well.
So the difference between your methods is that the one marked implicit will be inserted for you by the compiler when a Double is found but an Int is required.
implicit def doubleToInt(d: Double) = d.toInt
val x: Int = 42.0
will work the same as
def doubleToInt(d: Double) = d.toInt
val x: Int = doubleToInt(42.0)
In the second we've inserted the conversion manually; in the first the compiler did the same automatically. The conversion is required because of the type annotation on the left hand side.
Regarding your first snippet from Play:
Actions are explained on this page from the Play documentation (see also API docs). You are using
apply(block: (Request[AnyContent]) ā Result): Action[AnyContent]
on the Action object (which is the companion to the trait of the same name).
So we need to supply a Function as the argument, which can be written as a literal in the form
request => ...
In a function literal, the part before the => is a value declaration, and can be marked implicit if you want, just like in any other val declaration. Here, request doesn't have to be marked implicit for this to type check, but by doing so it will be available as an implicit value for any methods that might need it within the function (and of course, it can be used explicitly as well). In this particular case, this has been done because the bindFromRequest method on the Form class requires an implicit Request argument.
WARNING: contains sarcasm judiciously! YMMV...
Luigi's answer is complete and correct. This one is only to extend it a bit with an example of how you can gloriously overuse implicits, as it happens quite often in Scala projects. Actually so often, you can probably even find it in one of the "Best Practice" guides.
object HelloWorld {
case class Text(content: String)
case class Prefix(text: String)
implicit def String2Text(content: String)(implicit prefix: Prefix) = {
Text(prefix.text + " " + content)
}
def printText(text: Text): Unit = {
println(text.content)
}
def main(args: Array[String]): Unit = {
printText("World!")
}
// Best to hide this line somewhere below a pile of completely unrelated code.
// Better yet, import its package from another distant place.
implicit val prefixLOL = Prefix("Hello")
}
In scala implicit works as:
Converter
Parameter value injector
Extension method
There are some uses of Implicit
Implicitly type conversion : It converts the error producing assignment into intended type
val x :String = "1"
val y:Int = x
String is not the sub type of Int , so error happens in line 2. To resolve the error the compiler will look for such a method in the scope which has implicit keyword and takes a String as argument and returns an Int .
so
implicit def z(a:String):Int = 2
val x :String = "1"
val y:Int = x // compiler will use z here like val y:Int=z(x)
println(y) // result 2 & no error!
Implicitly receiver conversion: We generally by receiver call object's properties, eg. methods or variables . So to call any property by a receiver the property must be the member of that receiver's class/object.
class Mahadi{
val haveCar:String ="BMW"
}
class Johnny{
val haveTv:String = "Sony"
}
val mahadi = new Mahadi
mahadi.haveTv // Error happening
Here mahadi.haveTv will produce an error. Because scala compiler will first look for the haveTv property to mahadi receiver. It will not find. Second it will look for a method in scope having implicit keyword which take Mahadi object as argument and returns Johnny object. But it does not have here. So it will create error. But the following is okay.
class Mahadi{
val haveCar:String ="BMW"
}
class Johnny{
val haveTv:String = "Sony"
}
val mahadi = new Mahadi
implicit def z(a:Mahadi):Johnny = new Johnny
mahadi.haveTv // compiler will use z here like new Johnny().haveTv
println(mahadi.haveTv)// result Sony & no error
Implicitly parameter injection: If we call a method and do not pass its parameter value, it will cause an error. The scala compiler works like this - first will try to pass value, but it will get no direct value for the parameter.
def x(a:Int)= a
x // ERROR happening
Second if the parameter has any implicit keyword it will look for any val in the scope which have the same type of value. If not get it will cause error.
def x(implicit a:Int)= a
x // error happening here
To slove this problem compiler will look for a implicit val having the type of Int because the parameter a has implicit keyword.
def x(implicit a:Int)=a
implicit val z:Int =10
x // compiler will use implicit like this x(z)
println(x) // will result 10 & no error.
Another example:
def l(implicit b:Int)
def x(implicit a:Int)= l(a)
we can also write it like-
def x(implicit a:Int)= l
Because l has a implicit parameter and in scope of method x's body, there is an implicit local variable(parameters are local variables) a which is the parameter of x, so in the body of x method the method-signature l's implicit argument value is filed by the x method's local implicit variable(parameter) a implicitly.
So
def x(implicit a:Int)= l
will be in compiler like this
def x(implicit a:Int)= l(a)
Another example:
def c(implicit k:Int):String = k.toString
def x(a:Int => String):String =a
x{
x => c
}
it will cause error, because c in x{x=>c} needs explicitly-value-passing in argument or implicit val in scope.
So we can make the function literal's parameter explicitly implicit when we call the method x
x{
implicit x => c // the compiler will set the parameter of c like this c(x)
}
This has been used in action method of Play-Framework
in view folder of app the template is declared like
#()(implicit requestHreader:RequestHeader)
in controller action is like
def index = Action{
implicit request =>
Ok(views.html.formpage())
}
if you do not mention request parameter as implicit explicitly then you must have been written-
def index = Action{
request =>
Ok(views.html.formpage()(request))
}
Extension Method
Think, we want to add new method with Integer object. The name of the method will be meterToCm,
> 1 .meterToCm
res0 100
to do this we need to create an implicit class within a object/class/trait . This class can not be a case class.
object Extensions{
implicit class MeterToCm(meter:Int){
def meterToCm={
meter*100
}
}
}
Note the implicit class will only take one constructor parameter.
Now import the implicit class in the scope you are wanting to use
import Extensions._
2.meterToCm // result 200
Why and when you should mark the request parameter as implicit:
Some methods that you will make use of in the body of your action have an implicit parameter list like, for example, Form.scala defines a method:
def bindFromRequest()(implicit request: play.api.mvc.Request[_]): Form[T] = { ... }
You don't necessarily notice this as you would just call myForm.bindFromRequest() You don't have to provide the implicit arguments explicitly. No, you leave the compiler to look for any valid candidate object to pass in every time it comes across a method call that requires an instance of the request. Since you do have a request available, all you need to do is to mark it as implicit.
You explicitly mark it as available for implicit use.
You hint the compiler that it's "OK" to use the request object sent in by the Play framework (that we gave the name "request" but could have used just "r" or "req") wherever required, "on the sly".
myForm.bindFromRequest()
see it? it's not there, but it is there!
It just happens without your having to slot it in manually in every place it's needed (but you can pass it explicitly, if you so wish, no matter if it's marked implicit or not):
myForm.bindFromRequest()(request)
Without marking it as implicit, you would have to do the above. Marking it as implicit you don't have to.
When should you mark the request as implicit? You only really need to if you are making use of methods that declare an implicit parameter list expecting an instance of the Request. But to keep it simple, you could just get into the habit of marking the request implicit always. That way you can just write beautiful terse code.
Also, in the above case there should be only one implicit function whose type is double => Int. Otherwise, the compiler gets confused and won't compile properly.
//this won't compile
implicit def doubleToInt(d: Double) = d.toInt
implicit def doubleToIntSecond(d: Double) = d.toInt
val x: Int = 42.0
I had the exact same question as you had and I think I should share how I started to understand it by a few really simple examples (note that it only covers the common use cases).
There are two common use cases in Scala using implicit.
Using it on a variable
Using it on a function
Examples are as follows
Using it on a variable. As you can see, if the implicit keyword is used in the last parameter list, then the closest variable will be used.
// Here I define a class and initiated an instance of this class
case class Person(val name: String)
val charles: Person = Person("Charles")
// Here I define a function
def greeting(words: String)(implicit person: Person) = person match {
case Person(name: String) if name != "" => s"$name, $words"
case _ => "$words"
}
greeting("Good morning") // Charles, Good moring
val charles: Person = Person("")
greeting("Good morning") // Good moring
Using it on a function. As you can see, if the implicit is used on the function, then the closest type conversion method will be used.
val num = 10 // num: Int (of course)
// Here I define a implicit function
implicit def intToString(num: Int) = s"$num -- I am a String now!"
val num = 10 // num: Int (of course). Nothing happens yet.. Compiler believes you want 10 to be an Int
// Util...
val num: String = 10 // Compiler trust you first, and it thinks you have `implicitly` told it that you had a way to covert the type from Int to String, which the function `intToString` can do!
// So num is now actually "10 -- I am a String now!"
// console will print this -> val num: String = 10 -- I am a String now!
Hope this can help.
A very basic example of Implicits in scala.
Implicit parameters:
val value = 10
implicit val multiplier = 3
def multiply(implicit by: Int) = value * by
val result = multiply // implicit parameter wiil be passed here
println(result) // It will print 30 as a result
Note: Here multiplier will be implicitly passed into the function multiply. Missing parameters to the function call are looked up by type in the current scope meaning that code will not compile if there is no implicit variable of type Int in the scope.
Implicit conversions:
implicit def convert(a: Double): Int = a.toInt
val res = multiply(2.0) // Type conversions with implicit functions
println(res) // It will print 20 as a result
Note: When we call multiply function passing a double value, the compiler will try to find the conversion implicit function in the current scope, which converts Int to Double (As function multiply accept Int parameter). If there is no implicit convert function then the compiler will not compile the code.