I try to write a method which should use a variable from the surrounding scope. The problem is that I cannot access the part of the code where the variable is defined. Something like this:
object Example extends App {
val myvar = 1.0 // cannot change that code
def myMethod()(implicit value:Double) = {
print(value)
}
myMethod()
}
This fails because myMethod cannot find a suitable implicit for value.
is there a way to "mark" value as implicit after it has been defined, other than defining a new implicit variable pointing to value?
Background: We use Spark-Notebook where the SparkContext (named sc) is created automatically. As sc is a commonly known name for this variable in the community, we would prefer not to introduce another variable name.
If you just don't want to have an instance of SparkContext that isn't labeled as sc you could assign it to an implicit-underscore variable like this:
implicit val _: SparkContext = sc
That would create an implicit variable of the correct type in scope for your function to use.
Alternatively you can always pass implicit variables explicitly, in your example above you could do
myMethod()(myvar)
and that is the solution that I would use.
Related
I may have not asked the question correctly but let say I want to use the name of whatever parameter i pass in to my function, to have my variable within that function to be named ;
def myFunc(dfName: DataFrame): Unit = {
val "{dfName}_concatenated" = dfName
}
so if i passed in myFunc(testDf) the variable inside should be named testDf_concatenated
Values and variables have to be named at compile time and this process you're looking for needs to be done at runtime, so you can not do it this way, try using a simple name and the value would be a Map("{whatever}_concatenated" -> ...).
I am really not sure what is your question, will that work for you?
def myFunc(dfName: String, anotherParam: Int): Unit = {
val dfNameConcatenated = s"${dfName}_concatenated"
}
I am using avro4s to help with avro serialization and deserialization.
I have a case class that includes Timestamps and need those Timestamps to be converted to nicely formatted strings before I publish the records to Kafka; the default encoder is converting my Timestamps to Longs. I read that I needed to write a decoder and encoder (from the avro4s readme).
Here is my case class:
case class MembershipRecordEvent(id: String,
userHandle: String,
planId: String,
teamId: Option[String] = None,
note: Option[String] = None,
startDate: Timestamp,
endDate: Option[Timestamp] = None,
eventName: Option[String] = None,
eventDate: Timestamp)
I have written the following encoder:
Test.scala
def test() = {
implicit object MembershipRecordEventEncoder extends Encoder[MembershipRecordEvent] {
override def encode(t: MembershipRecordEvent, schema: Schema) = {
val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss")
val record = new GenericData.Record(schema)
record.put("id", t.id)
record.put("userHandle", t.userHandle)
record.put("teamId", t.teamId.orNull)
record.put("note", t.note.orNull)
record.put("startDate", dateFormat.format(t.startDate))
record.put("endDate", if(t.endDate.isDefined) dateFormat.format(t.endDate.get) else null)
record.put("eventName", t.eventName.orNull)
record.put("eventDate", dateFormat.format(t.eventDate))
record
}
}
val recordInAvro2 = Encoder[MembershipRecordEvent].encode(testRecord, AvroSchema[MembershipRecordEvent]).asInstanceOf[GenericRecord]
println(recordInAvro2)
}
If I declare the my implicit object in line, like I did above, it creates the GenericRecord that I am looking for just fine. I tried to abstract the implicit object to a file, wrapped in an object, and I import Implicits._ to use my custom encoder.
Implicits.scala
object Implicits {
implicit object MembershipRecordEventEncoder extends Encoder[MembershipRecordEvent] {
override def encode(t: MembershipRecordEvent, schema: Schema) = {
val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss")
val record = new GenericData.Record(schema)
record.put("id", t.id)
record.put("userHandle", t.userHandle)
record.put("teamId", t.teamId.orNull)
record.put("note", t.note.orNull)
record.put("startDate", dateFormat.format(t.startDate))
record.put("endDate", if(t.endDate.isDefined) dateFormat.format(t.endDate.get) else null)
record.put("eventName", t.eventName.orNull)
record.put("eventDate", dateFormat.format(t.eventDate))
record
}
}
}
Test.scala
import Implicits._
val recordInAvro2 = Encoder[MembershipRecordEvent].encode(testRecord, AvroSchema[MembershipRecordEvent]).asInstanceOf[GenericRecord]
println(recordInAvro2)
It fails to use my encoder (doesn't hit my breakpoints). I have tried a myriad of things to try and see why it fails to no avail.
How can I correctly import an implicit object?
Is there a simpler solution to encode my case class's Timestamps to Strings without writing an encoder for the entire case class?
TL;DR
As suggested in one of the comments above, you can place it in the companion object.
The longer version:
Probably you have another encoder, that is used instead of the encoder you defined in Implicits.
I'll quote some phrases from WHERE DOES SCALA LOOK FOR IMPLICITS?
When a value of a certain name is required, lexical scope is searched for a value with that name. Similarly, when an implicit value of a certain type is required, lexical scope is searched for a value with that type.
Any such value which can be referenced with its “simple” name, without selecting from another value using dotted syntax, is an eligible implicit value.
There may be more than one such value because they have different names.
In that case, overload resolution is used to pick one of them. The algorithm for overload resolution is the same used to choose the reference for a given name, when more than one term in scope has that name. For example, println is overloaded, and each overload takes a different parameter type. An invocation of println requires selecting the correct overloaded method.
In implicit search, overload resolution chooses a value among more than one that have the same required type. Usually this entails selecting a narrower type or a value defined in a subclass relative to other eligible values.
The rule that the value must be accessible using its simple name means that the normal rules for name binding apply.
In summary, a definition for x shadows a definition in an enclosing scope. But a binding for x can also be introduced by local imports. Imported symbols can’t override definitions of the same name in an enclosing scope. Similarly, wildcard imports can’t override an import of a specific name, and names in the current package that are visible from other source files can’t override imports or local definitions.
These are the normal rules for deciding what x means in a given context, and also determine which value x is accessible by its simple name and is eligible as an implicit.
This means that an implicit in scope can be disabled by shadowing it with a term of the same name.
Now I'll state the companion object logic:
Implicit syntax can avoid the import tax, which of course is a “sin tax,” by leveraging “implicit scope”, which depends on the type of the implicit instead of imports in lexical scope.
When an implicit of type T is required, implicit scope includes the companion object T:
When an F[T] is required, implicit scope includes both the companion of F and the companion of the type argument, e.g., object C for F[C].
In addition, implicit scope includes the companions of the base classes of F and C, including package objects, such as p for p.F.
Suppose I want to explicitly pass an ExecutionContext as a parameter to a method, but want to use that context implicitly in the body of that method. I can achieve this with the following simple code snippet:
def run(foo: Unit => Unit,
ex1: ScheduledExecutorService) {
import scala.concurrent.duration._
implicit val ex2 = ex1
scheduleAtFixedRate(foo, 1.seconds, 3.seconds) // pass ex2 implicitly, not explicitly
}
Is there a way to do this without reassigning ex1 to ex2, but still require the parameters to be passed explicitly?
No. If you want to require that a parameter be passed in explicitly, the only way to use it implicitly within the body of the function is to reassign it to an implicit variable. Why you would ever want to do that defies my reasoning, however.
Resolution of implicit parameters is type-based. The only way I can think of (especially since you want to receive the parameter from the caller) is to have run accept the executor service implicitly, as in:
def run(foo: Unit => Unit)(implicit ex1: ScheduledExecutorService) {
// no need for the below; ex1 already used for implicit resolution
// implicit val ex2 = ex1
}
Of course, that means that the caller for the run method must define the implicit value to used for the executor service, or pass it explicitly as in:
run(someFoo)(someScheduledExecutorService)
I have a class which takes a function
case class FunctionParser1Arg[T, U](func:(T => U))
def testFunc(name1:String):String = name1
val res = FunctionParser1Arg(testFunc)
I would like to know the type signature information on the function from inside the case class. I want to know both the parameter name and type. I have had success in finding the type using the runtime mirror objects, but not the name. Any suggestions?
Ok, let's say you got the symbol for the instance func points to:
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{currentMirror => m}
val im = m reflect res.func // Instance Mirror
You can get the apply method from its type members:
val apply = newTermName("apply")
val applySymbol = im.symbol.typeSignature member apply
And since we know it's a method, make it a method symbol:
val applyMethod = applySymbol.asMethod
It's parameters can be found through paramss, and we know there's only one parameter on one parameter list, so we can get the first parameter of the first parameter list:
val param = applyMethod.paramss(0)(0)
Then what you are asking for is:
val name = param.name.decoded // if you want "+" instead of "$plus", for example
val type = param.typeSignature
It's possible that you think that's the wrong answer because you got x$1 instead of name1, but what is passed to the constructor is not the named function testFunc, but, instead, an anonymous function representing that method created through a process called eta expansion. You can't find out the parameter name of the method because you can't pass the method.
If that's what you need, I suggest you use a macro instead. With a macro, you'll be able to see exactly what is being passed at compile time and get the name from it.
I would like to write a class looking like this:
class Store[+A](dest: Symbol)(implicit c: String => A) extends Action(dest) {
override def update(options: HashMap[Symbol,Any], arg: String): Unit = {
options += ((dest -> c(arg)))
}
}
object Store {
def apply[A](dest: Symbol)(c: String=>A) = new Store[A](dest)(c)
def apply[A](dest: Symbol) = new Store[A](dest)
}
Doing so, I have a few problems:
Using implicit with strings causes no end of trouble
Anyway, the system doesn't find the implicit if they are defined in my module, they would need to be defined in the module creating the class
the second apply method of the Store object just doesn't compile as A will be erased so the compiler has no way of finding a conversion from String to A
How would you create such an object that convert a string to some other type? I wouldn't want the user of the library to enter the type rwice (i.e. by specifying both the type and the conversion function).
I don't understand what you are trying with the second apply. To me, it looks like the first apply should have the implicit keyword, and you'd be done with it. You can either pass the parameter explicitly, or leave it out if an implicit is present. Also, you wouldn't need to pass c explicitly, since you'd already have it implicitly in the scope of the first apply.
I'd venture the second apply doesn't compile because there's no implicit String => A available in the scope of object Store.