I was receiving this error in my Hadoop job.
java.lang.NoSuchMethodException: <PackageName>.<ClassName>.<init>(<parameters>)
In most Scala code, you would have it in compile time. But since this job is called in runtime I was not catching it in compile time.
I would think default parameter would cause constructors with both signatures to be created, one taking a single argument.
class BasicDynamicBlocker(args: Args, evaluation: Boolean = false) extends Job(args) with HiveAccess {
//I NEEDED THIS TOO:
def this(args: Args) = {
this(args, false)
}
...
}
I learned the hard way that I needed to declare the overloaded constructor using this. (I wanted to write this out in case it helps someone else.)
I also have a small questions. It still seems redundant to me. Is there a reason Scala language's design restrictions require this?
It is not like when you have default parameter you will get overloads generated for each possible case, like for example:
def method(num: Int = 4, str: String = "") = ???
you expect compiler to generate
def method(num: Int) = method(num, "")
def method(str: String) = method(4, str)
def method() = method(4, "")
but that is not the case.
You will instead have generated methods (in companion object), for each default param
def method$default$1: Int = 4
def method$default$2: String = "a"
and whenever you say in your code
method(str = "a")
it will be just changed to
method(method$default$1, "a")
So in your case, constructor with signature this(args: Args) just did not exist, there was only the 2 param version.
You can read more here: http://docs.scala-lang.org/sips/completed/named-and-default-arguments.html
Related
A use case needs to compare the actual type of two type parameters at runtime time. That might be easily done with TypeTag, but it is deprecated in scala-3.
Therefore I tried with TypeTest but it requires the existence of an instance of the type, which I don't have.
So, I implemented TypeId, my own version of TypeTag. It is way way more limited and simple. It only supports type comparison and works only if a single instance of TypeId exists for each involved type.
import scala.collection.{mutable, immutable}
object TypeId {
private var idSequencer: Int = 0
private val typeIds: mutable.HashMap[Int, TypeId[?]] = mutable.HashMap.empty
private def registerNew[A](tm: TypeId[A]): Int = this.synchronized {
idSequencer += 1
typeIds.put(idSequencer, tm)
idSequencer
}
def knownTypeIds: immutable.HashMap[Int, TypeId[?]] = this.synchronized(immutable.HashMap.from(typeIds))
}
trait TypeId[A] {
val id: Int = registerNew(this)
def equalsTo(that: TypeId[?]): Boolean = this.id == that.id
}
As you can see, it requires synchronization to be thread-safe, which is not convenient; and the value of knownTypeIds is vulnerable to race conditions.
I wonder if it is possible to create the unique ids at compile using a macro. I suppose that to do that I need some kind of compile-time global variable to implement the sequencer. Does scala-3 macros support compile time global variables or an alternative to achieve my goal?
EDIT:
A couple of minutes after asking my question I came up with a very simple way to achieve my goal by making the identifier a string with the extended name of the type. Such a name can easily be generated in a macro.
However, I leave the question to know if scala-3 macros support compile-time global variables.
Here is the code that achieves my goal using the name of the type instead of sequential integer.
import scala.collection.{immutable}
object TypeId {
#volatile
private var typeIds: immutable.Set[String] = immutable.HashSet.empty
def knownTypeIds: immutable.Set[String] = typeIds
private inline def registerNew[A]: String = {
val id = TypeIdMacros.createId[A]
this.synchronized(typeIds += id)
id
}
}
trait TypeId[A] {
val id: String = registerNew[A]
def equalsTo(that: TypeId[?]): Boolean = this.id == that.id
}
import scala.quoted.{Quotes, Type, Expr}
object TypeIdMacros {
inline def createId[A]: String = ${createIdImpl[A]}
def createIdImpl[A](using typeA: Type[A], quotes: Quotes): Expr[String] =
Expr(Type.show[A])
}
It would be a very bad idea, because of incremental compilation - it's quite hard to tell exactly what will be compiled or when. You could easily get something that works on a clean compile, but fails on partial re-compiles during development as not every bit of code that should be triggering the sequencer gets fired.
(I'm not sure if it's doable at all, and there might be other reasons to avoid it, but this is a general reason to avoid any sort of imperative code in macros.)
I have read that null should not be used in scala.
How can I leave myVar uninitialized without the use of null?
class TestClass {
private var myVar: MyClass = null
}
I understand I can just make a dummy MyClass, that is never used in place of the null. But this can and does reduce the code's understandability.
As Rado has explained I can change null as shown below. I understand that I can now check to see if the variable is set during run-time, however, if I don't program that check then there is no benefit of using Option in this case.
Coming from Java, I feel there should be a way to simply leave the var uninitialized at compile-time and let it set during run-time without using the Option class, because as I mentioned above, if I don't code for the unset case then why use Option?
class TestClass {
private var myVar: Option[MyClass] = None
private def createVar() {
myVar = Some(new MyClass)
x: MyClass = myVar.get
}
}
I am thinking the only other way of doing what I am asking is:
class TestClass {
// Using dummy MyClass that will never be used.
private var myVar: MyClass = new MyClass
private def process(myVar: MyClass) {
this.myVar = myVar
myVar.useVarMethod()
}
}
The Scala way is to declare the variable as Option[MyClass]:
class TestClass {
private var myVar: Option[MyClass] = None
private def createVar() {
myVar = Some(new MyClass)
}
// Usage example:
def useMyVar(): Unit = {
myVar match {
case Some(myClass) => {
// Use myClass here ...
println(myClass.toString)
}
case None => // What to do if myVar is undefined?
}
}
}
That way you avoid NullPointerException. You make it explicit that the variable can be in undefined state. Everytime you use the myVar you have to define what to do if it is undefined.
http://www.scala-lang.org/api/current/index.html#scala.Option
I need myVar to be of type MyClass not Option[MyClass]. I see that I
could use Rado's updated answer and then use the get method, but is
there any other way?
When you use Option you can telling the compiler and everyone else who will read/use your code that it's okay not to define this value and the code will handle that condition and not fail at runtime.
The other way of dealing with is to do null checks every time before you access the variable because it could be null and therefore throw an exception at runtime.
When you use Option, the compiler will tell you if at compile time that you have not handled a condition where the value of a variable maybe undefined.
If you think about it, it's really a big deal. you have converted a runtime exception (which is deterministic) to a compile-time error.
If you want to extract the value out of something like an Option (which supports map and also flatMap), then you don't necessarily have to keep doing pattern matching on whether or not the Option contains a value (i.e. is a "Some") or not (i.e. is a "None").
Two methods are very useful - if you want just alter (or "map") the value within the Option then you can use the map method, which takes a function with a general type of:
f: A => B
so in your case at compile time would end up being:
f: MyClass => B
When you map an option, if the option is a "Some" then the contained value is passed through to the mapping function, and the function is applied (to change the MyClass to a B if you like...) and the result is passed back wrapped in an Option. If your Option is a None, then you just get a None back.
Here's a simple example:
scala> case class MyClass(value : String)
defined class MyClass
scala> val emptyOption : Option[MyClass] = None
emptyOption: Option[MyClass] = None
scala> val nonEmptyOption = Some(new MyClass("Some value"))
nonEmptyOption: Option[MyClass] = Some(MyClass(Some value)
Try and extract the value from both option instances using map:
scala> nonEmptyOption map { s => s.value + " mapped" }
res10: Option[String] = Some(Some value mapped)
scala> emptyOption map { s => s.value }
res11: Option[String] = None
So basically, if you map across an empty option, you always get a None. Map across a non-empty Option and you can extract the contained value and transform it and get the result wrapped back in an Option. Notice that you don't need to explicitly check any patterns...the map method takes care of it.
Flatmap is a bit more challenging to explain, but it basically isn't massively different except that it takes a function which has type:
g: A => M[B]
which basically allows you to take the value out of the Option (the 'A' in the type signature above), do something to it (i.e. a computation) and then return the result ('B') - wrapped in another container such as another Option, List...
The same notion (across Option anyway) that if the Option is a None then nothing really happens still applies. flatMap and map form the basis of Scala "for comprehensions" which you can read about (and are done far more justice than I could!!) in lots of other places.
I've been lately working on the DSL-style library wrapper over Apache POI functionality and faced a challenge which I can't seem to good solution for.
One of the goals of the library is to provide user with ability to build a spreadsheet model as a collection of immutable objects, i.e.
val headerStyle = CellStyle(fillPattern = CellFill.Solid, fillForegroundColor = Color.AquaMarine, font = Font(bold = true))
val italicStyle = CellStyle(font = Font(italic = true))
with the following assumptions:
User can optionally specify any parameter (that means, that you can create CellStyle with no parameters as well as with the full list of explicitly specified parameters);
If the parameter hasn't been specified explicitly by the user it is considered undefined and the default environment value (default value for the format we're converting to) will be used;
The 2nd point is important, as I want to convert this data model into multiple formats and i.e. the default font in Excel doesn't have to be the same as default font in HTML browser (and if user doesn't define the font family explicitly I'd like him to see the data using those defaults).
To deal with the requirements I've used the variation of the null pattern described here: Pattern for optional-parameters in Scala using null and also suggested here Scala default parameters and null (below a simplified example).
object ModelObject {
def apply(modelParam : String = null) : ModelObject = ModelObject(
modelParam = Option(modelParam)
)
}
case class ModelObject private(modelParam : Option[String])
Since null is used only internally in the companion object and very localized I decided to accept the null-sacrifice for the sake of the simplicity of the solution. The pattern works well with all the reference classes.
However for Scala primitive types wrappers null cannot be specified. This is especially a huge problem with Boolean for which I effectively consider 3 states (true, false and undefined). Wanting to provide the interface, where user still be able to write bold = true I decided to reach to Java wrappers which accept nulls.
object ModelObject {
def apply(boolParam : java.lang.Boolean = null) : ModelObject = ModelObject(
boolParam = Option(boolParam).map(_.booleanValue)
)
}
case class ModelObject private(boolParam : Option[Boolean])
This however doesn't right and I've been wondering whether there is a better approach to the problem. I've been thinking about defining the union types (with additional object denoting undefined value): How to define "type disjunction" (union types)?, however since the undefined state shouldn't be used explicitly the parameter type exposed by IDE to the user, it is going to be very confusing (ideally I'd like it to be Boolean).
Is there any better approach to the problem?
Further information:
More DSL API examples: https://github.com/norbert-radyk/spoiwo/blob/master/examples/com/norbitltd/spoiwo/examples/quickguide/SpoiwoExamples.scala
Sample implementation of the full class: https://github.com/norbert-radyk/spoiwo/blob/master/src/main/scala/com/norbitltd/spoiwo/model/CellStyle.scala
You can use a variation of the pattern I described here: How to provide helper methods to build a Map
To sum it up, you can use some helper generic class to represent optional arguments (much like an Option).
abstract sealed class OptArg[+T] {
def toOption: Option[T]
}
object OptArg{
implicit def autoWrap[T]( value: T ): OptArg[T] = SomeArg(value)
implicit def toOption[T]( arg: OptArg[T] ): Option[T] = arg.toOption
}
case class SomeArg[+T]( value: T ) extends OptArg[T] {
def toOption = Some( value )
}
case object NoArg extends OptArg[Nothing] {
val toOption = None
}
You can simply use it like this:
scala>case class ModelObject(boolParam: OptArg[Boolean] = NoArg)
defined class ModelObject
scala> ModelObject(true)
res12: ModelObject = ModelObject(SomeArg(true))
scala> ModelObject()
res13: ModelObject = ModelObject(NoArg)
However as you can see the OptArg now leaks in the ModelObject class itself (boolParam is typed as OptArg[Boolean] instead of Option[Boolean].
Fixing this (if it is important to you) just requires to define a separate factory as you have done yourself:
scala> :paste
// Entering paste mode (ctrl-D to finish)
case class ModelObject private(boolParam: Option[Boolean])
object ModelObject {
def apply(boolParam: OptArg[Boolean] = NoArg): ModelObject = new ModelObject(
boolParam = boolParam.toOption
)
}
// Exiting paste mode, now interpreting.
defined class ModelObject
defined module ModelObject
scala> ModelObject(true)
res22: ModelObject = ModelObject(Some(true))
scala> ModelObject()
res23: ModelObject = ModelObject(None)
UPDATE The advantage of using this pattern, over simply defining several overloaded apply methods as shown by #drexin is that in the latter case the number of overloads grows very fast with the number of arguments(2^N). If ModelObject had 4 parameters, that would mean 16 overloads to write by hand!
For scala case class with number of parameters (21!!)
e.g. case class Car(type: String, brand: String, door: Int ....)
where type = jeep, brand = toyota, door = 4 ....etc
And there is a copy method which allow override with named parameter: Car.copy(brand = Kia)
where would become type = jeep, brand = Kia, door = 2...etc
My question is, is there anyway I can provide the named parameter dynamically?
def copyCar(key: String, name: String) = {
Car.copy("key" = "name") // this is something I make up and want to see if would work
}
Is scala reflection library could provide a help here?
The reason I am using copy method is that I don't want to repeat the 21 parameters assignment every time when I create a case class which only have 1 or 2 parameter changed.
Many Thanks!
FWIW, I've just implemented a Java reflection version: CaseClassCopy.scala. I tried a TypeTag version but it wasn't that useful; TypeTag was too restrictive for this purpose.
def copy(o: AnyRef, vals: (String, Any)*) = {
val copier = new Copier(o.getClass)
copier(o, vals: _*)
}
/**
* Utility class for providing copying of a designated case class with minimal overhead.
*/
class Copier(cls: Class[_]) {
private val ctor = cls.getConstructors.apply(0)
private val getters = cls.getDeclaredFields
.filter {
f =>
val m = f.getModifiers
Modifier.isPrivate(m) && Modifier.isFinal(m) && !Modifier.isStatic(m)
}
.take(ctor.getParameterTypes.size)
.map(f => cls.getMethod(f.getName))
/**
* A reflective, non-generic version of case class copying.
*/
def apply[T](o: T, vals: (String, Any)*): T = {
val byIx = vals.map {
case (name, value) =>
val ix = getters.indexWhere(_.getName == name)
if (ix < 0) throw new IllegalArgumentException("Unknown field: " + name)
(ix, value.asInstanceOf[Object])
}.toMap
val args = (0 until getters.size).map {
i =>
byIx.get(i)
.getOrElse(getters(i).invoke(o))
}
ctor.newInstance(args: _*).asInstanceOf[T]
}
}
It is not possible using case classes.
Copy method generated at compile time and named parameters handled on compile time to. There is no possibility to do it ar runtime.
Dynamic may help to solve your issue: http://hacking-scala.tumblr.com/post/49051516694/introduction-to-type-dynamic
Yes, you would need to use reflection to do that.
It is a bit involved, because copy is a synthetic method and you'll have to invoke the getters for all fields except the one you want to replace.
To give you an idea, the copy method in this class does exactly that, except using an argument index instead of name. It calls the companion object's apply method, but the effect is the same.
I'm a bit confused - how is the following not what you need?
car: Car = ... // Retrieve an instance of Car somehow.
car.copy(type = "jeep") // Copied instance, only the type has been changed.
car.copy(door = 4) // Copied instance, only the number of doors has changed.
// ...
Is it because you have a lot of parameters for the initial instance creation? In that case, can you not use default values?
case class Car(type: String = "Jeep", door: Int = 4, ...)
You seem to know about both these features and feel that they don't fit your need - could you explain why?
I know there are multiple questions addressing related problems, but I'm not sure it does attack exactly what I'm looking for. I'm still new to Scala, after several years of Java development. I'm looking for the best way to test if an object has been initialized, and if not, initialize it then. For example, in Java:
private MyObject myObj = null;
and at some point in the future:
public void initMyObj(){
if (myObj == null){
myObj = new MyObj();
}
// do something with myObj
}
After this, I might reassign myObj to a different object, but it is unlikely. In Scala, I have this:
class Test {
var myObj: MyObj = _
}
I've read that I could use Option instead, something like:
var myObj = None : Option[MyObj]
and then my check:
myObj match {
case None => ...
case Some(value) => ...
}
but it feels ackward to use this pattern when I might not make this kind of check anywhere else at any other time - though being so new to Scala, I might be wrong. Is this the best way to achieve what I want or is there any other option not involving Option?
It is not generally ideal practice in Scala to leave partially-constructed objects lying around. You would normally rethink how your objects were getting instantiated to see if you can't use a different pattern that is less fragile. For example, instead of setting uninitialized variables in methods:
class Foo { var a: String = null; var b: String = null }
def initFooA(s: String, f: Foo) { if (f.a == null) f.a = s }
def initFooB(s: String, f: Foo) { if (f.b == null) f.b = s }
f
initFooA("salmon", f)
// Do stuff
initFooB("herring", f)
you would attempt to restructure your code to generate the values you need on demand, and delay the instantiation of foo until then:
case class Bar(a: String, b: String) {}
def initBarA(s: String) = s
def initBarB(s: String) = s
val iba = initBarA("halibut")
// Do stuff
val ibb = initBarB("cod")
Bar(iba, ibb)
Because Scala has easy access to tuples (and type inference), this can be a lot less painful than in Java.
Another thing you can do is defer the late initialization to someone else.
case class Baz(a: String)(bMaker: => String) {
lazy val b = bMaker
}
Now you pass in something that will make parameter b, and arrange for it to handle any late initialization stuff that needs to be handled. This doesn't always avoid needing to set vars, but it can help push it out of your class code into your initialization logic (which is usually a better place for it).
Doing this with vars is a little less straightforward. Realistically, you're probably best off just devoting a class to it e.g. by:
class LazyVar[A](initial: => A) {
private[this] var loaded = false
private[this] var variable: A = _
def apply() = { if (!loaded) { loaded = true; variable = initial }; variable }
def update(a: A) { loaded = true; variable = a }
}
where you then (sadly) have to use () on every read and write.
scala> val lv = new LazyVar({ println("Hi!"); 5 })
lv: LazyVar[Int] = LazyVar#2626ea08
scala> lv()
Hi!
res2: Int = 5
scala> lv() = 7
scala> lv()
res4: Int = 7
Then you use an instance of this class instead of the actual var and pass through the lazy initializer. (lazy val is very much like this under the hood; the compiler just protects you from noticing.)
Finally, if you want to have a fully-functional object that is occasionally missing a value, var x: Option[X] is the construct you want to use; if you can't find a way around the standard Java creation patterns (and you don't want to try something more exotic like objects that create each other with more and more information, either because performance is critical and you can't afford it, or you dislike writing that much boilerplate to allow type-checking to verify that your object is properly created) but you otherwise want to use it, var x: X = null is what I'd choose, not _. If X is a primitive, you probably need to choose the correct value wisely anyway (for example, Double.NaN instead of 0.0, -1 rather than 0 for Int) to indicate I-am-not-initialized. If it's generic code and you want Any instead of AnyRef, asInstanceOf-ing back and forth between Any and AnyRef is probably the best way out of poorly typechecked situation (assuming you really, really can't use Option, which at that point is much clearer).
Maybe a lazy variable is what you need.
lazy val myObj: MyObj = //here you put the object creation code
In this way the object creation is postponed to the first time the code tries to access it.