In Scala/Spark DataFrame
dfReduced.schema.fieldNames
is a java String array (String[]). However,
dfReduced.schema.fieldNames.asInstanceOf[Seq[String]]
throws
java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to
scala.collection.Seq
Assigning same array to a Seq[String] is fine.
val f3:Seq[String]=dfReduced.schema.fieldNames
As a Java programmer this surprises me as both would require casting in Java. Can someone explain why there is this distinction in Scala
(Note, I'm not being critical, I just want to understand Scala better)
The reason why val f3:Seq[String]=dfReduced.schema.fieldNames this is working is because In Scala there is implicit conversion available than can cast the Array[T] to Seq[T] implicitly
In Java there is no such type of implicit casting available.
As Leo C mention in comment The difference is run-time type cast versus compile-time type ascription. For more info you can refer to this link.
Hope this clears your dough
Thanks
Related
I have a List[Any] which I want to convert to a JsArray.
List with type works:
Json.arr(List("1"))
But:
Json.arr(List("1").asInstanceOf[List[Any]])
throws:
diverging implicit expansion for type play.api.libs.json.Reads[T1]
starting with method oFormatFromReadsAndOWrites in object OFormat
How can I convert List[Any] to JsArray?
I tried:
implicit val listAnyFormat: OFormat[List[Any]] = Json.format[List[Any]]
But I get thrown with:
No instance of Reads is available for scala.collection.immutable.Nil in the implicit scope
Using Play 2.8.x and Scala 2.11.8
You can't.
At least not without defining a Format[Any] which can be done technically but will likely not cover all the possible cases.
The question is why do you have a List[Any] in the first place? It has not much sense in Scala world.
It would be better if you could have a List[Something] where Something has a known set of subtypes and each of them has a Format.
In Scala REPL I can use Seq[String]() as a default value for a parameter of type Seq[T].
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101).
Type in expressions to have them evaluated.
Type :help for more information.
scala> def d[T](foo: Seq[T] = Seq[String]()) = 12
d: [T](foo: Seq[T])Int
scala> d()
res0: Int = 12
Trying the same in IDEA, it complains “Seq[String] doesn't conform to expected type Seq[T]”. Why?
IntelliJ IDEA 2016.2.4
Scala Plugin 2016.2.1
Scala 2.11.7
Note 1: Sorry, I know that my example function does not make much sense. However, my real (and useful) function is unnecessarily complex to post it here.
Note 2: At first, instead of type T my type name in the example was Any which was not a good idea (because it shadowed scala.Any) and caused some confusion. Thus I fixed that.
When you say def d[Any], the Any here is a generic place holder. It does not point to class Any in scala. It basically shadows the Any class defined globally in scala. So, when you assign Seq[String] to Seq[Any], the compiler does not know any relation between String and Any. Note that Any could be replaced with any character / word as generic place holder. The result would be the same.
Now coming, to why this works in REPL, I am not exactly sure why REPL accepts if Seq[String] when given as a default value, but I was able to reproduce the error in repl when I do the same operation inside the method body.
The following code in REPL throws error:
def d[Any](foo: Seq[Any]) = {
val a: Seq[Any] = Seq[String]()
}
<console>:12: error: type mismatch;
found : Seq[String]
required: Seq[Any]
val a: Seq[Any] = Seq[String]()
^
I am not sure why REPL was not able to catch the error while given as a default argument.
One alternative theory is, in general when you use generics, the value of the type will be determined based on the caller. For example,
def d[A](a:A) = {}
d(1) // Type of A is Int
d("a") // Type of A is String
So, when you give default value, it assigns the value of String to Any. hence the compilation success.Intellij's Type Checker works based on the first theory and shows an error. But strangely as someone pointed out earlier, the compilation succeeds.
val mutator=HFactory.createMutator(keyspace,StringSerializer.get())
mutator.addInsertion("rahul", "user", HFactory.createColumn("birth_year", 1990,
StringSerializer.get(), LongSerializer.get()))//error in LongSerializer.get() as
mutator.execute()
I am using LongSerializer like above and i am getting the following error.
Description Resource Path Location Type
type mismatch; found : me.prettyprint.cassandra.serializers.LongSerializer
required: me.prettyprint.hector.api.Serializer[Any] Note: Long <: Any (and
me.prettyprint.cassandra.serializers.LongSerializer <:
me.prettyprint.cassandra.serializers.AbstractSerializer[Long]), but Java-defined trait
Serializer is invariant in type T. You may wish to investigate a wildcard type such as _
<: Any. (SLS 3.2.10) User.scala /winoria/app/models line 22 Scala Problem
Tell me the solution .
There are a couple of things happening here.
First, Java does not allow primitive types as generics, so Hector's LongSerializer is an AbstractSerializer[java.lang.Long]. However you are working in Scala, so you need an AbstractSerializer[scala.Long]. Depending on circumstances Scala's Long can either be a primitive long or java.lang.Long. The good news is Scala is smart enough to figure out what to use and when. What you need here is a little type coercion: LongSerializer.get().asInstanceOf[Serializer[Long]]
The other suspect is that you need me.prettyprint.hector.api.Serializer[Any]. It looks like whatever you are calling your method on is lacking proper type declarations. A sample of surrounding code would help diagnose this further.
Edit:
Thanks for posting the surrounding code. It looks like you have a type disagreement. createColumn[N, V] is inferred as createColumn[String, Int], because the 1990 argument you have provided is an Int. This gets converted to java.lang.Int, which is a class and does not have type conversions like primitives do. This is why you get the error "int cannot be cast to long".
val mutator = HFactory.createMutator(keyspace, StringSerializer.get)
mutator.addInsertion(
"rahul", "user",
HFactory.createColumn( // Either do: HFactory.createColumn[String, Long]
"birth_year", 1990L, // Or make sure the type is inferred as Long
StringSerializer.get,
// Coerce serializer to scala.Long
LongSerializer.get.asInstanceOf[Serializer[Long]]))
mutator.execute()
The following Scala code works correctly:
val str1 = "hallo"
val str2 = "huhu"
val zipped: IndexedSeq[(Char, Char)] = str1.zip(str2)
However if I import the implicit method
implicit def stringToNode(str: String): xml.Node = new xml.Text(str)
then the Scala (2.10) compiler shows an error: value zip is not a member of String
It seems that the presence of stringToNode somehow blocks the implicit conversion of str1 and str2 to WrappedString. Why? And is there a way to modify stringToNode such that zip works but stringToNode is still used when I call a function that requires a Node argument with a String?
You have ambiguous implicits here. Both StringOps and xml.Node have the zip-method, therefore the implicit conversion is ambiguous and cannot be resolved. I don't know why it doesn't give a better error message.
Here are some links to back it up:
http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.StringOps
and
http://www.scala-lang.org/api/current/index.html#scala.xml.Node
edit: it was StringOps, not WrappedString, changed the links :) Have a look at Predef: http://www.scala-lang.org/api/current/index.html#scala.Predef$
to see predefined implicits in Scala.
I would avoid using implicits in this case. You want 2 different implicit conversions which both provide a method of the same name (zip). I don't think this is possible. Also, if you import xml.Text, you can convert with just Text(str) which should be concise enough for anyone. If you must have this implicit conversion to xml.Node, I would pack the implicit def into an object and then import it only in the places where you need it to make your code readable and to, possibly, avoid conflicts where you also need to zip strings. But basically, I would very much avoid using implicits just to make convenient conversions.
Like #Felix wrote, it is generally a bad idea to define implicit conversions between similar data types, like the one you used. Doing that weakens type system, leads to ambiguities like you encountered and may produce extremely unclear ("magic") code which is very hard to analyze and debug.
Implicit conversions in Scala are mostly used to define lightweight, short-lived wrappers in order to enrich API of wrapped type. Implicit conversion that converts String into WrappedString falls into that category.
Twitter's Effective Scala has a section about this issue.
I'm porting some java code across and have the following
val overnightChanges: java.util.Hashtable[String, Double] = ...
When I try
if (null != overnightChanges.get(...))
I get the following warning
warning: comparing values of types Null and Double using `!=' will always yield true
Primitive and reference types are much less different in scala than they are in java, and so the convention is that name starts with an uppercase for all of them. Double is scala.Double which is the primitive java double, not the reference java.lang.Double.
When you need "a double or no value" in scala, you would use Option[Double] most of the time. Option has strong library support, and the type system will not let you ignore that there might be no value. However, when you need to interact closely with java, as in your example, your table does contain java.lang.Double and you should say it so.
val a = new java.util.HashMap[String, java.lang.Double]
If java.lang.Double starts to appear everywhere in your code, you can alias to JDouble, either by importing
import java.lang.{Double => JDouble}
or by defining
type JDouble = java.lang.Double
There are implicit conversions between scala.Double and java.lang.Double, so interaction should be reasonably smooth. However, java.lang.Double should probably be confined to the scala/java interaction layer, it would be confusing to have it go deep into scala code.
In Scala Double are primitives and thus cannot be null. That's annoying when using directly java maps, because when a key is not defined, you get the default primitive value, (here 0.0):
scala> val a = new java.util.Hashtable[String,Double]()
a: java.util.Hashtable[String,Double] = {}
scala> a.get("Foo")
res9: Double = 0.0
If the value is a object like String or List, your code should work as expected.
So, to solve the problem, you can:
Use contains in an outer if condition.
Use one of the Scala maps (many conversions are defined in scala.collection.JavaConversions)
Use Scala "options", also known as "maybe" in Haskell:
http://blog.danielwellman.com/2008/03/using-scalas-op.html