I am trying to do something very simple and just cannot get past the second line.
Python 3.6 version throws an error "First argument must be callable or None" at >>> d = defaultdict(list)
Everywhere, the code is identified as exactly below, but for me the error comes up. This is so simple yet I am stumped. Please advise.
from collections import defaultdict
s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]
d = defaultdict(list)
d.default_factory
for k, v in s:
d[k].append(v)
d.items()
d
Related
I am doing a comparison between two BreezeDenseVectors with the following way a :< b and what i get as a return is a BitVector.
I haven't worked again with this and everything i read about it, was not helpful enough.
Can anyone explain to me how they work?
Additionally, by printing the output, i get: {0, 1, 2, 3, 4 }. What is this supposed to mean?
You can check BitVectorTest.scala for more detail usage.
Basically, a :< b gives you a BitVector, which indicates which elements in a smaller than the ones in b.
For example val a = DenseVector[Int](4, 9, 3); val b = DenseVector[Int](8, 2, 5); a :< b will gives you BitVector(0, 2), it means that a(0) < b(0) and a(2) < b(2), which is correct.
Update:
I had misunderstood an error message, so this question is invalid.
val inputIds = Array[Any](7, 1, 3)
is sufficient to create an Array[Any] in Scala.
===========
I'm trying to create an array that needs to be passed to a method that takes an Array[Any], but the values are numeric.
At first I tried just the normal way of creating an array:
val inputIds = Array(7, 1, 4)
and got the error:
Type mismatch, expected: Array[Any], actual: Array[Int]
(from the IntelliJ editor, I guess I haven't checked if it's correct)
Then I tried creating Integer values directly, e.g. by:
Integer.getInteger("7")
and passing those into the Array constructor, but I kept getting the same error.
Now I tried:
val inputIds: Array[Any] = Array[Any](7, 1, 4)
and it gave me:
Type mismatch, expected: Array[Any], actual: Array[Array[Int]]
As you can tell I'm going a bit spare, all I want is wrapper types and not primitives! I guess Scala is trying to optimize the array for iteration, but all I need is a tiny array I can pass to a var args taking arrays of mixed type.. Or maybe a better way of creating that vararg method?? Right now its type is:
Array[Any]*
========
Okay, so I got my original question resolved (though there's still a dispute in the comments over whether I correctly understood why the error was being caused). However, it didn't solve my problem, which is the following:
I am trying to transpose an array of arrays with different types (some are nested, but ultimately either Double or Ints) and can't figure out how to get the scala type system to do it. Here's the basic example:
val integerArray = Array(7, 1, 4)
val nestedIntegerArray = Array(
Array(6, 10, 60),
Array(5, 89, 57),
Array(15, 3, 5)
)
val nestedDoubleArray = Array(
Array(.13, .9, .8),
Array(.2, .777, .57),
Array(.15, .3, .5)
)
val brokenComputation = typeErrorExample(integerArray, nestedIntegerArray, nestedDoubleArray)
where the method being called is:
private def typeErrorExample(arrays: Array[_ <: Any]*)={
val arrayDataFrame = Array(arrays).transpose
}
This gives the error:
No implicit view available from Seq[Array[_]] => Array[U].
[INFO] val arrayDataFrame = Array(arrays).transpose
What's the right way to do this? Should I use a non-array data structure instead?
Hopefully having more of the code will help the experts understand what was causing my original error too. (inputIds was renamed to integerArray for consistency).
Explicitly casting is not ideal, but should do the trick.
Array(1, 2, 3).asInstanceOf[Array[Any]]
#Adair, this appears to arrange the input arrays into the Array[Array[Any]] type that you're looking for.
def transpose(arrays: Array[_]*) :Array[Array[_]] =
arrays.indices.map(x => arrays.map(_(x)).toArray).toArray
Now here's a word of advice: DON'T DO IT.
It's unsafe. This will throw an exception if the number of arrays passed exceeds the dimensions of each array.
Type Any is not what you want. When a data element becomes type Any then you've lost type information and it's a royal pain trying to get it back.
Jason's answer works, so does doing what the compiler error suggests when I ran it outside of intelliJ and changing the datatype in the method signature:
arrays: Array[_ <: Any]*
scala> Array(1, 2, 3).map(_.asInstanceOf[Any])
res6: Array[Any] = Array(1, 2, 3)
There are many answers which are correct. But why even need to do asInstanceOf. You can do it easily.
scala> val arr: Array[Any] = Array(1,2,3)
arr: Array[Any] = Array(1, 2, 3)
Simple, Work is done.
I'm trying to execute this line on a CoordinateMatrix...
test = test.entries.map(lambda (i, j, v): (j, (i, v)))
where the equivalent in Scala seems to work but fails in pyspark. The error I get when the line is executing...
'MatrixEntry' object is not iterable
And confirming that I am working with a CoordinateMatrix...
>>> test = test_coord.entries
>>> test.first()
>>> MatrixEntry(0, 0, 7.0)
Anyone know what might be off?
Suppose test is a CoordinatedMatrix, then:
test.entries.map(lambda e: (e.j, (e.i, e.value)))
A side note: you can't unpack a tuple in a lambda function. So map(lambda (x, y, z): ) is not going to work in this case even though it doesn't seem to be the reason that fails.
Example:
test = CoordinateMatrix(sc.parallelize([(1,2,3), (4,5,6)]))
test.entries.collect()
# [MatrixEntry(1, 2, 3.0), MatrixEntry(4, 5, 6.0)]
test.entries.map(lambda e: (e.j, (e.i, e.value))).collect()
# [(2L, (1L, 3.0)), (5L, (4L, 6.0))]
How to generate a list of n unique values (Gen[List[T]]) from a set of values (not generators) using ScalaCheck? This post uses Gen[T]* instead of a set of values, and I can't seem to rewrite it to make it work.
EDIT
At #Jubobs' request I now shamefully display what I have tried so far, revealing my utter novice status at using ScalaCheck :-)
I simply tried to replace gs: Gen[T] repeated parameter to a Set in what #Eric wrote as a solution here:
def permute[T](n: Int, gs: Set[T]): Gen[Seq[T]] = {
val perm = Random.shuffle(gs.toList)
for {
is <- Gen.pick(n, 1 until gs.size)
xs <- Gen.sequence[List[T], T](is.toList.map(perm(_)))
} yield xs
}
but is.toList.map(perm(_)) got underlined with red, with IntelliJ IDEA telling me that "You should read ScalaCheck API first before blind (although intuitive) trial and error", or maybe "Type mismatch, expected: Traversable[Gen[T]], actual List[T]", I can't remember.
I also tried several other ways, most of which I find ridiculous (and thus not worthy of posting) in hindsight, with the most naive being the using of #Eric's (otherwise useful and neat) solution as-is:
val g = for (i1 <- Gen.choose(0, myList1.length - 1);
i2 <- Gen.choose(0, myList2.length - 1))
yield new MyObject(myList1(i1), myList2(i2))
val pleaseNoDuplicatesPlease = permute(4, g, g, g, g, g)
After some testing I saw that pleaseNoDuplicatesPlease in fact contained duplicates, at which point I weighed my options of having to read through ScalaCheck API and understand a whole lot more than I do now (which will inevitably, and gradually come), or posting my question at StackOverflow (after carefully searching whether similar questions exist).
Gen.pick is right up your alley:
scala> import org.scalacheck.Gen
import org.scalacheck.Gen
scala> val set = Set(1,2,3,4,5,6)
set: scala.collection.immutable.Set[Int] = Set(5, 1, 6, 2, 3, 4)
scala> val myGen = Gen.pick(5, set).map { _.toList }
myGen: org.scalacheck.Gen[scala.collection.immutable.List[Int]] = org.scalacheck.Gen$$anon$3#78693eee
scala> myGen.sample
res0: Option[scala.collection.immutable.List[Int]] = Some(List(5, 6, 2, 3, 4))
scala> myGen.sample
res1: Option[scala.collection.immutable.List[Int] = Some(List(1, 6, 2, 3, 4))
I am new to scala..I came across a concept where it says like below:
{ val x = a; b.:::(x) }
In this block a is still evaluated before b, and then the result of
this evaluation is passed as an operand to b’s ::: method
What is the meaning of above statement..
I tried like below:
var a =10
var b =20
What should be the result i should expect.
Can somebody please give me an example...
Thanks in advance....
The ::: operator is defined on List trait and concatenates two lists. Using it on Int like in your example (var a=10) shouldn't work (unless you define such operator yourself).
Here is how it works on lists:
val a = List(1, 2);
val b = List(3, 4);
val c1 = a ::: b // List(1, 2, 3, 4)
val c2 = a.:::(b) // List(3, 4, 1, 2)
Calling ::: with the infix syntax (c1) and method call syntax (c2) differ in the order in which lists are concatenated (see Jörg's comment).
The statement "a is still evaluated before b" means that a is evaluated before passing it as an argument to the method call. Unless the method uses call by name, its arguments are evaluated before the call just like in Java.
This could give you some hint how to search for meaning of Scala operators and keywords.