Chained indexing in ND4S and Breeze produces a Scala compiler error:
var m = Nd4j.create(2, 3)
var a = m(0, ->)
var b = a(0)
var c = m(0, ->)(0)
The first two work but the third produces:
not enough arguments for method apply: (implicit ev: org.nd4s.NDArrayEvidence[org.nd4j.linalg.api.ndarray.INDArray,B], implicit ev2: scala.reflect.Manifest[B])org.nd4j.linalg.api.ndarray.INDArray in class RichINDArray.
I tried with (0, 0) instead of (0) since it is still 2d, but that didn't matter. And many variations of extra parenthesis.
Is this the Scala parser thinking it is some other construct or a bug in ND4S? Is there some workaround syntax that can make chaining work?
EDIT:
Under Breeze:
var m = DenseMatrix.zeros[Int](5,5)
var a = m(0, ::)
var b = a(0)
var c = m(0, ::)(0)
a and b work but c produces this compiler error:
type mismatch; found : Int(0) required: breeze.linalg.support.CanSlice2[breeze.linalg.DenseMatrix[Int],Int,collection.immutable.::.type,?]
Maybe this is parsing as curried function calls, or a macro expands in a way that doesn't work here. And while I guess a person never needs to chain indexing, since m(0, 0) works above and any indexing can be collapsed, is there some solution for the general case of tricking Scala in these situations?
Also, chaining works with multidimensional arrays:
var n = Array.ofDim[Double](2, 3)
var x = n(0)(0)
Both methods m(0, ->) and m(0, ::) take a second implicit parameter list. Thus chaining the calls is interpreted by Scala as trying to provide this implicit parameter list explicitly. That's why it reports the argument number mismatch and the argument type mismatch.
Try using (m(0, ->))(0) or m(0, ->).apply(0) (or the Breeze analogue). This forces Scala to provide the implicit parameter list implicitly, and your second call then becomes a normal function call as you had intended.
Related
In the below two scenarios, I have flatMap function invoked on a list. In both the cases, the map portion of the flatMap function returns an array which has a iterator. In the first case, the code errors out where in the second case, it produces the expected result.
Scenario-1
val x = List("abc","cde")
x flatMap ( e => e.toArray)
<console>:13: error: polymorphic expression cannot be instantiated to expected type;
found : [B >: Char]Array[B]
required: scala.collection.GenTraversableOnce[?]
x flatMap ( e => e.toArray)
Scenario-2
val x = List("abc,def")
x flatMap ( e => e.split(",") )
res1: List[String] = List(abc, def) //Result
Can you please help why in the first case, it is not behaving as expected ?
I think the difference is that in scenario 1 you actually have a Array[B] where B is some yet-to-be-decided supertype of Char. Normally the compiler would look for an implicit conversion to GenTraversableOnce but because B is not yet known you run into a type inference issue / limitation.
You can help type inference by filling in B.
List("abc", "cde").flatMap(_.toArray[Char])
Or even better, you don't need flatMap in this case. Just call flatten.
List("abc", "cde").flatten
It is worth keeping in mind that Array is not a proper Scala collection so compiler has to work harder to first convert it to something that fits with the rest of Scala collections
implicitly[Array[Char] <:< GenTraversableOnce[Char]] // error (Scala 2.12)
implicitly[Array[Char] <:< IterableOnce[Char]] // error (Scala 2.13)
Hence because flatMap takes a function
String => GenTraversableOnce[Char]
but we are passing in
String => Array[Char]
the compiler first has to find an appropriate implicit conversion of Array[Char] to GenTraversableOnce[Char]. Namely this should be wrapCharArray
scala.collection.immutable.List.apply[String]("abc", "cde").flatMap[Char](((e: String) => scala.Predef.wrapCharArray(scala.Predef.augmentString(e).toArray[Char]((ClassTag.Char: scala.reflect.ClassTag[Char])))))
or shorter
List("abc", "cde").flatMap(e => wrapCharArray(augmentString(e).toArray))
however due to type inference reasons explained by Jasper-M, the compiler is unable to select wrapCharArray conversion. As Daniel puts it
Array tries to be a Java Array and a Scala Collection at the same
time. It mostly succeeds, but fail at both for some corner cases.
Perhaps this is one of those corner cases. For these reasons it is best to avoid Array unless performance or compatibility reasons dictate it. Nevertheless another approach that seems to work in Scala 2.13 is to use to(Collection) method
List("abc","cde").flatMap(_.to(Array))
scala> val x = List("abc","cde")
x: List[String] = List(abc, cde)
scala> x.flatMap[Char](_.toArray)
res0: List[Char] = List(a, b, c, c, d, e)
As the error mentions, your type is wrong.
In the first case, if you appy map not flatmap, you obtain List[Array[Char]]. If you apply flatten to that, you get a List[Char]
In the second case, if you appy map not flatmap, you obtain List[Array[String]]. If you apply flatten to that, you get a List[String]
I suppose you need to transform String to Char in your Array, in order to make it work.
I use Scala 2.13 and still have the same error.
The scala documentation has a code example that includes the following line:
val numberFunc = numbers.foldLeft(List[Int]())_
What does the underscore after the method call mean?
It's a partially applied function. You only provide the first parameter to foldLeft (the initial value), but you don't provide the second one; you postpone it for later. In the docs you linked they do it in the next line, where they define squares:
val numberFunc = numbers.foldLeft(List[Int]())_
val squares = numberFunc((xs, x) => xs:+ x*x)
See that (xs, x) => xs:+ x*x, that's the missing second parameter which you omitted while defining numberFunc. If you had provided it right away, then numberFunc would not be a function - it would be the computed value.
So basically the whole thing can also be written as a one-liner in the curried form:
val squares = numbers.foldLeft(List[Int]())((xs, x) => xs:+ x*x)
However, if you want to be able to reuse foldLeft over and over again, having the same collection and initial value, but providing a different function every time, then it's very convinient to define a separate numbersFunc (as they did in the docs) and reuse it with different functions, e.g.:
val squares = numberFunc((xs, x) => xs:+ x*x)
val cubes = numberFunc((xs, x) => xs:+ x*x*x)
...
Note that the compiler error message is pretty straightforward in case you forget the underscore:
Error: missing argument list for method foldLeft in trait
LinearSeqOptimized Unapplied methods are only converted to functions
when a function type is expected. You can make this conversion
explicit by writing foldLeft _ or foldLeft(_)(_) instead of
foldLeft. val numberFunc = numbers.foldLeft(ListInt)
EDIT: Haha I just realized that they did the exact same thing with cubes in the documentation.
I don't know if it helps but I prefer this syntax
val numberFunc = numbers.foldLeft(List[Int]())(_)
then numberFunc is basically a delegate corresponding to an instance method (instance being numbers) waiting for a parameter. Which later comes to be a lambda expression in the scala documentation example
As far as I understand .
_ is a short lambda to omit a=>
i find this code (can find here scala-function-true-power)
val file = List("warn 2013 msg", "warn 2012 msg", "error 2013 msg", "warn 2013 msg")
val size = file.filter(_.contains("warn")).filter(_.contains("2013")).size
//val size1 = file.filter(_.contains("warn") && _.contains("2013")).size
val size2 = file.filter( a=> a.contains("warn") && a.contains("2013")).size
println("cat file | grep 'warn' | grep '2013' | wc : " +size )
the line to get size1 has syntax error,looks like it can't recognize the "_" ,it's not a element in fileList.
but i use a=>,the normal kind,it works good .
so,why the scala work by this way?
is there more difference in _ and a=> ?
In scala, any _ placeholder is matched against the passed arguments in the context of calling function. So for example if the signature of the function you are trying to use is f : A ⇒ B and you are calling something like collectionOfFunctA.map(_.f) - Scala compiler will infer the correct type of the function and will use the first underscore to put the actual item from a collection and call the function f over it. But if you will try to write it as collectionOfFunctA.map(_.f + _.size) - that will fail, because Scala compiler will pick up the first placeholder as of type that has function f defined, and the second underscore will not match any function in the context. So it will expect to have a function that takes two parameters instead of one.
More on this
As jdevelop says, but here in the words of the compiler/REPL:
scala> val size1 = file.filter(_.contains("warn") && _.contains("2013")).size
<console>:8: error: missing parameter type for expanded function ((x$1, x$2) => x$1.contains("warn").$amp$amp(x$2.contains("2013")))
val size1 = file.filter(_.contains("warn") && _.contains("2013")).size
^
<console>:8: error: missing parameter type for expanded function ((x$1: <error>, x$2) => x$1.contains("warn").$amp$amp(x$2.contains("2013")))
val size1 = file.filter(_.contains("warn") && _.contains("2013")).size
^
You see that hint: for expanded function ((x$1, x$2) => x$1.contains("warn").$amp$amp(x$2.contains("2013")))
It is expecting 2 parameters while there is just one.
You can think of the place holder as being matched with the lambda's arguments positionnally.
The first occurrence of the _ is matched with the first argument, the second occurence is matched with the second argument, etc.
As the other answers have shown, this means that using the placeholder twice will be desugared as trying to pass a lamba with 2 arguments to the filter which only expects one.
In your example :
val size = file.filter(_.contains("warn") && _.contains("2013")).size
would be desugared as
val size = file.filter((a,b)=>a.contains("warn") && b.contains("2013")).size
which will not compile since filter expects a predicate p: A => Boolean
Now, a reason the placeholder is matched positionnally is to avoid ambiguity in lambdas with more than one argument.
How can the compiler guess the correct implementation for the following case if the place holder can be reused multiple times for the same argument:
file.fold("")(_++_)
Should it be desugared as :
file.fold("")((a,b)=> a++b )
or as
file.fold("")((a,b)=> a++a )
or as
file.fold("")((a,b)=> b++b )
and worse, what would you expect for
file.fold("")(_++_++_)
There is no general way for the compiler to infer the correct implementation.
One might argue for relaxing the constraint when the expected lambda only accepts one argument. I suggest doing a more detailed research before taking the first steps to the scala improvement process as it seems likely that this particular design decision has been challenged and explained before.
If you are worried about the performance of iterating over the list twice (which is the case when you write)
file.filter(_.contains("warn")).filter(_.contains("2013")).size
In theory it should be possible for the compiler to detect that both filters can be applied within the same iteration.
In scala, the collections are eager by default but you can get the lazy evaluation by using views.
The current implementation has known issues which are being worked on. Other collection implementations in scala are actively being developed to be able to combine transformations and computations by default (see psp-std for example)
I'm looking at someone else's source code (Scala), where I see the operator :+= being called on a variable of type IndexedSeq. I am looking all over the scaladocs page for that class to figure out what that operator does, but I do not see it. I'm thinking that either it's defined in a class outside of IndexedSeq's inheritance hierarchy, or else the javascript on the scaladocs page is hiding it somewhere I can't see it. (Actually it's neither; see answer below.)
I've hit every button on the scaladocs page trying to unhide everything. I've looked in the web-page's HTML code. There has got to be a way to look up an operator from the documentation of a class to which it can be applied. Hasn't there?
(N.B.: I looked up that operator using symbolhound, so I know what that operator means now. This question is about scala documentation in general, not that particular operator.)
All operators in Scala are normal methods.
You cannot find it because it is compiler magic for re-assignement, it is not an operator. Or to say it another way: it looks like an operator of its own, but it is actually "an operator followed by the = character".
The compiler will magically turn that into a assignment if the operator (here :+) returns the proper type, and the original value was a var, obviously.
Since it is not provided by any implicit nor explicit method on Seq[T] or whatever, it does not appear anywhere in the generated scaladoc.
So to answer the general question:
It is a language construct, so the only place where it is documented is the specification, sadly,
but, if you find some "<?>=" unknown operator somewhere, look for the definition of "<?>", that one is sure to be documented.
Edit: I finally found where this is defined in the SLS:
§6.12.4:
An assignment operator is an operator symbol (syntax category op in (§1.1)) that
ends in an equals character “=”, with the exception of operators for which one of the
following conditions holds:
(1) the operator also starts with an equals character, or
(2) the operator is one of (<=), (>=), (!=).
It also says later on that it only happens when all other options have been tried (including potential implicits).
Is this value assigned to a variable? If it's the case I think this syntax sugar:
scala> var x = IndexedSeq(1,2,3)
x: IndexedSeq[Int] = Vector(1, 2, 3)
scala> x :+= 10
scala> x
res59: IndexedSeq[Int] = Vector(1, 2, 3, 10)
scala> val y = IndexedSeq(1,2,3)
y: IndexedSeq[Int] = Vector(1, 2, 3)
scala> y :+= 10
<console>:16: error: value :+= is not a member of IndexedSeq[Int]
y :+= 10
^
It is syntax sugar for "operation and assignment", like +=:
scala> var x = 10
x: Int = 10
scala> x += 1
scala> x
res63: Int = 11
Which de-sugars to x = x + 1.
I am attempting to add an update method to the Symbol class.
class SymbolUpdate(s: Symbol) {
def update(i: Int) = s.name + i
}
implicit def toSymbolUpdate(s: Symbol) = new SymbolUpdate(s)
But when I run the code I get the following
scala> 's = 1
<console>:327: error: value update is not a member of object Symbol
's = 1
^
But it does work when I call the method directly.
scala> 's.update(1)
res41: java.lang.String = s1
Or if I explicitly put an empty argument array.
scala> 's() = 1
res42: java.lang.String = s1
Not sure what the problem is with my code?
According to the Scala Language Spec:
An assignment f(args) = e with a function application to the left of the ‘=’ operator is interpreted as f.update(args, e), i.e. the invocation of an update function
defined by f.
It's especially clear if you read the corresponding section in Programming in Scala:
Similarly, when an assignment is made to a variable to which parenthesis and one or more arguments have been applied, the compiler will transform that into an invocation of an update methods that takes the arguments in parenthesis as well as the object to the right of the equals sign.
Together, I take it to mean that the parenthesis are required.