this code fails:
#user_pages, #users = paginate :users, :per_page => 40, :order => :name
rewriting it like this works:
#users = User.all.paginate(:page => params[:page], :per_page => 40)
but what does #user_pages, #users mean?
I take it that #users is being assigned to #user_pages?
joey
No, #user_pages and #users are two different values returned from an array of values. They are, in effect, value[0] and value[1].
An irb example should help:
MacBook-Pro:~ me$ irb
1.9.3-p429 :001 > a,b = [1,2]
=> [1, 2]
1.9.3-p429 :002 > a
=> 1
1.9.3-p429 :003 > b
=> 2
Related
Given df as below:
val df = spark.createDataFrame(Seq(
(1, 2, 3),
(3, 2, 1)
)).toDF("One", "Two", "Three")
with schema:
I would like to write a udf that takes Three columns as inout; and returns new column based on highest input value similar as below:
import org.apache.spark.sql.functions.udf
def udfScoreToCategory=udf((One: Int, Two: Int, Three: Int): Int => {
cols match {
case cols if One > Two && One > Three => 1
case cols if Two > One && Two > Three => 2
case _ => 0
}}
It will be interesting to see how to do similar with vector type as input:
import org.apache.spark.ml.linalg.Vector
def udfVectorToCategory=udf((cols:org.apache.spark.ml.linalg.Vector): Int => {
cols match {
case cols if cols(0) > cols(1) && cols(0) > cols(2) => 1,
case cols if cols(1) > cols(0) && cols(1) > cols(2) => 2
case _ => 0
}})
Some problems:
cols in the first example are not in the scope.
(...): T => ... is not valid syntax for anonymous function.
It is better to use val over def here.
One way to define this:
val udfScoreToCategory = udf[Int, (Int, Int, Int)]{
case (one, two, three) if one > two && one > three => 1
case (one, two, three) if two > one && two > three => 2
case _ => 0
}
and
val udfVectorToCategory = udf[Int, org.apache.spark.ml.linalg.Vector]{
_.toArray match {
case Array(one, two, three) if one > two && one > three => 1
case Array(one, two, three) if two > one && two > three => 2
case _ => 0
}}
In general, for the first case you should use ``when`
import org.apache.spark.sql.functions.when
when ($"one" > $"two" && $"one" > $"three", 1)
.when ($"two" > $"one" && $"two" > $"three", 2)
.otherwise(0)
where one, two, three are column names.
I was able to find the biggest element of the vector by:
val vectorToCluster = udf{ (x: Vector) => x.argmax }
However, I am still puzzled how to do pattern matching on multiple columns values.
This feels like it should be very simple but after several variations I'm unable to return the correct result.
I have a list of strings that I'm trying to filter on and a Set of strings that I'm using as a blacklist. If any of the strings in my list are in the black list then I want to ignore them and only return the ones that are not present. In addition to this I have another case that I need to check for so I need to make sure its not in the black list and also not condition two.
val strings = List("something", "something.else", "something-else" )
val blacklist = Set(".else", ".new", ".old")
I’m trying to do something like:
strings.filter(f => !f.endsWith("-else"))
to leave just "something" and "something.else" but I can’t seem to return "something" by itself.
strings.filter(
f => !f.endsWith("-else") && !blacklist.exists(suffix => f.endsWith(suffix))
)
Or a shorter (but identical) version:
strings.filter(f => !f.endsWith("-else") && !blacklist.exists(f.endsWith))
I understand you want to remove an element if it ends with something that is in your blacklist.
You could try something like
strings.filter {x => (blacklist map {word => !(x endsWith word)} reduce( (a,b) => a && b ) ) }
with another condition, for example x.length >= 2
strings.filter {x => (blacklist map {word => !(x endsWith word)} reduce( (a,b) => a && b ) ) && x.length >= 2}
In the book 'Scala for the Impatient' the author provides the following two examples for 'for-comprehension':
for (c <- "Hello"; i <- 0 to 1) yield (c + i).toChar
// Yields "HIeflmlmop"
for (i <- 0 to 1; c <- "Hello") yield (c + i).toChar
// Yields Vector('H', 'e', 'l', 'l', 'o', 'I', 'f', 'm', 'm', 'p')
However, he didn't mention why the output is a String in the first case, and Vector in the second. Could someone please explain? Thanks.
Your first example is translated into something like:
"Hello".flatMap(c => (0 to 1).map(i => (c + i).toChar))
and the second to
(0 to 1).flatMap(i => "Hello".map(c => (c + i).toChar))
StringOps.flatMap returns a String, so your first example returns a String as well. Range.flatMap returns an IndexedSeq instead.
I saw a very strange piece of code in a perl script used in my project, it's something like:
my $arrayRef = [
A => {AA => 11, AAA => 111},
B => {BB => 11, BBB => 111},
];
IMO, it tries to construct an anonymous array from a hash table. I try to print the array element and here is what I get:
foreach (#$arrayRef )
{
print;
print "\n";
}
A
HASH(0x1e60220)
B
HASH(0x1e71bd0)
which means it treats every element (key&value) in the hash table as a separate element in the anonymous array. However I am really confused about why do they want to save a hash into an array. The ONLY benefit for me is to save some memory if the hash table is really huge. Is this a wedely used perl tricks?
Thanks!
it tries to construct an anonymous array from a hash table.
No, it constructs an anonymous array from a four-element list.
A => {AA => 11, AAA => 111}, B => {BB => 11, BBB => 111}
is exactly the same thing as
'A', {AA => 11, AAA => 111}, 'B', {BB => 11, BBB => 111}
The use of => does imply some sort of relationship, so I suppose they could have used
{ A => {AA => 11, AAA => 111}, B => {BB => 11, BBB => 111} }
or
[ [ A => {AA => 11, AAA => 111} ], [ B => {BB => 11, BBB => 111} ] ]
or any of a million other data structures, but there's way to know why one was chosen over another from what you gave.
It's an anonymous array, in which alternates a string key with an anonymous hash. The answer to your question: it depends on context, a concrete data-structure should help to resolve a concrete problem. So if we have only the data-structure and forget the problem we are trying to resolve, it's harder to imagine why they used this construction.
Perhaps, they needed a "ordered hash of hashes", the array structure makes sure the order, hash not
I have the following lists :
case class myPair(ids:Int,vals:Int)
val someTable = List((20,30), (89,90), (40,65), (45,75), (35,45))
val someList:List[myPair] =
someTable.map(elem => myPair(elem._1, elem._2)).toList
I would like to filter all "ids" > 45 .
I tried something like this article filter using pattern matching):
someList.filter{ case(myPair) => ids >= 45 }
but without success.
appreciate your help
You don't need pattern matching at all, type is known at compile time:
someList.filter(_.ids >= 45)
or slightly more verbose/readable:
someList.filter(pair => pair.ids >= 45)
You mean like:
someList.filter{ case MyPair(ids,vals) => ids >= 45 }
Renamed myPair to MyPair, identifiers beginning with lowercase are considered variables, much like ids and vals in the above. --Actually this is not true, look at #RandallSchulz's comment.
Going further(1):
val someList = someTable.map(case (ids,vals) => MyPair(ids,vals)).toList
Even more(2):
val someList = someTable.map(elem => MyPair.tupled(elem)).toList
Way more(3):
val someList = someTable.map(MyPair.tupled).toList
Of course, only (1) is about pattern match. (2) and (3) is turning the arguments of MyPair.apply(Int,Int) into Tuple[Int,Int].
Here's one more variant using pattern matching
someTable collect {case (i, v) if i > 45 => MyPair(i, v)}
collect combines a filter operation and a map operation.
case class myPair(ids:Int,vals:Int)
val someTable = List((20,30), (89,90), (40,65), (45,75), (35,45))
val someList:List[myPair] = for( elem <- someTable; if elem._1 > 45) yield myPair(elem._1, elem._2)
Which gives
someList: List[myPair] = List(myPair(89,90))