Conditionally map a nullable field to None value in Slick - scala

In Slick 2.1, I need to perform a query/map operation where I convert a nullable field to None if it contains a certain non-null value. Not sure whether it matters or not, but in my case the column type in question is a mapped column type. Here is a code snippet which tries to illustrate what I'm trying to do. It won't compile, as the compiler doesn't like the None.
case class Record(field1: Int, field2: Int, field3: MyEnum)
sealed trait MyEnum
val MyValue: MyEnum = new MyEnum { }
// table is a TableQuery[Record]
table.map { r => (
r.field1,
r.field2,
Case If (r.field3 === MyValue) Then MyValue Else None // compile error on 'None'
)
}
The error is something like this:
type mismatch; found : None.type required: scala.slick.lifted.Column[MyEnum]
Actually, the reason I want to do this is that I want to perform a groupBy in which I count the number of records whose field3 contains a given value. I couldn't get the more complicated groupBy expression working, so I backed off to this simpler example which I still can't get working. If there's a more direct way to show me the groupBy expression, that would be fine too. Thanks!
Update
I tried the code suggested by #cvogt but this produces a compile error. Here is a SSCCE in case anyone can spot what I'm doing wrong here. Compile fails with "value ? is not a member of Int":
import scala.slick.jdbc.JdbcBackend.Database
import scala.slick.driver.H2Driver
object ExpMain extends App {
val dbName = "mydb"
val db = Database.forURL(s"jdbc:h2:mem:${dbName};DB_CLOSE_DELAY=-1", driver = "org.h2.Driver")
val driver = H2Driver
import driver.simple._
class Exp(tag: Tag) extends Table[(Int, Option[Int])](tag, "EXP") {
def id = column[Int]("ID", O.PrimaryKey)
def value = column[Option[Int]]("VALUE")
def * = (id, value)
}
val exp = TableQuery[Exp]
db withSession { implicit session =>
exp.ddl.create
exp += (1, (Some(1)))
exp += (2, None)
exp += (3, (Some(4)))
exp.map { record =>
Case If (record.value === 1) Then 1.? Else None // this will NOT compile
//Case If (record.value === 1) Then Some(1) Else None // this will NOT compile
//Case If (record.value === 1) Then 1 Else 0 // this will compile
}.foreach {
println
}
}
}

I need to perform a query/map operation where I convert a nullable field to None if it contains a certain non-null value
Given the example data you have in the update, and pretending that 1 is the "certain" value you care about, I believe this is the output you expect:
None, None, Some(4)
(for rows with IDs 1, 2 and 3)
If I've understood the problem correctly, is this what you need...?
val q: Query[Column[Option[Int]], Option[Int], Seq] = exp.map { record =>
Case If (record.value === 1) Then (None: Option[Int]) Else (record.value)
}
...which equates to:
select (case when ("VALUE" = 1) then null else "VALUE" end) from "EXP"

You need to wrap MyValue in an Option, so that both outcomes of the conditional are options. In Slick 2.1 you use the .? operator for that. In Slick 3.0 it will likely be Rep.Some(...).
Try
Case If (r.field3 === MyValue) Then MyValue.? Else None
or
Case If (r.field3 === MyValue) Then MyValue.? Else (None: Option[MyEnum])

Related

Syntactic sugar explanation of Scala'a unapply method

I am getting an error in the extractor step (unapply method call).
The error message is: Wrong number of arguments for the extractors. found 2; expected 0
Can someone please help what is causing the error (where my misunderstanding is).
class ABC(val name:String, val age:Int) //class is defined.
object ABC{
def apply(age:Int, name:String) = new ABC(name, age)
def unapply(x:ABC) = (x.name, x.age)
}
val ins = ABC(25, "Joe") //here apply method is in action.
val ABC(x,y) = ins //unapply is indirectly called. As per my understanding , 25 and Joe suppose to be captured in x and y respectively. But this steps gives error.
The error I get is
an unapply result must have a member def isEmpty: Boolean
The easiest way to fix this is to make unapply return an Option:
def unapply(x: ABC) = Option((x.name, x.age))
The unapply method in an extractor which binds values must return an Option. This is because there's no intrinsic guarantee that an extractor will always succeed. For instance consider this massively oversimplified example of an extractor for an email address:
object Email {
def unapply(s: String): Option[(String, String)] =
s.indexOf('#') match {
case idx if idx >= 0 =>
val (user, maybeSite) = s.splitAt(idx)
if (maybeSite.length < 2 || maybeSite.lastIndexOf('#') > 0) None
else Some(user -> maybeSite.tail)
case _ => None
}
}
At the application site:
val Email(u, s) = "user3103957#stackoverflow.example.xyz"
Turns into code that's basically (from the description in Programming In Scala (Odersky, Spoon, Venners (3rd ed))):
val _tmpTuple2 =
"user3103957#stackoverflow.example.xyz" match {
case str: String =>
Email.unapply(str).getOrElse(throw ???)
case _ => throw ???
}
val u = _tmpTuple2._1
val s = _tmpTuple2._2
Technically, since the compiler already knows that the value is a String, the type check is elided, but I've included the type check for generality. The desugaring of extractors in a pattern match also need not throw except for the last extractor attempt.

Scala test match case classes on some field

I have
case class Foo(field1: String, field2: String, field3: String)
expected: Seq[Foo] = Seq(...)
result: Seq[Foo] = Seq(...)
I want to write a Scala matcher that compare the elements of Seq without taking in consideration field1.
So two elements will match if field2 and field3 are equals. Field1 value is not considerated.
Example:
val expected = Seq(
Foo(
"f1", "f2", "f3"
)
)
val result = Seq(
Foo(
"fx", "f2", "f3"
)
)
This two sequence has to match.
result should matchWithoutId(expected)
The documentation is in Using custom Matchers, this example would be (for example)
def matchWithoutId(expected: Foo): Matcher[Foo] = Matcher { actual =>
MatchResult(
actual.field2 == expected.field2 && actual.field3 == expected.field3,
if (actual.field2 != expected.field2)
s"field2 of $actual was not equal to that of $expected"
else
s"field3 of $actual was not equal to that of $expected",
s"field2 and field3 of $actual were equal to those of $expected")
}
Adjust error messages to taste.
Or another approach (probably a better one in this case):
def matchWithoutId(expected: Foo): Matcher[Foo] = have(
'field2 (expected.field2),
'field3 (expected.field3)
)
Normally I'd say using Symbols to name properties and use reflection should be avoided, but here a change to Foo will make the tests fail to compile anyway because of access to expected's fields.
EDIT: I missed that you want to compare Seq[Foo]s and not Foos, so Mario Galic's answer is probably the one you want. Still, this can hopefully be useful as well.
Working with "sequences" states
if you want to change how containership is determined for an element
type E, place an implicit Equality[E] in scope or use the explicitly
DSL.
so the following should work
(expected should contain theSameElementsAs (result)) (decided by fooEqualityWithoutId)
Note about contains usage
Note that when you use the explicitly DSL with contain you need to
wrap the entire contain expression in parentheses
Here is the full example
import org.scalactic._
import org.scalatest._
import org.scalatest.matchers.should.Matchers
import org.scalactic.Explicitly._
class CustomizeEqualitySeqSpec extends FlatSpec with Matchers with Explicitly {
case class Foo(field1: String, field2: String, field3: String)
val expected = Seq(Foo("f1", "f2", "f3"))
val result = Seq(Foo("fx", "f2", "f3"))
val fooEqualityWithoutId = new Equality[Foo] {
override def areEqual(a: Foo, b: Any): Boolean = b match {
case Foo(_, field2, field3) => a.field2 == field2 && a.field3 == field3;
case _ => false
}
}
"Sequences" should "use custom Equality[E]" in {
(expected should contain theSameElementsAs (result)) (decided by fooEqualityWithoutId)
}
}

Type Mismatch in scala case match

Trying to create multiple dataframes in a single foreach, using spark, as below
I get values delivery and click out of row.getAs("type"), when I try to print them.
val check = eachrec.foreach(recrd => recrd.map(row => {
row.getAs("type") match {
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
case "click" => val click_data = delivery(row.get(0).toString,row.get(1).toString)
case _ => "not sure if this impacts"
}})
)
but getting below error:
Error:(41, 14) type mismatch; found : String("delivery") required: Nothing
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
^
My plan is to create dataframe using todf() once I create these individual delivery objects referenced by delivery_data and click_data by:
delivery_data.toDF() and click_data.toDF().
Please provide any clue regarding the error above (in match case).
How can I create two df's using todf() in val check?
val declarations make your first 2 cases return type to be unit, but in the third case you return a String
for instance, here the z type was inferred by the compiler, Unit:
def x = {
val z: Unit = 3 match {
case 2 => val a = 2
case _ => val b = 3
}
}
I think you need to cast this match clause to String.
row.getAs("type").toString

Is there any way to use immutable collections here + make the code look better?

I have to validate some variables manually for some reason and return a map with the sequance of the error messages for each variable. I've decided to use mutable collections for this because I think there is no other choise left:
val errors = collection.mutable.Map[String, ListBuffer[String]]()
//field1
val fieldToValidate1 = getData1()
if (fieldToValidate1 = "")
errors("fieldToValidate1") += "it must not be empty!"
if (validate2(fieldToValidate1))
errors("fieldToValidate1") += "validation2!"
if (validate3(fieldToValidate1))
errors("fieldToValidate1") += "validation3!"
//field2
val fieldToValidate2 = getData1()
//approximately the same steps
if (fieldToValidate2 = "")
errors("fieldToValidate2") += "it must not be empty!"
//.....
To my mind, it look kind of clumsy and there should other elegant solution. I'd also like to not use mutable collections if possible. Your ideas?
Instead of using mutable collections, you can define errors with var and update it in this way.
var errors = Map[String, List[String]]().withDefaultValue(Nil)
errors = errors updated ("fieldToValidate1", errors("fieldToValidate1") ++ List("it must not be empty!"))
errors = errors updated ("fieldToValidate1", errors("fieldToValidate1") ++ List("validation2"))
The code looks more tedious, but it gets out of mutable collections.
So what is a good type for your check? I was thinking about A => Option[String] if A is the type of your object under test. If your error messages do not depend on the value of the object under test, (A => Boolean, String) might be more convenient.
//for constructing checks from boolean test and an error message
def checkMsg[A](check: A => Boolean, msg: => String): A => Option[String] =
x => if(check(x)) Some(msg) else None
val checks = Seq[String => Option[String]](
checkMsg((_ == ""), "it must not be empty"),
//example of using the object under test in the error message
x => Some(x).filterNot(_ startsWith "ab").map(x => x + " does not begin with ab")
)
val objectUnderTest = "acvw"
val errors = checks.flatMap(c => c(objectUnderTest))
Error labels
As I just noted, you were asking for a map with a label for each check. In this case, you need to provide the check label, of course. Then the type of your check would be (String, A => Option[String]).
Although a [relatively] widespread way of doing-the-thing-right would be using scalaz's Validation (as #senia has shown), I think it is a little bit overwhelming approach (if you're bringing scalaz to your project you have to be a seasoned scala developer, otherwise it may bring you more harm than good).
Nice alternative could be using ScalaUtils which has Or and Every specifically made for this purpose, in fact if you're using ScalaTest you already have seen an example of them in use (it uses scalautils underneath). I shamefully copy-pasted example from their doc:
import org.scalautils._
def parseName(input: String): String Or One[ErrorMessage] = {
val trimmed = input.trim
if (!trimmed.isEmpty) Good(trimmed) else Bad(One(s""""${input}" is not a valid name"""))
}
def parseAge(input: String): Int Or One[ErrorMessage] = {
try {
val age = input.trim.toInt
if (age >= 0) Good(age) else Bad(One(s""""${age}" is not a valid age"""))
}
catch {
case _: NumberFormatException => Bad(One(s""""${input}" is not a valid integer"""))
}
}
import Accumulation._
def parsePerson(inputName: String, inputAge: String): Person Or Every[ErrorMessage] = {
val name = parseName(inputName)
val age = parseAge(inputAge)
withGood(name, age) { Person(_, _) }
}
parsePerson("Bridget Jones", "29")
// Result: Good(Person(Bridget Jones,29))
parsePerson("Bridget Jones", "")
// Result: Bad(One("" is not a valid integer))
parsePerson("Bridget Jones", "-29")
// Result: Bad(One("-29" is not a valid age))
parsePerson("", "")
// Result: Bad(Many("" is not a valid name, "" is not a valid integer))
Having said this, I don't think you can do any better than your current approach if you want to stick with core scala without any external dependencies.
In case you can use scalaz the best solution for aggregation errors is Validation:
def validate1(value: String) =
if (value == "") "it must not be empty!".failNel else value.success
def validate2(value: String) =
if (value.length > 10) "it must not be longer than 10!".failNel else value.success
def validate3(value: String) =
if (value == "error") "it must not be equal to 'error'!".failNel else value.success
def validateField(name: String, value: String): ValidationNel[(String, String), String] =
(
validate1(value) |#|
validate2(value) |#|
validate3(value)
).tupled >| value leftMap { _.map{ name -> _ } }
val result = (
validateField("fieldToValidate1", getData1()) |#|
validateField("fieldToValidate2", getData2())
).tupled
Then you could get optional errors Map like this:
val errors =
result.swap.toOption.map{
_.toList.groupBy(_._1).map{ case (k, v) => k -> v.map(_._2) }
}
// Some(Map(fieldToValidate2 -> List(it must not be equal to 'error'!), fieldToValidate1 -> List(it must not be empty!)))

Warning about an unchecked type argument in this Scala pattern match?

This file:
object Test extends App {
val obj = List(1,2,3) : Object
val res = obj match {
case Seq(1,2,3) => "first"
case _ => "other"
}
println(res)
}
Gives this warning:
Test.scala:6: warning: non variable type-argument A in type pattern Seq[A]
is unchecked since it is eliminated by erasure
case Seq(1,2,3) => "first"
Scala version 2.9.0.1.
I don't see how an erased type parameter is needed to perform the match. That first case clause is meant to ask if obj is a Seq with 3 elements equal to 1, 2, and 3.
I would understand this warning if I had written something like:
case strings : Seq[String] => ...
Why do I get the warning, and what is a good way to make it go away?
By the way, I do want to match against something with static type of Object. In the real code I'm parsing something like a Lisp datum - it might be an String, sequence of datums, Symbol, Number, etc.
Here is some insight to what happens behind the scene. Consider this code:
class Test {
new Object match { case x: Seq[Int] => true }
new Object match { case Seq(1) => true }
}
If you compile with scalac -Xprint:12 -unchecked, you'll see the code just before the erasure phase (id 13). For the first type pattern, you will see something like:
<synthetic> val temp1: java.lang.Object = new java.lang.Object();
if (temp1.isInstanceOf[Seq[Int]]())
For the Seq extractor pattern, you will see something like:
<synthetic> val temp3: java.lang.Object = new java.lang.Object();
if (temp3.isInstanceOf[Seq[A]]()) {
<synthetic> val temp4: Seq[A] = temp3.asInstanceOf[Seq[A]]();
<synthetic> val temp5: Some[Seq[A]] = collection.this.Seq.unapplySeq[A](temp4);
// ...
}
In both cases, there is a type test to see if the object is of type Seq (Seq[Int] and Seq[A]). Type parameters will be eliminated during the erasure phase. Thus the warning. Even though the second may be unexpected, it does make sense to check the type since if object is not of type Seq that clause won't match and the JVM can proceed to the next clause. If the type does match, then the object can be casted to Seq and unapplySeq can be called.
RE: thoredge comment on the type check. May be we are talking about different things. I was merely saying that:
(o: Object) match {
case Seq(i) => println("seq " + i)
case Array(i) => println("array " + i)
}
translates to something like:
if (o.isInstanceOf[Seq[_]]) { // type check
val temp1 = o.asInstanceOf[Seq[_]] // cast
// verify that temp1 is of length 1 and println("seq " + temp1(0))
} else if (o.isInstanceOf[Array[_]]) { // type check
val temp1 = o.asInstanceOf[Array[_]] // cast
// verify that temp1 is of length 1 and println("array " + temp1(0))
}
The type check is used so that when the cast is done there is no class cast exception.
Whether the warning non variable type-argument A in type pattern Seq[A] is unchecked since it is eliminated by erasure is justified and whether there would be cases where there could be class cast exception even with the type check, I don't know.
Edit: here is an example:
object SeqSumIs10 {
def unapply(seq: Seq[Int]) = if (seq.sum == 10) Some(seq) else None
}
(Seq("a"): Object) match {
case SeqSumIs10(seq) => println("seq.sum is 10 " + seq)
}
// ClassCastException: java.lang.String cannot be cast to java.lang.Integer
Declaring the match object outside at least makes it go away, but I'm not sure why:
class App
object Test extends App {
val obj = List(1,2,3) : Object
val MatchMe = Seq(1,2,3)
val res = obj match {
case MatchMe => "first"
case _ => "other"
}
println(res)
}