I'm writing a DSL that generates SQL. The syntax for loading a table is:
session.activateWhere( _.User.ID == 490 )
This will select from the User table where the ID column is 490. I can use "==" because I can override "equals()" to generate the correct SQL. My problem is that "!=" doesn't work because it calls equals() and then negates the result. Sadly "!=" is final so I can't override it. Is there any way in my equals() method to tell that it was called as part of !=?
I've implemented a "<>" operation that logically does the same thing as "!=" and it works fine:
session.activateWhere( _.User.ID <> 490 )
My issue is that not only is "!=" valid syntax (to the compiler) but it will run and generate the exact opposite of what the user intends.
As you say, != is (similarly to ==) final and so cannot be overridden - with a very good reason. That is why most DLSs use === as an alternative.
The == and != operators have a very well defines meaning for all objects in Scala. Changing the meaning for some objects would be, in my opinion, very dangerous.
If you want to use == and !=, you have to override equals, hashcode and canEqual (trait Equals)
It is not a simple as it looks, you have to be carreful.
There is some examples over the net.
For example : https://www.safaribooksonline.com/library/view/scala-cookbook/9781449340292/ch04s16.html
Another one : https://groups.google.com/forum/#!msg/scala-user/Qosfawmaecw/RUWigwvVtQ4J
Related
I'm currently working on my functional programming - I am fairly new to it. Am i using Options correctly here? I feel pretty insecure on my skills currently. I want my code to be as safe as possible - Can any one point out what am I doing wrong here or is it not that bad? My code is pretty straight forward here:
def main(args: Array[String]): Unit =
{
val file = "myFile.txt"
val myGame = Game(file) //I have my game that returns an Option here
if(myGame.isDefined) //Check if I indeed past a .txt file
{
val solutions = myGame.get.getAllSolutions() //This returns options as well
if(solutions.isDefined) //Is it possible to solve the puzzle(crossword)
{
for(i <- solutions.get){ //print all solutions to the crossword
i.solvedCrossword foreach println
}
}
}
}
-Thanks!! ^^
When using Option, it is recommended to use match case instead of calling 'isDefined' and 'get'
Instead of the java style for loop, use higher-order function:
myGame match {
case Some(allSolutions) =>
val solutions = allSolutions.getAllSolutions
solutions.foreach(_.solvedCrossword.foreach(println))
case None =>
}
As a rule of thumb, you can think of Option as a replacement for Java's null pointer. That is, in cases where you might want to use null in Java, it often makes sense to use Option in Scala.
Your Game() function uses None to represent errors. So you're not really using it as a replacement for null (at least I'd consider it poor practice for an equivalent Java method to return null there instead of throwing an exception), but as a replacement for exceptions. That's not a good use of Option because it loses error information: you can no longer differentiate between the file not existing, the file being in the wrong format or other types of errors.
Instead you should use Either. Either consists of the cases Left and Right where Right is like Option's Some, but Left differs from None in that it also takes an argument. Here that argument can be used to store information about the error. So you can create a case class containing the possible types of errors and use that as an argument to Left. Or, if you never need to handle the errors differently, but just present them to the user, you can use a string with the error message as the argument to Left instead of case classes.
In getAllSolutions you're just using None as a replacement for the empty list. That's unnecessary because the empty list needs no replacement. It's perfectly fine to just return an empty list when there are no solutions.
When it comes to interacting with the Options, you're using isDefined + get, which is a bit of an anti pattern. get can be used as a shortcut if you know that the option you have is never None, but should generally be avoided. isDefined should generally only be used in situations where you need to know whether an option contains a value, but don't need to know the value.
In cases where you need to know both whether there is a value and what that value is, you should either use pattern matching or one of Option's higher-order functions, such as map, flatMap, getOrElse (which is kind of a higher-order function if you squint a bit and consider by-name arguments as kind-of like functions). For cases where you want to do something with the value if there is one and do nothing otherwise, you can use foreach (or equivalently a for loop), but note that you really shouldn't do nothing in the error case here. You should tell the user about the error instead.
If all you need here is to print it in case all is good, you can use for-comprehension which is considered quite idiomatic Scala way
for {
myGame <- Game("mFile.txt")
solutions <- myGame.getAllSolutions()
solution <- solutions
crossword <- solution.solvedCrossword
} println(crossword)
Consider a repository/DAO method like this, which works great:
def countReports(customerId: Long, createdSince: ZonedDateTime) =
DB.withConnection {
implicit c =>
SQL"""SELECT COUNT(*)
FROM report
WHERE customer_id = $customerId
AND created >= $createdSince
""".as(scalar[Int].single)
}
But what if the method is defined with optional parameters:
def countReports(customerId: Option[Long], createdSince: Option[ZonedDateTime])
Point being, if either optional argument is present, use it in filtering the results (as shown above), and otherwise (in case it is None) simply leave out the corresponding WHERE condition.
What's the simplest way to write this method with optional WHERE conditions? As Anorm newbie I was struggling to find an example of this, but I suppose there must be some sensible way to do it (that is, without duplicating the SQL for each combination of present/missing arguments).
Note that the java.time.ZonedDateTime instance maps perfectly and automatically into Postgres timestamptz when used inside the Anorm SQL call. (Trying to extract the WHERE condition as a string, outside SQL, created with normal string interpolation did not work; toString produces a representation not understood by the database.)
Play 2.4.4
One approach is to set up filter clauses such as
val customerClause =
if (customerId.isEmpty) ""
else " and customer_id={customerId}"
then substitute these into you SQL:
SQL(s"""
select count(*)
from report
where true
$customerClause
$createdClause
""")
.on('customerId -> customerId,
'createdSince -> createdSince)
.as(scalar[Int].singleOpt).getOrElse(0)
Using {variable} as opposed to $variable is I think preferable as it reduces the risk of SQL injection attacks where someone potentially calls your method with a malicious string. Anorm doesn't mind if you have additional symbols that aren't referenced in the SQL (i.e. if a clause string is empty). Lastly, depending on the database(?), a count might return no rows, so I use singleOpt rather than single.
I'm curious as to what other answers you receive.
Edit: Anorm interpolation (i.e. SQL"...", an interpolation implementation beyond Scala's s"...", f"..." and raw"...") was introduced to allow the use $variable as equivalent to {variable} with .on. And from Play 2.4, Scala and Anorm interpolation can be mixed using $ for Anorm (SQL parameter/variable) and #$ for Scala (plain string). And indeed this works well, as long as the Scala interpolated string does not contains references to an SQL parameter. The only way, in 2.4.4, I could find to use a variable in an Scala interpolated string when using Anorm interpolation, was:
val limitClause = if (nameFilter="") "" else s"where name>'$nameFilter'"
SQL"select * from tab #$limitClause order by name"
But this is vulnerable to SQL injection (e.g. a string like it's will cause a runtime syntax exception). So, in the case of variables inside interpolated strings, it seems it is necessary to use the "traditional" .on approach with only Scala interpolation:
val limitClause = if (nameFilter="") "" else "where name>{nameFilter}"
SQL(s"select * from tab $limitClause order by name").on('limitClause -> limitClause)
Perhaps in the future Anorm interpolation could be extended to parse the interpolated string for variables?
Edit2: I'm finding there are some tables where the number of attributes that might or might not be included in the query changes from time to time. For these cases I'm defining a context class, e.g. CustomerContext. In this case class there are lazy vals for the different clauses that affect the sql. Callers of the sql method must supply a CustomerContext, and the sql will then have inclusions such as ${context.createdClause} and so on. This helps give a consistency, as I end up using the context in other places (such as total record count for paging, etc.).
Finally got this simpler approach posted by Joel Arnold to work in my example case, also with ZonedDateTime!
def countReports(customerId: Option[Long], createdSince: Option[ZonedDateTime]) =
DB.withConnection {
implicit c =>
SQL( """
SELECT count(*) FROM report
WHERE ({customerId} is null or customer_id = {customerId})
AND ({created}::timestamptz is null or created >= {created})
""")
.on('customerId -> customerId, 'created -> createdSince)
.as(scalar[Int].singleOpt).getOrElse(0)
}
The tricky part is having to use {created}::timestamptz in the null check. As Joel commented, this is needed to work around a PostgreSQL driver issue.
Apparently the cast is needed only for timestamp types, and the simpler way ({customerId} is null) works with everything else. Also, comment if you know whether other databases require something like this, or if this is a Postgres-only peculiarity.
(While wwkudu's approach also works fine, this definitely is cleaner, as you can see comparing them side to side in a full example.)
How do I test and see if a variable is set in Scala. In PHP you would use isset()
I am looking for a way to see if a key is set in an array.
First, Array in Scala does not have keys. They have indices, and all indices have values in them. See the edit below about how those values might be initialized, though.
You probably mean Map, which has keys. You can check whether a key is present (and, therefore, a value) by using isDefinedAt or contains:
map isDefinedAt key
map contains key
There's no practical difference between the two. Now, you see in the edit that Scala favors the use of Option, and there's just such a method when dealing with maps. If you do this:
map get key
You'll receive an Option back, which will be None if the key (and, therefore, the value) is not present.
EDIT
This is the original answer. I've noticed now that the question is not exactly about this.
As a practical matter, all fields on the JVM are pre-initialized by the JVM itself, which zeroes it. In practice, all reference fields end up pointing to null, booleans are initialized with false and all other primitives are initialized with their version of zero.
There's no such thing in Scala as an "undefined" field -- you cannot even write such a thing. You can write var x: Type = _, but that simply results in the JVM initialization value. You can use null to stand for uninitialized where it makes sense, but idiomatic Scala code tries to avoid doing so.
The usual way of indicating the possibility that a value is not present is using Option. If you have a value, then you get Some(value). If you don't, you get None. See other Stack Overflow questions about various ways of using Option, since you don't use it like variable.isDefined in idiomatic code either (though that works).
Finally, note that idiomatic Scala code don't use var much, preferring val. That means you won't set things, but, instead, produce a new copy of the thing with that value set to something else.
PHP and Scala are so different that there is no direct equivalent. First of all Scala promotes immutable variables (final in Java world) so typically we strive for variables that are always set.
You can check for null:
var person: Person = null
//...
if(person == null) {//not set
//...
}
person = new Person()
if(person == null) {//set
//...
}
But it is a poor practice. The most idiomatic way would be to use Option:
var person: Option[Person] = None
//...
if(person.isDefined) {//not set
//...
}
person = Some(new Person())
if(person.isDefined) {//set
//...
}
Again, using isDefined isn't the most idiomatic ways. Consider map and pattern matching.
What is the standard idiom for implementing the equals and hashCode methods in Scala?
I know the preferred approach is discussed in Programming in Scala, but I don't currently have access to the book.
There's a free 1st edition of PinS that discuss this subject just as well. However, I think the best source is this article by Odersky discussing equality in Java. The discussion in PinS is, iirc, an abbreviated version of this article.
After doing quite a bit of research, I couldn't find an answer with an essential and correct implementation of the equals and hashCode pattern for a Scala class (not a case class as that is automatically compiler generated and should not be overridden). I did find a 2.10 (stale) macro making a valiant attempt at solving this.
I ended up combining the patterns recommended in "Effective Java, 2nd Edition" (by Joshua Bloch) and in this article "How to Write an Equality Method in Java" (by Martin Odersky, Lex Spoon, and Bill Venners), and have created a standard default pattern I now use to implement equals and hashCode for my Scala classes.
The primary goal of the equals pattern is to minimize the number of actual comparison's required to execute to get to a valid and definitive true or false.
Additionally, the hashCode method should ALWAYS be overridden and re-implemented when the equals method is overridden (again, see "Effective Java, 2nd Edition" (by Joshua Bloch)). Hence, my inclusion of the hashCode method "pattern" in the code below which also incorporates critical advice about using ## instead of hashCode in the actual implementation.
It is worth mentioning that each of super.equals and super.hashCode must be called only if an ancestor has already overridden it. If not, then it is imperative to NOT call super.* as the default implementation in java.lang.Object (equals compares for the same class instance and hashCode most likely converts the memory address of the object into an integer), both of which will break the specified equals and hashCode contract for the now overridden methods.
class Person(val name: String, val age: Int) extends Equals {
override def canEqual(that: Any): Boolean =
that.isInstanceOf[Person]
//Intentionally avoiding the call to super.equals because no ancestor has overridden equals (see note 7 below)
override def equals(that: Any): Boolean =
that match {
case person: Person =>
( (this eq person) //optional, but highly recommended sans very specific knowledge about this exact class implementation
|| ( person.canEqual(this) //optional only if this class is marked final
&& (hashCode == person.hashCode) //optional, exceptionally execution efficient if hashCode is cached, at an obvious space inefficiency tradeoff
&& ( (name == person.name)
&& (age == person.age)
)
)
)
case _ =>
false
}
//Intentionally avoiding the call to super.hashCode because no ancestor has overridden hashCode (see note 7 below)
override def hashCode(): Int =
31 * (
name.##
) + age.##
}
The code has a number of nuances which are critically important:
Extending scala.Equals - Ensures the equals idiomatic pattern, which includes the canEqual method formalization, is being fully implemented. While extending it is technically optional, it remains highly recommended.
Same instance short circuit - Testing (this eq person) for true ensures no further (expensive) comparisons as it is literally the same instance. This test is required to be inside the pattern match as the eq method is available on AnyRef, not on Any (the type of that). And because AnyRef is an ancestor of Person, this technique is doing two simultaneous type validations by type validating the descendant, Person, which implies an automatic type validation of all its ancestors, including AnyRef, which is required for the eq check. While this test is technically optional, it remains highly recommended.
Check that's canEqual - It's very easy to get this backwards which is INCORRECT. It is crucial the check of canEqual be executed on the that instance with this provided as the parameter. And while it might seem redundant to the pattern match (given we get to this line of code, that must be a Person instance), we must still make the method call as we cannot assume that is an equals-compatible descendant of Person (all descendants of Person will successfully pattern match as a Person). If the class is marked final, this test optional and may be safely removed. Otherwise, it is required.
Checking hashCode short circuit - While not sufficient nor required, if this hashCode test is false, it eliminates the need to perform all the value level checks (item 5). If this test is true, then a field by field check is actually required. This test is optional, and may be excluded if the hashCode value isn't cached and the total cost of the per-field equality checks is low enough.
Per-field equality checks - Even if the hashCode test is provided and succeeds, all of the field level values must still be checked. This is because, although it is highly improbable, it remains possible for two different instances to generate the exact same hashCode value, and still not be actually equivalent at the field level. The parent's equals must also be invoked to ensure any additional field's defined in ancestors are also tested.
Pattern match case _ => - This is actually achieving two different effects. First, Scala pattern match guarantees a null is properly routed here so null doesn't have to appear anywhere within our pure Scala code. Secondly, the pattern match guarantees whatever that is, it isn't an instance of Person or one of its descendants.
When to call each of super.equals and super.hashCode is a bit tricky - If an ancestor has already overridden both (should never be either) of these, it is imperative you incorporate super.* in your own overridden implementations. And if an ancestor has not overridden both, then your overridden implementations must avoid invoking super.*. The Person code example above shows the case where there is no ancestor who has overridden both. So, calling each super.* method call would incorrectly fall all the way to the default java.lang.Object.* implementation which would invalidate the assumed combined contract for equals and hashCode.
This is the super.equals based code to use ONLY IF there is at least one ancestor who has explicitly overridden equals already.
override def equals(that: Any): Boolean =
...
case person: Person =>
( ...
//WARNING: including the next line ASSUMES at least one ancestor has already overridden equals; i.e. that this does not end up invoking java.lang.Object.equals
&& ( super.equals(person) //incorporate checking ancestor(s)' fields
&& (name == person.name)
&& (age == person.age)
)
...
)
...
This is the super.hashCode based code to use ONLY IF there is at least one ancestor who has explicitly overridden hashCode already.
override def hashCode(): Int =
31 * (
31 * (
//WARNING: including the next line ASSUMES at least one ancestor has already overridden hashCode; i.e. that this does not end up invoking java.lang.Object.hashCode
super.hashCode //incorporate adding ancestor(s)' hashCode (and thereby, their fields)
) + name.##
) + age.##
One final note: In my doing research for this, I couldn't believe how many erred implementations of this pattern exist. It is clearly still an area where it's difficult to get the details right:
Programming in Scala, First Edition - Missed 1, 2, and 4 above.
Alvin Alexander's Scala Cookbook - Missed 1, 2, and 4.
Code Examples for Programming in Scala - Incorrectly uses .hashCode instead of .## on class fields when generating class's hashCode override and implementation. See Tree3.scala
Yes, overriding equals and hashCode is a dauting task in both Java and Scala. I would recommend not to use equals at all, and instead use a type class (Eq/Eql etc.). It's more typesafe (compiler error when comparing unrelated types), easier to implement (no overriding and class checking) and more flexible (you can write type class instances separate from the data class). Dotty uses a concept of "multiversal equality" which offers a choice between catching some obviously incorrect usages of equals and strict equality checking a la Haskell.
So I'm wrapping bits of the Mechanical Turk API, and you need to specify qualification requirements such as:
Worker_Locale == "US"
Worker_PercentAssignmentsApproved > 95
...
In my code, I'd like to allow the syntax above and have these translated into something like:
QualificationRequirement("00000000000000000071", "LocaleValue.Country", "EqualTo", "US")
QualificationRequirement("000000000000000000L0", "IntegerValue", "GreaterThan", 95)
I can achieve most of what I want by declaring an object like:
object Worker_PercentAssignmentsApproved {
def >(x: Int) = {
QualificationRequirement("000000000000000000L0", "IntegerValue", "GreaterThan", x)
}
}
But I can't do the same thing for the "==" (equals) or "!=" (not equals) methods since they're declared final in AnyRef. Is there a standard workaround for this? Perhaps I should just use "===" and "!==" instead?
(I guess one good answer might be a summary of how a few different scala DSLs have chosen to work around this issue and then I can just do whatever the majority of those do.)
Edit: Note that I'm not trying to actually perform an equality comparison. Instead, I'm trying to observe the comparison operator the user indicated in scala code, save an object based description of that comparison, and give that description to the server. Specifically, the following scala code:
Worker_Locale == "US"
will result in the following parameters being added to my request:
&QualificationRequirement.1.QualificationTypeId=000000000000000000L0
&QualificationRequirement.1.Comparator=EqualTo
&QualificationRequirement.1.LocaleValue.Country=US
So I can't override equals since it returns a Boolean, and I need to return a structure that represents all these parameters.
If you look at the definition of == and != in the scala reference, (§ 12.1), you’ll find that they are defined in terms of eq and equals.
eq is the reference equality and is also final (it is only used to check for null in that case) but you should be able to override equals.
Note that you’ll probably also need to write the hashCode method to ensure
∀ o1, o2 with o1.equals(o2) ⇒ (o1.hashCode.equals(o2.hashCode)).
However, if you need some other return type for your DSL than Boolean or more flexibility in general, you should maybe use ===, as has been done in Squeryl for example.
Here's a little survey of what various DSLs use for this kind of thing.
Liftweb uses === in Javascript expressions:
JsIf(ValById("username") === value.toLowerCase, ...)
Squeryl uses === for SQL expressions:
authors.where(a=> a.lastName === "Pouchkine")
querydsl uses $eq for SQL expressions:
person.firstName $eq "Ben"
Prolog-in-Scala uses === for Prolog expressions:
'Z === 'A
Scalatest uses === to get an Option instead of a Boolean:
assert("hello" === "world")
So I think the consensus is mostly to use ===.
I've been considering a similar problem. I was thinking of creating a DSL for writing domain-specific formulas. The trouble is that users might want to do string manipulation too and you end up with expression like
"some string" + <someDslConstruct>
No matter what you do its going to lex this as a something like
stringToLiteralString("some string" + <someDslConstruct>)
I think the only potential way out of this pit would be to try using macros. In your example perhaps you could have a macro that wraps a scala expression and converts the raw AST into a query? Doing this for arbitrary expressions wouldn't be feasible but if your domain is sufficiently well constrained it might be a workable alternative solution.