So I'm wrapping bits of the Mechanical Turk API, and you need to specify qualification requirements such as:
Worker_Locale == "US"
Worker_PercentAssignmentsApproved > 95
...
In my code, I'd like to allow the syntax above and have these translated into something like:
QualificationRequirement("00000000000000000071", "LocaleValue.Country", "EqualTo", "US")
QualificationRequirement("000000000000000000L0", "IntegerValue", "GreaterThan", 95)
I can achieve most of what I want by declaring an object like:
object Worker_PercentAssignmentsApproved {
def >(x: Int) = {
QualificationRequirement("000000000000000000L0", "IntegerValue", "GreaterThan", x)
}
}
But I can't do the same thing for the "==" (equals) or "!=" (not equals) methods since they're declared final in AnyRef. Is there a standard workaround for this? Perhaps I should just use "===" and "!==" instead?
(I guess one good answer might be a summary of how a few different scala DSLs have chosen to work around this issue and then I can just do whatever the majority of those do.)
Edit: Note that I'm not trying to actually perform an equality comparison. Instead, I'm trying to observe the comparison operator the user indicated in scala code, save an object based description of that comparison, and give that description to the server. Specifically, the following scala code:
Worker_Locale == "US"
will result in the following parameters being added to my request:
&QualificationRequirement.1.QualificationTypeId=000000000000000000L0
&QualificationRequirement.1.Comparator=EqualTo
&QualificationRequirement.1.LocaleValue.Country=US
So I can't override equals since it returns a Boolean, and I need to return a structure that represents all these parameters.
If you look at the definition of == and != in the scala reference, (§ 12.1), you’ll find that they are defined in terms of eq and equals.
eq is the reference equality and is also final (it is only used to check for null in that case) but you should be able to override equals.
Note that you’ll probably also need to write the hashCode method to ensure
∀ o1, o2 with o1.equals(o2) ⇒ (o1.hashCode.equals(o2.hashCode)).
However, if you need some other return type for your DSL than Boolean or more flexibility in general, you should maybe use ===, as has been done in Squeryl for example.
Here's a little survey of what various DSLs use for this kind of thing.
Liftweb uses === in Javascript expressions:
JsIf(ValById("username") === value.toLowerCase, ...)
Squeryl uses === for SQL expressions:
authors.where(a=> a.lastName === "Pouchkine")
querydsl uses $eq for SQL expressions:
person.firstName $eq "Ben"
Prolog-in-Scala uses === for Prolog expressions:
'Z === 'A
Scalatest uses === to get an Option instead of a Boolean:
assert("hello" === "world")
So I think the consensus is mostly to use ===.
I've been considering a similar problem. I was thinking of creating a DSL for writing domain-specific formulas. The trouble is that users might want to do string manipulation too and you end up with expression like
"some string" + <someDslConstruct>
No matter what you do its going to lex this as a something like
stringToLiteralString("some string" + <someDslConstruct>)
I think the only potential way out of this pit would be to try using macros. In your example perhaps you could have a macro that wraps a scala expression and converts the raw AST into a query? Doing this for arbitrary expressions wouldn't be feasible but if your domain is sufficiently well constrained it might be a workable alternative solution.
Related
I'm not an expert on Haskell. And this question is not exactly a Haskell question, but I know Haskell people would have a better understanding of what I'm trying to achieve.
So I'm building a dynamic language and I want it to be pure... Totally pure. (With support for IO effects, and I already know why, but that's not part of this question)
Also, I would like it to have some form of polymorphism, so I'm toying with the idea of adding class support.
(Also, everything in the language is supposed to be an expression, so yep, no statements)
While exploring the idea I ended up realizing that in order for it to be referentially transparent, class expressions should be able to be substituted too.
The thing with class expressions is that one of its main functionalities is to check whether some value is instance of it.
So
val Person =class {...}
val person1 =Person(blabla)
Person.instantiated(person1) // returns true
// Equivalent to
class {...}.
instantiated(class{...}(blabla))
Yet! That last part makes no sense... It feels wrong, like I created two different classes
So!
Is there an expression such that
val expr = <<expression>>
expr == expr // true
But <<expression>> == <<expression>> is false?
In a pure language?
I think that what I'm asking is equivalent to asking if the newtype Haskell statement could become an expression
The way you've worded your question, you're likely to get at least a few answers that talk about peculiarities of the == operator (and, as I write this, you've already gotten one comment to that effect). But, that's not what you're asking, so forget about ==. Go back to your class example.
Referential transparency implies that after:
val Person = class {<PERSONCLASSDEFN>}
val person1 = Person(<PERSONARGS>)
the two expressions:
Person.instantiated(person1)
and:
(class {<PERSONCLASSDEFN>}).instantiated((class {<PERSONCLASSDEFN>})(<PERSONARGS>))
should be indistinguishable. That is, a program's meaning should not change if one is substituted for the other and vice versa.
Therefore, the identity of classes must depend only on their definition (the part in the curly braces), not where or how many times they are (re)defined or the names they are given.
As a simpler example, you should also consider the implications of:
val Person = class {<CLASSDEFN>}
val Automobile = class {<CLASSDEFN>}
val person = Person(<ARGS>)
val automobile = Automobile(<ARGS>)
after which, the two objects person and automobile should be indistinguishable.
I find it difficult to see what this question actually is about, but maybe the problem is that you're talking about equalities when you actually mean equivalence relations?
Two objects that are an instance of the same class are typically not equal, and correspondingly == will yield False. Yet they are equivalent in the sense of being instances of the same class. They're members of the same equivalence class (mathematical term; the usage of the word “class” in both OO and Haskell descends from this).
You can just have that equivalence class as a different operator. Like, in Python
def sameclassinstances(a, b):
return (type(a) is type(b))
Depending on your language's syntax that could of course also be a custom infix operator like
infix 4 ~=.
A separate issue is that equality itself can be interpreted as either value equality (always in Haskell), or some form of implementation equality or reference equality, which is fairly common in other languages. But if you want your language to be pure, you should probably stay away from the latter, or give it a telling name like Haskell's reallyUnsafePtrEquality.
First of all, I'm very new to Scala and don't have any experience writing production code with it, so I lack understanding of what is considered a good/best practice among community. I stumbled upon these resources:
https://github.com/alexandru/scala-best-practices
https://nrinaudo.github.io/scala-best-practices/
It is mentioned there that throwing exceptions is not very good practice, which made me think what would be a good way to define preconditions for function then, because
A function that throws is a bit of a lie: its type implies it’s total function when it’s not.
After a bit of research, it seems that using Option/Either/Try/Or(scalactic) is a better approach, since you can use something like T Or IllegalArgumentException as return type to clearly indicate that function is actually partial, using exception as a way to store message that can be wrapped in other exceptions.
However lacking Scala experience I don't quite understand if this is actually viable approach for a real project or using Predef.require is a way to go. I would appreciate if someone explained how things are usually done in Scala community and why.
I've also seen Functional assertion in Scala, but while the idea itself looks interesting, I think PartialFunction is not very suitable for the purpose as it is, because often more than one argument is passed and tuples look like a hack in this case.
Option or Either is definitely the way to go for functional programming.
With Option it is important to document why None might be returned.
With Either, the left side is the unsuccessful value (the "error"), while the right side is the successful value. The left side does not necessarily have to be an Exception (or a subtype of it), it can be a simple error message String (type aliases are your friend here) or a custom data type that is suitable for you application.
As an example, I usually use the following pattern when error handling with Either:
// Somewhere in a package.scala
type Error = String // Or choose something more advanced
type EitherE[T] = Either[Error, T]
// Somewhere in the program
def fooMaybe(...): EitherE[Foo] = ...
Try should only be used for wrapping unsafe (most of the time, plain Java) code, giving you the ability to pattern-match on the result:
Try(fooDangerous()) match {
case Success(value) => ...
case Failure(value) => ...
}
But I would suggest only using Try locally and then go with the above mentioned data types from there.
Some advanced datatypes like cats.effect.IO or monix.reactive.Observable contain error handling natively.
I would also suggest looking into cats.data.EitherT for typeclass-based error handling. Read the documentation, it's definitely worth it.
As a sidenote, for everyone coming from Java, Scala treats all Exceptions as Java treats RuntimeExceptions. That means, even when an unsafe piece of code from one of your dependencies throws a (checked) IOException, Scala will never require you to catch or otherwise handle the exception. So as a rule of thumb, when using Java - dependencies, almost always wrap them in a Try (or an IO if they execute side effects or block the thread).
I think your reasoning is correct. If you have a simple total (opposite of partial) function with arguments that can have invalid types then the most common and simple solution is to return some optional result like Option, etc.
It's usually not advisable to throw exceptions as they break FP laws. You can use any library that can return a more advanced type than Option like Scalaz Validation if you need to compose results in ways that are awkward with Option.
Another two alternatives I could offer is to use:
Type constrained arguments that enforce preconditions. Example: val i: Int Refined Positive = 5 based on https://github.com/fthomas/refined. You can also write your own types which wrap primitive types and assert some properties. The problem here is if you have arguments that have multiple interdependent valid values which are mutually exclusive per argument. For instance x > 1 and y < 1 or x < 1 and y > 1. In such case you can return an optional value instead of using this approach.
Partial functions, which in the essence resemble optional return types: case i: Int if i > 0 => .... Docs: https://www.scala-lang.org/api/2.12.1/scala/PartialFunction.html.
For example:
PF's def lift: (A) ⇒ Option[B] converts PF to your regular function.
Turns this partial function into a plain function returning an Option
result.
Which is similar to returning an option. The problem with partial functions that they are a bit awkward to use and not fully FP friendly.
I think Predef.require belongs to very rare cases where you don't want to allow any invalid data to be constructed and is more of a stop-everything-if-this-happens kind of measure. Example would be that you get arguments you never supposed to get.
You use the return type of the function to indicate the type of the result.
If you want to describe a function that can fail for whatever reason, of the types you mentioned you would probably return Try or Either: I am going to "try" to give your a result, or I am going to return "either" a success or an failure.
Now you can specify a custom exception
case class ConditionException(message: String) extends RuntimeException(message)
that you would return if your condition is not satisfied, e.g
import scala.util._
def myfunction(a: String, minLength: Int): Try[String] = {
if(a.size < minLength) {
Failure(ConditionException(s"string $a is too short")
} else {
Success(a)
}
}
and with Either you would get
import scala.util._
def myfunction(a: String, minLength: Int): Either[ConditionException,String] = {
if(a.size < minLength) {
Left(ConditionException(s"string $a is too short")
} else {
Right(a)
}
}
Not that the Either solution clearly indicates the error your function might return
I'm writing a DSL that generates SQL. The syntax for loading a table is:
session.activateWhere( _.User.ID == 490 )
This will select from the User table where the ID column is 490. I can use "==" because I can override "equals()" to generate the correct SQL. My problem is that "!=" doesn't work because it calls equals() and then negates the result. Sadly "!=" is final so I can't override it. Is there any way in my equals() method to tell that it was called as part of !=?
I've implemented a "<>" operation that logically does the same thing as "!=" and it works fine:
session.activateWhere( _.User.ID <> 490 )
My issue is that not only is "!=" valid syntax (to the compiler) but it will run and generate the exact opposite of what the user intends.
As you say, != is (similarly to ==) final and so cannot be overridden - with a very good reason. That is why most DLSs use === as an alternative.
The == and != operators have a very well defines meaning for all objects in Scala. Changing the meaning for some objects would be, in my opinion, very dangerous.
If you want to use == and !=, you have to override equals, hashcode and canEqual (trait Equals)
It is not a simple as it looks, you have to be carreful.
There is some examples over the net.
For example : https://www.safaribooksonline.com/library/view/scala-cookbook/9781449340292/ch04s16.html
Another one : https://groups.google.com/forum/#!msg/scala-user/Qosfawmaecw/RUWigwvVtQ4J
I am confused.
I thought everything is expression because the statement returns a value. But I also heard that everything is an object in scala.
What is it in reality? Why did scala choose to do it one way or the other? What does that mean to a scala developer?
I thought everything is expression because the statement returns a value.
There are things that don't have values, but for the most part, this is correct. That means that we can basically drop the distinction between "statement" and "expression" in Scala.
The term "returns a value" is not quite fitting, however. We say everything "evaluates" to a value.
But I also heard that everything is an object in scala.
That doesn't contradict the previous statement at all :) It just means that every possible value is an object (so every expression evaluates to an object). By the way, functions, as first-class citizens in Scala, are objects, too.
Why did scala choose to do it one way or the other?
It has to be noted that this is in fact a generalization of the Java way, where statements and expressions are distinct things and not everything is an object. You can translate every piece of Java code to Scala without a lot of adaptions, but not the other way round. So this design decision makes Scala is in fact more powerful in terms of conciseness and expressiveness.
What does that mean to a scala developer?
It means, for example, that:
You often don't need return, because you can just put the return value as the last expression in a method
You can exploit the fact that if and case are expressions to make your code shorter
An example would be:
def mymethod(x: Int) = if (x > 2) "yay!" else "too low!"
// ...
println(mymethod(10)) // => prints "yay!"
println(mymethod(0)) // => prints "too low!"
We can also assign the value of such a compound expression to a variable:
val str = value match {
case Some(x) => "Result: " + x
case None => "Error!"
}
The distinction here is that the assertion "everything is a expression" is being made about blocks of code, whereas "everything is an object" is being made about the values in your program.
Blocks of Code
In the Java language, there are both expressions and statements. That is, any "block" of code is either an expression or a statement;
//the if-else-block is a statement whilst (x == 4) is an expression
if (x == 4) {
foo();
}
else {
bar();
}
Expressions have a type; statements do not; statements are invoked purely for their side-effects.
In scala, every block of code is an expression; that is, it has a type:
if (x == 4) foo else bar //has the type lub of foo and bar
This is extremely important! I cannot stress this enough; it's one of the things which makes scala a pleasure to work with because I can assign a value to an expression.
val x = if (x == 4) foo else bar
Values
By value, I mean something that we might reference in the program:
int i = foo(); //i is a reference to a value
java.util.TimeUnit.SECONDS;
In Java, the i above is not an object - it is a primitive. Furthermore I can access the field SECONDS of TimeUnit, but TimeUnit is not an object either; it is a static (for want of a better phrase). In scala:
val j = 4
Map.empty
As far as the language is concerned, j is an object (and I may dispatch methods to it), as is Map - the module (or companion object) for the type scala.collection.immutable.Map.
Is everything a function or expression or object in scala?
None of it.
There are things that are not objects. e.g. classes, methods.
There are things that are not expressions. e.g. class Foo { } is a statement, and does not evaluate to any value. (This is basically the same point as above.)
There are things that are not functions. I don't need to mention any examples for this one as there would be plenty in sight in any Scala code.
In other words, "Everything is a X" is nothing more than a sales pitch (in case of Scala).
I am struggling to create a SQL DSL for Scala. The DSL is an extension to Querydsl, which is a popular Query abstraction layer for Java.
I am struggling now with really simple expressions like the following
user.firstName == "Bob" || user.firstName == "Ann"
As Querydsl supports already an expression model which can be used here I decided to provide conversions from Proxy objects to Querydsl expressions. In order to use the proxies I create an instance like this
import com.mysema.query.alias.Alias._
var user = alias(classOf[User])
With the following implicit conversions I can convert proxy instances and proxy property call chains into Querydsl expressions
import com.mysema.query.alias.Alias._
import com.mysema.query.types.expr._
import com.mysema.query.types.path._
object Conversions {
def not(b: EBoolean): EBoolean = b.not()
implicit def booleanPath(b: Boolean): PBoolean = $(b);
implicit def stringPath(s: String): PString = $(s);
implicit def datePath(d: java.sql.Date): PDate[java.sql.Date] = $(d);
implicit def dateTimePath(d: java.util.Date): PDateTime[java.util.Date] = $(d);
implicit def timePath(t: java.sql.Time): PTime[java.sql.Time] = $(t);
implicit def comparablePath(c: Comparable[_]): PComparable[_] = $(c);
implicit def simplePath(s: Object): PSimple[_] = $(s);
}
Now I can construct expressions like this
import com.mysema.query.alias.Alias._
import com.mysema.query.scala.Conversions._
var user = alias(classOf[User])
var predicate = (user.firstName like "Bob") or (user.firstName like "Ann")
I am struggling with the following problem.
eq and ne are already available as methods in Scala, so the conversions aren't triggered when they are used
This problem can be generalized as the following. When using method names that are already available in Scala types such as eq, ne, startsWith etc one needs to use some kind of escaping to trigger the implicit conversions.
I am considering the following
Uppercase
var predicate = (user.firstName LIKE "Bob") OR (user.firstName LIKE "Ann")
This is for example the approach in Circumflex ORM, a very powerful ORM framework for Scala with similar DSL aims. But this approach would be inconsistent with the query keywords (select, from, where etc), which are lowercase in Querydsl.
Some prefix
var predicate = (user.firstName :like "Bob") :or (user.firstName :like "Ann")
The context of the predicate usage is something like this
var user = alias(classOf[User])
query().from(user)
.where(
(user.firstName like "Bob") or (user.firstName like "Ann"))
.orderBy(user.firstName asc)
.list(user);
Do you see better options or a different approach for SQL DSL construction for Scala?
So the question basically boils down to two cases
Is it possible to trigger an implicit type conversion when using a method that exists in the super class (e.g. eq)
If it is not possible, what would be the most Scalaesque syntax to use for methods like eq, ne.
EDIT
We got Scala support in Querydsl working by using alias instances and a $-prefix based escape syntax. Here is a blog post on the results : http://blog.mysema.com/2010/09/querying-with-scala.html
There was a very good talk at Scala Days: Type-safe SQL embedded in Scala by Christoph Wulf.
See the video here: Type-safe SQL embedded in Scala by Christoph Wulf
Mr Westkämper - I was pondering this problem, and I wondered if would be possible to use 'tracer' objects, where the basic data types such as Int and String would be extended such that they contained source information, and the results of combining them would likewise hold within themselves their sources and the nature of the combination.
For example, your user.firstName method would return a TracerString, which extends String, but which also indicates that the String corresponds to a column in a relation. The == method would be overwritten such that it returns an EqualityTracerBoolean which extends Boolean. This would preserve the standard Scala semantics. However, the constructor for EqualityTracerBoolean would record the fact that the result of the expression was derived by comparing a column in a relation to a string constant. Your 'where' method could then analyse the EqualityTracerBoolean object returned by the conditional expression evaluated over a dummy argument in order to derive the expression used to create it.
There would have to be override defs for inequality operators, as well as plus and minus, for Ints, and whatever else you wished to represent from sql, and corresponding tracer classes for each of these. It would be a bit of a project!
Anyway, I decided not to bother, and use squeryl instead.
I didn't have the exact same problem with jOOQ, as I'm using a bit more verbose operator names: equal, notEqual, etc instead of eq, ne. On the other hand, there is a val operator in jOOQ for explicitly creating bind values, which I had to overload with value, as val is a keyword in Scala. Is overloading operators an option for you? I documented my attempts of running jOOQ in Scala here:
http://lukaseder.wordpress.com/2011/12/11/the-ultimate-sql-dsl-jooq-in-scala/
Just like you, I had also thought about capitalising all keywords in a major release (including SELECT, FROM, etc). But that will leave an open question about whether "compound" keywords should be split in two method calls, or connected by an underscore: GROUP().BY() or GROUP_BY(). WHEN().MATCHED().THEN().UPDATE() or WHEN_MATCHED_THEN_UPDATE(). Since the result is not really satisfying, I guess it's not worth to break backwards-compatibility for such a fix, even if the two-method-call option would look very very nice in Scala, as . and () can be omitted. So maybe, jOOQ and QueryDSL should both be "wrapped" (as opposed to "extended") by a dedicated Scala-API?
What about decompiling the bytecode at runtime? I started to write such a tool:
http://h2database.com/html/jaqu.html#natural_syntax
I know it's a hack, so please don't vote -1 :-) I just wanted to mentioned it. It's a relatively novel approach. Instead of decompiling at runtime, it might be possible to do it at compile time using an annotation processor, not sure if that's possible using Scala (and not sure if it's really possible with Java, but Project Lombok seems to do something like that).