I'm trying to support arbitrary filters for a REST API that fetches a list of documents from MongoDB.
For instance
//example.com/users <- list all
//example.com/users?age=30 <- all users who are 30
//example.com/users?age=30&name=John <- all users who are 30 and called John
...
I'm using Play-ReactiveMongo and dealing with JSONCollection objects only.
So in my routes I put
GET /users controllers.Users.list(id: Option[String], name: Option[String], age: Option[Int])
But there are two problem with that, first I'll need to have a pretty long list of optional parameters, and then in my controller I need to use pattern matching on all of them to check whether they're empty or not, and also build the selector that I use to filter my collection.
var filters = JsObject(Nil)
name match {
case Some(x) => filters += ("name" -> JsString(x))
case None => None
}
I realized that I can get the full query string from the request object, which is a Map[String, Seq[String]]. But then I don't know a good way to check whether values are String or something else.
Is there another better and idiomatic way to do what I want?
Possible solution can be:
Use POST instead of GET:
POST /example.com/users
"data"={"age":25, "name":"xyz", ... }
OR single param in GET:
GET /example.com/users?filter={"age":25, "name":"xyz", ... }
On the server side, just validate against your model class OR just pass the same json in your reactivemongo find method.
Maybe request binders will help you to create complex objects from varying request parameters.
https://www.playframework.com/documentation/2.5.x/ScalaRequestBinders#QueryStringBindable
For example, you could build something like this (from the docs):
case class AgeRange(from: Int, to: Int)
for requests like this:
/age?from=1&to=10
Now you could change these attributes to Option and create a function that creates a reactivemongo query based on the values at hand.
Related
I have a 3-level nested case class model with a bunch of options that represents some data in a database. It's essentially:
case class User(settings: Option[Settings])
case class Settings(keys: Option[List[KeySet]])
case class KeySet(privateKey: String, publicKey: String)
I understand how to get deeply nested fields out of this using some for comprehension or chains of flatMap (Scala Option object inside another Option object) and I also understand how to update it using a lens library, but I want to figure out how to update the fields even if some stuff in the tree is None and automatically make Somes of those if they don't exist yet.
For example, how would I handle the case where I want to add to the keys List but the user hasn't yet set any settings? Is it possible to, in some sense, automatically create a Some(settings) field and also a Some(keys) field?
I have an idea of how to do it with a lot of pattern matching, but this seems wrong because of 1. rightward drift of code and 2. not using map or flatMap very much with the options.
Is this possible using a lens library on its own? I read here that it might not be possible: https://github.com/julien-truffaut/Monocle/issues/215 as in the case of Monocle it can't update an Option that is a None. Maybe I need to think about the problem another way?
Thanks
I'm not sure why you use Option[List[KeySet]]. Is there an important distinction between None and an empty List?
In any case, I find fold to be a handy tool when working with Options.
def updateUser(u :User, ks :KeySet) :User = {
u.copy(settings =
Some(u.settings.fold(Settings(Some(ks::Nil))) (stngs =>
stngs.copy(keys = Some(stngs.keys.fold(ks::Nil) (ks::_))))))
}
val pat = updateUser(User(None), KeySet("a","b"))
//pat: User = User(Some(Settings(Some(List(KeySet(a,b))))))
val upat = updateUser(pat, KeySet("c","d"))
//upat: User = User(Some(Settings(Some(List(KeySet(c,d), KeySet(a,b))))))
In my Scala Application, I am trying to find() all Objects for a List of ObjectIds.
My DB Structure looks like that:
case class Parent(_id: ObjectId, name: String)
case class Child(_id: ObjectId, name: String, parents: List[ObjectId])
Now, I have a list of ParentIds and want to find all Parents.find(in("_id", foundChild.parents)).
Unfortunately, parents will be empty. I have searched the Internet for a in("_id", List[ObjectId])-Example, but couldn't find a single one.
Reading the documentation, it seems like it is not supported to check for lists, but only for TItem*. Is it possible to achieve a "is in List" Check somehow?
Pass in the List as varargs:
Parents.find(in("_id", foundChild.parents: _*))
The title pretty much sums it up. Option as a singleton collection can sometimes be confusing, but sometimes it allows for an interesting application. I have one example on top of my head, and would like to learn more of such examples.
My only example is running for comprehension on the Option[List[T]]. We can do the following:
val v = Some(List(1, 2, 3))
for {
list <- v.toList
elem <- list
} yield elem + 1
Without having Option.toList, it wouldn't be possible to stay in the same for comprehension, and I'd be forced to write something like this:
for {
list <- v
} yield for {
elem <- list
} yield elem + 1
The first example is cleaner, and it's an advantage of Option being a collection. Of course, the result type will be different in these 2 examples, but let's assume it doesn't matter for the sake of discussion.
Any other examples? I'd especially like to concentrate on collection-like usage, and not usage of Option's monadic properties - those are pretty much obvious. In other words, map and flatMap functions are out of scope of this question. They're definitely very useful, just coming from elsewhere.
I find that working with Option[T] as a collection's main benefit is that you get to use operations defined on a collection, such as map, flatmap, filter, foreach etc. This makes it easier to do operations on a given option, instead of using pattern matching or checking Option[T].isDefined to see if a value exists.
For example, let's take the user repository example from Daniel Westheide blog post about Option[T]:
Say you have a UserRepository object which returns users based on their ID. The user may or may not exist, hence it returns an Option[Person]. Now let's say we want to search a person by id and then filter their age. We can do:
val age: Some[Int] = UserRepository.findById(1).map(_.age)
Now let's say that a Person also has a gender property of type Option[String]. If you wanted to extract that out, you could use map:
val gender: Option[Option[String]] = UserRepository.findById(1).map(_.gender)
But working with nested options isn't too convenient. For that, you have flatMap:
val gender: Option[String] = UserRepository.findById(1).flatMap(_.gender)
And if we want to print out the gender if it exists, we can use foreach:
gender.foreach(println)
You'll find yourself working with scala types that have nested Option[T] fields defined and it's really handy to have collection like methods which help you remove out boilerplate and noise for extracting the actual value out of the operation.
A more real life use case I just encountered the other day was working with the awscala SDK, where I wanted to retrieve an object from S3 storage:
val bucket: Option[Bucket] = s3.bucket(amazonConfig.bucketName)
val result: Option[S3Object] = bucket.flatMap(_.get(amazonConfig.offsetKey))
result.flatMap(s3Object =>
Source.fromInputStream(s3Object.content).mkString.decodeOption[Array[KafkaOffset]])
So what happens here is that you query the S3 service for a bucket, which may or may not exist. Then, you want to extract an S3Object out of it which actually contains the data, but the API itself returns an Option[S3Object], so it's handy to use flatMap to flat out get an Option[S3Object] instead of Option[Option[S3Object]]. Finally, I want to deserialize the S3Object which actually contains a JSON, and using the Argonaut library, it returns an Option[MyObject], so then again using flatMap to the rescue of extracting the inner option type.
Edit:
As you pointed out, map and flatMap belong to the monadic property of Option[T]. I've written a blog post describing the reduction of two options where the final solution was:
def reduce[T](a: Option[T], b: Option[T], f: (T, T) => T): Option[T] = {
(a ++ b).reduceLeftOption(f)
}
Which takes advantage of the ++ operator defined on any collection which is also specifically defined on Option[T], being a collection.
I'd suggest to take a look at the corresponding chapter of The Neophyte's Guide to Scala.
In my experience, most useful use-cases of Option-as-collection are to filter an option and to make flatMap that implicitly filters None values.
I have a controller defined like this:
def registerCompany = Action.async(BodyParsers.parse.json) { request =>
request.body.validate[Company].fold(
errors => Future {
BadRequest(errors.mkString)
},
company => Future {
registrationService.registerCompany
Ok("saved")
}
)
}
Company is a simple case class
case class Company(name: String, address: Address, adminUser: Option[User] = None,
venues: Option[Set[Venue]] = None, _id: Option[Long]) {
}
so that I can take advantage of
implicit val companyFormatter = Json.format[Company]
So far so good, but now I want to have validation in the Company class. I've been googling a bit and the best I found was this:
http://koff.io/posts/292173-validation-in-scala/
So many solutions, yet, I'm not happy with any of them. Most of these solutions have known limitations or are a bit messy. I'd like to have declarative validation (annotation based), as that means I write less code and it looks cleaner.
I could mix java with scala and use JSR-303, but it doesn't work for case classes and I don't want to implement Reads and Writes for simple objects.
This is the closest I could find to what I want, but it doesn't support NotNull: https://github.com/bean-validation-scala/bean-validation-scala
Seems a bit like a luxury problem, with so many different solutions, but the truth is that in Java I can get the best of both worlds.
Is there anything else that I could use? Or any work around to the possibilities I'm listing here that could allow me to use both annotation based validation and case classes?
It all depends on what you mean by validation.
You gave an example of a validate Json in request body and I think you would like validate Json request (with Company constraints) but not case class Company (although the topic is called "Scala case class validation"). So you need use Validation with Reads
For example you may use:
( __ \ "name").read[String] for Required constraint.
( __ \ "_id").read[Long](min(0) keepAnd max(150)) for 0 < x < 150 constraint.
You may implement own Reads[Company] and Writes[Company] but not macros Json.format[Company]
Update for comment "The difference is that you rarely need to write a custom deserializer, as jackson knows how to handle any object"
If you don't want implement own deserialization and want use format macros, you may implement Company validation, but still using Reads:
val f(company: Company): Boolean = {... company constraints to boolean ...}
request.body.validate[Company](
companyFormatter.filter(ValidationError("Company validation error"))(f)
)
but this constraints will be applied after full Company deserialization.
Don't even know which is better: universal deserialization and "post" constraints or own deserialization and "pre" constraints. But both are working.
You could also roll your own, something like this:
https://gist.github.com/eirirlar/b40bd07a71044d3776bc069f210798c6
It will validate both case classes and incomplete case classes (hlists of options of types tagged with keys, where types and keys match the case class).
It won't give you annotations, but you're free to declare different validators for different contexts.
I'm admittedly very new to Scala, and I'm having trouble with the syntactical sugar I see in many Scala examples.
It often results in a very concise statement, but honestly so far (for me) a bit unreadable.
So I wish to take a typical use of the Option class, safe-dereferencing, as a good place to start for understanding, for example, the use of the underscore in a particular example I've seen.
I found a really nice article showing examples of the use of Option to avoid the case of null.
https://medium.com/#sinisalouc/demystifying-the-monad-in-scala-cc716bb6f534#.fhrljf7nl
He describes a use as so:
trait User {
val child: Option[User]
}
By the way, you can also write those functions as in-place lambda
functions instead of defining them a priori. Then the code becomes
this:
val result = UserService.loadUser("mike")
.flatMap(user => user.child)
.flatMap(user => user.child)
That looks great! Maybe not as concise as one can do in groovy, but not bad.
So I thought I'd try to apply it to a case I am trying to solve.
I have a type Person where the existence of a Person is optional, but if we have a person, his attributes are guaranteed. For that reason, there are no use of the Option type within the Person type itself.
The Person has an PID which is of type Id. The Id type consists of two String types; the Id-Type and the Id-Value.
I've used the Scala console to test the following:
class Id(val idCode : String, val idVal : String)
class Person(val pid : Id, val name : String)
val anId: Id = new Id("Passport_number", "12345")
val person: Person = new Person(anId, "Sean")
val operson : Option[Person] = Some(person)
OK. That setup my person and it's optional instance.
I learned from the above linked article that I could get the Persons Id-Val by using flatMap; Like this:
val result = operson.flatMap(person => Some(person.pid)).flatMap(pid => Some(pid.idVal)).getOrElse("NoValue")
Great! That works. And if I infact have no person, my result is "NoValue".
I used flatMap (and not Map) because, unless I misunderstand (and my tests with Map were incorrect) if I use Map I have to provide an alternate or default Person instance. That I didn't want to have to do.
OK, so, flatMap is the way to go.
However, that is really not a very concise statement.
If I were writing that in more of a groovy style, I guess i'd be able to do something like this:
val result = person?.pid.idVal
Wow, that's a bit nicer!
Surely Scala has a means to provide something at least nearly as nice as Groovy?
In the above linked example, he was able to make his statement more concise using some of that syntactical sugar I mentioned before. The underscore:
or even more concise:
val result = UserService.loadUser("mike")
.flatMap(_.child)
.flatMap(_.child)
So, it seems in this case the underscore character allows you to skip specifying the type (as the type is inferred) and replace it with underscore.
However, when I try the same thing with my example:
val result = operson.flatMap(Some(_.pid)).flatMap(Some(_.idVal)).getOrElse("NoValue")
Scala complains.
<console>:15: error: missing parameter type for expanded function ((x$2) => x$2.idVal)
val result = operson.flatMap(Some(_.pid)).flatMap(Some(_.idVal)).getOrElse("NoValue")
Can someone help me along here?
How am I misunderstanding this?
Is there a short-hand method of writing my above lengthy statement?
Is flatMap the best way to achieve what I am after? Or is there a better more concise and/or readable way to do it ?
thanks in advance!
Why do you insist on using flatMap? I'd just use map for your example instead:
val result = operson.map(_.pid).map(_.idVal).getOrElse("NoValue")
or even shorter:
val result = operson.map(_.pid.idVal).getOrElse("NoValue")
You should only use flatMap with functions that return Options. Your pid and idVals are not Options, so just map them instead.
You said
I have a type Person where the existence of a Person is optional, but if we have a person, his attributes are guaranteed. For that reason, there are no use of the Option type within the Person type itself.
This is the essential difference between your example and the User example. In the User example, both the existence of a User instance, and its child field are options. This is why, to get a child, you need to flatMap. However, since in your example, only the existence of a Person is not guaranteed, after you've retrieved an Option[Person], you can safely map to any of its fields.
Think of flatMap as a map, followed by a flatten (hence its name). If I mapped on child:
val ouser = Some(new User())
val child: Option[Option[User]] = ouser.map(_.child)
I would end up with an Option[Option[User]]. I need to flatten that to a single Option level, that's why I use flatMap in the first place.
If you looking for the most concise solution, consider this:
val result = operson.fold("NoValue")(_.pid.idVal)
Though one could find it not clear or confusing