Let's say I have a case class with the optional field nickName and codec like this:
final case class Person(name: String, nickName: Option[String])
object Person {
implicit val personCodec: JsonCodec[Person] = DeriveJsonCodec.gen
}
I want to encode it using zio-json (v1.5.0) and have this as result:
{"name":"SomeName"}
And this is my test for it:
encoder.encodeJson(Person("SomeName", None), None).toString shouldBe """{"name":"SomeName"}""".stripMargin
Looks like the zio-json encode None with null and I've get the test error:
Expected :"{"name":"SomeName"[]}"
Actual :"{"name":"SomeName"[,"nickName":null]}"
I checked the code and found the encoder for Option https://github.com/zio/zio-json/blob/52d007ee22f214d12e1706b016f149c3243c632c/zio-json/shared/src/main/scala/zio/json/encoder.scala#L188-L202
Any idea how I can encode it as a missing JSON field?
implicit val OptionStringCodec: JsonCodec[Option[String]] = JsonCodec.string.xmap( s =>
s match {
case null | "" => None
case s => Some(s)
},
_.getOrElse("")
)
Note: I'm not familiar sith zio-json, there might be another way like a configuration to achieve the same thing.
Given the sample of code you linked, you can easily write an encoder that doesn't write anything in case of none by copying most of the code but modifying:
def unsafeEncode(oa: Option[A], indent: Option[Int], out: Write): Unit = oa match {
case None => () // out.write("null")
case Some(a) => A.unsafeEncode(a, indent, out)
}
Related
I have a very large file of json lines, which I intend to read into a list of case classes. Due to the size of the file, rather than reading the entire file into a variable first and then filtering, I would like to filter within the json decoding pattern matching. Currently the code looks like this:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser.decode
case class Person(name: String, age: Int, country: String)
val personList: List[Person] =
Source.fromResource("Persons.json").getLines.toList.map { line =>
implicit val jsonDecoder: Decoder[Person] = deriveDecoder[Person]
val decoded = decode[Person](line)
decoded match {
case Right(decodedJson) =>
Person(
decodedJson.name,
decodedJson.age,
decodedJson.country
)
case Left(ex) => throw new RuntimeException(ex)
}
}
however, if I wanted to only include Person instances with a country of "us", what would be the best way to accomplish this? Should I have nested pattern matching, that will specifically look for Person(_, _, "us") (im not sure how I would accomplish this), or is there some way I can implement Option handling?
You could do something like this:
import io.circe.Decoder
import io.circe.generic.semiauto.deriveDecoder
import io.circe.parser.decode
case class Person(name: String, age: Int, country: String)
implicit val jsonDecoder: Decoder[Person] = deriveDecoder[Person]
val personList: List[Person] =
Source
.fromResource("Persons.json")
.getLines
.flatMap { line =>
val decoded = decode[Person](line)
decoded match {
case Right(person # Person(_, _, "us")) => Some(person)
case Right(_) => None
case Left(ex) =>
println(s"couldn't decode: $line, will skip (error: ${ex.getMessage})")
None
}
}
.toList
println(s"US people: $personList")
A few things to note:
I moved the .toList to the end. In your implementation, you called it right after .getLines which kind of loses the lazyness of the whole thing. Assuming there's only a few US people out of huge number of people in the JSON file, this can be beneficial for performance & efficiency.
Wrapping each iteration's result in an Option along with flatMap over the original Iterator we're running upon is very helpful to get this kind collection filtering.
I didn't throw an exception upon an error, but rather logged it and moved on with a None. You could also accumulate errors and do whatever you want with them after all iterations are done, if that's helpful to you.
The # in person # Person(_, _, "us") can be used for something like "match & bind" upon the whole object in question.
As the comment to the original question noted - no need to re-instantiate the implicit Decoder upon each iteration. You can just pull it one layer up, as I did in my example.
I am playing around with Scala at the moment and the pattern matching. I have the general idea behind it and can get the basics working. My issue is with Option[]. It is possible to use pattern matching on Option[]'s?
What I am trying to do is make a little function that will take in an option[String] parameter and then based on the input return the string if its a string and a heads up if not. I am not too sure on how to go about this though, I have tried a few thing but it either gives out or in the case below will never hit the second case.
def getString(someString: Option[String]): String =
someString match {
case s: Option[String] => someString //also tried things like case: String => ...
case _ => s"no string entered" //and things like case _ => ...
}
This is the easiest way to implement your function:
def getString(someString: Option[String]): String =
someString.getOrElse("no string entered")
If you want to use match it looks like this:
def getString(someString: Option[String]): String =
someString match {
case Some(s) => s
case _ => "no string entered"
}
We'd like to run a patch/partial UPDATE with Slick (3.0.0) so that we only modify some of the fields in a record. Exactly which fields will be updated exactly will only be known at runtime.
For example, for a REST PATCH request.
Currently we run a SELECT first to get the original record then run an UPDATE but it would be nicer to do this in a single SQL statement.
Something like this:
def patchPerson(name: Option[String], age: Option[Int]) = {
people.filter(_.name === "M Odersky")
.map(p =>
(name, age) match {
case (Some(_), Some(_)) => (p.name, p.age)
case (Some(_), None) => (p.name)
case (None , Some(_)) => (p.age)
}
)
.update(
(name, age) match {
case (Some(_), Some(_)) => (name.get, age.get)
case (Some(_), None) => (name.get)
case (None , Some(_)) => (age.get)
}
)
}
(Please ignore the ugly code here)
The above does not compile with the following error message:
No matching Shape found. Slick does not know how to map the given
types. Possible causes: T in Table[T] does not match your *
projection. Or you use an unsupported type in a Query (e.g. scala
List). Required level: slick.lifted.FlatShapeLevel Source type:
Object Unpacked type: T Packed type: G
And:
not enough arguments for method map: (implicit shape:
slick.lifted.Shape[_ <: slick.lifted.FlatShapeLevel, Object, T,
G])slick.lifted.Query[G,T,Seq]. Unspecified value parameter shape.
I assume this is because Slick expects the tuple length and type to match the results for both the filter and update functions.
We've tried using the Slick heterogeneous list class but this also seems to expect the length and types to match.
Is there a way to write this in Slick so that we can update an arbitrary number of fields in a record with one database call?
Why not do the pattern matching before constructing the update query?
def patchPerson(name: Option[String], age: Option[Int]) = {
val query = people.filter(_.name === "M Odersky")
(name, age) match {
case (Some(name), Some(age)) =>
query.map(p => (p.name, p.age)).update(name, age)
case (Some(name), None) =>
query.map(p => p.name).update(name)
case (None, Some(age)) =>
query.map(p => p.age).update(age)
}
}
My best guess would be to run a plain SQL query
Even if the SQL query has 2 parts, the relational database management system (postgresql, mysql, etc) is able to tune the query under the hoods.
I'm not sure if in this case Slick is able to optimize, but in several cases it also optimizes the queries by itself.
Typical update:
def updateRecord(id: Long, field1: Int) = {
db.withSession {
self.filter(_.id === id).map(_.field1).update(field1)
}
}
Doing your type of update would require a bit more logic like you did. Don't think it's possible to simplify if you only know at runtime which fields to change. But you can force the update, using existing value for the field on the record as a fallback (may lead to more updates on DB than it should)
def updateRecord(id: Long, field1: Option[Int], field2: Option[Int]) = {
db.withSession {
self.filter(_.id === id).map(_.field1, _.field2).update(field1.getOrElse(existingValue1), field2.getOrElse(existingValue2))
}
}
You already have answers as written by #pedrorijo91 and #thirstycow but i'll try to explain why this doesnt work.
I'm havent used slick 3 but im guessing its because the map function doesnt return a consistent type for it to run update against. As a thought experiment, if you cut your call at map what do you think the type will be?
val partialQuery:??? = people.filter(_.name === "M Odersky")
.map(p =>
(name, age) match {
case (Some(_), Some(_)) => (p.name, p.age)
case (Some(_), None) => (p.name)
case (None , Some(_)) => (p.age)
}
);
val fullQuery:??? = partialQuery.update {
(name, age) match {
case (Some(_), Some(_)) => (name.get, age.get)
case (Some(_), None) => (name.get)
case (None , Some(_)) => (age.get)
}
}
The matcher returns different "shapes" at compile time which im guessing will revert to Any type.
I am struggling with using the extractor pattern in a certain use case where it seems that it could be very powerful.
I start with an input of Map[String, String] coming from a web request. This is either a searchRequest or a countRequest to our api.
searchRequest has keys
query(required)
fromDate(optional-defaulted)
toDate(optional-defaulted)
nextToken(optional)
maxResults(optional-defaulted)
countRequest has keys
query(required)
fromDate(optional-defaulted)
toDate(optional-defaulted)
bucket(optional-defaulted)
Then, I want to convert both of these to a composition type structure like so
protected case class CommonQueryRequest(
originalQuery: String,
fromDate: DateTime,
toDate: DateTime
)
case class SearchQueryRequest(
commonRequest: CommonQueryRequest,
maxResults: Int,
nextToken: Option[Long])
case class CountRequest(commonRequest: CommonQueryRequest, bucket: String)
As you can see, I am sort of converting Strings to DateTimes and Int, Long, etc. My issue is that I really need errors for invalid fromDate vs. invalid toDate format vs. invalid maxResults vs. invalid next token IF available.
At the same time, I need to stick in defaults(which vary depending on if it is a search or count request).
Naturally, with the Map being passed in, you can tell search vs. count so in my first go at this, I added a key="type" with value of search or count so that I could match at least on that.
Am I even going down the correct path? I thought perhaps using matching could be cleaner than our existing implementation but the further I go down this path, it seems to be getting a bit uglier.
thanks,
Dean
I would suggest you to take a look at scalaz.Validation and ValidationNel. It's super nice way to collect validation errors, perfect fit for input request validation.
You can learn more about Validation here: http://eed3si9n.com/learning-scalaz/Validation.html. However in my example I use scalaz 7.1 and it can be a little bit different from what described in this article. However main idea remains the same.
Heres small example for your use case:
import java.util.NoSuchElementException
import org.joda.time.DateTime
import org.joda.time.format.DateTimeFormat
import scala.util.Try
import scalaz.ValidationNel
import scalaz.syntax.applicative._
import scalaz.syntax.validation._
type Input = Map[String, String]
type Error = String
case class CommonQueryRequest(originalQuery: String,
fromDate: DateTime,
toDate: DateTime)
case class SearchQueryRequest(commonRequest: CommonQueryRequest,
maxResults: Int,
nextToken: Option[Long])
case class CountRequest(commonRequest: CommonQueryRequest, bucket: String)
def stringField(field: String)(input: Input): ValidationNel[Error, String] =
input.get(field) match {
case None => s"Field $field is not defined".failureNel
case Some(value) => value.successNel
}
val dateTimeFormat = DateTimeFormat.fullTime()
def dateTimeField(field: String)(input: Input): ValidationNel[Error, DateTime] =
Try(dateTimeFormat.parseDateTime(input(field))) recover {
case _: NoSuchElementException => DateTime.now()
} match {
case scala.util.Success(dt) => dt.successNel
case scala.util.Failure(err) => err.toString.failureNel
}
def intField(field: String)(input: Input): ValidationNel[Error, Int] =
Try(input(field).toInt) match {
case scala.util.Success(i) => i.successNel
case scala.util.Failure(err) => err.toString.failureNel
}
def countRequest(input: Input): ValidationNel[Error, CountRequest] =
(
stringField ("query") (input) |#|
dateTimeField("fromDate")(input) |#|
dateTimeField("toDate") (input) |#|
stringField ("bucket") (input)
) { (query, from, to, bucket) =>
CountRequest(CommonQueryRequest(query, from, to), bucket)
}
val validCountReq = Map("query" -> "a", "bucket" -> "c")
val badCountReq = Map("fromDate" -> "invalid format", "bucket" -> "c")
println(countRequest(validCountReq))
println(countRequest(badCountReq))
scalactic looks pretty cool as well and I may go that route (though not sure if we can use that lib or not but I think I will just proceed forward until someone says no).
Everything started from a couple of considerations:
Extractors are Scala objects that implements some unapply methods with certain peculiarities (directly from «Programming in Scala 2nd edition», I've checked)
Objects are singletons lazy initialised on the static scope
I've tried to implement a sort of «parametric extractors» under the form of case classes to try to have an elegant pattern for SHA1 checking.
I'd like to check a list of SHA1s against a buffer to match which of them apply. I'd like to write something like this:
val sha1: Array[Byte] = ...
val sha2: Array[Byte] = ...
buffer match {
case SHA1(sha1) => ...
case SHA1(sha2) => ...
...
}
Ok, it looks weird, but don't bother now.
I've tried to solve the problem by simply implementing a case class like this
case class SHA1(sha1: Array[Byte]) {
def unapply(buffer: Array[Byte]): Boolean = ...
}
and use it like
case SHA1(sha1)() =>
and even
case (SHA1(sha1)) =>
but it doesn't work: compiler fails.
Then I've a little changed the code in:
val sha1 = SHA1(sha1)
val sha2 = SHA1(sha2)
buffer match {
case sha1() => println("sha1 Match")
case sha2() => println("sha2 Match")
...
}
and it works without any issue.
Questions are:
Q1: There are any subtle implications in using such a kind of «extractors»
Q2: Provided the last example works, which syntax was I supposed to use to avoid to define temporary vals? (if any provided compiler's job with match…case expressions)
EDIT
The solution proposed by Aaron doesn't work either. A snippet:
case class SHA1(sha1: Array[Byte]) {
def unapply(buffer: Array[Byte]) = buffer.length % 2 == 0
}
object Sha1Sample {
def main(args: Array[String]) {
println("Sha1 Sample")
val b1: Array[Byte] = Array(0, 1, 2)
val b2: Array[Byte] = Array(0, 1, 2, 3)
val sha1 = SHA1(b1)
List(b1, b2) map { b =>
b match {
case sha1() => println("Match") // works
case `sha1` => println("Match") // compile but it is semantically incorrect
case SHA1(`b1`) => println("SOLVED") // won't compile
case _ => println("Doesn't Match")
}
}
}
}
Short answer: you need to put backticks around lowercase identifiers if you don't want them to be interpreted as pattern variables.
case Sha1(`sha1`) => // ...
See this question.