I want to use ujson of the upickle library to extract an optional string from a json document. In some documents the json field exists, in others not.
When acessing the field and the field does not exist I get a NoSuchElementException:
val json = ujson.read(jsonString)
json("attributename").str
results in: java.util.NoSuchElementException: key not found: attributename
What is the idiomatic way to deal with optional json attributes in ujson?
If you want to return default value in case of any exception you can use Try with getOrElse:
val result = Try(json("attributename").str).getOrElse("defaultValue")
The result will be value of attributename key or defaultValue string if there is no such key.
I think the idiomatic way is to have a case class instead of going against the JSON AST manually.
In your case class you could then have an Option[String] field.
case class MyModel( attributeName: Option[String] )
implicit val rw: ReadWriter[MyModel] = macroRW
read[MyModel](jsonString)
But from the looks of it, you could do
json.obj.value.get("attributename").map(_.str)
to get an Option[String] back.
Don't forget, a json is an object.
So we can check it like this:
val json = ujson.read(jsonString)
if (json.obj.get("attributename").nonEmpty) {
json("attributename").str
...
}
And btw, you can get the keySet like this:
json.obj.keySet
According to liahoy
the canonical way is to add None defaults
for example,
import upickle.default._
case class User(name: String, age: Option[Int] = None)
implicit val userRW: ReadWriter[User] = macroRW
read[User]("""{ "name": "Picard" }""")
outputs
res0: User = User(Picard,None)
I am not sure, but, I was looking the GitHub repo here
It seems at line 62 and from line 87 till line 99, it just call x.obj(i).
It doesn't perform any check, and just call it. This lead to a java.util.NoSuchElementException because trying to access without a check. I didn't see around any version to get the Option, or to even perform a check if this value exists.
I would suggest to go via a Try/Success/Failure idiom on scala
val tryAttr = Try{json("attributename").str}
tryAttr match {
case Success(_) => doYourThing
case Failure(t: NoSuchElementException) => DoSomethingElse
}
Related
my Task is to read registrations from a file given like:
Keri,345246,2
Ingar,488058,2
Almeta,422016,1
and insert them into a list(Tuple of (String, Int, Int).
So far I wrote this:
The problem is that I don‘t understand why I can't try to cast value2 and value3 to Int even tho they should be Strings because they come from an Array of Strings. Could someone tell me, what my mistake is, I am relatively new to Scala
What is the point of using Scala if you are going to write Java code?
This is how you would properly read a file as a List of case classes.
import scala.io.Source
import scala.util.Using
// Use proper names for the fields.
final case class Registration(field1: String, field2: Int, field3: Int)
// You may change the error handling logic.
def readRegistrationsFromFile(fileName: String): List[Registration] =
Using(Source.fromFile(fileName)) { source =>
source.getLines().map(line => line.split(',').toList).flatMap {
case field1Raw :: field2Raw :: field3Raw :: Nil =>
for {
field2 <- field2Raw.toIntOption
field3 <- field3Raw.toIntOption
} yield Registration(field1 = field1Raw.trim, field2, field3)
case _ =>
None
}.toList
}.getOrElse(default = List.empty)
(feel free to ask any question you may have about this code)
In Scala, in order to convert a String to an Int you need explicit casting.
This can be achieved like this if you are sure the string can be parsed into a integer:
val values = values(1).toInt
If you cannot trust the input (and you probably should not), you can use .toIntOption which will give you a Option[Int] defined if the value was converted successfully or undefined if the string did not represent an integer.
The previous answers are correct. I would add a few more points.
saveContent is declared as a val. This is means it cannot be changed (assigned another value). You can use the Scala REPL (command-line) tool to check:
scala> val saveContent = Nil
val v: collection.immutable.Nil.type = List()
scala> saveContent = 3
^
error: reassignment to val
Instead, you could use a var, although it would be more idiomatic to have an overall pattern like the one provided by Luis Miguel's answer - with pattern-matching and a for-comprehension.
You can use the Scala REPL to check the types of the variables, too. Splitting a String will always lead to more Strings, not Ints, etc.
> val values = "a,2,3".split(",")
val values: Array[String] = Array(a, 2, 3)
> values(2)
val res3: String = 3
This is why a cast like Gael's is necessary.
Array-type access is done with parentheses and not square brackets, in Scala. See above, and http://scalatutorials.com/tour/interactive_tour_of_scala_lists for more details.
This question is based upon Scala 2.12.12
scalaVersion := "2.12.12"
using play-json
"com.typesafe.play" %% "play-json" % "2.9.1"
If I have a Json object that looks like this:
{
"UpperCaseKey": "some value",
"AnotherUpperCaseKey": "some other value"
}
I know I can create a case class like so:
case class Yuck(UpperCaseKey: String, AnotherUpperCaseKey: String)
and follow that up with this chaser:
implicit val jsYuck = Json.format[Yuck]
and that will, of course, give me both reads[Yuck] and writes[Yuck] to and from Json.
I'm asking this because I have a use case where I'm not the one deciding the case of the keys and I've being handed a Json object that is full of keys that start with an uppercase letter.
In this use case I will have to read and convert millions of them so performance is a concern.
I've looked into #JsonAnnotations and Scala's transformers. The former doesn't seem to have much documentation for use in Scala at the field level and the latter seems to be a lot of boilerplate for something that might be very simple another way if I only knew how...
Bear in mind as you answer this that some Keys will be named like this:
XXXYyyyyZzzzzz
So the predefined Snake/Camel case conversions will not work.
Writing a custom conversion seems to be an option yet unsure how to do that with Scala.
Is there a way to arbitrarily request that the Json read will take Key "XXXYyyyZzzz" and match it to a field labeled "xxxYyyyZzzz" in a Scala case class?
Just to be clear I may also need to convert, or at least know how, a Json key named "AbCdEf" into field labeled "fghi".
Just use provided PascalCase.
case class Yuck(
upperCaseKey: String,
anotherUpperCaseKey: String)
object Yuck {
import play.api.libs.json._
implicit val jsonFormat: OFormat[Yuck] = {
implicit val cfg = JsonConfiguration(naming = JsonNaming.PascalCase)
Json.format
}
}
play.api.libs.json.Json.parse("""{
"UpperCaseKey": "some value",
"AnotherUpperCaseKey": "some other value"
}""").validate[Yuck]
// => JsSuccess(Yuck(some value,some other value),)
play.api.libs.json.Json.toJson(Yuck(
upperCaseKey = "foo",
anotherUpperCaseKey = "bar"))
// => JsValue = {"UpperCaseKey":"foo","AnotherUpperCaseKey":"bar"}
I think that the only way play-json support such a scenario, is defining your own Format.
Let's assume we have:
case class Yuck(xxxYyyyZzzz: String, fghi: String)
So we can define Format on the companion object:
object Yuck {
implicit val format: Format[Yuck] = {
((__ \ "XXXYyyyZzzz").format[String] and (__ \ "AbCdEf").format[String]) (Yuck.apply(_, _), yuck => (yuck.xxxYyyyZzzz, yuck.fghi))
}
}
Then the following:
val jsonString = """{ "XXXYyyyZzzz": "first value", "AbCdEf": "second value" }"""
val yuck = Json.parse(jsonString).validate[Yuck]
println(yuck)
yuck.map(yuckResult => Json.toJson(yuckResult)).foreach(println)
Will output:
JsSuccess(Yuck(first value,second value),)
{"XXXYyyyZzzz":"first value","AbCdEf":"second value"}
As we can see, XXXYyyyZzzz was mapped into xxxYyyyZzzz and AbCdEf into fghi.
Code run at Scastie.
Another option you have, is to usd JsonNaming, as #cchantep suggested in the comment. If you define:
object Yuck {
val keysMap = Map("xxxYyyyZzzz" -> "XXXYyyyZzzz", "fghi" -> "AbCdEf")
implicit val config = JsonConfiguration(JsonNaming(keysMap))
implicit val fotmat = Json.format[Yuck]
}
Running the same code will output the same. Code ru nat Scastie.
I'm trying to encode a list of objects using Circe, something that looks similar to:
val test = Seq(MyObject("hello", None, 1, 2, None)
I'm trying to parse this using Circe:
test.asJson
But this creates the JSON object:
[
{
name: "hello",
someVal: null,
someNum: 1,
anotherNum: 2,
anotherVal: null
}
]
I've tried running asJson with .dropNullValues, but that doesn't seem to access the null values inside of the object. Is there a way to drop the null values inside of the objects?
I'm expecting something more like this:
[
{
name: "hello",
someNum: 1,
anotherNum: 2
}
]
Circe provides the function deepDropNullValues in the Json class.
Example: test.asJson.deepDropNullValues
You see field: null because circe turns Option[T] to t.asJson on Some[T] and JsonNull on None, and default case class encoder just puts all fields to the JsonObject. In a way that circe uses to encode sealed trait family, it may use these null fields to distinguish classes like
sealed trait Foo
case class Bar(a: Option[String])
case class Baz(a: Option[String], b: Option[String])
So, if you really want to drop this information and need one-way conversion with information loss, you can map resulting Json to drop all and every null field with code like that:
implicit val fooEnc: Encoder[Foo] = deriveEncoder[Foo].mapJsonObject{jsonObj => jsonObj.filter{case (k,v) => !v.isNull}}
However, you should write such a custom codec for any class you want to drop null fields. To post-process your json, you can use fold on resulting json:
val json: Json = ???
json.fold[Json](
Json.Null,
Json.fromBoolean,
{_.asJson},
{_.asJson},
{_.asJson},
{jsonObj => jsonObj.filter{case (k,v) => !v.isNull}.asJson}
)
or implement a custom folder.
The answer that was suggested by #iva-kmm is good, but it can be done better!
This mechanic is already implemented in circe, just call it:
implicit val fooEnc: Encoder[Foo] = deriveEncoder[Foo].mapJson(_.dropNullValues) // or _.deepDropNullValues
For example my case class is
case class Test(id: String, myValues: List[Item])
case class Item(id: Long, order: Long)
and I get string value like
val checkValue: String = "id"
I want sort Tests by items and I want it to look like
val test= Test("0", List(Item(0, 14), Item(1, 34))
val sortedItems = test.myValues.map(_.*checkValue*).sorted
Its about get field of class like someInstanceOfClass.checkValue
Scala is not an interpreted language, therefore you can't just use strings as variable names. The easiest way to solve your problem is to map the string value to the variable:
scala> def get(item: Items, str: String) = str match {
| case "id" => item.id
| case "order" => item.order
| }
get: (item: Items, str: String)Long
scala> test.myValues.map(get(_, checkValue)).sorted
res0: List[Long] = List(0, 1)
scala> test.myValues.map(get(_, "order")).sorted
res1: List[Long] = List(14, 34)
Of course there are more ways to solve the problem. You could use Reflection to read the name of the variable at runtime. In case you already know at compile time the name of the variable you want to read, you could also use macros to generate the code that is doing what you want. But these are both very specialized solutions, I would go with the runtime matching as shown above.
You may wish to rethink how you're going about this. What good does the string "id" actually do you? If you just need the capability to pull out a particular bit of data, why not use a function?
val f: Item => Long = _.id
Do you not want to have to type the function type over and over again? That's fine too; you can use a method to request the compiler's help filling in the type arguments:
def pick[A](x: Item => A) = x
val f = pick(_.id)
Now you can use f anywhere you would have used "id". (You can even name it id instead of f if that will help, or something that reminds you that it's actually a function that gets an id, not an id itself, like idF or getId.)
Assume I have
var mp = Map[String,String]()
.....
val n = mp("kk")
The above will throw runtime error in case key "kk" did not exist.
I expected n will be null in case key did not exist. I want n to be null if key did not exist.
What is the proper way to handle this situation in scala with a short code sample?
First of all, you probably don't really want null, as that's almost always a sign of bad coding in Scala. What you want is for n to be of type Option[String], which says that the value is either a String or is missing. The right way to do that is with the .get() method on you map
val n = mp.get("kk")
If you really do need null (for interop with Java libraries, for example), you can use .getOrElse()
val n = mp.getOrElse("kk", null)
Try this:
val valueOpt = mp.get("kk")
Your result is of type Option[String] and can be either None or Some(actualValue). You can use pattern matching to find out:
valueOpt match {
case Some(value) => println(value)
case None => println("default")
}
A more appropriate way to do that kind of things, however, is to use the methods on Option, e.g.:
println(valueOpt.getOrElse("default"))
Look for the API docs for Option there.
Edit: Note that Mapitself directly defines a getOrElse method, too, as shown in Dave's answer.
val conversionRatios:mutable.Map[String, Double] = mutable.Map[String, Double](
"USD" -> 2.0,
"CNY" -> 3.0
)
val currentRate = conversionRatios.apply(key) // it will return you value or NoSuchElementException will be thrown