I have a JSON string which is parsed and a typecaseted to a map. I'm using this map to get a List[Map[String, Any]]. Here to make my code error free I have used getOrElse while type casting.
JSON string looks similar to
{
"map-key" : [
{
"list-object-1-key" : "list-object-1-value"
},
{
"list-object-2-key" : "list-object-2-value"
},
]
}
My code
val json = JSON.parseFull(string) match {
case Some(e) =>
val list = e.asInstanceOf[Map[String, Any]]
.getOrElse("map-key", List[Map[String,Any]]) // Error here
val info = list.asInstanceOf[List[Map[String, Any]]]
//iterate over each element in the list and perform my operations
case None => string
}
I can understand that whenever there is no result present in list object then info object is repeated code.
How can I improve this programme by giving the default value to list object?
Do it in more functional way, without asInstanceOf:
val parsed = JSON.parseFull(string)
parsed match {
case Some(e: Map[String, Any]) =>
e.get("map-key") match {
case Some(a: List[Any]) =>
a.foreach {
case inner: Map[String, Any] => println(inner.toList)
}
case _ =>
}
case None => string
}
Your default value is wrong. You're passing a type, not an empty list.
e.asInstanceOf[Map[String, Any]].getOrElse("map-key", List.empty[Map[String,Any]])
Unfortunately i don't have the environment at this machine but try something like that
first thing you need to convert json to map
def jsonStrToMap(jsonStr: String): Map[String, Any] = {
implicit val formats = org.json4s.DefaultFormats
parse(jsonStr).extract[Map[String, Any]]
}
and the second thing you will need to iterate over map to get values of list
val list= jsonStrToMap.map{ case(k,v) => (k.getBytes, v) }. toList
Related
I have following HOCON config:
a {
b.c.d = "val1"
d.f.g = "val2"
}
HOCON represents paths "b.c.d" and "d.f.g" as objects. So, I would like to have a reader, which reads these configs as Map[String, String], ex:
Map("b.c.d" -> "val1", "d.f.g" -> "val2")
I've created a reader and trying to do it recursively:
import scala.collection.mutable.{Map => MutableMap}
private implicit val mapReader: ConfigReader[Map[String, String]] = ConfigReader.fromCursor(cur => {
def concat(prefix: String, key: String): String = if (prefix.nonEmpty) s"$prefix.$key" else key
def toMap(): Map[String, String] = {
val acc = MutableMap[String, String]()
def go(
cur: ConfigCursor,
prefix: String = EMPTY,
acc: MutableMap[String, String]
): Result[Map[String, Object]] = {
cur.fluent.mapObject { obj =>
obj.value.valueType() match {
case ConfigValueType.OBJECT => go(obj, concat(prefix, obj.pathElems.head), acc)
case ConfigValueType.STRING =>
acc += (concat(prefix, obj.pathElems.head) -> obj.asString.right.getOrElse(EMPTY))
}
obj.asRight
}
}
go(cur, acc = acc)
acc.toMap
}
toMap().asRight
})
It gives me the correct result but is there a way to avoid MutableMap here?
P.S. Also, I would like to keep implementation by "pureconfig" reader.
The solution given by Ivan Stanislavciuc isn't ideal. If the parsed config object contains values other than strings or objects, you don't get an error message (as you would expect) but instead some very strange output. For instance, if you parse a typesafe config document like this
"a":[1]
The resulting value will look like this:
Map(a -> [
# String: 1
1
])
And even if the input only contains objects and strings, it doesn't work correctly, because it erroneously adds double quotes around all the string values.
So I gave this a shot myself and came up with a recursive solution that reports an error for things like lists or null and doesn't add quotes that shouldn't be there.
implicit val reader: ConfigReader[Map[String, String]] = {
implicit val r: ConfigReader[String => Map[String, String]] =
ConfigReader[String]
.map(v => (prefix: String) => Map(prefix -> v))
.orElse { reader.map { v =>
(prefix: String) => v.map { case (k, v2) => s"$prefix.$k" -> v2 }
}}
ConfigReader[Map[String, String => Map[String, String]]].map {
_.flatMap { case (prefix, v) => v(prefix) }
}
}
Note that my solution doesn't mention ConfigValue or ConfigReader.Result at all. It only takes existing ConfigReader objects and combines them with combinators like map and orElse. This is, generally speaking, the best way to write ConfigReaders: don't start from scratch with methods like ConfigReader.fromFunction, use existing readers and combine them.
It seems a bit surprising at first that the above code works at all, because I'm using reader within its own definition. But it works because the orElse method takes its argument by name and not by value.
You can do the same without using recursion. Use method entrySet as following
import scala.jdk.CollectionConverters._
val hocon =
"""
|a {
| b.c.d = "val1"
| d.f.g = val2
|}""".stripMargin
val config = ConfigFactory.load(ConfigFactory.parseString(hocon))
val innerConfig = config.getConfig("a")
val map = innerConfig
.entrySet()
.asScala
.map { entry =>
entry.getKey -> entry.getValue.render()
}
.toMap
println(map)
Produces
Map(b.c.d -> "val1", d.f.g -> "val2")
With given knowledge, it's possible to define a pureconfig.ConfigReader that reads Map[String, String] as following
implicit val reader: ConfigReader[Map[String, String]] = ConfigReader.fromFunction {
case co: ConfigObject =>
Right(
co.toConfig
.entrySet()
.asScala
.map { entry =>
entry.getKey -> entry.getValue.render()
}
.toMap
)
case value =>
//Handle error case
Left(
ConfigReaderFailures(
ThrowableFailure(
new RuntimeException("cannot be mapped to map of string -> string"),
Option(value.origin())
)
)
)
}
I did not want to write custom readers to get a mapping of key value pairs. I instead changed my internal data type from a map to list of pairs (I am using kotlin), and then I can easily change that to a map at some later internal stage if I need to. My HOCON was then able to look like this.
additionalProperties = [
{first = "sasl.mechanism", second = "PLAIN"},
{first = "security.protocol", second = "SASL_SSL"},
]
additionalProducerProperties = [
{first = "acks", second = "all"},
]
Not the best for humans... but I prefer it to having to build custom parsing components.
I have a quite odd problem to solve, I have a String, a custom Type and a Map of Maps.
The string needs to have a few values replaced based on mapping between a value in custom type (which is a key in the map of maps).
This is the current structure:
case class Students(favSubject: String)
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val studentInfo: List[Students] = List(Students("English"))
val data: String = "John is the favourite hobby"
I tried the following:
mapping.foldLeft(data){ case (outputString, (studentName, favSubject)) => outputString.replace(studentName, favSubject.getOrElse(studentInfo.map(x => x.favSubject).toString, "")) }
What I need to get is:
"Soccer is the favourite hobby"
What I get is:
" is the favourite hobby"
So looks like I am getting the map of maps traversal right but the getOrElse part is having issues.
What I would do, would be to first change the structure of mappings so it makes more sense for the problem.
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val mapping2 =
mapping.iterator.flatMap {
case (student, map) => map.iterator.map {
case (info, value) => (info, student, value)
}
}.toList
.groupBy(_._1)
.view
.mapValues { group =>
group.iterator.map {
case (_, student, value) => student -> value
}.toList
}.toMap
// mapping2: Map[String, List[(String, String)]] = Map("English" -> List(("John", "Soccer")))
Then I would just traverse the students informativo, making all the necessary replacements.
final case class StudentInfo(favSubject: String)
val studentsInformation: List[StudentInfo] = List(StudentInfo("English"))
val data: String = "John is the favourite hobby"
val result =
studentsInformation.foldLeft(data) { (acc, info) =>
mapping2
.getOrElse(key = info.favSubject, default = List.empty)
.foldLeft(acc) { (acc2, tuple) =>
val (key, replace) = tuple
acc2.replace(key, replace)
}
}
// result: String = "Soccer is the favourite hobby"
When you map() a List, you get a List back. It's toString has a format "List(el1,el2,...)". Surely you cannot use it as a key for your sub-map, you would want just el1.
Here is a version of the working code. It might not be a solution you are looking for(!), just a solution to your question:
case class Students(favSubject: String)
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val studentInfo: List[Students] = List(Students("English"))
val data: String = "John is the favourite hobby"
val res = mapping.foldLeft(data) {
case (outputString, (studentName, favSubjectDict)) =>
outputString.replace(
studentName,
favSubjectDict.getOrElse(studentInfo.map(x => x.favSubject).head, "?")
)
}
println(s"$res") //prints "Soccer is the favourite hobby"
val notMatchingSubject = studentInfo.map(x => x.favSubject).toString
println(s"Problem in previous code: '$notMatchingSubject' !== 'English'")
Try it here: https://scastie.scala-lang.org/flQNRrUQSXWPxSTXOPPFgA
The issue
It is a bit unclear why StudentInfo is a List in this form... If I guessed, it was designed to be a list of StudentInfo containing both, name and favSubject and you would need to search it by name to find favSubject. But it is just a guess.
I went with the simplest working solution, to get a .head (first element) of the sequence from the map. Which will always be "English" even if you add more Studends to the list.
I have a function in my EmployeesRepository class with the following signature:
def findAllEmployeesById(ids: List[String]): Array[RedisFuture[String]] {...}
I am basically trying to pass this to an EmployeeREST class which has a GET method that takes in a List[String] (the employee ids) and is supposed to return a JSON array of corresponding employees retrieved from Redis.
Am I allowed to pass in an Array[RedisFuture[String]]? Is there a way for me to actually implement this or am I thinking about it the wrong way? Is there a different way to achieve the functionality I'm trying to implement here?
Extending some of the comments to the question: you can first convert your RedisFuture to a scala Future using the converter:
import scala.compat.java8.FutureConverters.toScala
val redisFutureToScalaFuture : RedisFuture[String] => Future[String] = toScala
This can be used to convert the entire RedisFuture Array:
val redisArrayToScala : Array[RedisFuture[String]] => Array[Future[String]] =
_ map redisFutureToScalaFuture
Now, the Future.sequence function will be handy to unwrap the Futures inside of the Array:
val unwrapArray : Array[Future[String]] => Future[Array[String]] = Future.sequence
Combine all of these together along with the original query function:
val idsToFuture : List[String] => Future[Array[String]] =
(findAllEmployeesById _) andThen redisArrayToScala andThen unwrapArray
Finally, the idsToFuture can be used within a Directive:
val entityToList : RequestEntity => List[String] = ???
val arrayToResponse : Array[String] => String = ???
val route : Route =
get {
extractRequestEntity { entity =>
onComplete(idsToFuture(entityToList(entity))) {
case Success(arr) => complete(200 -> arrayToResponse(arr))
case Failure(ex) => complete(500 -> ex.toString)
}
}
}
I'm new to the Scala landscape after spending the last 10 years in Java and the last ~year in Groovy. Hi Scala!
For the life of me I can't seem to get the following code snippet to compile, and its just complicated enough to the point where the Google Gods aren't helping me.
I have a map that will contain Strings for keys and Lists of Tuples for values. The tuples will be a String-Long pair. In Groovy this would look like:
Map<String,List<Tuple2<String,Long>>> data = [:]
I need to be able to add and modify keys and values for this map. Specifically, I need to:
Add to the List of Tuples for existing keys
If a key doesn't exist, instantiate a new List of Tuples, and then add the key and list as a map entry
In Groovy this would look like:
Map<String,List<String,Long>> data = [:]
def addData(String key, String message) {
Long currTime = System.currentTimestampInMillis()
Tuple2<String,Long> tuple = new Tuple2<String,Long>(message, tuple)
if(data.contains(key)) {
data.key << tuple
} else {
data[key] = new List<Tuple2<String,Long>>()
data.key << tuple
}
}
I'm trying to do this in Scala, albeit unsuccessfully.
My best attempt thus far:
object MapUtils {
// var data : Map[String,ListBuffer[(String,Long)]] = Map()
val data = collection.mutable.Map[String, ListBuffer[(String, Long)]]()
def addData(key : String, message : String) : Unit = {
val newTuple = (message, System.currentTimeMillis())
val optionalOldValue = data.get(key)
optionalOldValue match {
case Some(olderBufferList) => olderBufferList += newTuple
case None => data
.put(key, ListBuffer[(String, Long)](newTuple))
}
}
}
Complains with this compiler error on the case Some(olderBufferList) => olderBufferList += newTuple line:
value += is not a member of Any
Any ideas what I can do to get this compiling & working?
You are missing an import for ListBuffer. The following code works perfectly fine in 2.9.1 (tested on TryScala), 2.11.7 (tested on IDEOne) and 2.11.8. Note the only addition is the first line adding the import:
import collection.mutable.ListBuffer
object MapUtils {
// var data : Map[String,ListBuffer[(String,Long)]] = Map()
val data = collection.mutable.Map[String, ListBuffer[(String, Long)]]()
def addData(key : String, message : String) : Unit = {
val newTuple = (message, System.currentTimeMillis())
val optionalOldValue = data.get(key)
optionalOldValue match {
case Some(olderBufferList) => olderBufferList += newTuple
case None => data
.put(key, ListBuffer[(String, Long)](newTuple))
}
}
}
MapUtils.addData("123", "message 1")
MapUtils.addData("456", "message 2")
MapUtils.data
//=> Map(456 -> ListBuffer((message 2,1472925061065)), 123 -> ListBuffer((message 1,1472925060926)))
The short version for your needs will be:
val map = mutable.Map[String, ListBuffer[(String, Long)]]()
map.put(key, map.getOrElse(key, ListBuffer[(String, Long)]()) += ((message, System.currentTimeMillis())))
You have some syntax issues with your code, If I'll try to change addData it would look like this:
def addData(key : String, message : String) : Unit = {
val newTuple = (message, System.currentTimeMillis())
val optionalOldValue = map.get(key)
optionalOldValue match {
case Some(olderBufferList) => olderBufferList += newTuple
case None => map.put(key, ListBuffer[(String, Long)](newTuple))
}
}
I have the following url : http://localhost/api/books/?bookId=21&bookId=62?authorId=2
I want to retrieve all the bookId values with Scala and then use Squeryl to do a fetch in a the database.
I'm using the PlayFrameWork as the WebServer, so here's my code :
val params = request.queryString.map { case (k, v) => k -> v(0) } // Retrieve only one the first occurence of a param
So params.get("bookId") will only get the last value in the bookId params. e-g : 62.
To retrieve all my bookId params i tried this :
val params = request.queryString.map { case (k, v) => k -> v } so i can get a Seq[String], but what about the authorId which is not a Seq[String]? .
At the end i want to fetch the bookIds and authorId in my DB using Squeryl :
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map(bookIds: Seq[String] => b.bookId in bookIds))
In my controller i get the params and open the DB connection :
val params = request.queryString.map { case (k, v) => k -> v(0) }
DB.withTransaction() { where(Library.whereHelper(params)}
In my model i use the queries :
def whereHelper(params : Map[String,String]) = {
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map{bookIds: Seq[String] => b.bookId in bookIds})
}
Since bookIds is a list, i need to use the Seq[String]. There's a way to use request.queryString.map { case (k, v) => k -> v } for both a string (authorId) and a list of strings (bookIds) ?
Thanks,
If I really understand what you are trying to do, you want to know how to get the parameters from queryString. This is pretty simple and you can do the following at your controller:
def myAction = Action { request =>
// get all the values from parameter named bookId and
// transforming it to Long. Maybe you don't want the map
// and then you can just remove it.
val bookIds: Seq[Long] = request.queryString("bookId").map(_.toLong)
// Notice that now I'm using getQueryString which is a helper
// method to access a queryString parameter. It returns an
// Option[String] which we are mapping to a Option[Long].
// Again, if you don't need the mapping, just remove it.
val authorId: Option[Long] = request.getQueryString("authorId").map(_.toLong)
DB.withTransaction() { where(Library.whereHelper(authorId, bookIds) }
// Do something with the result
}
At your model you will have:
def whereHelper(authorId: Option[Long], booksId: List[Long]) = authorId match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in bookIds)
case None =>
(b.bookId in bookIds)
}
I've left explicit types to help you understand what is happen. Now, since you have both values, you can just use the values at your query.
Edit after chat:
But, since you want to receive a params: Map[String, Seq[String]] at your models and is just having problems about how to get the authorId, here is what you can do:
def whereHelper(params: Map[String, Seq[String]]) = {
// Here I'm being defensive to the fact that maybe there is no
// "booksIds" key at the map. So, if there is not, an Seq.empty
// will be returned. map method will run only if there is something
// at the Seq.
val booksIds = params.getOrElse("booksIds", Seq.empty).map(_.toLong)
// The same defensive approach is being used here, and also getting
// the head as an Option, so if the Seq is empty, a None will be
// returned. Again, the map will be executed only if the Option
// is a Some, returning another Some with the value as a Long.
val authorId = params.getOrElse("authorId", Seq.empty).headOption
authorId.map(_.toLong) match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in booksIds)
case None =>
(b.bookId in booksIds)
}
}
Of course, more parameters you have, more complicated this method will be.