Related
How to Convert Seq[Option[Map[String, Any]]] to Option[Map[String, Any]]
So skip any option that is None, and keep the valid Maps and merge them. If every option is None, then the final option should be None as well.
A flatMap along with groupMapReduce, followed by an Option filter should do the job:
val listOfMaps: List[Option[Map[String, Any]]] =
List(Some(Map("a"->"p", "b"->2)), None, Some(Map("a"->"q", "c"->"r")))
val mergedMap = listOfMaps.
flatMap(_.getOrElse(Map.empty[String, Any])).
groupMapReduce(_._1)(t => List[Any](t._2))(_ ::: _)
// mergedMap: Map[String, List[Any]] =
// Map("a" -> List("p", "q"), "b" -> List(2), "c" -> List("r"))
Option(mergedMap).filter(_.nonEmpty)
// res1: Option[Map[String, List[Any]]] =
// Some(Map("a" -> List("p", "q"), "b" -> List(2), "c" -> List("r")))
A few notes:
groupMapReduce is available only on Scala 2.13+.
If you must stick to Seq instead of List, simply replace method ::: with ++ in groupMapReduce.
It is assumed that merging of the Maps means aggregating the Map values of a common key into a List. Replace groupMapReduce with toMap if keeping only one of the Map values of a common key is wanted instead.
This solution treats Some(Map(.empty[String, Any])) the same as None.
This is a one-line solution
val in: Seq[Option[Map[String, Any]]] = ???
val out: Option[Map[String, Any]] =
in.flatten.headOption.map(_ => in.flatten.reduce(_ ++ _))
The in.flatten.headOption is a simple way of getting Some if at least one of the elements is Some or None if they are all None.
The reduce just combines all the Maps into one.
This can clearly be optimised by avoiding the duplicate in.flatten call.
Here is an example with flatten,fold, ++ and a match at the end to provide Some or None.
baz.scala
package baz
object baz {
// fixtures
val x0 = Seq[Option[Map[String, Any]]]()
val x1 = Seq[Option[Map[String, Any]]](None)
val x2 = Seq[Option[Map[String, Any]]](Some(Map("a" -> 1, "b" -> "two")))
val x3 = Seq[Option[Map[String, Any]]](Some(Map("a" -> 1, "b" -> "two")), Some(Map("c" -> 3.0)), None)
def f(x: Seq[Option[Map[String, Any]]]) =
x.flatten.fold(Map[String, Any]())((a,b) => a ++ b) match { case m if m.isEmpty => None case m => Some(m) }
}
Sample run
bash-3.2$ scalac baz.scala && scala -classpath .
Welcome to Scala 2.13.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_66).
Type in expressions for evaluation. Or try :help.
scala> import baz.baz._
import baz.baz._
scala> x0 -> f(x0)
res0: (Seq[Option[Map[String,Any]]], Option[Map[String,Any]]) = (List(),None)
scala> x1 -> f(x1)
res1: (Seq[Option[Map[String,Any]]], Option[Map[String,Any]]) = (List(None),None)
scala> x2 -> f(x2)
res2: (Seq[Option[Map[String,Any]]], Option[Map[String,Any]]) = (List(Some(Map(a -> 1, b -> two))),Some(Map(a -> 1, b -> two)))
scala> x3 -> f(x3)
res3: (Seq[Option[Map[String,Any]]], Option[Map[String,Any]]) = (List(Some(Map(a -> 1, b -> two)), Some(Map(c -> 3.0)), None),Some(Map(a -> 1, b -> two, c -> 3.0)))
scala> :quit
I am currently working on a function that takes in a Map[String, List[String]] and a String as arguments. The map contains a user Id and the IDs of films that they liked. What I need to do is, to return a List[List[String]] which contains the other movies that where liked by the user who liked the movie that was passed into the function.
The function declaration looks as follows:
def movies(m: Map[String, List[String]], mov: String) : List[List[String]]= {
}
So lets imagine the following:
val m1 : [Map[Int, List[String]]] = Map(1 ‐> List("b", "a"), 2 ‐> List("y", "x"), 3 ‐> List("c", "a"))
val movieID = "a"
movies(m1, movieId)
This should return:
List(List("b"), List("c"))
I have tried using
m1.filter(x => x._2.contains(movieID))
So that only Lists containing movieID are kept in the map, but my problem is that I need to remove movieID from every list it occurs in, and then return the result as a List[List[String]].
You could use collect:
val m = Map("1" -> List("b", "a"), "2" -> List("y", "x"), "3" -> List("c", "a"))
def movies(m: Map[String, List[String]], mov: String) = m.collect {
case (_, l) if l.contains(mov) => l.filterNot(_ == mov)
}
movies(m, "a") //List(List(b), List(c))
Problem with this approach is, that it would iterate over every movie list twice, the first time with contains and the second time with filterNot. We could optimize it tail-recursive function, which would look for element and if found just return list without it:
import scala.annotation.tailrec
def movies(m: Map[String, List[String]], mov: String) = {
#tailrec
def withoutElement[T](l: List[T], mov: T, acc: List[T] = Nil): Option[List[T]] = {
l match {
case x :: xs if x == mov => Some(acc.reverse ++ xs)
case x :: xs => withoutElement(xs, mov, x :: acc)
case Nil => None
}
}
m.values.flatMap(withoutElement(_, mov))
}
The solution from Krzysztof is a good one. Here's an alternate way to traverse every List just once.
def movies(m: Map[String, List[String]], mov: String) =
m.values.toList.flatMap{ss =>
val tpl = ss.foldLeft((false, List.empty[String])){
case ((_,res), `mov`) => (true, res)
case ((keep,res), str) => (keep, str::res)
}
if (tpl._1) Some(tpl._2) else None
}
This should work for you:
object DemoAbc extends App {
val m1 = Map(1 -> List("b", "a"), 2 -> List("y", "x"), 3 -> List("c", "a"))
val movieID = "a"
def movies(m: Map[Int, List[String]], mov: String): List[List[String]] = {
val ans = m.foldLeft(List.empty[List[String]])((a: List[List[String]], b: (Int, List[String])) => {
if (b._2.contains(mov))
b._2.filter(_ != mov) :: a
else a
})
ans
}
print(movies(m1, movieID))
}
I'm working on getting the nearest key value in both ways with a given input integer.
Example:
The defined map is as below.
val m = Map(10 -> "W",20 -> "W",30 -> "I",40 -> "A",50 -> "P",60 -> "S",70 -> "A",80 -> "A",90 -> "A",100 -> "I",110 -> "A",120 -> "E")
The keys are integers and they increase in order.
If say, 95 is given as input, then I should get the tuple output as follows
((90->"A"), (100->"I"))
scala> m.map( x=> (x._1,x._2)).toList.sortBy(_._1).filter( _._1<=95 ).last
res74: (Int, String) = (90,A)
scala> m.map( x=> (x._1,x._2)).toList.sortBy(_._1).filter( _._1>=95 ).head
res75: (Int, String) = (100,I)
scala>
The Map size will be big in real scenario(1K) and I want to avoid the sortBy(). Are there any possible foldLeft or map solutions available to this?.
Here's a pretty straight forward solution. No sorting required.
def nearest[V](m :Map[Int,V], k :Int) :Seq[(Int,V)] =
m.get(k).fold {
val (before, after) = m.keys.partition(_ < k)
Seq(before.maxOption, after.minOption).flatten.map(x => (x,m(x)))
}(v => Seq((k,v)))
testing:
val m = Map(10 -> "W",20 -> "W",30 -> "I",40 -> "A",50 -> "P",60 -> "S",70 -> "A",80 -> "A",90 -> "A",100 -> "I",110 -> "A",120 -> "E")
nearest(m, -7) //res0: Seq[(Int, String)] = List((10,W))
nearest(m, 60) //res1: Seq[(Int, String)] = List((60,S))
nearest(m, 93) //res2: Seq[(Int, String)] = List((90,A), (100,I))
nearest(m, 121) //res3: Seq[(Int, String)] = List((120,E))
Here's one approach that leverages TreeMap's sorting and range query features as shown below:
def nearestValues(m: Map[Int, String], key: Int) = {
import scala.collection.immutable.TreeMap
val tm = TreeMap(m.toSeq: _*)
Seq(tm.to(key).lastOption, tm.from(key).headOption).flatten.distinct
}
val m = Map(
10 -> "W", 20 -> "W", 30 -> "I", 40 -> "A", 50 -> "P", 60 -> "S",
70 -> "A", 80 -> "A", 90 -> "A", 100 -> "I", 110 -> "A", 120 -> "E"
)
nearestValues(m, 95)
// res1: Seq[(Int, String)] = List((90,A), (100,I))
nearestValues(m, 20)
// res2: Seq[(Int, String)] = List((20,W))
nearestValues(m, 125)
// res3: Seq[(Int, String)] = List((120,E))
Note that the above method returns a Seq rather than a Tuple to accommodate cases of exact or one-sided matches. To return a Tuple, one could go with something similar to the following:
def nearestValues(m: Map[Int, String], key: Int) = {
import scala.collection.immutable.TreeMap
val tm = TreeMap(m.toSeq: _*)
Seq(tm.to(key).lastOption, tm.from(key).headOption) match {
case Seq(None, None) => (0 -> "", 0 -> "") // Default tuple for empty Map
case Seq(x, None) => (x.get, Int.MaxValue -> "")
case Seq(None, y) => (Int.MinValue -> "", y.get)
case Seq(x, y) => (x.get, y.get)
}
}
nearestValues(m, 95)
// res1: ((Int, String), (Int, String)) = ((90,A),(100,I))
nearestValues(m, 20)
// res2: ((Int, String), (Int, String)) = ((20,W),(20,W))
nearestValues(m, 125)
// res3: ((Int, String), (Int, String)) = ((120,E),(2147483647,""))
UPDATE:
Starting Scala 2.13, methods to and from for TreeMap are replaced with rangeTo and rangeFrom, respectively.
You can use sorted collection as Leo C proposed, however that requires collection construction and then search, so complexity of such algorithm will be O(n*log n).
Approaching this task in algorithmic way, you can calculate the difference for the keys and then find the closest negative and positive values to 0. In this case complexity will be lower O(n).
Below example returns nearest keys from the left and the right excluding exact match (you may change that by changing a condition in the filter):
val data = Map(1-> "a", 5->"b")
val key = 4
val diffs = data.keys.map(_ - key)
val rightKeyOpt = diffs.filter(_ > 0).reduceOption(_ min _).map(_ + key)
val leftKeyOpt = diffs.filter(_ < 0).reduceOption(_ max _).map(_ + key)
val result = (leftKeyOpt.map(k=> k->data(k)), rightKeyOpt.map(k=> k->data(k)))
println (leftKeyOpt, rightKeyOpt)
println result
I bet this could be done in a single line if you need that very much:)
I'm trying to take a iterator of Strings, and turn it into an iterator of collections of strings based on an arbitrary splitting function.
So say I have
val splitter: String => Boolean = s => s.isEmpty
then I want it to take
val data = List("abc", "def", "", "ghi", "jkl", "mno", "", "pqr").iterator
and have
def f[A] (input: Iterator[A], splitFcn: A => Boolean): Iterator[X[A]]
where X can be any collection-like class you want, so long as it can be converted into a Seq, such that
f(data, splitter).foreach(println(_.toList))
outputs
List("abc", "def")
List("ghi", "jkl", "mno")
List("pqr")
Is there a clean way to do this, that does not require collecting the results of the input iterator entirely into memory?
This should do what you want:
val splitter: String => Boolean = s => s.isEmpty
val data = List("abc", "def", "", "ghi", "jkl", "", "mno", "pqr")
def splitList[A](l: List[A], p: A => Boolean):List[List[A]] = {
l match {
case Nil => Nil
case _ =>
val (h, t) = l.span(a => !p(a))
h :: splitList(t.drop(1), p)
}
}
println(splitList(data, splitter))
//prints List(List(abc, def), List(ghi, jkl), List(mno, pqr))
Here it is:
scala> val data = List("abc", "def", "", "ghi", "jkl", "mno", "", "pqr").iterator
data: Iterator[String] = non-empty iterator
scala> val splitter: String => Boolean = s => s.isEmpty
splitter: String => Boolean = <function1>
scala> def f[A](in: Iterator[A], sf: A => Boolean): Iterator[Iterator[A]] =
in.hasNext match {
| case false => Iterator()
| case true => Iterator(in.takeWhile(x => !sf(x))) ++ f(in, sf)
| }
f: [A](in: Iterator[A], sf: A => Boolean)Iterator[Iterator[A]]
scala> f(data, splitter) foreach (x => println(x.toList))
List(abc, def)
List(ghi, jkl, mno)
List(pqr)
UPDATE #2: Travis Brown answered another question using Scalaz-streams, an interesting package that might be helpful to you here. I am just starting to look at the package, but was quickly able to use it to read data from a file containing this:
abc
def
ghi
jkl
mno
pqr
and produce another file that looked like this:
Vector(abc, def, )
Vector(ghi, jkl, mno, )
Vector(pqr)
The library only holds the Vector being accumulated in memory. Here's my code (which should be considered dangerous, as I barely know anything about Scalaz-streams):
import scalaz.stream._
io.linesR("/tmp/a")
.pipe( process1.chunkBy(_.nonEmpty) )
.map( _.toString + "\n" )
.pipe(text.utf8Encode)
.to( io.fileChunkW("/tmp/b") )
.run.run
Key to your task is the chunkBy(_.nonEmpty), which accumulates lines into a Vector until it hits an empty line. I have no idea at this point why you have to say run twice.
Old stuff below.
UPDATE #1: Ah! I just discovered the new constraint that it not all be read into memory. This solution isn't for you, then; you'd want Iterators or Streams.
I'm guessing that you'd want to enrich Traversable. And with the function in a separate argument list, the compiler can infer the types. For performance you would probably only want to make one pass over the data. And to avoid crashing with large datasets (and for performance), you wouldn't want any recursion that is not tail-recursion. Given this enricher:
implicit class EnrichedTraversable[A]( val xs:Traversable[A] ) extends AnyVal {
def splitWhere( f: A => Boolean ) = {
#tailrec
def loop( xs:Traversable[A], group:Seq[A], groups:Seq[Seq[A]] ):Seq[Seq[A]] =
if ( xs.isEmpty ) {
groups :+ group
} else {
val x = xs.head
val rest = xs.tail
if ( f(x) ) loop( rest, Vector(), groups :+ group )
else loop( rest, group :+ x, groups )
}
loop( xs, Vector(), Vector() )
}
}
you can do this:
List("a","b","","c","d") splitWhere (_.isEmpty)
Here are some tests you might want to check out, to be sure the semantics are what you want (I personally like splits to behave this way):
val xs = List("a","b","","d","e","","f","g") //> xs : List[String] = List(a, b, "", d, e, "", f, g)
xs splitWhere (_.isEmpty) //> res0: Seq[Seq[String]] = Vector(Vector(a, b), Vector(d, e), Vector(f, g))
List("a","b","") splitWhere (_.isEmpty) //> res1: Seq[Seq[String]] = Vector(Vector(a, b), Vector())
List("") splitWhere (_.isEmpty) //> res2: Seq[Seq[String]] = Vector(Vector(), Vector())
List[String]() splitWhere (_.isEmpty) //> res3: Seq[Seq[String]] = Vector(Vector())
Vector("a","b","","c") splitWhere (_.isEmpty) //> res4: Seq[Seq[String]] = Vector(Vector(a, b), Vector(c))
I think Stream is what you want since they are evaluated lazily (not everything in memory).
def split[A](inputStream: Stream[A], splitter: A => Boolean): Stream[List[A]] = {
var accumulationList: List[A] = Nil
def loop(inputStream: Stream[A]): Stream[List[A]] = {
if (inputStream.isEmpty) {
if (accumulationList.isEmpty)
Stream.empty[List[A]]
else
accumulationList.reverse #:: Stream.empty[List[A]]
} else if (splitter(inputStream.head)) {
val outputList = accumulationList.reverse
accumulationList = Nil
if (outputList.isEmpty)
loop(inputStream.tail)
else
outputList #:: loop(inputStream.tail)
} else {
accumulationList = inputStream.head :: accumulationList
loop(inputStream.tail)
}
}
loop(inputStream)
}
val splitter = { s: String => s.isEmpty }
val list = List("asdf", "aa", "", "fw", "", "wfwf", "", "")
val stream = split(list.toStream, splitter)
stream foreach println
The output is:
List(asdf, aa)
List(fw)
List(wfwf)
EDIT:
I have not looked at it in detail, but I guess my recursive method loop could be replaced by a foldLeft or foldRight.
I am new to scala, my objective is to iterate over list and check if items in the list exist the map as its keys and if they exist return the value for that key.
I did the following:
def getMatchingValues(listItmes: List[String]) = {
for (item <- listItems) {
theMap.keys.foreach { i =>
if (i.equals(item)) {
theMap(i)
}
"NoMatch"
}
}
}
I am trying to figure if there is a better way to do this in scala?
Map has a getOrElse method which does what you want:
def getMatchingValues(listItems: List[String]) = listItems map (theMap.getOrElse(_,"NoMatch"))
At least I think that is what you want. Here's an example:
scala> val theMap = Map("a"->"A", "b" -> "B")
theMap: scala.collection.immutable.Map[String,String] = Map(a -> A, b -> B)
scala> val listItems = List("a","b","c")
listItems: List[String] = List(a, b, c)
scala> listItems map (theMap.getOrElse(_,"NoMatch"))
res0: List[String] = List(A, B, NoMatch)
A possible solution with flatMap:
/* Return Some[String] if found, None if not found, then flatten */
def getMatchingValues(listItems: List[String], theMap: Map[String, String]): List[String] =
listItems.flatMap(item => theMap.get(item))
/* Same thing with some syntactic sugar */
def getMatchingValuesSmartass(listItems: List[String], theMap: Map[String, String]): List[String] =
listItems flatMap theMap.get
val l = List("1", "3", "5", "7")
val m = Map("5" -> "five", "2" -> "two", "1" -> "one")
getMatchingValues(l, m)
getMatchingValuesSmartass(l, m)
You could use the map.get method and handle the result with pattern matching
list.map { x => map.get(x) match {
case None => "No match"
case Some(i) => (x, i)
}}
The above code returns a list of pairs where each pair represents the elements of the list and the value associated in the map ("No match" if not found)
If I was you, I would do two steps. Given this Map:
val map = Map("a" -> "b", "b" -> "c", "c" -> "d", "d" -> "e")
and this List:
val list = List("a", "c", "e")
At first I would map an Option value to every item in the List. Giving you if there is a value for the item.
val mapped = list.map(item => item -> map.get(item))
This will give you this:
mapped: List[(String, Option[String])] =
List(("a",Some("b")), ("c",Some("d")), ("e", None))
Calling get on the map returns a wrapped result. If there is a result, you will get the result wrapped in a Some. Otherwise you will get a None. Both are subclasses of Option Option is a closure-construct, that provides you a null-value without having to deal with null. Now you are able to map again, to reach your goal.
val result = mapped.map(tuple => tuple._1 -> tuple._2.getOrElse("No match"))
result: List[(String, String)] = List(("a","b"), ("c","d"), ("e","No match"))
getOrElse extracts the value of a Some or falls back to the parameter if it is a None.
To make it look more professional, we can write this expression in one line ;)
val result = list.map(item => item -> map.get(item).getOrElse("No match"))
This will give you the exact same result.