Getting a Scala Map from a Java Properties - scala

I was trying to pull environment variables into a scala script using java Iterators and / or Enumerations and realised that Dr Frankenstein might claim parentage, so I hacked the following from the ugly tree instead:
import java.util.Map.Entry
import System._
val propSet = getProperties().entrySet().toArray()
val props = (0 until propSet.size).foldLeft(Map[String, String]()){(m, i) =>
val e = propSet(i).asInstanceOf[Entry[String, String]]
m + (e.getKey() -> e.getValue())
}
For example to print the said same environment
props.keySet.toList.sortWith(_ < _).foreach{k =>
println(k+(" " * (30 - k.length))+" = "+props(k))
}
Please, please don't set about polishing this t$#d, just show me the scala gem that I'm convinced exists for this situation (i.e java Properties --> scala.Map), thanks in advance ;#)

Scala 2.10.3
import scala.collection.JavaConverters._
//Create a variable to store the properties in
val props = new Properties
//Open a file stream to read the file
val fileStream = new FileInputStream(new File(fileName))
props.load(fileStream)
fileStream.close()
//Print the contents of the properties file as a map
println(props.asScala.toMap)

Scala 2.7:
val props = Map() ++ scala.collection.jcl.Conversions.convertMap(System.getProperties).elements
Though that needs some typecasting. Let me work on it a bit more.
val props = Map() ++ scala.collection.jcl.Conversions.convertMap(System.getProperties).elements.asInstanceOf[Iterator[(String, String)]]
Ok, that was easy. Let me work on 2.8 now...
import scala.collection.JavaConversions.asMap
val props = System.getProperties() : scala.collection.mutable.Map[AnyRef, AnyRef] // or
val props = System.getProperties().asInstanceOf[java.util.Map[String, String]] : scala.collection.mutable.Map[String, String] // way too many repetitions of types
val props = asMap(System.getProperties().asInstanceOf[java.util.Map[String, String]])
The verbosity, of course, can be decreased with a couple of imports. First of all, note that Map will be a mutable map on 2.8. On the bright side, if you convert back the map, you'll get the original object.
Now, I have no clue why Properties implements Map<Object, Object>, given that the javadocs clearly state that key and value are String, but there you go. Having to typecast this makes the implicit option much less attractive. This being the case, the alternative is the most concise of them.
EDIT
Scala 2.8 just acquired an implicit conversion from Properties to mutable.Map[String,String], which makes most of that code moot.

In Scala 2.9.1 this is solved by implicit conversions inside collection.JavaConversions._ . The other answers use deprecated functions. The details are documented here. This is a relevant snippet out of that page:
scala> import collection.JavaConversions._
import collection.JavaConversions._
scala> import collection.mutable._
import collection.mutable._
scala> val jul: java.util.List[Int] = ArrayBuffer(1, 2, 3)
jul: java.util.List[Int] = [1, 2, 3]
scala> val buf: Seq[Int] = jul
buf: scala.collection.mutable.Seq[Int] = ArrayBuffer(1, 2, 3)
scala> val m: java.util.Map[String, Int] = HashMap("abc" -> 1, "hello" -> 2)
m: java.util.Map[String,Int] = {hello=2, abc=1}
Getting from a mutable map to an immutable map is a matter of calling toMap on it.

In Scala 2.8.1 you can do it with asScalaMap(m : java.util.Map[A, B]) in a more concise way:
var props = asScalaMap(System.getProperties())
props.keySet.toList.sortWith(_ < _).foreach { k =>
println(k + (" " * (30 - k.length)) + " = " + props(k))
}

In Scala 2.13.2:
import scala.jdk.javaapi.CollectionConverters._
val props = asScala(System.getProperties)

Looks like in the most recent version of Scala (2.10.2 as of the time of this answer), the preferred way to do this is using the explicit .asScala from scala.collection.JavaConverters:
import scala.collection.JavaConverters._
val props = System.getProperties().asScala
assert(props.isInstanceOf[Map[String, String]])

Related

Scala function does not return a value

I think I understand the rules of implicit returns but I can't figure out why splithead is not being set. This code is run via
val m = new TaxiModel(sc, file)
and then I expect
m.splithead
to give me an array strings. Note head is an array of strings.
import org.apache.spark.SparkContext
import org.apache.spark.rdd.RDD
class TaxiModel(sc: SparkContext, dat: String) {
val rawData = sc.textFile(dat)
val head = rawData.take(10)
val splithead = head.slice(1,11).foreach(splitData)
def splitData(dat: String): Array[String] = {
val splits = dat.split("\",\"")
val split0 = splits(0).substring(1, splits(0).length)
val split8 = splits(8).substring(0, splits(8).length - 1)
Array(split0).union(splits.slice(1, 8)).union(Array(split8))
}
}
foreach just evaluates expression, and do not collect any data while iterating. You probably need map or flatMap (see docs here)
head.slice(1,11).map(splitData) // gives you Array[Array[String]]
head.slice(1,11).flatMap(splitData) // gives you Array[String]
Consider also a for comprehension (which desugars in this case into flatMap),
for (s <- head.slice(1,11)) yield splitData(s)
Note also that Scala strings are equipped with ordered collections methods, thus
splits(0).substring(1, splits(0).length)
proves equivalent to any of the following
splits(0).drop(1)
splits(0).tail

Flattening nested java lists in Scala

I am working in Scala with java libraries. One of these libraries returns a list of lists. I want to flatten the list.
Example:
import scala.collection.JavaConverters._
var parentList : util.List[util.List[Int]] = null
parentList = new util.ArrayList[util.List[Int]]
parentList.asScala.flatten // error
I have used asScala converter but I'm still meeting an error.
You need to call .asScala on every inner list :
scala> parentList.asScala.map(_.asScala)
res0: scala.collection.mutable.Buffer[scala.collection.mutable.Buffer[Int]] = ArrayBuffer()
scala> parentList.asScala.map(_.asScala).flatten
res1: scala.collection.mutable.Buffer[Int] = ArrayBuffer()
Note that calling .map and then .flatten can be done in one step using .flatMap :
scala> parentList.asScala.flatMap(_.asScala)
res2: scala.collection.mutable.Buffer[Int] = ArrayBuffer()
You also need to convert the inner List[Int]:
parentList.asScala.flatMap(_.asScala)
Try like this
import scala.jdk.CollectionConverters._
parentList.asScala.flatMap.map(_.toSeq)
This will do the trick.

Scala initialize collection from Java iterable

In scala, how can I initialize a scala collection from a Java iterable, in a clean idiomatic way?
Here's somewhat lame code taking a less functional approach for that:
var collection = Seq[MyClass]()
while (iterator.hasNext) {
val asArray: Array[String] = iterator.next.toArray
val val2 = asArray(2)
val val3 = asArray(3)
collection = collection :+ new MyClass(val2, val3)
}
How can initialization of a collection from a Java iterable take place more idiomatically?
import scala.collection.JavaConverters._
val collection = iterator.asScala.map{ x =>
val asArray = x.toArray
new MyClass(asArray(2), asArray(3))
}.toIndexedSeq
Scala can convert to and from Java collections seamlessly, provided you have imported the conversion helpers like below:
import scala.collection.JavaConversions._
val jl = new java.util.ArrayList[String]()
jl.add("Hello")
jl.add("There")
val collection = j1.map{ x => new MyClass(x(2), x(3)) }.toList

How do I provide basic configuration for a Scala application?

I am working on a small GUI application written in Scala. There are a few settings that the user will set in the GUI and I want them to persist between program executions. Basically I want a scala.collections.mutable.Map that automatically persists to a file when modified.
This seems like it must be a common problem, but I have been unable to find a lightweight solution. How is this problem typically solved?
I do a lot of this, and I use .properties files (it's idiomatic in Java-land). I keep my config pretty straight-forward by design, though. If you have nested config constructs you might want a different format like YAML (if humans are the main authors) or JSON or XML (if machines are the authors).
Here's some example code for loading props, manipulating as Scala Map, then saving as .properties again:
import java.io._
import java.util._
import scala.collection.JavaConverters._
val f = new File("test.properties")
// test.properties:
// foo=bar
// baz=123
val props = new Properties
// Note: in real code make sure all these streams are
// closed carefully in try/finally
val fis = new InputStreamReader(new FileInputStream(f), "UTF-8")
props.load(fis)
fis.close()
println(props) // {baz=123, foo=bar}
val map = props.asScala // Get to Scala Map via JavaConverters
map("foo") = "42"
map("quux") = "newvalue"
println(map) // Map(baz -> 123, quux -> newvalue, foo -> 42)
println(props) // {baz=123, quux=newvalue, foo=42}
val fos = new OutputStreamWriter(new FileOutputStream(f), "UTF-8")
props.store(fos, "")
fos.close()
Here's an example of using XML and a case class for reading a config. A real class can be nicer than a map. (You could also do what sbt and at least one project do, take the config as Scala source and compile it in; saving it is less automatic. Or as a repl script. I haven't googled, but someone must have done that.)
Here's the simpler code.
This version uses a case class:
case class PluginDescription(name: String, classname: String) {
def toXML: Node = {
<plugin>
<name>{name}</name>
<classname>{classname}</classname>
</plugin>
}
}
object PluginDescription {
def fromXML(xml: Node): PluginDescription = {
// extract one field
def getField(field: String): Option[String] = {
val text = (xml \\ field).text.trim
if (text == "") None else Some(text)
}
def extracted = {
val name = "name"
val claas = "classname"
val vs = Map(name -> getField(name), claas -> getField(claas))
if (vs.values exists (_.isEmpty)) fail()
else PluginDescription(name = vs(name).get, classname = vs(claas).get)
}
def fail() = throw new RuntimeException("Bad plugin descriptor.")
// check the top-level tag
xml match {
case <plugin>{_*}</plugin> => extracted
case _ => fail()
}
}
}
This code reflectively calls the apply of a case class. The use case is that fields missing from config can be supplied by default args. No type conversions here. E.g., case class Config(foo: String = "bar").
// isn't it easier to write a quick loop to reflect the field names?
import scala.reflect.runtime.{currentMirror => cm, universe => ru}
import ru._
def fromXML(xml: Node): Option[PluginDescription] = {
def extract[A]()(implicit tt: TypeTag[A]): Option[A] = {
// extract one field
def getField(field: String): Option[String] = {
val text = (xml \\ field).text.trim
if (text == "") None else Some(text)
}
val apply = ru.newTermName("apply")
val module = ru.typeOf[A].typeSymbol.companionSymbol.asModule
val ts = module.moduleClass.typeSignature
val m = (ts member apply).asMethod
val im = cm reflect (cm reflectModule module).instance
val mm = im reflectMethod m
def getDefault(i: Int): Option[Any] = {
val n = ru.newTermName("apply$default$" + (i+1))
val m = ts member n
if (m == NoSymbol) None
else Some((im reflectMethod m.asMethod)())
}
def extractArgs(pss: List[List[Symbol]]): List[Option[Any]] =
pss.flatten.zipWithIndex map (p => getField(p._1.name.encoded) orElse getDefault(p._2))
val args = extractArgs(m.paramss)
if (args exists (!_.isDefined)) None
else Some(mm(args.flatten: _*).asInstanceOf[A])
}
// check the top-level tag
xml match {
case <plugin>{_*}</plugin> => extract[PluginDescription]()
case _ => None
}
}
XML has loadFile and save, it's too bad there seems to be no one-liner for Properties.
$ scala
Welcome to Scala version 2.10.0-RC5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_06).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import reflect.io._
import reflect.io._
scala> import java.util._
import java.util._
scala> import java.io.{StringReader, File=>JFile}
import java.io.{StringReader, File=>JFile}
scala> import scala.collection.JavaConverters._
import scala.collection.JavaConverters._
scala> val p = new Properties
p: java.util.Properties = {}
scala> p load new StringReader(
| (new File(new JFile("t.properties"))).slurp)
scala> p.asScala
res2: scala.collection.mutable.Map[String,String] = Map(foo -> bar)
As it all boils down to serializing a map / object to a file, your choices are:
classic serialization to Bytecode
serialization to XML
serialization to JSON (easy using Jackson, or Lift-JSON)
use of a properties file (ugly, no utf-8 support)
serialization to a proprietary format (ugly, why reinvent the wheel)
I suggest to convert Map to Properties and vice versa. "*.properties" files are standard for storing configuration in Java world, why not use it for Scala?
The common way are *. properties, *.xml, since scala supports xml natively, so it would be easier using xml config then in java.

What's the new way to iterate over a Java Map in Scala 2.8.0?

How does scala.collection.JavaConversions supercede the answers given in Stack Overflow question Iterating over Java collections in Scala (it doesn't work because the "jcl" package is gone) and in Iterating over Map with Scala (it doesn't work for me in a complicated test which I'll try to boil down and post here later).
The latter is actually a Scala Map question, but I think I need to know both answers in order to iterate over a java.util.Map.
In 2.8, you import scala.collection.JavaConversions._ and use as a Scala map. Here's an example (in 2.8.0.RC1):
scala> val jmap:java.util.Map[String,String] = new java.util.HashMap[String,String]
jmap: java.util.Map[String,String] = {}
scala> jmap.put("Hi","there")
res0: String = null
scala> jmap.put("So","long")
res1: String = null
scala> jmap.put("Never","mind")
res2: String = null
scala> import scala.collection.JavaConversions._
import scala.collection.JavaConversions._
scala> jmap.foreach(kv => println(kv._1 + " -> " + kv._2))
Hi -> there
Never -> mind
So -> long
scala> jmap.keys.map(_.toUpperCase).foreach(println)
HI
NEVER
SO
If you specifically want a Scala iterator, use jmap.iterator (after the conversions import).