Making a folder/files and writing to them in scala? [duplicate] - scala

This question already has answers here:
How to write to a file in Scala?
(19 answers)
Closed 9 years ago.
How can I make files and folders in scala and write to them? I understand how it works in other languages, but Scala is a bit trickier :/
This is different than the other question because it also asks how to make directories and files in Scala.

Paul Phillips just mentioned parenthetically on ML, "I wrote a library which does a bunch of wrapping of java.nio.file.Path", so you should wait to see what he came up with.
In the meantime, his previous work on java.io has wandered like a restless spirit but now haunts:
scala> import reflect.io._
import reflect.io._
scala> val f = File("my-test.txt")
f: scala.reflect.io.File = my-test.txt
scala> f writeAll ("hello", ",", " ", "world", "\n")
scala> :q
apm#mara:~/tmp$ cat my-test.txt
hello, world
There are also abstractions for Path and Directory.
scala> val d = Directory(".")
d: scala.reflect.io.Directory = .
scala> d.list
res0: Iterator[scala.reflect.io.Path] = non-empty iterator
scala> d.deepList()
res2: Iterator[scala.reflect.io.Path] = non-empty iterator
scala> d.deepFiles.toList
res4: List[scala.reflect.io.File] = List(./mkarray.scala, ./for29.scala, ./nofunc-wrapped.scala, ./inlined.scala, ./xmlex.scala, ./pet.scala, ./lookup.scala, ./CompilerTool.scala, ./badvargs.scala, ./tz0.scala, ./discontinuations.jar, ./serious.scala, ./badjunk.scala, ./version.scala, ./shapeless_2.11.jar, ./callbacks.scala, ./delayed.scala, ./privctor0.scala, ./nestedtags.scala, ./hw-repl.scala, ./schema-one.txt, ./tstest.scala, ./badimp.scala, ./matchprim.scala, ./sxemata-all.txt, ./arrow.scala, ./continuations2.jar, ./inliner.scala, ./oneq.scala, ./impmag.scala, ./enumuser.scala, ./auto.save, ./xmlregex.scala, ./strtyp.scala, ./orelse.scala, ./scalac-plugin.xml, ./futuremap.scala, ./applied.save, ./looker.scala, ./pat1, ./sounds.scala, ./shapeless_2.10.jar, ./funkstr.scala, ./stupid....
scala> val p = Path(".")
p: scala.reflect.io.Path = .
scala> val f = p / "mkarray.scala"
f: scala.reflect.io.Path = ./mkarray.scala

Related

Spark Shell allowing to redeclare the same immutable variable [duplicate]

This question already has answers here:
Why is it possible to declare variable with same name in the REPL?
(2 answers)
Closed 3 years ago.
I am working with Spark-shell for Scala and found a strange behaviour in Spark-shell REPL which is not there if i use any IDE.
I can declare the same immutable variable again and again in REPL, but the same is not allowed in IDE.
Here is the code in REPL:
scala> val rdd = sc.textFile("README.md")
rdd: org.apache.spark.rdd.RDD[String] = README.md MapPartitionsRDD[5] at textFile at <console>:24
scala> val rdd = sc.textFile("README.md")
rdd: org.apache.spark.rdd.RDD[String] = README.md MapPartitionsRDD[7] at textFile at <console>:24
scala> val rdd = sc.textFile("README.md")
rdd: org.apache.spark.rdd.RDD[String] = README.md MapPartitionsRDD[9] at textFile at <console>:24
scala> val rdd = sc.textFile("README.md")
rdd: org.apache.spark.rdd.RDD[String] = README.md MapPartitionsRDD[11] at textFile at <console>:24
And here is the same thing I am trying in Eclipse IDE, and it shows compile time error:
Is there anything i am missing any configuration for Spark-shell REPL?
Or is it the expected behaviour?
In your REPL, your code is actually translated as follows:
object SomeName {
val rdd = sc.textFile("README.md")
}
object Some_Other_Name {
val rdd = sc.textFile("README.md")
}
Since both your rdd vals are defined in separate Singletons, there is no name collision between them. and since this happens behind the scenes in the REPL, you feel as if you are making reassignments to the same val.
In an IDE, all our code is written inside Classes or Scala singletons(Objects), so in the same scope (inside the resp. Class/Object) you are returned an error.

Get objects out of scala interpreter

I spent hours investigating one topic. I am definitely out of my depth here. What I want is to run the scala interpreter programmatically and be able to extract object values from the interpreter. for example, if I send
val a = 1
val b = a + 1
I want to be able to read out b as an Int, not just a string printed out like
b = 2
The source code is dense. So far I don't see any part which would allow such an extraction. Any experts here who can give me a tip or tell me this is utter nonsense?
How do I get typed objects out of the scala interpreter between sessions?
Use JSR 223.
Welcome to Scala version 2.11.7 [...]
scala> import javax.script._
import javax.script._
scala> val engine = (new ScriptEngineManager).getEngineByName("scala")
engine: javax.script.ScriptEngine = scala.tools.nsc.interpreter.IMain#4233e892
scala> engine.eval("val a = 1")
res0: Object = 1
scala> engine.eval("val b = a + 1")
res1: Object = 2
scala> engine.eval("b").asInstanceOf[Int]
res2: Int = 2

How to explain that "Set(someList : _*)" results the same as "Set(someList).flatten"

I found a piece of code I wrote some time ago using _* to create a flattened set from a list of objects.
The real line of code is a bit more complex and as I didn't remember exactly why was that there, took a bit of experimentation to understand the effect, which is actually very simple as seen in the following REPL session:
scala> val someList = List("a","a","b")
someList: List[java.lang.String] = List(a, a, b)
scala> val x = Set(someList: _*)
x: scala.collection.immutable.Set[java.lang.String] = Set(a, b)
scala> val y = Set(someList).flatten
y: scala.collection.immutable.Set[java.lang.String] = Set(a, b)
scala> x == y
res0: Boolean = true
Just as a reference of what happens without flatten:
scala> val z = Set(someList)
z: scala.collection.immutable.Set[List[java.lang.String]] = Set(List(a, a, b))
As I can't remember where did I get that idiom from I'd like to hear about what is actually happening there and if there is any consequence in going for one way or the other (besides the readability impact)
P.S.: Maybe as an effect of the overuse of underscore in Scala language (IMHO), it is kind of difficult to find documentation about some of its use cases, specially if it comes together with a symbol commonly used as a wildcard in most search engines.
_* is for expand this collection as if it was written here literally, so
val x = Set(Seq(1,2,3,4): _*)
is the same as
val x = Set(1,2,3,4)
Whereas, Set(someList) treats someList as a single argument.
To lookup funky symbols, you could use symbolhound

Not sure where my assignment is going

I ran into some issues today making assignments to a var field in a case class instance stored in a map. Here's a simple session in the repl demonstrating the problem:
scala> case class X(var x: Int)
defined class X
scala> val m = Map('x -> X(1))
m: scala.collection.immutable.Map[Symbol,X] = Map('x -> X(1))
scala> m
res0: scala.collection.immutable.Map[Symbol,X] = Map('x -> X(1))
scala> m('x).x = 7
scala> m
res1: scala.collection.immutable.Map[Symbol,X] = Map('x -> X(1))
scala> val x = m('x)
x: X = X(1)
scala> x.x = 7
x.x: Int = 7
scala> x
res2: X = X(7)
scala> m
res3: scala.collection.immutable.Map[Symbol,X] = Map('x -> X(7))
scala> m('x).x_=(8)
scala> m
res5: scala.collection.immutable.Map[Symbol,X] = Map('x -> X(8))
The first attempt at assignment does nothing. However, storing the instance in a val and then doing the assignment works, as does directly calling the assignment method for the field.
I'm using Scala 2.9.2.
If this is expected behavior, it would be nice if someone could explain it to me because I can't seem to make sense of it right now. If this is a bug then that would be good to know as well.
Either way, it would also be interesting to know where that first m('x).x = 7 assignment is going. I assume something is getting mutated somewhere—I just have no idea what that something could be.
Update: It looks like this only happens in the repl. I just tried compiling the code and the assignment happens as expected. So, what is the repl doing to my assignment?
This seems to be a bug. If one executes this with a 2.10 nightly an error message is thrown:
scala> m('x).x = 7
<console>:10: error: ')' expected but string literal found.
+ "m(scala.Symbol("x")).x: Int = " + `$ires0` + "\n"
^
I created a ticket for this.

Elegant way to express xs.sort.head

Several combinations of methods on a collection can be expressed more succinctly in Scala. For example, xs.filter(f).headOption can be expressed as xs.find(f), and xs.map.filter can usually be better expressed through xs.collect.
I find myself writing xs.sortWith(f).head, and this feels to me like the sort of thing that could be expressed as a single method, "find me the least element in this collection, according to this sorting function".
However, I can't see any obvious methods on Seq or TraversableLike. Is there a single method that captures my intent, or is .sort.head the more elegant way to find the "least" element?
scala> val xs = List("hello", "bye", "hi")
xs: List[java.lang.String] = List(hello, bye, hi)
scala> xs.sortWith(_.length < _.length).head
res10: java.lang.String = hi
scala> xs.min(Ordering.fromLessThan[String](_ > _))
res11: java.lang.String = hi
scala> xs.min(Ordering.by((_: String).length))
res12: java.lang.String = hi
scala> xs.minBy(_.length)
res13: java.lang.String = hi