I have a set of names in a file. I need to implement a Generator that iterates through them continually. However, the code is hanging indefinitely at if (iter.hasNext) after the first pass.
Gen code
var asStream = getClass.getResourceAsStream("/firstnames/female/en_US_sample.txt")
var source: BufferedSource = Source.fromInputStream(asStream)
var iter: Iterator[String] = Iterator.continually(source.getLines()).flatten
val genLastName: Gen[String] = {
genCannedData
}
def genCannedData: Gen[String] = {
println("Generating names: " + iter)
Gen.delay {
if (iter.hasNext) {
println("In if")
Gen.const(iter.next)
}
else {
println("In else")
Gen.const(iter.next)
}
}
}
Sample Property test
property("FirstNames") = {
forAll(genLastName) {
a => {
println(a)
a == a
}
}
}
en_US_sample.txt file contents
Abbie
Abby
Abigail
Ada
Adah
EDIT- Temporary working code
The following code works if I recreate the iterator but I was wondering why Iterator.continually is hanging?
def genCannedData: Gen[String] = {
Gen.delay {
if (iter.hasNext) {
Gen.const(iter.next)
}
else {
asStream = getClass.getResourceAsStream("/firstnames/female/en_US_sample.txt")
source = Source.fromInputStream(asStream)
iter = source.getLines()
Gen.const(iter.next)
}
}
}
After first iteration, an iterator returned by source.getLines() returns false for hasNext, which means an empty iterator.
Iterator.continually() continually evaluate source.getLines() expecting a next iterator, but it continues to return an empty iterator. Then it forms an infinite loop.
Related
My problem statement is as below:
1) I have a path till a folder. I have to traverse using that path. Check whether there are any subfolders and files in that path. If yes, match the contents of the folder(Lists) with the array. If it matches take it as a new path.
2) Using new path. List down the files in that path.
I am able to do everything except comparing lists and array. Below is my code:
import java.text.SimpleDateFormat
import java.util.Calendar
import java.io.File
class ListingDirectories {
def getListOfDirectories(dir: String): List[File] = {
val d = new File(dir)
if (d.exists && d.isDirectory) {
d.listFiles().filter(_.isDirectory()).toList
} else {
List[File]()
}
}
def getListOfFiles(dir: String): List[File] = {
val d = new File(dir)
if (d.exists && d.isDirectory()) {
d.listFiles().filter(_.isFile()).toList
} else {
List[File]()
}
}
}
object FirstSample {
def main(args: Array[String]) {
val ld = new ListingDirectories()
val directoriesList = ld.getListOfDirectories("C:/Users/Siddheshk2/Desktop/CENSUS").toList
println(directoriesList + "\n")
val directoriesListReplaced = directoriesList.toString().replace("//", "/")
// println(directoriesListReplaced.indexOf("C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE"))
var finalString = ""
var s = Array("C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE")
for (x <- s) {
if (x.equals(directoriesListReplaced)) {
finalString = s(0)
} else {
println("No matching strings")
}
}
val filesList = ld.getListOfFiles(finalString)
println(filesList.toString())
}
}
I just need to compare the values from the list and array and take it as a new path in finalString variable in order to pass in the next method which is getListOfFiles. I figured out since I am returning List[file] in methods I am not able to access the elements inside it. Can anyone help me to understand where am I going wrong? TIA
Your directoriesListReplaced will be a string which looks like "List(C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE,C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE1,C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE2)" and s won't equal it. It isn't at all clear what you want to do with directoriesListReplaced; maybe it should just be
for (x <- s) {
if (directoriesList.contains(x)) {
...
} else {
println("No matching strings")
}
}
After lot of struggle, I solved it using below code:
import java.io.File
object PlayingWithLists {
def main(ar: Array[String]) {
var s = "C:/Users/Siddheshk2/Desktop/CENSUS/SAMPLE2"
var finalValue = ""
var valuesReplaced = ""
var filePath = ""
for (file <- new File("C:/Users/Siddheshk2/Desktop/CENSUS").listFiles) {
valuesReplaced = file.toString.replace("\\", "/")
if (valuesReplaced.contains(s.trim)) {
finalValue = file.toString
} else {
}
}
for (file <- new File(finalValue).listFiles) {
filePath = file.toString.trim
}
}
}
I have a Scala function as shown below. Input neighborhood is array of strings. However, sometimes it (i.e. neighborhood) can be empty. In that case I get "ArrayIndexOutOfBoundsException", which is understandable. I want to avoid this exception. I mean, my code has to skip this error and move on to the next job (not shown here).
I tried this:
if(neighborhood.isEmpty){
true
} else {
val key = neighborhood(0)
neighborhood
.filterNot { _.equals(key) }
.combinations(k - 1)
}
But IntelliJ shows 'type mismatch between iterator and boolean.'
How to deal with this? I am newbie in Scala. Thanks!
Here is original function:
private def scanData(neighborhood: Array[String], k: Int): Iterator[Array[String]] = {
val key = neighborhood(0)
neighborhood
.filterNot { _.equals(key) }
.combinations(k - 1)
}
```scala
You can make use of headOption for a clean approach.
neighborhood.headOption.map { key =>
neighborhood.tail
.filterNot(_ == key)
.combinations(k-1)
}.getOrElse(Iterator.empty)
Use Option. You may find information here. Here you are en example :
object Run {
def main(args:Array[String]):Unit = {
val neighborhood:Array[String] = Array("1", "2","3")
val k = 1
val isa = geta(emptyArray) match {
case Some(isa) => scanData(neighborhood,k)
case None => Array.empty
}
}
def scanData(neighborhood: Array[String], k: Int): Iterator[Array[String]] = {
val key = neighborhood(0)
neighborhood
.filterNot { _.equals(key) }
.combinations(k - 1)
}
def geta(neighborhood:Array[String]):Option[Array[String]] = {
if(neighborhood.isEmpty){
return None;
} else {
return Some(neighborhood)
}
}
}
In the following code, I am reading no. of lines from a file. If something goes wrong, I'll like to close the file pointer. But how can I find out if f contains valid pointer or not?
def countLines(filename:String):Option[Int] = {
try{
val f = Source.fromFile(filename)
println(s"no. of lines ${f.getLines().size}")
Some(f.getLines.size)
} catch {
case ex: FileNotFoundException => {
println(s"file ${filename} not found")
None
}
} finally {
//f might not be a valid pointer depending on when the error occured
}
}
The book I am reading uses var to maintain state (if f is valid or not) but I am trying to avoid it for sake of using only immutable variables.
def countLines(filename:String):Option[Int] = {
var f:Option[Source] = None
try{
f = Some(Source.fromFile(filename))
println(s"no. of lines ${f.get.getLines().size}")
Some(f.get.getLines.size)
} catch {
case ex: FileNotFoundException => {
println(s"file ${filename} not found")
None
}
} finally {
for(i<-f){
println("closing file")
i.close()
}
}
}
A double Try(). This closes the io resource even if the getLines() fails, but only if the fromFile() succeeds.
import scala.util.Try
def countLines(filename: String): Option[Int] =
Try(io.Source.fromFile(filename)).fold(_ => None, {f =>
val count = Try(f.getLines().length)
f.close()
count.toOption
})
What do you think about this?
If you want Scala-way - i think it's good example for your task:
def countLines(filename: String): Try[Int] = Try(Source.fromFile(filename).getLines.toList.size)
def test() = {
val filename = "/etc/passwd"
countLines(filename) match {
case Success(n) => println(n)
case Failure(f) => println(f)
}
}
When n - is a number of our lines, and f - is a Throwable.
How about this:
def countLines(filename: String): Option[Int] = {
val file = Try(Source.fromFile(filename))
val count = file.map(_.getLines().size)
(for {
_ <- count.recoverWith { case _ => file.map(_.close()) }
lineCount <- count
} yield lineCount).toOption
}
Let's analyze it:
If file does not exist we will have failed Try instance and method returns None. In this case you do not need to clear any resources as no actual stream was created.
If getLines fails for any reason or anything else during processing goes south we will close created stream in first line of for comprehension
Hope it helps
Simply, how about this:
def numLines(fileName:String):Option[Int] = {
try {
val f = scala.io.Source.fromFile(fileName)
try { Some(f.getLines.size) }
catch { case ex: IOException =>
Console.err.println("i/o excetion")
None
}
finally { f.close() }
}
catch {
case ex: FileNotFoundException =>
Console.err.println("file not found")
None
}
}
Below code just returns a random String via the getFileName method :
import scala.collection.mutable.ListBuffer
object RTest extends App {
var currentFileNameList: ListBuffer[String] = new ListBuffer
def getFileName(): String = {
var fileName = java.util.UUID.randomUUID().toString();
while (true) {
if (!currentFileNameList.contains(fileName)) {
currentFileNameList = currentFileNameList :+ fileName
return fileName
} else {
fileName = java.util.UUID.randomUUID().toString();
}
}
throw new RuntimeException("Error - filename not generated")
}
println(getFileName())
}
If I remove the return statement in line : return fileName then the method getFileName appears to get caught in an infinite loop and does not terminate.
Why is the explicit return required ? Should the scala compiler not infer the return of var fileName ?
As om-nom-nom already said in a comment, it's not really about the type inference, rather it's a control flow problem.
By using return you're breaking the (otherwise) infinite while loop.
Also, not really part of the question, but since you're using a random UUID, you can probably drop the duplicate-checking code and simply return the generated UUID as it is, unless you have insanely high requirements.
According to Wikipedia,
[the probability of generating a duplicate string] is about 0.00000000006 (6 × 10−11), equivalent to the odds of creating a few tens of trillions of UUIDs in a year and having one duplicate.
In a "normal" case I would simply do
def getFileName: String = java.util.UUID.randomUUID.toString
and be done with it.
Because of this block...
while (true) {
if (!currentFileNameList.contains(fileName)) {
currentFileNameList = currentFileNameList :+ fileName
return fileName
} else {
fileName = java.util.UUID.randomUUID().toString();
}
}
You are in an endless loop, as "true" never changes. By returning the function you are jumping out of this loop (as well as completely out of the function).
You could do something like...
object RTest extends App {
var currentFileNameList: ListBuffer[String] = new ListBuffer
def getFileName(): String = {
var fileName = java.util.UUID.randomUUID().toString();
while (currentFileNameList.contains(fileName)) {
fileName = java.util.UUID.randomUUID().toString();
}
currentFileNameList = currentFileNameList :+ fileName;
return fileName;
}
println(getFileName())
}
Which should have the same effect.
No loops, vars, or (explicit) recursion
def getFileName(): String = {
val fileName = Iterator.continually(java.util.UUID.randomUUID().toString())
.find(!currentFileNameList.contains(_)).get
currentFileNameList += fileName
fileName
}
You're not supposed to be looping or using vars:
import scala.collection.mutable.ListBuffer
object Test extends App {
val currentFileNameList: ListBuffer[String] = new ListBuffer
#annotation.tailrec
def getFileName(): String = {
val fileName = java.util.UUID.randomUUID().toString();
if (!currentFileNameList.contains(fileName)) {
currentFileNameList += fileName
fileName
} else {
getFileName()
}
}
println(getFileName())
}
I am trying to save a large data set (50MB) into file. But I get the following error:
java.lang.StackOverflowError
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:350)
at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:350)
...
It looks to me like a recursion problem but I can't find where it would come from. Plase help. Function that makes the Iterator[String]:
override def toJsonIterator: Iterator[String] = {
val lastId = listMap.keys.toList.sorted.last
def evalJS(id: Int): String = {
Json.prettyPrint(listMap(id).toJsonValue) + (if(id != lastId) "," else "")
}
listMap.keys.toList.sorted
.foldLeft(Iterator("{"))( (sum, id) => sum ++ Iterator(evalJS(id)) ) ++ Iterator("}")
}
The save to file code:
private def write(st: Iterator[String], file: String): Unit = {
if (st.isEmpty) return ()
val f = new File(file)
if (!f.getParentFile().exists()) f.getParentFile().mkdirs()
if (!f.exists()) f.createNewFile()
val p = new java.io.PrintWriter(f)
try {
while(st.hasNext) {
p.println(st.next())
}
}
catch {
case e:Exception => println("Error: could not write to file \""+file+"\" because: "+e)
}
finally { p.close() }
}