Using Scala 2.10.2------------
I am a Java engineer, and started to learn Scala from yesterday, but I got stuck now, this simply code does not work for me, but when I use java to write it, it works fine:
package lesson4
import scala.io.Source
import scala.reflect.io.File
object Test {
def main(args: Array[String]): Unit = {
var filePath = Source.getClass().getResource("/lesson4/test.txt")
var file = Source.fromFile(filePath.getFile())
var lines = file.getLines
lines.foreach(println)
}
}
The file is in the right path:
But the code just does not work:
Exception in thread "main" java.io.FileNotFoundException: /Users/wenjiezhang/Desktop/source_files/git_hub%20workspace/Learning%20Scala/ScalaLearning/bin/lesson4/test.txt (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at scala.io.Source$.fromFile(Source.scala:90)
at scala.io.Source$.fromFile(Source.scala:75)
at scala.io.Source$.fromFile(Source.scala:53)
at lesson4.Test$.main(Test.scala:20)
at lesson4.Test.main(Test.scala)
As you can see from the error log, the path you're providing is being converted to a URL (for example SPACE becomes %20). You should use the fromURL method Source.fromURL(Source.getClass().getResource("/lesson4/test.txt"))
scala.io.Source docs
You have spaces in your path converted to encoded from the URL.
Related
I have following scala code:
val file = new FileReader("myfile.txt")
try {
// do operations on file
} finally {
file.close() // close the file
}
How do I handle FileNotFoundException thrown when I read the file? If I put that line inside try block, I am not able to access the file variable inside finally.
For scala 2.13:
you can just use Using to acquire some resource and release it automatically without error handling if it's an AutoClosable:
import java.io.FileReader
import scala.util.Using
val newStyle: Try[String] = Using(new FileReader("myfile.txt")) {
reader: FileReader =>
// do something with reader
"something"
}
newStyle
// will be
// Failure(java.io.FileNotFoundException: myfile.txt (No such file or directory))
// if file is not found or Success with some value it will not fall
scala 2.12:
You can wrap your reader creation by scala.util.Try and if it will fall on creation you will get Failure with FileNotFoundException inside.
import java.io.FileReader
import scala.util.Try
val oldStyle: Try[String] = Try{
val file = new FileReader("myfile.txt")
try {
// do operations on file
"something"
} finally {
file.close() // close the file
}
}
oldStyle
// will be
// Failure(java.io.FileNotFoundException: myfile.txt (No such file or directory))
// or Success with your result of file reading inside
I recommend not to use try ... catch blocks in scala code. It's not type safety for some cases and can lead to non-obvious results but for release some resource in old scala versions there is the only way to do it - using try-finally.
I'm trying to read text file located in resources directory using Scala version 2.12.3.
However I'm getting file not found error.
my project in eclipse
my scala code:
package main.scala
import scala.io.Source
import scala.io.Codec
object Application {
def main(args: Array[String]) {
try {
val source = Source.fromFile("sample.txt")(Codec.UTF8)
for (line <- source.getLines) {
println(line.toUpperCase)
}
source.close
} catch {
case e: Throwable => e.printStackTrace()
}
}
}
I also tried using
val source = Source.fromFile("sample.txt")(Codec.UTF8)
but got the same error.
If you want to read file from src/main/resources directory you should use Source.fromResource method, so try this:
Source.fromResource("sample.txt")(Codec.UTF8)
Update
In your case you have to use either Source.fromFile("src/main/resources/sample.txt") or
Source.fromFile("sample.txt") if you put your file in root project directory
I have a folder structure like below:
- main
-- java
-- resources
-- scalaresources
--- commandFiles
and in that folders I have my files that I have to read.
Here is the code:
def readData(runtype: String, snmphost: String, comstring: String, specificType: String): Unit = {
val realOrInvFile = "/commandFiles/snmpcmds." +runtype.trim // these files are under commandFiles folder, which I have to read.
try {
if (specificType.equalsIgnoreCase("Cisco")) {
val specificDeviceFile: String = "/commandFiles/snmpcmds."+runtype.trim+ ".cisco"
val realOrInvCmdsList = scala.io.Source.fromFile(realOrInvFile).getLines().toList.filterNot(line => line.startsWith("#")).map{
//some code
}
val specificCmdsList = scala.io.Source.fromFile(specificDeviceFile).getLines().toList.filterNot(line => line.startsWith("#")).map{
//some code
}
}
} catch {
case e: Exception => e.printStackTrace
}
}
}
Resources in Scala work exactly as they do in Java.
It is best to follow the Java best practices and put all resources in src/main/resources and src/test/resources.
Example folder structure:
testing_styles/
├── build.sbt
├── src
│ └── main
│ ├── resources
│ │ └── readme.txt
Scala 2.12.x && 2.13.x reading a resource
To read resources the object Source provides the method fromResource.
import scala.io.Source
val readmeText : Iterator[String] = Source.fromResource("readme.txt").getLines
reading resources prior 2.12 (still my favourite due to jar compatibility)
To read resources you can use getClass.getResource and getClass.getResourceAsStream .
val stream: InputStream = getClass.getResourceAsStream("/readme.txt")
val lines: Iterator[String] = scala.io.Source.fromInputStream( stream ).getLines
nicer error feedback (2.12.x && 2.13.x)
To avoid undebuggable Java NPEs, consider:
import scala.util.Try
import scala.io.Source
import java.io.FileNotFoundException
object Example {
def readResourceWithNiceError(resourcePath: String): Try[Iterator[String]] =
Try(Source.fromResource(resourcePath).getLines)
.recover(throw new FileNotFoundException(resourcePath))
}
good to know
Keep in mind that getResourceAsStream also works fine when the resources are part of a jar, getResource, which returns a URL which is often used to create a file can lead to problems there.
in Production
In production code I suggest to make sure that the source is closed again.
For Scala >= 2.12, use Source.fromResource:
scala.io.Source.fromResource("located_in_resouces.any")
One-liner solution for Scala >= 2.12
val source_html = Source.fromResource("file.html").mkString
Important taken from comments (thanks to #anentropic): with Source.fromResource you do not put the initial forward slash.
import scala.io.Source
object Demo {
def main(args: Array[String]): Unit = {
val ipfileStream = getClass.getResourceAsStream("/folder/a-words.txt")
val readlines = Source.fromInputStream(ipfileStream).getLines
readlines.foreach(readlines => println(readlines))
}
}
The required file can be accessed as below from resource folder in scala
val file = scala.io.Source.fromFile(s"src/main/resources/app.config").getLines().mkString
For Scala 2.11, if getLines doesn't do exactly what you want you can also copy the a file out of the jar to the local file system.
Here's a snippit that reads a binary google .p12 format API key from /resources, writes it to /tmp, and then uses the file path string as an input to a spark-google-spreadsheets write.
In the world of sbt-native-packager and sbt-assembly, copying to local is also useful with scalatest binary file tests. Just pop them out of resources to local, run the tests, and then delete.
import java.io.{File, FileOutputStream}
import java.nio.file.{Files, Paths}
def resourceToLocal(resourcePath: String) = {
val outPath = "/tmp/" + resourcePath
if (!Files.exists(Paths.get(outPath))) {
val resourceFileStream = getClass.getResourceAsStream(s"/${resourcePath}")
val fos = new FileOutputStream(outPath)
fos.write(
Stream.continually(resourceFileStream.read).takeWhile(-1 !=).map(_.toByte).toArray
)
fos.close()
}
outPath
}
val filePathFromResourcesDirectory = "google-docs-key.p12"
val serviceAccountId = "[something]#drive-integration-[something].iam.gserviceaccount.com"
val googleSheetId = "1nC8Y3a8cvtXhhrpZCNAsP4MBHRm5Uee4xX-rCW3CW_4"
val tabName = "Favorite Cities"
import spark.implicits
val df = Seq(("Brooklyn", "New York"),
("New York City", "New York"),
("San Francisco", "California")).
toDF("City", "State")
df.write.
format("com.github.potix2.spark.google.spreadsheets").
option("serviceAccountId", serviceAccountId).
option("credentialPath", resourceToLocal(filePathFromResourcesDirectory)).
save(s"${googleSheetId}/${tabName}")
The "resources" folder must be under the source root. if using intellj check for the blue folder in the project folders on the left side. eg AppName/src/main/scala or Project/scala/../main/ etc.
If using val stream: InputStream = getClass.getResourceAsStream("/readme.txt") don't forget the "/" (forward slash), given readme.txt is the file inside resources
I'm trying to get basic file attributes using Scala, and my reference is this Java question:
Determine file creation date in Java
and this piece of code I'm trying to rewrite in Scala:
static void getAttributes(String pathStr) throws IOException {
Path p = Paths.get(pathStr);
BasicFileAttributes view
= Files.getFileAttributeView(p, BasicFileAttributeView.class)
.readAttributes();
System.out.println(view.creationTime()+" is the same as "+view.lastModifiedTime());
}
The thing I just can't figure out is this line of code..I don't understand how to pass a class in this way using scala... or why Java is insisting upon this in the first place instead of using an actual constructed object as the parameter. Can someone please help me write this line of code to function properly? I must be using the wrong syntax
val attr = Files.readAttributes(f,Class[BasicFileAttributeView])
Try this:
def attrs(pathStr:String) =
Files.getFileAttributeView(
Paths.get(pathStr),
classOf[BasicFileAttributes] //corrected
).readAttributes
Get file creation date in Scala, from Basic Files Attributes:
// option 1,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributes
val pathStr = "/tmp/test.sql"
Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime
res3: java.nio.file.attribute.FileTime = 2018-03-06T00:25:52Z
// option 2,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributeView
val pathStr = "/tmp/test.sql"
{
Files
.getFileAttributeView(Paths.get(pathStr), classOf[BasicFileAttributeView])
.readAttributes.creationTime
}
res20: java.nio.file.attribute.FileTime = 2018-03-07T19:00:19Z
I'm a newbie in Scala, and I wanted to write some sourcecodes from myself for me to get better.
I've written a simple object (with a main entry) in order to simulate a "grep" call on all files of the current directory. (I launch the program from Eclipse Indigo, and in Debian Squeeze) :
package com.gmail.bernabe.laurent.scala.tests
import java.io.File
import scala.io.Source
object DealWithFiles {
def main(args:Array[String]){
for (result <- grepFilesHere(".*aur.*"))
println(result)
}
private def grepFilesHere(pattern:String):Array[String] = {
val filesHere = new File(".").listFiles
def linesOfFile(file:File) =
Source.fromFile(file).getLines.toList
for (file <- filesHere;
if file.isFile
)
yield linesOfFile(file)(0)
}
}
But I get a java.nio.charset.MalformedInputException, which I am not able to solve :
Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:319)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.readLine(BufferedReader.java:299)
at java.io.BufferedReader.readLine(BufferedReader.java:362)
at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:67)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.io.BufferedSource$BufferedLineIterator.foreach(BufferedSource.scala:43)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:130)
at scala.collection.TraversableOnce$class.toList(TraversableOnce.scala:242)
at scala.io.BufferedSource$BufferedLineIterator.toList(BufferedSource.scala:43)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.linesOfFile$1(DealWithFiles.scala:18)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$$anonfun$grepFilesHere$2.apply(DealWithFiles.scala:23)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$$anonfun$grepFilesHere$2.apply(DealWithFiles.scala:20)
at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:697)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:38)
at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:696)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.grepFilesHere(DealWithFiles.scala:20)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.main(DealWithFiles.scala:10)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles.main(DealWithFiles.scala)
Thanks in advance for helps :)
From the JavaDoc:
MalformedInputException
thrown when an input byte sequence is not legal for given charset, or
an input character sequence is not a legal sixteen-bit Unicode
sequence.
Pass the currect encoding as parameter to Source.fromFile method.
You can handle this character encoding exception by adding below snippet in your code
import scala.io.Codec
import java.nio.charset.CodingErrorAction
implicit val codec = Codec("UTF-8")
codec.onMalformedInput(CodingErrorAction.REPLACE)
codec.onUnmappableCharacter(CodingErrorAction.REPLACE)