I'm a newbie in Scala, and I wanted to write some sourcecodes from myself for me to get better.
I've written a simple object (with a main entry) in order to simulate a "grep" call on all files of the current directory. (I launch the program from Eclipse Indigo, and in Debian Squeeze) :
package com.gmail.bernabe.laurent.scala.tests
import java.io.File
import scala.io.Source
object DealWithFiles {
def main(args:Array[String]){
for (result <- grepFilesHere(".*aur.*"))
println(result)
}
private def grepFilesHere(pattern:String):Array[String] = {
val filesHere = new File(".").listFiles
def linesOfFile(file:File) =
Source.fromFile(file).getLines.toList
for (file <- filesHere;
if file.isFile
)
yield linesOfFile(file)(0)
}
}
But I get a java.nio.charset.MalformedInputException, which I am not able to solve :
Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:260)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:319)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.readLine(BufferedReader.java:299)
at java.io.BufferedReader.readLine(BufferedReader.java:362)
at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:67)
at scala.collection.Iterator$class.foreach(Iterator.scala:772)
at scala.io.BufferedSource$BufferedLineIterator.foreach(BufferedSource.scala:43)
at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at scala.collection.mutable.ListBuffer.$plus$plus$eq(ListBuffer.scala:130)
at scala.collection.TraversableOnce$class.toList(TraversableOnce.scala:242)
at scala.io.BufferedSource$BufferedLineIterator.toList(BufferedSource.scala:43)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.linesOfFile$1(DealWithFiles.scala:18)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$$anonfun$grepFilesHere$2.apply(DealWithFiles.scala:23)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$$anonfun$grepFilesHere$2.apply(DealWithFiles.scala:20)
at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:697)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:38)
at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:696)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.grepFilesHere(DealWithFiles.scala:20)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles$.main(DealWithFiles.scala:10)
at com.gmail.bernabe.laurent.scala.tests.DealWithFiles.main(DealWithFiles.scala)
Thanks in advance for helps :)
From the JavaDoc:
MalformedInputException
thrown when an input byte sequence is not legal for given charset, or
an input character sequence is not a legal sixteen-bit Unicode
sequence.
Pass the currect encoding as parameter to Source.fromFile method.
You can handle this character encoding exception by adding below snippet in your code
import scala.io.Codec
import java.nio.charset.CodingErrorAction
implicit val codec = Codec("UTF-8")
codec.onMalformedInput(CodingErrorAction.REPLACE)
codec.onUnmappableCharacter(CodingErrorAction.REPLACE)
Related
I have following scala code:
val file = new FileReader("myfile.txt")
try {
// do operations on file
} finally {
file.close() // close the file
}
How do I handle FileNotFoundException thrown when I read the file? If I put that line inside try block, I am not able to access the file variable inside finally.
For scala 2.13:
you can just use Using to acquire some resource and release it automatically without error handling if it's an AutoClosable:
import java.io.FileReader
import scala.util.Using
val newStyle: Try[String] = Using(new FileReader("myfile.txt")) {
reader: FileReader =>
// do something with reader
"something"
}
newStyle
// will be
// Failure(java.io.FileNotFoundException: myfile.txt (No such file or directory))
// if file is not found or Success with some value it will not fall
scala 2.12:
You can wrap your reader creation by scala.util.Try and if it will fall on creation you will get Failure with FileNotFoundException inside.
import java.io.FileReader
import scala.util.Try
val oldStyle: Try[String] = Try{
val file = new FileReader("myfile.txt")
try {
// do operations on file
"something"
} finally {
file.close() // close the file
}
}
oldStyle
// will be
// Failure(java.io.FileNotFoundException: myfile.txt (No such file or directory))
// or Success with your result of file reading inside
I recommend not to use try ... catch blocks in scala code. It's not type safety for some cases and can lead to non-obvious results but for release some resource in old scala versions there is the only way to do it - using try-finally.
I'm trying to get basic file attributes using Scala, and my reference is this Java question:
Determine file creation date in Java
and this piece of code I'm trying to rewrite in Scala:
static void getAttributes(String pathStr) throws IOException {
Path p = Paths.get(pathStr);
BasicFileAttributes view
= Files.getFileAttributeView(p, BasicFileAttributeView.class)
.readAttributes();
System.out.println(view.creationTime()+" is the same as "+view.lastModifiedTime());
}
The thing I just can't figure out is this line of code..I don't understand how to pass a class in this way using scala... or why Java is insisting upon this in the first place instead of using an actual constructed object as the parameter. Can someone please help me write this line of code to function properly? I must be using the wrong syntax
val attr = Files.readAttributes(f,Class[BasicFileAttributeView])
Try this:
def attrs(pathStr:String) =
Files.getFileAttributeView(
Paths.get(pathStr),
classOf[BasicFileAttributes] //corrected
).readAttributes
Get file creation date in Scala, from Basic Files Attributes:
// option 1,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributes
val pathStr = "/tmp/test.sql"
Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime
res3: java.nio.file.attribute.FileTime = 2018-03-06T00:25:52Z
// option 2,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributeView
val pathStr = "/tmp/test.sql"
{
Files
.getFileAttributeView(Paths.get(pathStr), classOf[BasicFileAttributeView])
.readAttributes.creationTime
}
res20: java.nio.file.attribute.FileTime = 2018-03-07T19:00:19Z
I'm having some trouble doing a Plain SQL Query using String Interpolation. I'm trying to query a MS-SQL 2008 database using Microsofts sqljdbc4.jar.
The table is simple:
[serv_num]
number (nvarchar(15)) | name (nvarchar(50))
=============================================
1234 | AA
+345 | BB
The code I'm using is as follows:
import java.sql.{Connection, DriverManager}
import org.slf4j.LoggerFactory
import scala.collection.JavaConversions._
import scala.slick.session.Database
import Database.threadLocalSession
import scala.slick.jdbc.{GetResult, StaticQuery => Q}
import Q.interpolation
object NumImport extends App {
def logger = LoggerFactory.getLogger("NumImport")
implicit val getServNumResult = GetResult(r => ServNum(r.<<, r.<<))
override def main(args: Array[String]) {
val uri = "jdbc:sqlserver://192.168.1.2;databaseName=slicktestdb;user=yyyy;password=xxxxxx"
val drv = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
Database.forURL(uri, driver = drv) withSession {
def getServNum(n: String) = sql"select number, name from serv_num where number = $n".as[ServNum]
val result = getServNum("1234")
println(result.firstOption)
}
}
case class ServNum(num: String, name: String)
}
My issue is that I'm getting a java.lang.NullPointerException on result.firstOption (I think!). This happens when a row is found. If try to select a number that doesn't exist, say getServNum("34"), then result.firstOption is a None, as expected.
Stacktrace:
08:54:47.932 TKD [run-main] DEBUG scala.slick.session.BaseSession - Preparing statement: select number, name from serv_num where number = ?
[error] (run-main) java.lang.NullPointerException
java.lang.NullPointerException
at scala.slick.jdbc.StaticQuery.extractValue(StaticQuery.scala:16)
at scala.slick.jdbc.StatementInvoker$$anon$1.extractValue(StatementInvoker.scala:37)
at scala.slick.session.PositionedResultIterator.foreach(PositionedResult.scala:212)
at scala.slick.jdbc.Invoker$class.foreach(Invoker.scala:91)
at scala.slick.jdbc.StatementInvoker.foreach(StatementInvoker.scala:10)
at scala.slick.jdbc.Invoker$class.firstOption(Invoker.scala:41)
at scala.slick.jdbc.StatementInvoker.firstOption(StatementInvoker.scala:10)
at scala.slick.jdbc.UnitInvoker$class.firstOption(Invoker.scala:148)
at scala.slick.jdbc.StaticQuery0.firstOption(StaticQuery.scala:95)
I tried debugging and following the stacktrace, but I'm quite new to both Scala and Slick, so I'm lost. If anyone could help me or point me in the right direction I would appreciate it a lot.
Thanks.
In looking at the slick source code where the exception was happening, I could see that it was trying to invoke apply on the GetResult object, but that object seemed to be null. In thinking about what might of caused it, I'm suggesting that you move the implicit inside of the main method. It seems that the way things are structured, slick is losing track of the implicit ResultSet to case class converter (the GetResult)
I am working on a small GUI application written in Scala. There are a few settings that the user will set in the GUI and I want them to persist between program executions. Basically I want a scala.collections.mutable.Map that automatically persists to a file when modified.
This seems like it must be a common problem, but I have been unable to find a lightweight solution. How is this problem typically solved?
I do a lot of this, and I use .properties files (it's idiomatic in Java-land). I keep my config pretty straight-forward by design, though. If you have nested config constructs you might want a different format like YAML (if humans are the main authors) or JSON or XML (if machines are the authors).
Here's some example code for loading props, manipulating as Scala Map, then saving as .properties again:
import java.io._
import java.util._
import scala.collection.JavaConverters._
val f = new File("test.properties")
// test.properties:
// foo=bar
// baz=123
val props = new Properties
// Note: in real code make sure all these streams are
// closed carefully in try/finally
val fis = new InputStreamReader(new FileInputStream(f), "UTF-8")
props.load(fis)
fis.close()
println(props) // {baz=123, foo=bar}
val map = props.asScala // Get to Scala Map via JavaConverters
map("foo") = "42"
map("quux") = "newvalue"
println(map) // Map(baz -> 123, quux -> newvalue, foo -> 42)
println(props) // {baz=123, quux=newvalue, foo=42}
val fos = new OutputStreamWriter(new FileOutputStream(f), "UTF-8")
props.store(fos, "")
fos.close()
Here's an example of using XML and a case class for reading a config. A real class can be nicer than a map. (You could also do what sbt and at least one project do, take the config as Scala source and compile it in; saving it is less automatic. Or as a repl script. I haven't googled, but someone must have done that.)
Here's the simpler code.
This version uses a case class:
case class PluginDescription(name: String, classname: String) {
def toXML: Node = {
<plugin>
<name>{name}</name>
<classname>{classname}</classname>
</plugin>
}
}
object PluginDescription {
def fromXML(xml: Node): PluginDescription = {
// extract one field
def getField(field: String): Option[String] = {
val text = (xml \\ field).text.trim
if (text == "") None else Some(text)
}
def extracted = {
val name = "name"
val claas = "classname"
val vs = Map(name -> getField(name), claas -> getField(claas))
if (vs.values exists (_.isEmpty)) fail()
else PluginDescription(name = vs(name).get, classname = vs(claas).get)
}
def fail() = throw new RuntimeException("Bad plugin descriptor.")
// check the top-level tag
xml match {
case <plugin>{_*}</plugin> => extracted
case _ => fail()
}
}
}
This code reflectively calls the apply of a case class. The use case is that fields missing from config can be supplied by default args. No type conversions here. E.g., case class Config(foo: String = "bar").
// isn't it easier to write a quick loop to reflect the field names?
import scala.reflect.runtime.{currentMirror => cm, universe => ru}
import ru._
def fromXML(xml: Node): Option[PluginDescription] = {
def extract[A]()(implicit tt: TypeTag[A]): Option[A] = {
// extract one field
def getField(field: String): Option[String] = {
val text = (xml \\ field).text.trim
if (text == "") None else Some(text)
}
val apply = ru.newTermName("apply")
val module = ru.typeOf[A].typeSymbol.companionSymbol.asModule
val ts = module.moduleClass.typeSignature
val m = (ts member apply).asMethod
val im = cm reflect (cm reflectModule module).instance
val mm = im reflectMethod m
def getDefault(i: Int): Option[Any] = {
val n = ru.newTermName("apply$default$" + (i+1))
val m = ts member n
if (m == NoSymbol) None
else Some((im reflectMethod m.asMethod)())
}
def extractArgs(pss: List[List[Symbol]]): List[Option[Any]] =
pss.flatten.zipWithIndex map (p => getField(p._1.name.encoded) orElse getDefault(p._2))
val args = extractArgs(m.paramss)
if (args exists (!_.isDefined)) None
else Some(mm(args.flatten: _*).asInstanceOf[A])
}
// check the top-level tag
xml match {
case <plugin>{_*}</plugin> => extract[PluginDescription]()
case _ => None
}
}
XML has loadFile and save, it's too bad there seems to be no one-liner for Properties.
$ scala
Welcome to Scala version 2.10.0-RC5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_06).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import reflect.io._
import reflect.io._
scala> import java.util._
import java.util._
scala> import java.io.{StringReader, File=>JFile}
import java.io.{StringReader, File=>JFile}
scala> import scala.collection.JavaConverters._
import scala.collection.JavaConverters._
scala> val p = new Properties
p: java.util.Properties = {}
scala> p load new StringReader(
| (new File(new JFile("t.properties"))).slurp)
scala> p.asScala
res2: scala.collection.mutable.Map[String,String] = Map(foo -> bar)
As it all boils down to serializing a map / object to a file, your choices are:
classic serialization to Bytecode
serialization to XML
serialization to JSON (easy using Jackson, or Lift-JSON)
use of a properties file (ugly, no utf-8 support)
serialization to a proprietary format (ugly, why reinvent the wheel)
I suggest to convert Map to Properties and vice versa. "*.properties" files are standard for storing configuration in Java world, why not use it for Scala?
The common way are *. properties, *.xml, since scala supports xml natively, so it would be easier using xml config then in java.
I was trying to do some scripting in Scala, to process some log files:
scala> import io.Source
import io.Source
scala> import java.io.File
import java.io.File
scala> val f = new File(".")
f: java.io.File = .
scala> for (l <- f.listFiles) {
| val src = Source.fromFile(l).getLines
| println( (0 /: src) { (i, line) => i + 1 } )
| }
3658
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Unknown Source)
at java.nio.HeapCharBuffer.get(Unknown Source)
at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:86)
at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:74)
at scala.io.Source$$anon$6.next(Source.scala:307)
at scala.io.Source$$anon$6.next(Source.scala:301)
at scala.Iterator$cla...
Why do I get this java.nio.BufferUnderflowException?
NOTE - I'm processing 10 log files, each about 1MB in size
I got BufferUnderflowException exception when I opened a file with the wrong enconding. It contained illegal characters (according to the wrong encoding) and this misleading exception was thrown.
I'd also be interested as to exactly why this is happening but I'd guess it's to do with the fact that Source is an object (i.e. a singleton) and how it is gets transparently reset. You can fix the problem as follows:
for (l <- g.listFiles if !l.isDirectory) {
| val src = Source.fromFile(l)
| println( (0 /: src.getLines) { (i, line) => i + 1 } )
| src.reset
| }
The important bit is the reset - which should probably be in a try-finally block (although the isDirectory test is probably useful too)
This is essentially a restatement of Elazar's answer, but you will also get this exception if you try to read a binary file using scala.io.Source.fromFile.
I just ran into this (accidentally trying to read a .jpg with fromFile) due to a very stupid bug in something I wrote...