I have the following implementation where I'm trying to handle proper resource closing during any fatal exceptions:
private def loadPrivateKey(keyPath: String) = {
def tryReadCertificate(file: File): Try[BufferedReader] = Try { new BufferedReader(new FileReader(file)) }
def tryLoadPemParser(reader: BufferedReader): Try[PEMParser] = Try { new PEMParser(reader) }
def createXXX(buffReader: BufferedReader, pemParser: PEMParser) = try {
...
} finally {
buffReader.close()
pemParser.close()
}
tryReadCertificate(new File(keyPath, "myKey.pem")) match {
case Success(buffReader) => tryLoadPemParser(buffReader) match {
case Success(pemParser) => createXXX(buffReader, pemParser)
case Failure(fail) =>
}
case Failure(fail) =>
}
}
I already see that my nested case blocks are a mess. Is there a better way to do this? In the end, I just want to make sure that I close the BufferedReader and the PEMParser !
You could restructure your code a little like this, using a for-comprehension to clean up some of the nested case statements:
def tryReadCertificate(file: File): Try[BufferedReader] = Try { new BufferedReader(new FileReader(file)) }
def tryLoadPemParser(reader: BufferedReader): Try[PEMParser] = Try { new PEMParser(reader) }
def createXXX(buffReader: BufferedReader, pemParser: PEMParser) = {
...
}
val certReaderTry = tryReadCertificate(new File(keyPath, "myKey.pem"))
val pemParserTry = for{
certReader <- certReaderTry
pemParser <- tryLoadPemParser(certReader)
} yield {
createXXX(certReader, pemParser)
pemParser
}
certReaderTry foreach(_.close)
pemParserTry foreach (_.close)
Structured like this, you will only ever end up calling close on things you are sure were opened successfully.
And even better, if your PEMParser happened to extend java.io.Closeable, meaning that the Trys both wrapped Closeable objects, then you could swap those last two lines for a single line like this:
(certReaderTry.toOption ++ pemParserTry.toOption) foreach (_.close)
EDIT
In response to the OP's comment: In the first example, if tryreadCertificate succeeds, then certReaderTry will be a Success[BufferedReader] and because it's successful, calling foreach on it will yield the BufferedReader which will then have close called on it. If certReaderTry is Success, then (via the for-comp) we will call tryLoadPemParser and if that also succeeds, we can move on to createXXX and assign the tryLoadPemParser to the pemParserTry val. Then, later, if pemParserTry is a Success, the same thing happens where foreach yields the PEMParser and we can close it. Per this example, as long as the those Trys are successes and something else unexpected does not happen (in createXXX for example) that would throw an exception all the way out, then you can guarantee that the closing related code at the end will do its job and close those resources.
EDIT2
If you wanted the value from createXXX in a separate Try, then you could do something like this:
val certReaderTry = tryReadCertificate(new File(keyPath, "myKey.pem"))
val pemParserTry = certReaderTry.flatMap(tryLoadPemParser)
val resultTry = for{
certReader <- certReaderTry
pemParser <- pemParserTry
} yield createXXX(certReader, pemParser)
Related
I have a piece of Scala code using DB connection:
def getAllProviderCodes()(implicit conf : Configuration) : List[String] = {
var conn: java.sql.Connection = null
try {
conn = DriverManager.getConnection(DBInfo.dbUrl(conf), DBInfo.dbUserName(conf), DBInfo.dbPassword(conf))
return ResultSetIterator.create(
conn.prepareStatement("SELECT pcode FROM providers").executeQuery()
){_.getString("pcode")}.toList
} catch {
case e: Exception =>
logger.warn("Something went wrong with creating the connection: " + e.getStackTrace)
} finally {
if (conn != null) {
conn.close()
}
}
List()
}
It's very OOP-Java-like style, so I'd like to know is there a way to write it in more functional way? I tried to succeed in applying Try monad, but failed: my biggest concern is that we have state here, as well as finally block. Maybe there's some kind of pattern for such cases?
Thank you in advance.
UPD: Here's the example from here of what IMHO the solution will look like:
val connection = database.getConnection()
val data: Seq[Data] = Try{
val results = connection.query("select whatever")
results.map(convertToWhatIneed)
} recover {
case t: Throwable =>
Seq.empty[Data]
} get
connection.close()
But as I've mentioned in the comment, I have to close the connection, then I have to place all the things regarding to connection inside Try to keep it pure... and then I to the variant with "try-catch-finally" inside Try block.
I've never played around with the Java SQL Connection library so the syntax of my answer has been written as pseudocode, but if I understand your question correctly here is how I would implement what you have done:
def getAllProviderCodes()(implicit conf : Configuration): List[String] = {
val conn: Connection = DriverManager.getConnection(???) // replace ??? with parameters
val result: List[String] = Try {
??? // ResultSetIterator stuff
} match {
case Success(output) => output // or whatever .toList thing
case Failure(_) => List.empty // add logging here
}
if(conn != null) conn.close()
result // will be whatever List you make (or an empty List if Try fails)
}
Instead of a Java-like try-catch-finally block, one Scala-like way of doing things would be to put the stuff which could explode in a Try block and assigning the response to a value using case Success(out) and case Failure(ex).
Just pull the connection outside of the try:
val conn = getConnection()
try {
doStuff(conn)
} finally {
conn.close
}
If you want the result of whole thing to be a Try, just wrap it into a Try:
def doDBStuff = Try {
val conn = getConnection()
try {
doStuff(conn)
} finally {
conn.close
}
}
Or with a bit less nesting (but this will throw connection exceptions):
def doDBStuff = {
val conn = getConnection()
val result = Try { doStuff(conn) }
conn.close
result
}
What is the best way to write the following in Scala? It doesn't look quite right to me - first the forward declaration of the 2 vals, then the long PrintWriter creation line, then thefinallyblock. The only thing that's idiomatic, is the catch block...
val outputStream = Try(fs.create(tmpFile))
val writer = new PrintWriter(new BufferedWriter(new OutputStreamWriter(outputStream.get)))
if (outputStream.isFailure) {
logger.error(s"Couldn't open file: $tmpFile")
}
try {
features.foreach {
case (sectionName, modelRDD) =>
writer.append("{" + sectionName + ", " + modelRDD.getNumPartitions + "}")
}
} catch {
case e: Exception =>
logger.error(s"Got exception", e)
throw e
} finally {
outputStream.get.close()
writer.close()
}
We can further use the context of the initial Try to execute the complete I/O operation:
First, we define a function that encapsulates our process:
def safeFilePrint(tf: => OutputStream)(op: PrintWriter => Unit): Try[Unit] = {
val os = Try(tf)
val write = {
val writer = os.map(f => new PrintWriter(f))
val writeOp = writer.map(op)
val flushOp = writer.map(_.flush)
writeOp.flatMap(_ => flushOp)
}
val close = os.map(_.close)
write.flatMap(_ => close)
}
And usage:
val collection = Seq(...)
val writeResult = safeFilePrint(new FileOutputStream(new File("/tmp/foo.txt"))){w =>
collection.foreach(elem => w.write(e)
}
Note that in contrast with the original code, we have a result of the write operation. Either writeResult will be Success(()) if everything went well or Failure(exception) is something went wrong. Based on that our application can further decide what to do.
One may wonder: "Where is the finally?" In Java, finally is used to ensure that some code (typically resource management) would be executed, even in the case that an exception thrown in the try scope would cause an exception-handling path to be followed.
In Scala, using constructs like Try, Either or our own ADT, we lift error handling to the application level. finally becomes unnecessary as our program is able to deal with the failure as just another valid state of the program.
Finally settled on that code after reading #maasg's answer, which highlights the monadic flow and is more "symmetric". It looks much, much better than the code in the OP!
def safePrintToStream(gen: => OutputStream)(op: PrintWriter => Unit): Try[Unit] = {
val os = Try(gen)
val writer = os.map(stream => new PrintWriter(stream))
val write = writer.map(op(_))
val flush = writer.map(_.flush)
val close = os.map(_.close)
write.flatMap(_ => flush).flatMap(_ => close)
}
I try to read a file in scala like this:
def parseFile(filename: String) = {
val source = scala.io.Source.fromFile(filename)
try {
val lines = source.getLines().map(line => line.trim.toDouble)
return lines
} catch {
// re-throw exception, but make source source is closed
case
t: Throwable => {
println("error during parsing of file")
throw t
}
} finally {
source.close()
}
}
When I access the result later, I get an
java.io.IOException: Stream Closed
I understand that this arises because source.getLines() only returns an (lazy) Iterator[String], and I already close the BufferedSource in the finally clause.
How can I avoid this error, i.e. how can a "evaluate" the Stream before closing the source?
EDIT: I tried to call source.getLines().toSeq which did not help.
Maybe, you can try the following solution, which makes the codes more functional and takes the advantage of lazy evaluation.
First, define a helper function using, which takes care of open/close the file.
def using[A <: {def close() : Unit}, B](param: A)(f: A => B): B =
try f(param) finally param.close()
Then, you can refactor your code in functional programming style:
using(Source.fromFile(filename)) {
source =>
val lines = Try(source.getLines().map(line => line.trim.toDouble))
val result = lines.flatMap(l => Try(processOrDoWhatYouWantForLines(l)))
result.get
}
Actually, the using function can be used for handling all resources which need to be closed at the end of the operation.
List is not lazy so change:
val lines = source.getLines().map(line => line.trim.toDouble)
to
val lines = source.getLines().toList.map(line => line.trim.toDouble)
in order to force computing.
Here below is my attempt to implement a class that provides functionality for compressing/decompressing strings:
object GZipHelper {
def deflate(txt: String): Try[String] = {
try {
val arrOutputStream = new ByteArrayOutputStream()
val zipOutputStream = new GZIPOutputStream(arrOutputStream)
zipOutputStream.write(txt.getBytes)
new Success(Base64.encodeBase64String(arrOutputStream.toByteArray))
} catch {
case _: e => new Failure(e)
}
}
def inflate(deflatedTxt: String): Try[String] = {
try {
val bytes = Base64.decodedBase64(deflatedTxt)
val zipInputStream = GZIPInputStream(new ByteArrayInputStream(bytes))
new success(IOUtils.toString(zipInputStream))
} catch {
case _: e => new Failure(e)
}
}
}
As you can see, the finally blocks that close GZIPOutputStream and GZIPInputStream are missing... how could I implement this in the ''scala'' way? How could I improve the code?
Since you're using the "old fashioned" try statement and explicitly turning it into a scala.util.Try, there really is no reason not to add a finally block after your try.
In this specific case though, there is little point in closing, for example, your ByteArrayInputStream - it's not really an open resource and does not need to be closed. In which case you can simplify your code and make it much more idiomatic this way:
def inflate(deflatedTxt: String): Try[String] = Try {
val bytes = Base64.decodedBase64(deflatedTxt)
val zipInputStream = GZIPInputStream(new ByteArrayInputStream(bytes))
IOUtils.toString(zipInputStream)
}
I personally would not declare bytes and zipInputStream since they're only used once, but that's a matter of preference.
The trick here is having a finally block with a call to scala.util.Try.apply - I'm not sure that's possible without going through a call to map that doesn't actually modify anything, which seems like a bit of an oversight to me. I was expecting to see an andThen or eventually method in scala.util.Try, but it doesn't seem to be there (yet?).
Just for completeness, here is the deflate method converted (original version was also missing the close() call on the GZIP class):
def deflate(txt: String): Try[String] = Try {
val arrOutputStream = new ByteArrayOutputStream()
val zipOutputStream = new GZIPOutputStream(arrOutputStream)
zipOutputStream.write(txt.getBytes)
zipOutputStream.close()
Base64.encodeBase64String(arrOutputStream.toByteArray)
}
https://github.com/jsuereth/scala-arm/wiki/basic-usage
this looks like a good approach
You can use scala-compress https://github.com/gekomad/scala-compress
compress string:
val aString: String = "foo"
val compressed: Try[Array[Byte]] = zipString(aString, charSetName = "UTF-8")
decompress:
val compressedArray: Array[Byte] = ???
val decompressed: Try[Array[Byte]] = unzipString(compressedArray)
new String(decompressed, "UTF-8")
In the code below, if I uncomment the for loop the file no longer gets deleted
val file = "myfile.csv"
//for (line <- Source.fromFile(file).getLines()) { }
new File(file).delete()
If so is there some type of close function that I should be calling?
There is some sort of close that you should be calling:
val file = "myfile.csv"
val source = Source.fromFile(file)
for (line <- source.getLines()) { }
source.close
new File(file).delete
but this is a bit tedious. If you rewrite the for loop as
source.getLines().foreach{ line => }
you can then
class CloseAfter[A <: { def close(): Unit }](a: A) {
def closed[B](f: A => B) = try { f(a) } finally { a.close }
}
implicit def close_things[A <: { def close(): Unit }](a: A) = new CloseAfter(a)
and now your code would become
val file = "myfile.csv"
Source.fromFile(file).closed(_.foreach{ line => })
new File(file).delete
(which would be a benefit if you're doing it many times in your code, or if you already maintain your own library of helpful functions and it would be easy to add the closing implicit just once there so you could use it everywhere).
As others have said, yes, you need to close the Source when you're done with it. Another good solution is to use scala-arm to automagically close the file for you.
import resource._
val file = "myfile.csv"
for {
source <- managed(Source.fromFile(file))
line <- source.getLines()
} {
}
new File(file).delete
After reading "Why doesn't Scala Source close the underlying InputStream?", use instead "scala-incubator / scala-io".
It includes a delete operation on a Path which takes care of everything. That library always always ensures that files are safely closed after each use.