Following these examples and especially this code:
object Control {
def using[A <: { def close(): Unit }, B](resource: A)(f: A => B): B =
try {
f(resource)
} finally {
resource.close()
}
}
...
using(io.Source.fromFile("example.txt")) { source => { .....
I wanted to extend the using method so instead of a type which implements close it receives a string (filename), a function to open a source, and the processing function. In this way, I would avoid the exception which would be thrown in the above example in case the given file does not exist.
So I ended up with this code:
object Control
{
def using[A <: { def close(): Unit }, B](opener: String => A)(name:String)(func: A => B): Unit =
{
var resource:A
// ^ Error: 'Block cannot contain declarations'
try
{
resource = opener(name)
func(resource)
}
catch
{
case e: (_) => println(s"Failed to open resource '${name}' (${e})")
}
finally
{
println("Closing file ...")
resource.close()
}
}
}
So I am defining a method, which takes as first parameter an opener-function, which receives a string, and returns an object which implements close, a string (for the opener function), and a processing function.
However it won't let me declare the resource variable outside of the try-catch block (so I can reach it in the finally block). It will work if I just put it into the try block like var resource:A = opener(name), however then I cannot reach resource in the finally block.
How can I solve it? I have to say that I am still a beginner in Scala, so I am a bit lost here.
Here is a revised example that you can also run on Scastie:
import scala.util.control.NonFatal
import scala.language.reflectiveCalls
type Resource = { def close(): Unit }
def using[A <: Resource, B](opener: String => A)(name: String)(func: A => B): Unit = {
var resource = null.asInstanceOf[A]
try {
resource = opener(name)
func(resource)
} catch {
case NonFatal(e) => println(s"Failed to open resource '${name}' (${e.getMessage})")
} finally {
println("Closing resource...")
resource.close()
}
}
final class SomeKindOfResource(n: String) {
def use(): Int = n.toInt
def close(): Unit = {}
}
using(new SomeKindOfResource(_))("42")(n => println(n.use()))
using(new SomeKindOfResource(_))("NaN")(n => println(n.use()))
The piece that you were lacking is that initialization:
var resource = null.asInstanceOf[A]
Please note that despite what you may think, this does not throw a NullPointerException. You can read more about it here.
I've added a few more things you may be interested in:
explicitly importing scala.language.reflectiveCalls: structural typing is achieved at runtime through reflective calls (on the JVM, at least) and the compiler will tell you at compile time with a warning
naming the { def close(): Unit } to something that makes it a little bit more readable in the method signature using type
using NonFatal to handle exception (you can read more about it here)
I am trying to write a function which can add a context to those functions given in parameters.
The idea is here
object example {
def withOne(f : => T) = {
val a = 1 //some context
f
}
def foo() = withOne {
println(a)
}
}
I think the context could be passed in implicit.
The idea is to not have the content of f constraint by the surrounding function f should be able to use the context or not.
For now the only way i seen to do that is like that
object example {
def withOne(f : => Int => T) = {
val a = 1 //some context
f(a)
}
def foo() = withOne { a =>
println(a)
}
}
But this forces to declare a 'a' witch is not obvious for others devs :x
I'm afraid you cannot work around this, since you cannot inject an implicit into a function.
There's a proposal to add this feature in the typelevel/scala fork, but it seems hard to achieve as of today.
My suggestion here is to use proper naming, so that you won't surprise your users. For instance if you provide a method like:
def withConnection[A](f: Connection => A): A = {
try {
val conn = ???
f(conn)
} finally {
conn.close()
}
}
it won't surprise me to do:
withConnection { implicit c =>
// db stuff
}
Is there a better way to ensure resources are properly released - a better way to write the following code ?
val out: Option[FileOutputStream] = try {
Option(new FileOutputStream(path))
} catch {
case _ => None
}
if (out.isDefined) {
try {
Iterator.continually(in.read).takeWhile(-1 != _).foreach(out.get.write)
} catch {
case e => println(e.getMessage)
} finally {
in.close
out.get.flush()
out.get.close()
}
}
Something like that is a good idea, but I'd make it a method:
def cleanly[A,B](resource: => A)(cleanup: A => Unit)(code: A => B): Option[B] = {
try {
val r = resource
try { Some(code(r)) }
finally { cleanup(r) }
} catch {
case e: Exception => None
}
}
(note that we only catch once; if you really want a message printed in one case and not the other, then you do have to catch both like you did). (Also note that I only catch exceptions; catching Error also is usually unwise, since it's almost impossible to recover from.) The method is used like so:
cleanly(new FileOutputStream(path))(_.close){ fos =>
Iterator.continually(in.read).takeWhile(_ != -1).foreach(fos.write)
}
Since it returns a value, you'll get a Some(()) if it succeeded here (which you can ignore).
Edit: to make it more general, I'd really have it return an Either instead, so you get the exception. Like so:
def cleanly[A,B](resource: => A)(cleanup: A => Unit)(code: A => B): Either[Exception,B] = {
try {
val r = resource
try { Right(code(r)) } finally { cleanup(r) }
}
catch { case e: Exception => Left(e) }
}
Now if you get a Right, all went okay. If you get a Left, you can pick out your exception. If you don't care about the exception, you can use .right.toOption to map it into an option, or just use .right.map or whatever to operate on the correct result only if it is there (just like with Option). (Pattern matching is a useful way to deal with Eithers.)
Have a look at Scala-ARM
This project aims to be the Scala Incubator project for Automatic-Resource-Management in the scala library ...
... The Scala ARM library allows users to ensure opening closing of resources within blocks of code using the "managed" method. The "managed" method essentially takes an argument of "anything that has a close or dispose method" and constructs a new ManagedResource object.
Alternatively you can do this with Choppy's Lazy TryClose monad.
val output = for {
fin <- TryClose(in)
fout <- TryClose.wrapWithCloser(new FileOutputStream(path))(out => {out.flush(); out.close();})
} yield wrap(Iterator.continually(fin.read).takeWhile(-1 != _).foreach(fout.get.write))
// Then execute it like this:
output.resolve
More info here: https://github.com/choppythelumberjack/tryclose
(just be sure to import tryclose._ and tryclose.JavaImplicits._)
I've defined 'using' function as following:
def using[A, B <: {def close(): Unit}] (closeable: B) (f: B => A): A =
try { f(closeable) } finally { closeable.close() }
I can use it like that:
using(new PrintWriter("sample.txt")){ out =>
out.println("hellow world!")
}
now I'm curious how to define 'using' function to take any number of parameters, and be able to access them separately:
using(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt")){ (in, out) =>
out.println(in.readLIne)
}
Starting Scala 2.13, the standard library provides a dedicated resource management utility: Using.
More specifically, the Using#Manager can be used when dealing with several resources.
In our case, we can manage different resources such as your PrintWriter or BufferedReader as they both implement AutoCloseable, in order to read and write from a file to another and, no matter what, close both the input and the output resource afterwards:
import scala.util.Using
import java.io.{PrintWriter, BufferedReader, FileReader}
Using.Manager { use =>
val in = use(new BufferedReader(new FileReader("input.txt")))
val out = use(new PrintWriter("output.txt"))
out.println(in.readLine)
}
// scala.util.Try[Unit] = Success(())
Someone has already done this—it's called Scala ARM.
From the readme:
import resource._
for(input <- managed(new FileInputStream("test.txt")) {
// Code that uses the input as a FileInputStream
}
I've been thinking about this and I thought maybe there was an other way to address this. Here is my take on supporting "any number" of parameters (limited by what tuples provide):
object UsingTest {
type Closeable = {def close():Unit }
final class CloseAfter[A<:Product](val x: A) {
def closeAfter[B](block: A=>B): B = {
try {
block(x);
} finally {
for (i <- 0 until x.productArity) {
x.productElement(i) match {
case c:Closeable => println("closing " + c); c.close()
case _ =>
}
}
}
}
}
implicit def any2CloseAfter[A<:Product](x: A): CloseAfter[A] =
new CloseAfter(x)
def main(args:Array[String]): Unit = {
import java.io._
(new BufferedReader(new FileReader("in.txt")),
new PrintWriter("out.txt"),
new PrintWriter("sample.txt")) closeAfter {case (in, out, other) =>
out.println(in.readLine)
other.println("hello world!")
}
}
}
I think I'm reusing the fact that 22 tuple/product classes have been written in the library... I don't think this syntax is clearer than using nested using (no pun intended), but it was an interesting puzzle.
using structural typing seems like a little overkill since java.lang.AutoCloseable is predestined for usage:
def using[A <: AutoCloseable, B](resource: A)(block: A => B): B =
try block(resource) finally resource.close()
or, if you prefer extension methods:
implicit class UsingExtension[A <: AutoCloseable](val resource: A) extends AnyVal {
def using[B](block: A => B): B = try block(resource) finally resource.close()
}
using2 is possible:
def using2[R1 <: AutoCloseable, R2 <: AutoCloseable, B](resource1: R1, resource2: R2)(block: (R1, R2) => B): B =
using(resource1) { _ =>
using(resource2) { _ =>
block(resource1, resource2)
}
}
but imho quite ugly - I would prefer to simply nest these using statements in the client code.
Unfortunately, there isn't support for arbitrary-length parameter lists with arbitrary types in standard Scala.
You might be able to do something like this with a couple of language changes (to allow variable parameter lists to be passed as HLists; see here for about 1/3 of what's needed).
Right now, the best thing to do is just do what Tuple and Function do: implement usingN for as many N as you need.
Two is easy enough, of course:
def using2[A, B <: {def close(): Unit}, C <: { def close(): Unit}](closeB: B, closeC: C)(f: (B,C) => A): A = {
try { f(closeB,closeC) } finally { closeB.close(); closeC.close() }
}
If you need more, it's probably worth writing something that'll generate the source code.
Here is an example that allows you to use the scala for comprehension as an automatic resource management block for any item that is a java.io.Closeable, but it could easily be expanded to work for any object with a close method.
This usage seems pretty close to the using statement and allows you to easily have as many resources defined in one block as you want.
object ResourceTest{
import CloseableResource._
import java.io._
def test(){
for( input <- new BufferedReader(new FileReader("/tmp/input.txt")); output <- new FileWriter("/tmp/output.txt") ){
output.write(input.readLine)
}
}
}
class CloseableResource[T](resource: =>T,onClose: T=>Unit){
def foreach(f: T=>Unit){
val r = resource
try{
f(r)
}
finally{
try{
onClose(r)
}
catch{
case e =>
println("error closing resource")
e.printStackTrace
}
}
}
}
object CloseableResource{
implicit def javaCloseableToCloseableResource[T <: java.io.Closeable](resource:T):CloseableResource[T] = new CloseableResource[T](resource,{_.close})
}
It is a good idea to detatch the cleanup algorithm from the program path.
This solution lets you accumulate closeables in a scope.
The scope cleanup will happen on after the block is executed, or the scope can be detached. The cleaning of the scope can then be done later.
This way we get the same convenience whitout being limited to single thread programming.
The utility class:
import java.io.Closeable
object ManagedScope {
val scope=new ThreadLocal[Scope]();
def managedScope[T](inner: =>T):T={
val previous=scope.get();
val thisScope=new Scope();
scope.set(thisScope);
try{
inner
} finally {
scope.set(previous);
if(!thisScope.detatched) thisScope.close();
}
}
def closeLater[T <: Closeable](what:T): T = {
val theScope=scope.get();
if(!(theScope eq null)){
theScope.closeables=theScope.closeables.:+(what);
}
what;
}
def detatchScope(): Scope={
val theScope=scope.get();
if(theScope eq null) null;
else {
theScope.detatched=true;
theScope;
}
}
}
class Scope{
var detatched=false;
var closeables:List[Closeable]=List();
def close():Unit={
for(c<-closeables){
try{
if(!(c eq null))c.close();
} catch{
case e:Throwable=>{};
}
}
}
}
The usage:
def checkSocketConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
doWork(socket);
}
def checkFutureConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
val future:Future[Boolean]=doAsyncWork(socket);
// Detatch the scope and use it in the future.
val scope=detatchScope();
future.onComplete(v=>scope.close());
}
This solution doesn't quite have the syntax you desire, but I think it's close enough :)
def using[A <: {def close(): Unit}, B](resources: List[A])(f: List[A] => B): B =
try f(resources) finally resources.foreach(_.close())
using(List(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt"))) {
case List(in: BufferedReader, out: PrintWriter) => out.println(in.readLine())
}
Of course the down side is you have to type out the types BufferedReader and PrintWrter in the using block. You might be able to add some magic so that you just need List(in, out) by using multiple ORed type bounds for type A in using.
By defining some pretty hacky and dangerous implicit conversions you can get around having to type List (and another way to get around specifying types for the resources), but I haven't documented the detail as it's too dangerous IMO.
here is my solution to the resource management in Scala:
def withResources[T <: AutoCloseable, V](r: => T)(f: T => V): V = {
val resource: T = r
require(resource != null, "resource is null")
var exception: Throwable = null
try {
f(resource)
} catch {
case NonFatal(e) =>
exception = e
throw e
} finally {
closeAndAddSuppressed(exception, resource)
}
}
private def closeAndAddSuppressed(e: Throwable,
resource: AutoCloseable): Unit = {
if (e != null) {
try {
resource.close()
} catch {
case NonFatal(suppressed) =>
e.addSuppressed(suppressed)
}
} else {
resource.close()
}
}
I used this in multiple Scala apps including managing resources in Spark executors. and one should be aware that we are other even better ways to manage resource like in CatsIO: https://typelevel.org/cats-effect/datatypes/resource.html. if you are ok with pure FP in Scala.
to answer your last question, you can definitely nest the resource like this:
withResource(r: File)(
r => {
withResource(a: File)(
anotherR => {
withResource(...)(...)
}
)
}
)
this way, not just that those resources are protected from leaking, they will also be released in the correct order(like stack). same behaviour like the Resource Monad from CatsIO.
I have a helper method:
def controlStructure[T <: SomeObject](exceptions: Class[_]*)(body: => T) = {
try {
val tempObject = body
tempObject.callSomeMethod
Some(tempObject)
} catch {
case e if (exceptions.contains(e.getClass)) => None
}
}
called with:
controlStructure[MySomeObject](classOf[Exception]) { getMySomeObjectSomehow }
the main point of which is to call the 'callSomeMethod' on the entity passed in (for example loaded from ORM), it incidentally wraps things up in exception handling too.
I would now like to add a new method which does the same thing but for a collection (java.util.List) of T.
I am unsure of the syntax, and structures to work with a collection of T in the method signature, and abstract type param definitions.
Thanks for your help.
With a scala list, you are wanting something like this (I think):
def controlStructure[T <: SomeObject](exceptions: Class[_]*)(body: => List[T]) = {
try {
val tempObject = body
tempObject.foreach { _.callSomeMethod() }
Some(tempObject)
}
catch {
case e if (exceptions.contains(e.getClass)) => None
}
}
I haven't worked with Java lists in scala, so I'm guessing you could do it with java.util.List like this:
def controlStructure[T <: SomeObject](exceptions: Class[_]*)(body: => java.util.List[T]) = {
import scala.collection.JavaConversions._
try {
val tempObject = body
tempObject foreach { _.callSomeMethod() }
Some(tempObject)
}
catch {
case e if (exceptions.contains(e.getClass)) => None
}
}
There's no pass by name vararg in Scala. You have to pass a function if you want this. See this ticket of an enhancement request to this effect.
Thanks for your help Mitch. It turns out the answer is in this case to specify the return type of the method, as java.util.List[T], as for once Scala is not using its magic type inference to sort everything out.