Following these examples and especially this code:
object Control {
def using[A <: { def close(): Unit }, B](resource: A)(f: A => B): B =
try {
f(resource)
} finally {
resource.close()
}
}
...
using(io.Source.fromFile("example.txt")) { source => { .....
I wanted to extend the using method so instead of a type which implements close it receives a string (filename), a function to open a source, and the processing function. In this way, I would avoid the exception which would be thrown in the above example in case the given file does not exist.
So I ended up with this code:
object Control
{
def using[A <: { def close(): Unit }, B](opener: String => A)(name:String)(func: A => B): Unit =
{
var resource:A
// ^ Error: 'Block cannot contain declarations'
try
{
resource = opener(name)
func(resource)
}
catch
{
case e: (_) => println(s"Failed to open resource '${name}' (${e})")
}
finally
{
println("Closing file ...")
resource.close()
}
}
}
So I am defining a method, which takes as first parameter an opener-function, which receives a string, and returns an object which implements close, a string (for the opener function), and a processing function.
However it won't let me declare the resource variable outside of the try-catch block (so I can reach it in the finally block). It will work if I just put it into the try block like var resource:A = opener(name), however then I cannot reach resource in the finally block.
How can I solve it? I have to say that I am still a beginner in Scala, so I am a bit lost here.
Here is a revised example that you can also run on Scastie:
import scala.util.control.NonFatal
import scala.language.reflectiveCalls
type Resource = { def close(): Unit }
def using[A <: Resource, B](opener: String => A)(name: String)(func: A => B): Unit = {
var resource = null.asInstanceOf[A]
try {
resource = opener(name)
func(resource)
} catch {
case NonFatal(e) => println(s"Failed to open resource '${name}' (${e.getMessage})")
} finally {
println("Closing resource...")
resource.close()
}
}
final class SomeKindOfResource(n: String) {
def use(): Int = n.toInt
def close(): Unit = {}
}
using(new SomeKindOfResource(_))("42")(n => println(n.use()))
using(new SomeKindOfResource(_))("NaN")(n => println(n.use()))
The piece that you were lacking is that initialization:
var resource = null.asInstanceOf[A]
Please note that despite what you may think, this does not throw a NullPointerException. You can read more about it here.
I've added a few more things you may be interested in:
explicitly importing scala.language.reflectiveCalls: structural typing is achieved at runtime through reflective calls (on the JVM, at least) and the compiler will tell you at compile time with a warning
naming the { def close(): Unit } to something that makes it a little bit more readable in the method signature using type
using NonFatal to handle exception (you can read more about it here)
Related
I wrote simple callback(handler) function which i pass to async api and i want to wait for result:
object Handlers {
val logger: Logger = Logger("Handlers")
implicit val cs: ContextShift[IO] =
IO.contextShift(ExecutionContext.Implicits.global)
class DefaultHandler[A] {
val response: IO[MVar[IO, A]] = MVar.empty[IO, A]
def onResult(obj: Any): Unit = {
obj match {
case obj: A =>
println(response.flatMap(_.tryPut(obj)).unsafeRunSync())
println(response.flatMap(_.isEmpty).unsafeRunSync())
case _ => logger.error("Wrong expected type")
}
}
def getResponse: A = {
response.flatMap(_.take).unsafeRunSync()
}
}
But for some reason both tryPut and isEmpty(when i'd manually call onResult method) returns true, therefore when i calling getResponse it sleeps forever.
This is the my test:
class HandlersTest extends FunSuite {
test("DefaultHandler.test") {
val handler = new DefaultHandler[Int]
handler.onResult(3)
val response = handler.getResponse
assert(response != 0)
}
}
Can somebody explain why tryPut returns true, but nothing puts. And what is the right way to use Mvar/channels in scala?
IO[X] means that you have the recipe to create some X. So on your example, yuo are putting in one MVar and then asking in another.
Here is how I would do it.
object Handlers {
trait DefaultHandler[A] {
def onResult(obj: Any): IO[Unit]
def getResponse: IO[A]
}
object DefaultHandler {
def apply[A : ClassTag]: IO[DefaultHandler[A]] =
MVar.empty[IO, A].map { response =>
new DefaultHandler[A] {
override def onResult(obj: Any): IO[Unit] = obj match {
case obj: A =>
for {
r1 <- response.tryPut(obj)
_ <- IO(println(r1))
r2 <- response.isEmpty
_ <- IO(println(r2))
} yield ()
case _ =>
IO(logger.error("Wrong expected type"))
}
override def getResponse: IO[A] =
response.take
}
}
}
}
The "unsafe" is sort of a hint, but every time you call unsafeRunSync, you should basically think of it as an entire new universe. Before you make the call, you can only describe instructions for what will happen, you can't actually change anything. During the call is when all the changes occur. Once the call completes, that universe is destroyed, and you can read the result but no longer change anything. What happens in one unsafeRunSync universe doesn't affect another.
You need to call it exactly once in your test code. That means your test code needs to look something like:
val test = for {
handler <- TestHandler.DefaultHandler[Int]
_ <- handler.onResult(3)
response <- handler.getResponse
} yield response
assert test.unsafeRunSync() == 3
Note this doesn't really buy you much over just using the MVar directly. I think you're trying to mix side effects inside IO and outside it, but that doesn't work. All the side effects need to be inside.
Edit2
Okay, so maybe I should parse out two desires here.
I had in mind that when it came time to get to the setSendTimeout(0) part, I would be using something like implicitly[Socket].
new ZContext(1) {
createSocket(ZMQ.PUB).setSendTimeout(0).//RATS!
}
I also had in mind a more generic approach to it, that would be (in pseudo code terms):
This is how you wrap a reference of T at a point in time without copying it, so that moving forward, you can tease out state of the reference after potential state changes from the value of whatever expression used it.
If it could be thought of as a chain of map map map s from T to where ever it ended up, then it is easy to append / apply a value onto it - just map again...
This is the motivating example.
override def createLogic(inheritedAttributes: Attributes): GraphStageLogic = new GraphStageLogic(shape) {
logger.info("Initializing ZMQ context.")
val context = new ZContext(1)
logger.info(s"Binding PUB socket to ${endpoint}")
val socket = {
val s = context.createSocket(ZMQ.PUB)
s.setSendTimeOut(0)
s.bind(endpoint)
s
}
Look at socket down there. For some reason that feels uglier than it needs to be to me, but is a consequence of the fact that setters don't return stuff like setSendTimeOut.
I would normally try to improve it as follows:
new ZContext(1) {
createSocket(ZMQ.PUB).setSendTimeout(0).//RATS!
}
Here a version of #Dima's answer. Again the setup:
trait Instance {
def createPort(): Port
}
trait Port {
def makeSpecial(): Unit
def bindTo(address: Any): Unit
}
trait Provider {
def getTheInstance(i: Int): Instance
}
Now the trick:
implicit class InstanceOps(i: Instance) {
def withCreatePort(fun: (Unit => Port) => Any): Port = {
val res = i.createPort()
fun(_ => res)
res
}
}
And if you add an implicit modifier to the argument of the function passed into withCreatePort, you "import" the implicit conversion:
trait ConnectTest extends Provider {
getTheInstance(2).withCreatePort { implicit p =>
().makeSpecial().bindTo("foo")
}
}
This is potentially more dangerous, because you have an implicit conversion from Unit to Port, although it is locally encapsulated. This is generic because Connect is generic.
This trick is perhaps too clever and difficult to understand by some outside standing person reading your code.
Yes, you can create two wrappers, one giving you withCreatePort, the other giving you variants of the port method that return this:
trait Instance {
def createPort(): Port
}
trait Port {
def makeSpecial(): Unit
def bindTo(address: Any): Unit
}
class PortOps(p: Port) {
def makeSpecial() : this.type = { p.makeSpecial() ; this }
def bindTo(address: Any): this.type = { p.bindTo(address); this }
}
implicit class InstanceOps(i: Instance) {
def withCreatePort[A](fun: PortOps => A): A = fun(new PortOps(i.createPort()))
}
Example:
trait Provider {
def getTheInstance(i: Int): Instance
}
trait Plain extends Provider {
val instance = getTheInstance(2)
val port = instance.createPort()
port.makeSpecial()
port.bindTo("foo")
}
trait Rich extends Provider {
getTheInstance(2).withCreatePort { p =>
p.makeSpecial().bindTo("foo")
}
}
The question is if the effort is worth it. You can also experiment with import:
trait Import extends Provider {
val instance = getTheInstance(2)
val port = instance.createPort()
locally {
import port._
makeSpecial(); bindTo("foo")
}
}
I am not sure where you are going with this Zipped thingy ... But what you described in the beginning of your question (assuming that port in the end of that snippet is a typo, and you really meant to return instance) can be done with something like this:
object Taps {
implicit class Tap[T](t: T) extends Anyval {
def tap(f: T => Unit) = { f(t); t }
}
}
Then you can write:
import Taps._
val instance = getTheInstance(2).tap {
_.createPort
.makeSpecial
.bindTo(...)
}
Is this what you are looking for?
I am struggling with the type system. I get a "error: type mismatch" at the line
handler.addJob(job1)
It says found "MessageEvent" required "Event"
I think that I need to somehow change the addJob method to pass in any Job with a type that extends Event but I can't figure out how to do that.
Also the line
var jobs = List[Job[Event]]()
should probably take a job with a subtype of Event but again I don't know how to do that. Any help is appreciated.
-Eric
class EventHandler {
var jobs = List[Job[Event]]()
def receive(event: Event) {
jobs.foreach {
_.processEvent(event)
}
}
def addJob(job: Job[Event]) {
jobs = job :: jobs
}
}
class Job[T <: Event] {
var steps = List[(T => Unit)]()
def addStep(step: (T => Unit)) {
steps = step :: steps
}
def processEvent(event: T): Boolean = {
steps.foreach(_.apply(event))
return true
}
}
class AppTest {
def testApp {
val handler = new EventHandler()
val job1 = new Job[MessageEvent]
job1.addStep {
println(_)
}
handler.addJob(job1)
handler.receive(new MessageEvent(new Message()))
}
}
The problems you mention are easy to fix:
class EventHandler {
var jobs = List[Job[_]]()
def receive(event: Event) {
jobs.foreach {
_.processEvent(event)
}
}
def addJob(job: Job[_]) {
jobs = job :: jobs
}
}
But this shows another problem with the receive method: you need each job to process any Event. This can be fixed using Manifests to work around type erasure:
class Job[T <: Event : ClassManifest] {
val clazz: Class[T] = implicitly[ClassManifest[T]].asInstanceOf[Class[T]]
var steps = List[(T => Unit)]()
def addStep(step: (T => Unit)) {
steps = step :: steps
}
def processEvent1(event: Event): Boolean = {
try {
processEvent(clazz.cast(event))
}
catch {
case e: ClassCastException => false
}
}
def processEvent(event: T): Boolean = {
steps.foreach(_.apply(event))
return true
}
}
Changing addJobs:
def addJob[T <: Event](job: Job[T]) {
jobs = job :: jobs
}
But jobs won't work with that, since Job[MessageEvent] is not a Job[Event]. The only way to get that is to make Job co-variant, but, unfortunately, you can't make Job co-variant as it is.
Why don't you, instead, completely removes Job's parameterization and use Event internally? You can then use T <: Event (like above in addJob) with addStep and processEvent, if necessary.
Based on your example, it looks as though you'll be building the Job and EventHandler instances statically. In this case, you really don't need those classes at all!
Starting with Job. This performs two roles:
maintain a list of T => Unit functions
execute those functions
(it's also worth noting that :: prepends, so steps will be executed in the reverse of the order they were added)
Building and maintaining that list of functions at runtime (within a mutable list) can be completely avoided if you already know what they'll be when the thing compiles. This is most naturally done with an aggregate function:
val job = (m: MessageEvent) => {
log.debug(m)
println(m)
somethingElse(m)
}
Instead of holding a List[Job[Event]], this means that EventHandler now holds a List[(T => Unit)] (as Job previously did). So rinse and repeat...
Is there more elegant way to write the following?
try {
... // Some throwing code
return first
}
catch {
case e:ExceptionType => {} // No code to execute. Ignore error.
}
return second
scala.util.control.Exception.ignoring(classOf[ExceptionType]) {
... // Some throwing code
}
#Daniel has already provided the canonical method to use to do this. Look through the other methods in scala.util.control.Exception--they are quite helpful and generic!
If you need to get a return value out of the try block, use failing instead of ignoring (but be aware that the result is an Any, i.e. not typesafe).
You can also write your own exception-catcher, which will be a little slow for heavy-duty work but otherwise nice to use:
class DefaultOn[E <: Exception] {
def apply[A](default: => A)(f: => A)(implicit m: Manifest[E]) = {
try { f } catch { case x if (m.erasure.isInstance(x)) => default }
}
}
object DefaultOn { def apply[E <: Exception] = new DefaultOn[E] }
scala> DefaultOn[NumberFormatException](0) { "Hi".toInt }
res0: Int = 0
Or if you like options:
class TryOption[E <: Exception] {
def apply[A](f: => A)(implicit m: Manifest[E]) = {
try { Some(f) } catch { case x if (m.erasure.isInstance(x)) => None }
}
}
object TryOption { def apply[E <: Exception] = new TryOption[E] }
scala> TryOption[NumberFormatException] { "Hi".toInt }
res1: Option[Int] = None
Or you can be inspired by this plus the library routines and create your own methods to ignore multiple different exceptions and preserve types on the return value.
In Scala all exceptions are not checked, so if you don't want, you may just skip handling them (and thus exception will be escalated to a higher level). Silently ignoring an exception the way you want to do is generally a bad practice. However, your code can be shortened to:
try {
... // Some throwing code
} catch {
case e:ExceptionType =>
}
Hows about:
Try {
// some throwing code
}
This will ignore all non fatal exceptions, which is what you want to do most of the time.
I've defined 'using' function as following:
def using[A, B <: {def close(): Unit}] (closeable: B) (f: B => A): A =
try { f(closeable) } finally { closeable.close() }
I can use it like that:
using(new PrintWriter("sample.txt")){ out =>
out.println("hellow world!")
}
now I'm curious how to define 'using' function to take any number of parameters, and be able to access them separately:
using(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt")){ (in, out) =>
out.println(in.readLIne)
}
Starting Scala 2.13, the standard library provides a dedicated resource management utility: Using.
More specifically, the Using#Manager can be used when dealing with several resources.
In our case, we can manage different resources such as your PrintWriter or BufferedReader as they both implement AutoCloseable, in order to read and write from a file to another and, no matter what, close both the input and the output resource afterwards:
import scala.util.Using
import java.io.{PrintWriter, BufferedReader, FileReader}
Using.Manager { use =>
val in = use(new BufferedReader(new FileReader("input.txt")))
val out = use(new PrintWriter("output.txt"))
out.println(in.readLine)
}
// scala.util.Try[Unit] = Success(())
Someone has already done this—it's called Scala ARM.
From the readme:
import resource._
for(input <- managed(new FileInputStream("test.txt")) {
// Code that uses the input as a FileInputStream
}
I've been thinking about this and I thought maybe there was an other way to address this. Here is my take on supporting "any number" of parameters (limited by what tuples provide):
object UsingTest {
type Closeable = {def close():Unit }
final class CloseAfter[A<:Product](val x: A) {
def closeAfter[B](block: A=>B): B = {
try {
block(x);
} finally {
for (i <- 0 until x.productArity) {
x.productElement(i) match {
case c:Closeable => println("closing " + c); c.close()
case _ =>
}
}
}
}
}
implicit def any2CloseAfter[A<:Product](x: A): CloseAfter[A] =
new CloseAfter(x)
def main(args:Array[String]): Unit = {
import java.io._
(new BufferedReader(new FileReader("in.txt")),
new PrintWriter("out.txt"),
new PrintWriter("sample.txt")) closeAfter {case (in, out, other) =>
out.println(in.readLine)
other.println("hello world!")
}
}
}
I think I'm reusing the fact that 22 tuple/product classes have been written in the library... I don't think this syntax is clearer than using nested using (no pun intended), but it was an interesting puzzle.
using structural typing seems like a little overkill since java.lang.AutoCloseable is predestined for usage:
def using[A <: AutoCloseable, B](resource: A)(block: A => B): B =
try block(resource) finally resource.close()
or, if you prefer extension methods:
implicit class UsingExtension[A <: AutoCloseable](val resource: A) extends AnyVal {
def using[B](block: A => B): B = try block(resource) finally resource.close()
}
using2 is possible:
def using2[R1 <: AutoCloseable, R2 <: AutoCloseable, B](resource1: R1, resource2: R2)(block: (R1, R2) => B): B =
using(resource1) { _ =>
using(resource2) { _ =>
block(resource1, resource2)
}
}
but imho quite ugly - I would prefer to simply nest these using statements in the client code.
Unfortunately, there isn't support for arbitrary-length parameter lists with arbitrary types in standard Scala.
You might be able to do something like this with a couple of language changes (to allow variable parameter lists to be passed as HLists; see here for about 1/3 of what's needed).
Right now, the best thing to do is just do what Tuple and Function do: implement usingN for as many N as you need.
Two is easy enough, of course:
def using2[A, B <: {def close(): Unit}, C <: { def close(): Unit}](closeB: B, closeC: C)(f: (B,C) => A): A = {
try { f(closeB,closeC) } finally { closeB.close(); closeC.close() }
}
If you need more, it's probably worth writing something that'll generate the source code.
Here is an example that allows you to use the scala for comprehension as an automatic resource management block for any item that is a java.io.Closeable, but it could easily be expanded to work for any object with a close method.
This usage seems pretty close to the using statement and allows you to easily have as many resources defined in one block as you want.
object ResourceTest{
import CloseableResource._
import java.io._
def test(){
for( input <- new BufferedReader(new FileReader("/tmp/input.txt")); output <- new FileWriter("/tmp/output.txt") ){
output.write(input.readLine)
}
}
}
class CloseableResource[T](resource: =>T,onClose: T=>Unit){
def foreach(f: T=>Unit){
val r = resource
try{
f(r)
}
finally{
try{
onClose(r)
}
catch{
case e =>
println("error closing resource")
e.printStackTrace
}
}
}
}
object CloseableResource{
implicit def javaCloseableToCloseableResource[T <: java.io.Closeable](resource:T):CloseableResource[T] = new CloseableResource[T](resource,{_.close})
}
It is a good idea to detatch the cleanup algorithm from the program path.
This solution lets you accumulate closeables in a scope.
The scope cleanup will happen on after the block is executed, or the scope can be detached. The cleaning of the scope can then be done later.
This way we get the same convenience whitout being limited to single thread programming.
The utility class:
import java.io.Closeable
object ManagedScope {
val scope=new ThreadLocal[Scope]();
def managedScope[T](inner: =>T):T={
val previous=scope.get();
val thisScope=new Scope();
scope.set(thisScope);
try{
inner
} finally {
scope.set(previous);
if(!thisScope.detatched) thisScope.close();
}
}
def closeLater[T <: Closeable](what:T): T = {
val theScope=scope.get();
if(!(theScope eq null)){
theScope.closeables=theScope.closeables.:+(what);
}
what;
}
def detatchScope(): Scope={
val theScope=scope.get();
if(theScope eq null) null;
else {
theScope.detatched=true;
theScope;
}
}
}
class Scope{
var detatched=false;
var closeables:List[Closeable]=List();
def close():Unit={
for(c<-closeables){
try{
if(!(c eq null))c.close();
} catch{
case e:Throwable=>{};
}
}
}
}
The usage:
def checkSocketConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
doWork(socket);
}
def checkFutureConnect(host:String, portNumber:Int):Unit = managedScope {
// The close later function tags the closeable to be closed later
val socket = closeLater( new Socket(host, portNumber) );
val future:Future[Boolean]=doAsyncWork(socket);
// Detatch the scope and use it in the future.
val scope=detatchScope();
future.onComplete(v=>scope.close());
}
This solution doesn't quite have the syntax you desire, but I think it's close enough :)
def using[A <: {def close(): Unit}, B](resources: List[A])(f: List[A] => B): B =
try f(resources) finally resources.foreach(_.close())
using(List(new BufferedReader(new FileReader("in.txt")), new PrintWriter("out.txt"))) {
case List(in: BufferedReader, out: PrintWriter) => out.println(in.readLine())
}
Of course the down side is you have to type out the types BufferedReader and PrintWrter in the using block. You might be able to add some magic so that you just need List(in, out) by using multiple ORed type bounds for type A in using.
By defining some pretty hacky and dangerous implicit conversions you can get around having to type List (and another way to get around specifying types for the resources), but I haven't documented the detail as it's too dangerous IMO.
here is my solution to the resource management in Scala:
def withResources[T <: AutoCloseable, V](r: => T)(f: T => V): V = {
val resource: T = r
require(resource != null, "resource is null")
var exception: Throwable = null
try {
f(resource)
} catch {
case NonFatal(e) =>
exception = e
throw e
} finally {
closeAndAddSuppressed(exception, resource)
}
}
private def closeAndAddSuppressed(e: Throwable,
resource: AutoCloseable): Unit = {
if (e != null) {
try {
resource.close()
} catch {
case NonFatal(suppressed) =>
e.addSuppressed(suppressed)
}
} else {
resource.close()
}
}
I used this in multiple Scala apps including managing resources in Spark executors. and one should be aware that we are other even better ways to manage resource like in CatsIO: https://typelevel.org/cats-effect/datatypes/resource.html. if you are ok with pure FP in Scala.
to answer your last question, you can definitely nest the resource like this:
withResource(r: File)(
r => {
withResource(a: File)(
anotherR => {
withResource(...)(...)
}
)
}
)
this way, not just that those resources are protected from leaking, they will also be released in the correct order(like stack). same behaviour like the Resource Monad from CatsIO.