For practice, I've gotten some straightforward JDBC stuff from like 100 lines down to this, but it won't typecheck. Any ideas? Better approaches?
def withResultSet[T](sql: String, f: ResultSet => T)(implicit info: ConnectionInfo): Try[T] = {
for {
conn <- Try(connect(info))
stmt <- Try(conn.createStatement()) orElse { case err: SQLException => {conn.close(); err} }
results <- Try(stmt.executeQuery(sql)) orElse { case err: SQLException => { conn.close(); stmt.close(); err }}
} yield f(results)
}
and my error is
missing parameter type for expanded function
The argument types of an anonymous function must be fully known. (SLS 8.5)
Expected type was: scala.util.Try[?]
stmt <- Try(conn.createStatement()) orElse { case err: SQLException => {conn.close(); err} }
^
I don't know that Try is the right tool to dispose of resources once they are no longer needed. At least I cannot see an obvious way to do it. You may want to look at https://github.com/jsuereth/scala-arm. Then your code may look like this:
def withResultSet[T](sql: String, f: ResultSet => T)(
implicit info: ConnectionInfo): ManagedResource[T] = for {
conn <- managed(connect(info))
stmt <- managed(conn.createStatement())
results <- managed(stmt.executeQuery(sql))
} yield f(results)
You can keep using map or flatMap (or for comprehension) to work with possibly other resources and at the end you can get an Option or an Either out of it and at the same time close everything that needs to be closed.
You can split your function into different parts, one for each resource type, and use it together with a separate resource manager trait.
The generalized resource manager (can be used with files etc. as well):
trait ResourceManager {
def withResource[T <: {def close()}, R](resource: T)(code: (T) => R): R = {
try {
code(resource)
} finally {
import scala.language.reflectiveCalls
resource.close()
}
}
is mixed into something like this
class Db extends ResourceManager {
private def getConnection = ...
def withConnection[T](f: (Connection) => T): T = {
withResource(getConnection) {
conn =>
f(conn)
}
}
def withStatement[T](f: (Statement) => T): T = {
withConnection {
conn =>
withResource(conn.createStatement()) {
stmnt =>
f(stmnt)
}
}
}
def withResultSet[T](selectSqlCmd: String)(f: (ResultSet) => T): T = {
withStatement {
stmnt => {
withResource(stmnt.executeQuery(selectSqlCmd)) {
rs =>
f(rs)
}
}
}
}
}
which stacks the resources, and gives entry points at each level.
On top of this another level
def mapResultSet[T, C <: Iterable[T]](selectSqlCmd: String)
(f: (ResultSet) => T)
(implicit cbf: CanBuildFrom[Nothing, T, C]): C = {
withResultSet(selectSqlCmd) {
rs =>
val builder = cbf()
while (rs.next()) {
builder += f(rs)
}
builder.result()
}
}
Each method goes only one step further down, the for comprehension is decomposed into separate functions, and the Try does not interfere.
Use it like this:
val db = new Db()
val result = db.mapResultSet("select * from pg_user")(rs => rs.getString(1))
to read from the database in a single line.
recoverWith is what i needed; you don't actually need to recover you can just rethrow.
def withResultSet[T](sql: String, f: ResultSet => T)(implicit info: ConnectionInfo): Try[T] = {
for {
conn <- Try(connect(info))
stmt <- Try(conn.createStatement()) recoverWith { case err => {conn.close(); Failure(err)} }
results <- Try(stmt.executeQuery(sql)) recoverWith { case err => { conn.close(); stmt.close(); Failure(err) }}
} yield f(results)
}
Related
I'm not sure whether I chose the right title for my question..
I'm interested as to why the collection in the companion object is defined. Am I mistaken that this collection will have only one f in it? What I am seeing is a collection with exactly one element.
Here's the Future I'm dealing with:
trait Future[+T] { self =>
def onComplete(callback: Try[T] => Unit): Unit
def map[U](f: T => U) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete (t => callback(t.map(f)))
}
def flatMap[U](f: T => Future[U]) = new Future[U] {
def onComplete(callback: Try[U] => Unit) =
self onComplete { _.map(f) match {
case Success(fu) => fu.onComplete(callback)
case Failure(e) => callback(Failure(e))
} }
}
def filter(p: T => Boolean) =
map { t => if (!p(t)) throw new NoSuchElementException; t }
}
Its companion object:
object Future {
def apply[T](f: => T) = {
val handlers = collection.mutable.Buffer.empty[Try[T] => Unit]
var result: Option[Try[T]] = None
val runnable = new Runnable {
def run = {
val r = Try(f)
handlers.synchronized {
result = Some(r)
handlers.foreach(_(r))
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handlers.synchronized {
result match {
case None => handlers += f
case Some(r) => f(r)
}
}
}
}
}
In my head I was imagining something like the following instead of the above companion object (notice how I replaced the above val handlers .. with var handler ..):
object Future {
def apply[T](f: => T) = {
var handler: Option[Try[T] => Unit] = None
var result: Option[Try[T]] = None
val runnable = new Runnable {
val execute_when_ready: Try[T] => Unit = r => handler match {
case None => execute_when_ready(r)
case Some(f) => f(r)
}
def run = {
val r = Try(f)
handler.synchronized {
result = Some(r)
execute_when_ready(r)
}
}
}
(new Thread(runnable)).start()
new Future[T] {
def onComplete(f: Try[T] => Unit) = handler.synchronized {
result match {
case None => handler = Some(f)
case Some(r) => f(r)
}
}
}
}
}
So why does the function execute_when_ready leads to stackoverflow, but that's not the case with handlers.foreach? what is the collection is offering me which I can't do without it? And is it possible to replace the collection with something else in the companion object?
The collection is not in the companion object, it is in the apply method, so there is a new instance for each Future. It is there because there can be multiple pending onComplete handlers on the same Future.
Your implementation only allows a single handler and silently removes any existing handler in onComplete which is a bad idea because the caller has no idea if a previous function has added an onComplete handler or not.
As noted in the comments, the stack overflow is because execute_when_ready calls itself if handler is None with no mechanism to stop the recursion.
I'm new to Scala, and I'm running into this strange situation.
def bar[A, B](implicit foo: A => B): B = {
// do something
foo
}
And then I got error like
require B but found A => B
How should I get B form A => B
Here's the reason why I did this, I have two functions:
def funcA: String = {
def getStrA: String = "A"
// then there's the same operation in both functions
Try{ } match {
case Success(_) => getStrA
case Failure(_) => // exactlly same error handler in both function
}
}
def funcB: Int = {
def doSomething(x: Int): Int = {
// do something
x / 1
}
val x = 1
Try{ } match {
case Success(_) => doSomething(1)
case Failure(_) => // exactlly same error handler in both function
}
}
Here's what I want to achieve
def funcA: String = {
implicit def getStrA: String = "A"
bar
}
def funcB: Int = {
val x = 1
implicit def doSomething(x: Int): Int = {
// do something
x / 1
}
bar
}
def bar[A, B](implicit foo: A => B): B = {
Try{ } match {
case Success(_) => foo
case Failure(_) => // exactlly same error handler in both function
}
}
You have a conversion from A to B. You need to return B. The only way to do this is to pass A into the function. This signature has an implied assumption that you have some valid A value (most likely hardcoded) that you will always use here.
def bar[A, B](implicit foo: A => B): B = {
val a: A = ... // hmm...
foo(a)
}
Considering, that A is parametric, then you are either missing some information, or this A is impossible to create (it cannot be null because not all types can take null as a value), so you might need to throw exception in such case. Probably you are either missing some A provider or you should always fail this operation.
UPDATE:
There is no need for using implicits at all in your code:
def bar[B](f: onSuccess: A => B) =
Try{ some operations } match {
case Success(value) => onSuccess(value)
case Failure(_) => // error handler
}
def funcA = bar(_ => "A")
def funcB = bar(_ => 1)
In Scala we have a by-name-parameters where we can write
def foo[T](f: => T):T = {
f // invokes f
}
// use as:
foo(println("hello"))
I now want to do the same with an array of methods, that is I want to use them as:
def foo[T](f:Array[ => T]):T = { // does not work
f(0) // invokes f(0) // does not work
}
foo(println("hi"), println("hello")) // does not work
Is there any way to do what I want? The best I have come up with is:
def foo[T](f:() => T *):T = {
f(0)() // invokes f(0)
}
// use as:
foo(() => println("hi"), () => println("hello"))
or
def foo[T](f:Array[() => T]):T = {
f(0)() // invokes f(0)
}
// use as:
foo(Array(() => println("hi"), () => println("hello")))
EDIT: The proposed SIP-24 is not very useful as pointed out by Seth Tisue in a comment to this answer.
An example where this will be problematic is the following code of a utility function trycatch:
type unitToT[T] = ()=>T
def trycatch[T](list:unitToT[T] *):T = list.size match {
case i if i > 1 =>
try list.head()
catch { case t:Any => trycatch(list.tail: _*) }
case 1 => list(0)()
case _ => throw new Exception("call list must be non-empty")
}
Here trycatch takes a list of methods of type ()=>T and applies each element successively until it succeeds or the end is reached.
Now suppose I have two methods:
def getYahooRate(currencyA:String, currencyB:String):Double = ???
and
def getGoogleRate(currencyA:String, currencyB:String):Double = ???
that convert one unit of currencyA to currencyB and output Double.
I use trycatch as:
val usdEuroRate = trycatch(() => getYahooRate("USD", "EUR"),
() => getGoogleRate("USD", "EUR"))
I would have preferred:
val usdEuroRate = trycatch(getYahooRate("USD", "EUR"),
getGoogleRate("USD", "EUR")) // does not work
In the example above, I would like getGoogleRate("USD", "EUR") to be invoked only if getYahooRate("USD", "EUR") throws an exception. This is not the intended behavior of SIP-24.
Here is a solution, although with a few restrictions compared to direct call-by-name:
import scala.util.control.NonFatal
object Main extends App {
implicit class Attempt[+A](f: => A) {
def apply(): A = f
}
def tryCatch[T](attempts: Attempt[T]*): T = attempts.toList match {
case a :: b :: rest =>
try a()
catch {
case NonFatal(e) =>
tryCatch(b :: rest: _*)
}
case a :: Nil =>
a()
case Nil => throw new Exception("call list must be non-empty")
}
def a = println("Hi")
def b: Unit = sys.error("one")
def c = println("bye")
tryCatch(a, b, c)
def d: Int = sys.error("two")
def e = { println("here"); 45 }
def f = println("not here")
val result = tryCatch(d, e, f)
println("Result is " + result)
}
The restrictions are:
Using a block as an argument won't work; only the last expression of the block will be wrapped in an Attempt.
If the expression is of type Nothing (e.g., if b and d weren't annotated), the conversion to Attempt is not inserted since Nothing is a subtype of every type, including Attempt. Presumably the same would apply for an expression of type Null.
As of Scala 2.11.7, the answer is no. However, there is SIP-24, so in some future version your f: => T* version may be possible.
I have this code that I use in a spray handler
get {
def callService = {
val p = Promise[Option[DocumentRef]]
val fut = p.future
archiveService.getByHash(ZeroHash, {
result => p success result
})
fut
}
onComplete(OnCompleteFutureMagnet(callService)){
case Success(docRef) => {
val doc = docRef map {
x => x.title
} getOrElse "nothing"
complete("Done with " + doc)
}
case Failure(ex) => complete("error ${ex.getMessage}")
}
}
so I had the bright idea of writing the following function to encapsulate the work done to create a future out of a promise:
def callback2Future[T](funToCall: (T => Unit) => Any): Future[T] = {
val p = Promise[T]
val resultFuture = p future
def callbacklistener(arg: T): Unit = {
arg: T => p success arg
}
funToCall(callbacklistener)
resultFuture
}
And restructure the onComplete as:
onComplete(OnCompleteFutureMagnet(callback2Future(archiveService.getByHash(ZeroHash, _: Option[DocumentRef] => Unit)))) {
case Success(docRef) => {
...
}
In the original implementation with callservice, it works great (with great throughput too), with the callback2Future implementation I get a forever wait and it eventually times out. They seem the same to me, can anyone spot the error?
I believe that your problem is due to the infamous auto-Unit feature of Scala. Your function:
def callbacklistener(arg: T): Unit = {
arg: T => p success arg
}
will probably be interpreted as:
def callbacklistener(arg: T): Unit = {
{ arg: T => p success arg }
()
}
What you really want is probably:
def callbacklistener(arg: T): Unit = p success arg
To be clear, in your implementation you are defining a function callbackListener with return type Unit; in the body of this function you have an expression, { arg: T => p success arg }, whose value is of type T => Unit and is discarded; the Scala compiler will then put a free () in your code as the return type of the callbackListener is supposed to be Unit.
Is there a way to turn a Seq[Future[X]] into an Enumerator[X] ? The use case is that I want to get resources by crawling the web. This is going to return a Sequence of Futures, and I'd like to return an Enumerator that will push the futures in the order in which they are first finished on to the Iteratee.
It looks like Victor Klang's Future select gist could be used to do this - though it looks pretty inefficient.
Note: The Iteratees and Enumerator's in question are those given by the play framework version 2.x, ie with the following imports: import play.api.libs.iteratee._
Using Victor Klang's select method:
/**
* "Select" off the first future to be satisfied. Return this as a
* result, with the remainder of the Futures as a sequence.
*
* #param fs a scala.collection.Seq
*/
def select[A](fs: Seq[Future[A]])(implicit ec: ExecutionContext):
Future[(Try[A], Seq[Future[A]])] = {
#scala.annotation.tailrec
def stripe(p: Promise[(Try[A], Seq[Future[A]])],
heads: Seq[Future[A]],
elem: Future[A],
tail: Seq[Future[A]]): Future[(Try[A], Seq[Future[A]])] = {
elem onComplete { res => if (!p.isCompleted) p.trySuccess((res, heads ++ tail)) }
if (tail.isEmpty) p.future
else stripe(p, heads :+ elem, tail.head, tail.tail)
}
if (fs.isEmpty) Future.failed(new IllegalArgumentException("empty future list!"))
else stripe(Promise(), fs.genericBuilder[Future[A]].result, fs.head, fs.tail)
}
}
I can then get what I need with
Enumerator.unfoldM(initialSeqOfFutureAs){ seqOfFutureAs =>
if (seqOfFutureAs.isEmpty) {
Future(None)
} else {
FutureUtil.select(seqOfFutureAs).map {
case (t, seqFuture) => t.toOption.map {
a => (seqFuture, a)
}
}
}
}
A better, shorter and I think more efficient answer is:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] = {
Future.sequence(seqFutureX).flatMap { seqX: Seq[X] =>
seqX.foldLeft(Future.successful(i)) {
case (i, x) => i.flatMap(_.feed(Input.El(x)))
}
}
}
}
I do realise that the question is a bit old already, but based on Santhosh's answer and the built-in Enumterator.enumerate() implementation I came up with the following:
def enumerateM[E](traversable: TraversableOnce[Future[E]])(implicit ec: ExecutionContext): Enumerator[E] = {
val it = traversable.toIterator
Enumerator.generateM {
if (it.hasNext) {
val next: Future[E] = it.next()
next map {
e => Some(e)
}
} else {
Future.successful[Option[E]] {
None
}
}
}
}
Note that unlike the first Viktor-select-based-solution this one preserves the order, but you can still start off all computations asynchronously before. So, for example, you can do the following:
// For lack of a better name
def mapEachM[E, NE](eventuallyList: Future[List[E]])(f: E => Future[NE])(implicit ec: ExecutionContext): Enumerator[NE] =
Enumerator.flatten(
eventuallyList map { list =>
enumerateM(list map f)
}
)
This latter method was in fact what I was looking for when I stumbled on this thread. Hope it helps someone! :)
You could construct one using the Java Executor Completeion Service (JavaDoc). The idea is to use create a sequence of new futures, each using ExecutorCompletionService.take() to wait for the next result. Each future will start, when the previous future has its result.
But please b e aware, that this might be not that efficient, because a lot of synchronisation is happening behind the scenes. It might be more efficient, to use some parallel map reduce for calculation (e.g. using Scala's ParSeq) and let the Enumerator wait for the complete result.
WARNING: Not compiled before answering
What about something like this:
def toEnumerator(seqFutureX: Seq[Future[X]]) = new Enumerator[X] {
def apply[A](i: Iteratee[X, A]): Future[Iteratee[X, A]] =
Future.fold(seqFutureX)(i){ case (i, x) => i.flatMap(_.feed(Input.El(x)))) }
}
Here is something I found handy,
def unfold[A,B](xs:Seq[A])(proc:A => Future[B])(implicit errorHandler:Throwable => B):Enumerator[B] = {
Enumerator.unfoldM (xs) { xs =>
if (xs.isEmpty) Future(None)
else proc(xs.head) map (b => Some(xs.tail,b)) recover {
case e => Some((xs.tail,errorHandler(e)))
}
}
}
def unfold[A,B](fxs:Future[Seq[A]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
(unfold(Seq(fxs))(fxs => fxs)(errorHandler1)).flatMap(unfold(_)(proc)(errorHandler))
}
def unfoldFutures[A,B](xsfxs:Seq[Future[Seq[A]]])(proc:A => Future[B]) (implicit errorHandler1:Throwable => Seq[A], errorHandler:Throwable => B) :Enumerator[B] = {
xsfxs.map(unfold(_)(proc)).reduceLeft((a,b) => a.andThen(b))
}
I would like to propose the use of a Broadcast
def seqToEnumerator[A](futuresA: Seq[Future[A]])(defaultValue: A, errorHandler: Throwable => A): Enumerator[A] ={
val (enumerator, channel) = Concurrent.broadcast[A]
futuresA.foreach(f => f.onComplete({
case Success(Some(a: A)) => channel.push(a)
case Success(None) => channel.push(defaultValue)
case Failure(exception) => channel.push(errorHandler(exception))
}))
enumerator
}
I added errorHandling and defaultValues but you can skip those by using onSuccess or onFailure, instead of onComplete