Often I find myself wanting to chain a side-effecting function to the end of another method call in a more functional-looking way, but I don't want to transform the original type to Unit. Suppose I have a read method that searches a database for a record, returning Option[Record].
def read(id: Long): Option[Record] = ...
If read returns Some(record), then I might want to cache that value and move on. I could do something like this:
read(id).map { record =>
// Cache the record
record
}
But, I would like to avoid the above code and end up with something more like this to make it more clear as to what's happening:
read(id).withSideEffect { record =>
// Cache the record
}
Where withSideEffect returns the same value as read(id). After searching high and low, I can't find any method on any type that does something like this. The closest solution I can come up with is using implicit magic:
implicit class ExtendedOption[A](underlying: Option[A]) {
def withSideEffect(op: A => Unit): Option[A] = {
underlying.foreach(op)
underlying
}
}
Are there any Scala types I may have overlooked with methods like this one? And are there are any potential design flaws from using such a method?
Future.andThen (scaladoc) takes a side-effect and returns a future of the current value to facilitate fluent chaining.
The return type is not this.type.
See also duplicate questions about tap.
You can use scalaz for "explicit annotation" of side-effectful functions. In scalaz 7.0.6 it's IO monad: http://eed3si9n.com/learning-scalaz/IO+Monad.html
It's deprecated in scalaz 7.1. I would do something like that with Task
val readAndCache = Task.delay(read(id)).map(record => cacheRecord(record); record)
readAndCache.run // Run task for it's side effects
Related
Is there some sort of 'tee' operation on Option in Scala's standard library available? The best I could find is foreach, however its return type is Unit, therefore it cannot be chained.
This is what I am looking for: given an Option instance, perform some operation with side effects on its value if the option is not empty (Some[A]), otherwise do nothing; return the option in any case.
I have a custom implementation using an implicit class, but I am wondering whether there is a more common way to do this without implicit conversion:
object OptionExtensions {
implicit class TeeableOption[A](value: Option[A]) {
def tee(action: A => Unit): Option[A] = {
value foreach action
value
}
}
}
Example code:
import OptionExtensions._
val option: Option[Int] = Some(42)
option.tee(println).foreach(println) // will print 42 twice
val another: Option[Int] = None
another.tee(println).foreach(println) // does nothing
Any suggestions?
In order to avoid implicit conversion, instead of using method chaining you can use function composition with k-combinator.
k-combinator gives you an idiomatic way to communicate the fact that you are going to perform a side effect.
Here is a short example:
object KCombinator {
def tap[A](a: A)(action: A => Any): A = {
action(a)
a
}
}
import KCombinator._
val func = ((_: Option[Int]).getOrElse(0))
.andThen(tap(_)(println))
.andThen(_ + 3)
.andThen(tap(_)(println))
If we call our func with an argument of Option(3) the result will be an Int with the value of 6
and this is how the console will look like:
3
6
There is not an existing way to accomplish this in the standard library, because side effects are minimized and isolated in functional programming. Depending on what your actual goals are, there are a couple different ways to idiomatically accomplish your task.
In the case of doing a lot of println commands, instead of sprinkling them throughout your algorithm, you would typically gather them in a collection, then do one foreach println at the end. This minimizes the side effects to the smallest possible impact. That goes with any other side effect. Try to find a way to squeeze it into the smallest possible space.
If you are trying to chain a series of "actions," you should look into futures. Futures basically treat an action as a value, and provide a lot of useful functions to work with them.
Simply use map, and make your side effecting functions conform to action: A => A rather than action: A => Unit.
def tprintln[A](a: A): A = {
println(a)
a
}
another.map(tprintln).foreach(println)
I'm attempting to write a method which accepts multiple generic types and takes as an argument a unit of work to execute.
The idea is that the unit of work is a common function that itself is generic. For the sake of example, let's say it's something like the following:
def loadModelRdd[T: TypeTag](sc: SparkContext): RDD[T] = {
...
}
loadModelRdd() will construct an RDD of the given type after some internal processing like loading the Model information, etc.
A prototype method I've been hacking on looks something like the following (non-working):
def forkAll[A : Manifest, B : Manifest](work: => RDD[_]): (RDD[A], RDD[B]) = {
def aFuture = Future { work } // How can I notify that this work call returns type A?
def bFuture = Future { work } // How can I notify that this work call returns type B?
val res = for {
a <- aFuture
b <- bFuture
} yield (a.asInstanceOf[A], b.asInstanceOf[B])
Await.result(res, 10.seconds)
}
This is a shortened version of the code I'm working on as I'm actually looking at accepting as many as 10 different types.
As you can see, the overall goal of the forkAll method is to wrap the unit of work in a Future, fork-join the execution of the unit of work for each type, then return the results as a Tuple'd result. An example consumer statement would be:
val (a, b) = forkAll[ClassA, ClassB](loadModelRdd)
i.e I want to fork-join at this point and wait for the results, but I want the executions to be executed in parallel and then collected back to the Driver (Spark Driver to be specific).
The problem is I'm not sure how to coerce the type returned by the unit of work within forkAll when constructing the Future {} blocks. Without the forkAll, the implementation looked like the following:
val resA = loadModelRdd[ClassA](sc)
val resB = loadModelRdd[ClassB](sc)
...
I am looking at doing this for two reasons:
To abstract the details of fork-join for any unit of work which matches this model.
A version of this code, which explicitly states what the unit of work is, is working in Production and was responsible for cutting execution of a long-running block by close to half. I have a couple of execution steps where this pattern could be applied
Is this something that is possible in Scala's type system? Or should I look at this problem from a different perspective? I've tried a couple of implementations (including one described here) but I haven't quite found one that fits my current view of the problem
Please let me know if there is any additional information needed.
Thanks!
Short answer: Scala does not allow functions with type parameters, so what you want is not exactly possible.
You are attempting to pass a method with a type parameter. Although methods are allowed to have type parameters, functions are not. When you try to pass a method, it acts like an anonymous function, so you must specify a type.
However, since methods do allow type parameters, you can take advantage of this by creating an abstract class that will do your fork/join
abstract class ForkJoin {
protected def work[T]: RDD[T]
def apply[A, B]: (RDD[A], RDD[B]) = {
// Write implementation of fork/join here
(work[A], work[B])
}
}
then overriding the type generic work method so that it does what you want, such as calling some other pre-defined method.
val forkJoin = new ForkJoin {
override protected def work[T]: RDD[T] =
loadModelRdd[T](sc)
}
val (intRdd, stringRdd) = forkJoin[Int, String]
Check out this for a prototype implementation that compiles and runs without issues.
I have a following function:
def getIntValue(x: Int)(implicit y: Int ) : Int = {x + y}
I see above declaration everywhere. I understand what above function is doing. It is a currying function which takes two arguments. If you omit the second argument, it will invoke implicit definition which returns int instead. So I think it is something very similar to defining a default value for the argument.
implicit val temp = 3
scala> getIntValue(3)
res8: Int = 6
I was wondering what are the benefits of above declaration?
Here's my "pragmatic" answer: you typically use currying as more of a "convention" than anything else meaningful. It comes in really handy when your last parameter happens to be a "call by name" parameter (for example: : => Boolean):
def transaction(conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
conn.startTransaction // start transaction
val booleanResult = codeToExecuteInTransaction //invoke the code block they passed in
//deal with errors and rollback if necessary, or commit
//return connection to connection pool
}
What this is saying is "I have a function called transaction, its first parameter is a Connection and its second parameter will be a code-block".
This allows us to use this method like so (using the "I can use curly brace instead of parenthesis rule"):
transaction(myConn) {
//code to execute in a transaction
//the code block's last executable statement must be a Boolean as per the second
//parameter of the transaction method
}
If you didn't curry that transaction method, it would look pretty unnatural doing this:
transaction(myConn, {
//code block
})
How about implicit? Yes it can seem like a very ambiguous construct, but you get used to it after a while, and the nice thing about implicit functions is they have scoping rules. So this means for production, you might define an implicit function for getting that database connection from the PROD database, but in your integration test you'll define an implicit function that will superscede the PROD version, and it will be used to get a connection from a DEV database instead for use in your test.
As an example, how about we add an implicit parameter to the transaction method?
def transaction(implicit conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
}
Now, assuming I have an implicit function somewhere in my code base that returns a Connection, like so:
def implicit getConnectionFromPool() : Connection = { ...}
I can execute the transaction method like so:
transaction {
//code to execute in transaction
}
and Scala will translate that to:
transaction(getConnectionFromPool) {
//code to execute in transaction
}
In summary, Implicits are a pretty nice way to not have to make the developer provide a value for a required parameter when that parameter is 99% of the time going to be the same everywhere you use the function. In that 1% of the time you need a different Connection, you can provide your own connection by passing in a value instead of letting Scala figure out which implicit function provides the value.
In your specific example there are no practical benefits. In fact using implicits for this task will only obfuscate your code.
The standard use case of implicits is the Type Class Pattern. I'd say that it is the only use case that is practically useful. In all other cases it's better to have things explicit.
Here is an example of a typeclass:
// A typeclass
trait Show[a] {
def show(a: a): String
}
// Some data type
case class Artist(name: String)
// An instance of the `Show` typeclass for that data type
implicit val artistShowInstance =
new Show[Artist] {
def show(a: Artist) = a.name
}
// A function that works for any type `a`, which has an instance of a class `Show`
def showAListOfShowables[a](list: List[a])(implicit showInstance: Show[a]): String =
list.view.map(showInstance.show).mkString(", ")
// The following code outputs `Beatles, Michael Jackson, Rolling Stones`
val list = List(Artist("Beatles"), Artist("Michael Jackson"), Artist("Rolling Stones"))
println(showAListOfShowables(list))
This pattern originates from a functional programming language named Haskell and turned out to be more practical than the standard OO practices for writing a modular and decoupled software. The main benefit of it is it allows you to extend the already existing types with new functionality without changing them.
There's plenty of details unmentioned, like syntactic sugar, def instances and etc. It is a huge subject and fortunately it has a great coverage throughout the web. Just google for "scala type class".
There are many benefits, outside of your example.
I'll give just one; at the same time, this is also a trick that you can use on certain occasions.
Imagine you create a trait that is a generic container for other values, like a list, a set, a tree or something like that.
trait MyContainer[A] {
def containedValue:A
}
Now, at some point, you find it useful to iterate over all elements of the contained value.
Of course, this only makes sense if the contained value is of an iterable type.
But because you want your class to be useful for all types, you don't want to restrict A to be of a Seq type, or Traversable, or anything like that.
Basically, you want a method that says: "I can only be called if A is of a Seq type."
And if someone calls it on, say, MyContainer[Int], that should result in a compile error.
That's possible.
What you need is some evidence that A is of a sequence type.
And you can do that with Scala and implicit arguments:
trait MyContainer[A] {
def containedValue:A
def aggregate[B](f:B=>B)(implicit ev:A=>Seq[B]):B =
ev(containedValue) reduce f
}
So, if you call this method on a MyContainer[Seq[Int]], the compiler will look for an implicit Seq[Int]=>Seq[B].
That's really simple to resolve for the compiler.
Because there is a global implicit function that's called identity, and it is always in scope.
Its type signature is something like: A=>A
It simply returns whatever argument is passed to it.
I don't know how this pattern is called. (Can anyone help out?)
But I think it's a neat trick that comes in handy sometimes.
You can see a good example of that in the Scala library if you look at the method signature of Seq.sum.
In the case of sum, another implicit parameter type is used; in that case, the implicit parameter is evidence that the contained type is numeric, and therefore, a sum can be built out of all contained values.
That's not the only use of implicits, and certainly not the most prominent, but I'd say it's an honorable mention. :-)
Scala in Depth demonstrates the Loaner Pattern:
def readFile[T](f: File)(handler: FileInputStream => T): T = {
val resource = new java.io.FileInputStream(f)
try {
handler(resource)
} finally {
resource.close()
}
}
Example usage:
readFile(new java.io.File("test.txt")) { input =>
println(input.readByte)
}
This code appears simple and clear. What is an "anti-pattern" of the Loaner pattern in Scala so that I know how to avoid it?
Make sure that whatever you compute is evaluated eagerly and no longer depends on the resource. Scala makes lazy computation fairly easy. For instance, if you wrap scala.io.Source.fromFile in this way, you might try
readFile("test.txt")(_.getLines)
Unfortunately, this doesn't work because getLines is lazy (returns an iterator). And Scala doesn't have any great way to indicate which methods are lazy and which are not. So you just have to know (docs will tend to tell you), and you have to actually do the work before returning:
readFile("test.txt")(_.getLines.toVector)
Overall, it's a very useful pattern. Just make sure that all accesses to the resource are completed before exiting the block (so no uncompleted futures, no lazy vals that depend on the resource, no iterators, no returning the resource itself, no streams that haven't been fully read, etc.; of course any of these things are okay if they do not depend on the open resource but only on some fully-computed quantity based upon the resource).
With the Loan pattern it is important to know when the "bit" of code that is going to actually call your loaned resource is going to use it.
If you want to return a future from a loan pattern I advise to not create it inside the function that is passed to the loan pattern function.
Don't write
readFile("text.file")(future { doSomething })
but do:
future { readFile("text.file")( doSomething ) }
what I usually do is that I define two types of loan pattern functions: Synchronous and Async
So in your case I would have:
def asyncReadFile[T](f: File)(handler: FileInputStream => T): Future[T] = {
future{
readFile(f)(handler)
}
}
This way you avoid calling closed resources. And you reuse your already tested and hopefully correct code of the Synchronous function.
I've been reading about the OO 'fluent interface' approach in Java, JavaScript and Scala and I like the look of it, but have been struggling to see how to reconcile it with a more type-based/functional approach in Scala.
To give a very specific example of what I mean: I've written an API client which can be invoked like this:
val response = MyTargetApi.get("orders", 24)
The return value from get() is a Tuple3 type called RestfulResponse, as defined in my package object:
// 1. Return code
// 2. Response headers
// 2. Response body (Option)
type RestfulResponse = (Int, List[String], Option[String])
This works fine - and I don't really want to sacrifice the functional simplicity of a tuple return value - but I would like to extend the library with various 'fluent' method calls, perhaps something like this:
val response = MyTargetApi.get("customers", 55).throwIfError()
// Or perhaps:
MyTargetApi.get("orders", 24).debugPrint(verbose=true)
How can I combine the functional simplicity of get() returning a typed tuple (or similar) with the ability to add more 'fluent' capabilities to my API?
It seems you are dealing with a client side API of a rest style communication. Your get method seems to be what triggers the actual request/response cycle. It looks like you'd have to deal with this:
properties of the transport (like credentials, debug level, error handling)
providing data for the input (your id and type of record (order or customer)
doing something with the results
I think for the properties of the transport, you can put some of it into the constructor of the MyTargetApi object, but you can also create a query object that will store those for a single query and can be set in a fluent way using a query() method:
MyTargetApi.query().debugPrint(verbose=true).throwIfError()
This would return some stateful Query object that stores the value for log level, error handling. For providing the data for the input, you can also use the query object to set those values but instead of returning your response return a QueryResult:
class Query {
def debugPrint(verbose: Boolean): this.type = { _verbose = verbose; this }
def throwIfError(): this.type = { ... }
def get(tpe: String, id: Int): QueryResult[RestfulResponse] =
new QueryResult[RestfulResponse] {
def run(): RestfulResponse = // code to make rest call goes here
}
}
trait QueryResult[A] { self =>
def map[B](f: (A) => B): QueryResult[B] = new QueryResult[B] {
def run(): B = f(self.run())
}
def flatMap[B](f: (A) => QueryResult[B]) = new QueryResult[B] {
def run(): B = f(self.run()).run()
}
def run(): A
}
Then to eventually get the results you call run. So at the end of the day you can call it like this:
MyTargetApi.query()
.debugPrint(verbose=true)
.throwIfError()
.get("customers", 22)
.map(resp => resp._3.map(_.length)) // body
.run()
Which should be a verbose request that will error out on issue, retrieve the customers with id 22, keep the body and get its length as an Option[Int].
The idea is that you can use map to define computations on a result you do not yet have. If we add flatMap to it, then you could also combine two computations from two different queries.
To be honest, I think it sounds like you need to feel your way around a little more because the example is not obviously functional, nor particularly fluent. It seems you might be mixing up fluency with not-idempotent in the sense that your debugPrint method is presumably performing I/O and the throwIfError is throwing exceptions. Is that what you mean?
If you are referring to whether a stateful builder is functional, the answer is "not in the purest sense". However, note that a builder does not have to be stateful.
case class Person(name: String, age: Int)
Firstly; this can be created using named parameters:
Person(name="Oxbow", age=36)
Or, a stateless builder:
object Person {
def withName(name: String)
= new { def andAge(age: Int) = new Person(name, age) }
}
Hey presto:
scala> Person withName "Oxbow" andAge 36
As to your use of untyped strings to define the query you are making; this is poor form in a statically-typed language. What is more, there is no need:
sealed trait Query
case object orders extends Query
def get(query: Query): Result
Hey presto:
api get orders
Although, I think this is a bad idea - you shouldn't have a single method which can give you back notionally completely different types of results
To conclude: I personally think there is no reason whatsoever that fluency and functional cannot mix, since functional just indicates the lack of mutable state and the strong preference for idempotent functions to perform your logic in.
Here's one for you:
args.map(_.toInt)
args map toInt
I would argue that the second is more fluent. It's possible if you define:
val toInt = (_ : String).toInt
That is; if you define a function. I find functions and fluency mix very well in Scala.
You could try having get() return a wrapper object that might look something like this
type RestfulResponse = (Int, List[String], Option[String])
class ResponseWrapper(private rr: RestfulResponse /* and maybe some flags as additional arguments, or something? */) {
def get : RestfulResponse = rr
def throwIfError : RestfulResponse = {
// Throw your exception if you detect an error
rr // And return the response if you didn't detect an error
}
def debugPrint(verbose: Boolean, /* whatever other parameters you had in mind */) {
// All of your debugging printing logic
}
// Any and all other methods that you want this API response to be able to execute
}
Basically, this allows you to put your response into a contain that has all of these nice methods that you want, and, if you simply want to get the wrapped response, you can just call the wrapper's get() method.
Of course, the downside of this is that you will need to change your API a bit, if that's worrisome to you at all. Well... you could probably avoid needing to change your API, actually, if you, instead, created an implicit conversion from RestfulResponse to ResponseWrapper and vice versa. That's something worth considering.