So I've finally understood why Applicatives are very useful to represent parallel execution, while Monads very useful to represent sequential execution.
That being said, I've also understood that Monads are more powerful than Applicatives, so can I represent the ap function in terms of the bind function?
In other words... can I represent parallel execution with Monads?
The Monad laws have something to say about this:
Furthermore, the Monad and Applicative operations should relate as
follows:
pure = return
(<*>) = ap
Given that ap is defined to compose computations sequentially,
ap mf mx = do
f <- mf
x <- mx
return (f x)
there's only one way to read that law: a type which exposes a monadic interface cannot use Applicative to do parallel computation. You could provide a newtype wrapper for your monad which has a parallel Applicative instance and no Monad instance, but you can't do both at the same time.
In theory, theory and practice are the same, but in practice, they are not. In the real world you do in fact see people bending these rules and interpreting the above law to mean that (<*>) should be morally equivalent to ap, even if it's not exactly equivalent.
The best example of this that I know happens to be the one which directly addresses your question. Haxl is a library implementing a domain-specific language for concurrent IO. The GenHaxl monad's <*> automatically parallelises two computations where possible, whereas its >>= runs them in sequence (because it has to). This clearly goes against the letter of the law, but since Haxl is meant to be used for database reads which don't mutate anything (rather than writes, which do) you can kinda get away with it and the world doesn't explode.
You can implement <*>, and therefore also ap, from Functor and Monad:
class Functor m => Monad m where
join :: m (m a) -> m a
return :: a -> m a
(>>=) :: Monad m => m a -> (a -> m b) -> m b
m >>= f = join $ fmap f m
(<*>) :: Monad m => m (a -> b) -> m a -> m b
fs <*> xs = fs >>= \f -> xs >>= \x -> return (f x)
ap :: Monad m => m (a -> b) -> m a -> m b
ap = (<*>)
This examples hides Haskell's standard Monad definition, and instead defines Monad in terms of join and return, but you can also define join from (>>=); both ways work.
Consider a case: Lets say we have two futures.
val future1 = Future {
//some long running computation
1
}
val future2 = Future {
// some othe long running computation
2
}
future1.flatMap(_ => future2)
In the above case, future1 and future2 run parallelly provided enough threads are there in the execution pool.
We are able to run the futures parallelly. So, what does this mean?
Monads come into picture when there is data dependence between previous and current tasks (monads). But, if there is no data dependence when they can be run parallelly (at least in case of futures).
Now consider a case:
val future1 = Future {
// long running task
1
}
def compute(value: Int) = Future { value + 1}
future1.flatMap(value => compute(value))
Now one future runs after the completion of others. Now execution has to be serial because of data dependency. Second future depends on the value of first future.
Related
I was recently reading Category Theory for Programmers and in one of the challenges, Bartosz proposed to write a function called memoize which takes a function as an argument and returns the same one with the difference that, the first time this new function is called, it stores the result of the argument and then returns this result each time it is called again.
def memoize[A, B](f: A => B): A => B = ???
The problem is, I can't think of any way to implement this function without resorting to mutability. Moreover, the implementations I have seen uses mutable data structures to accomplish the task.
My question is, is there a purely functional way of accomplishing this? Maybe without mutability or by using some functional trick?
Thanks for reading my question and for any future help. Have a nice day!
is there a purely functional way of accomplishing this?
No. Not in the narrowest sense of pure functions and using the given signature.
TLDR: Use mutable collections, it's okay!
Impurity of g
val g = memoize(f)
// state 1
g(a)
// state 2
What would you expect to happen for the call g(a)?
If g(a) memoizes the result, an (internal) state has to change, so the state is different after the call g(a) than before.
As this could be observed from the outside, the call to g has side effects, which makes your program impure.
From the Book you referenced, 2.5 Pure and Dirty Functions:
[...] functions that
always produce the same result given the same input and
have no side effects
are called pure functions.
Is this really a side effect?
Normally, at least in Scala, internal state changes are not considered side effects.
See the definition in the Scala Book
A pure function is a function that depends only on its declared inputs and its internal algorithm to produce its output. It does not read any other values from “the outside world” — the world outside of the function’s scope — and it does not modify any values in the outside world.
The following examples of lazy computations both change their internal states, but are normally still considered purely functional as they always yield the same result and have no side effects apart from internal state:
lazy val x = 1
// state 1: x is not computed
x
// state 2: x is 1
val ll = LazyList.continually(0)
// state 1: ll = LazyList(<not computed>)
ll(0)
// state 2: ll = LazyList(0, <not computed>)
In your case, the equivalent would be something using a private, mutable Map (as the implementations you may have found) like:
def memoize[A, B](f: A => B): A => B = {
val cache = mutable.Map.empty[A, B]
(a: A) => cache.getOrElseUpdate(a, f(a))
}
Note that the cache is not public.
So, for a pure function f and without looking at memory consumption, timings, reflection or other evil stuff, you won't be able to tell from the outside whether f was called twice or g cached the result of f.
In this sense, side effects are only things like printing output, writing to public variables, files etc.
Thus, this implementation is considered pure (at least in Scala).
Avoiding mutable collections
If you really want to avoid var and mutable collections, you need to change the signature of your memoize method.
This is, because if g cannot change internal state, it won't be able to memoize anything new after it was initialized.
An (inefficient but simple) example would be
def memoizeOneValue[A, B](f: A => B)(a: A): (B, A => B) = {
val b = f(a)
val g = (v: A) => if (v == a) b else f(v)
(b, g)
}
val (b1, g) = memoizeOneValue(f, a1)
val (b2, h) = memoizeOneValue(g, a2)
// ...
The result of f(a1) would be cached in g, but nothing else. Then, you could chain this and always get a new function.
If you are interested in a faster version of that, see #esse's answer, which does the same, but more efficient (using an immutable map, so O(log(n)) instead of the linked list of functions above, O(n)).
Let's try(Note: I have change the return type of memoize to store the cached data):
import scala.language.existentials
type M[A, B] = A => T forSome { type T <: (B, A => T) }
def memoize[A, B](f: A => B): M[A, B] = {
import scala.collection.immutable
def withCache(cache: immutable.Map[A, B]): M[A, B] = a => cache.get(a) match {
case Some(b) => (b, withCache(cache))
case None =>
val b = f(a)
(b, withCache(cache + (a -> b)))
}
withCache(immutable.Map.empty)
}
def f(i: Int): Int = { print(s"Invoke f($i)"); i }
val (i0, m0) = memoize(f)(1) // f only invoked at first time
val (i1, m1) = m0(1)
val (i2, m2) = m1(1)
Yes there is pure functional ways to implement polymorphic function memoization. The topic is surprisingly deep and even summons the Yoneda Lemma, which is likely what Bartosz had in mind with this exercise.
The blog post Memoization in Haskell gives a nice introduction by simplifying the problem a bit: instead of looking at arbitrary functions it restricts the problem to functions from the integers.
The following memoize function takes a function of type Int -> a and
returns a memoized version of the same function. The trick is to turn
a function into a value because, in Haskell, functions are not
memoized but values are. memoize converts a function f :: Int -> a
into an infinite list [a] whose nth element contains the value of f n.
Thus each element of the list is evaluated when it is first accessed
and cached automatically by the Haskell runtime thanks to lazy
evaluation.
memoize :: (Int -> a) -> (Int -> a)
memoize f = (map f [0 ..] !!)
Apparently the approach can be generalised to function of arbitrary domains. The trick is to come up with a way to use the type of the domain as an index into a lazy data structure used for "storing" previous values. And this is where the Yoneda Lemma comes in and my own understanding of the topic becomes flimsy.
Scala has a very nice support of partial functions, mainly because in Scala when you define a partial function it also defines an isDefinedAt function for it. And also Scala has orElse and andThen functions to work with partial functions.
Haskell does support partial functions by simply non-exhaustively defining a function (though they are strongly discouraged in Haskell community). But to define isDefinedAt function in general you have to use some sort of exception handling, which I'm not being able to figure out. Once isDefinedAt function is defined then it can be used to define orElse and andThen function is already there as (.).
In short, I want to define a function,
isDefinedAt :: (a -> b) -> a -> Bool
isDefinedAt f x = -- returns True if f is defined at x else False
Can anyone please tell me how such a function can be written.
Note, I can define a function with signature
isDefinedAt :: (a -> b) -> a -> IO Bool
for generic b. But I want a function without IO in co-domain.
A nice article on Scala's Partial Functions is - How to create and use partial functions in Scala By Alvin Alexander
I recommend that, like in Scala, you use a separate type for partial functions.
import Control.Arrow
import Data.Maybe
type Partial = Kleisli Maybe
isDefinedAt :: Partial a b -> a -> Bool
isDefinedAt f x = isJust $ runKleisli f x
-- laziness should save some of the work, if possible
orElse :: Partial a b -> Partial a b -> Partial a b
orElse = (<+>)
andThen :: Partial a b -> Partial b c -> Partial a c
andThen = (>>>)
Your versions of isDefinedAt are not what Scala does (even in signature); it's very possible (though discouraged) for a PartialFunction to throw an exception when isDefinedAt is true. Or, when you define one explicitly (not using a literal), vice versa: apply doesn't have to throw when isDefinedAt is false, it's user responsibility not to call it then. So the direct equivalent would just be
data PartialFunction a b = PartialFunction { apply :: a -> b, isDefinedAt :: a -> Boolean }
which isn't particularly useful.
apply and isDefinedAt are only really linked in Scala for PartialFunction literals which requires compiler support:
A PartialFunction's value receives an additional isDefinedAt member, which is derived from the pattern match in the function literal, with each case's body being replaced by true, and an added default (if none was given) that evaluates to false.
You can emulate this by using Template Haskell, I believe, but I honestly think using the more Haskell-like approach as described in Daniel Wagner's answer is better. As he mentions, laziness helps.
Though it works even better if you make sure runKleisli f x is executed only once; optimizing cases where you have both isDefinedAt and runKleisli requires Common Subexpression Elimination, and the compiler is cautious about doing that, see Under what circumstances could Common Subexpression Elimination affect the laziness of a Haskell program?
You could do something like this (DISCLAIMER: I have not checked the laws of the relevant typeclasses, and the presence of a string in the constructor for the exception in Alternative makes me wonder if it is lawful). Scala's heterogeneous andThen is covered by fmap; its homogeneous andThen / compose are covered by the >>> / <<< from Category; orElse is covered by <|>; lift is runToMaybe.
However, without a deep compiler integration such as exists in Scala, the pattern incompleteness warnings will interact poorly with this. Haskell only has module-level pragmas for these things, and you won't want to just indiscriminately turn them off in any module where you declare inexhaustive functions, or you may get nasty surprises. Depending on your usecase, you may find optics more idiomatic and less problematic; you can have the boilerplate generated for you through Template Haskell.
(Note: I called it Inexhaustive because PartialFunction is something of a misnomer, in that it implies that Function is total. But Scala has no termination or positivity checkers, so the compiler is not actually able to talk about totality; so you get this weird situation where a function that is not a partial function is just a "regular" Function, whereas you should be able to call it a "total Function". The question here is not partially or totality, which is a broader idea, but inexhaustivity of pattern matches.)
{-# LANGUAGE TypeApplications #-}
module Inexhaustive
( Inexhaustive, inexhaustive
, runToMaybe, isDefinedAt
) where
import Prelude hiding ((.), id)
import Control.Applicative
import Control.Exception
import Control.Category
import Data.Maybe
import System.IO.Unsafe (unsafePerformIO)
newtype Inexhaustive a b = Inexhaustive (a -> b)
inexhaustive :: (a -> b) -> Inexhaustive a b
inexhaustive = Inexhaustive
runToMaybe :: Inexhaustive a b -> a -> Maybe b
runToMaybe (Inexhaustive f) x =
let io = fmap Just $ evaluate $ f x
in unsafePerformIO $ catch #PatternMatchFail io (\_ -> return Nothing)
isDefinedAt :: Inexhaustive a b -> a -> Bool
isDefinedAt f = isJust . runToMaybe f
instance Functor (Inexhaustive z) where
fmap f (Inexhaustive g) = inexhaustive (f . g)
instance Applicative (Inexhaustive z) where
pure x = inexhaustive (const x)
(Inexhaustive zab) <*> (Inexhaustive za) = Inexhaustive (\z -> zab z $ za z)
instance Alternative (Inexhaustive z) where
empty = inexhaustive (\_ -> throw $ PatternMatchFail "inexhaustive empty")
f <|> g =
inexhaustive $ \x ->
case runToMaybe f x <|> runToMaybe g x of
Just y -> y
instance Category Inexhaustive where
id = inexhaustive id
(Inexhaustive f) . (Inexhaustive g) = Inexhaustive (f . g)
I am reading scala cats book from underscore.io. It says following about Monad and Functor:
While monads and functors are the most widely used sequencing data
types..
I can see that Monad is using for sequencing data but Functor not at all. Could someone please show about sequencing computation on functors?
Seq(1, 2, 3).map(_ * 2).map(_.toString).foreach(println)
Here: you have a sequence of operations on a sequence of data.
Every monad is actually a functor, because you could implement map with flatMap and unit/pure/whatever your implementation calls it. So if you agree that monads are "sequencing data types", then you should agree on functors being them too.
Taken out of context, this statement is less clear than it could be.
A fuller version of the quote is:
While monads and functors are the most widely used sequencing data types
[...], semigroupals and applicatives are the most general.
The goal of this statement is not to erase the difference between functorial and monadic notions of "sequencing", but rather to contrast them with obviously non-sequential operations provided by Semigroupal.
Both Functor and Monad do support (different) kinds of "sequencing".
Given a value x of type F[X] for some functor F and some type X, we can "sequence" pure functions
f: X => Y
g: Y => Z
like this:
x map f map g
You can call this "sequencing", at least "elementwise". The point is that g has to wait until f produces at least a single y of type Y in order to do anything useful. However, this does not mean that all invocations of f must be finished before g is invoked for the first time, e.g. if x is a long list, one could process each element in parallel - that's why I called it "elementwise".
With monads that represent monadic effects, the notion of "sequencing" is usually taken a bit more seriously. For example, if you are working with a value x of type M[X] = Writer[Vector[String], X], and you have two functions with the "writing"-effect
f: X => M[Y]
g: Y => M[Z]
and then you sequence them like this:
x flatMap f flatMap g
you really want f to finish completely, until g begins to write anything into the Vector[String]-typed log. So here, this is literally just "sequencing", without any fine-print.
Now, contrast this with Semigroupal.
Suppose that F is semigroupal, that is, we can always form a F[(A,B)] from F[A] and F[B]. Given two functions
f: X => F[A]
g: X => F[B]
we can build a function
(x: X) => product(f(x), g(x))
that returns results of type F[(A, B)]. Note that now f and g can process x completely independently: whatever it is, it is definitely not sequencing.
Similarly, for an Applicative F and functions
f: A => X
g: B => Y
c: (X, Y) => Z
and two independent values a: F[A], b: F[B], you can process a and b completely independently with f and g, and then combine the results in the end with c into a single F[Z]:
map2(a, b){ (a, b) => c(f(a), g(b)) }
Again: f and g don't know anything about each other's inputs and outputs, they work completely independently until the very last step c, so this is again not "sequencing".
I hope it clarifies the distinction somewhat.
I was reading the purescript wiki and found following section which explains do in terms of >>=.
What does >>= mean?
Do notation
The do keyword introduces simple syntactic sugar for monadic
expressions.
Here is an example, using the monad for the Maybe type:
maybeSum :: Maybe Number -> Maybe Number -> Maybe Number
maybeSum a b = do
n <- a
m <- b
let result = n + m
return result
maybeSum takes two
values of type Maybe Number and returns their sum if neither number is
Nothing.
When using do notation, there must be a corresponding
instance of the Monad type class for the return type. Statements can
have the following form:
a <- x which desugars to x >>= \a -> ...
x which desugars to x >>= \_ -> ... or just x if this is the last statement.
A let binding let a = x. Note the lack of the in keyword.
The example maybeSum desugars to ::
maybeSum a b =
a >>= \n ->
b >>= \m ->
let result = n + m
in return result
>>= is a function, nothing more. It resides in the Prelude module and has type (>>=) :: forall m a b. (Bind m) => m a -> (a -> m b) -> m b, being an alias for the bind function of the Bind type class. You can find the definitions of the Prelude module in this link, found in the Pursuit package index.
This is closely related to the Monad type class in Haskell, which is a bit easier to find resources. There's a famous question on SO about this concept, which is a good starting point if you're looking to improve your knowledge on the bind function (if you're starting on functional programming now, you can skip it for a while).
Let's say that we have a recursive data-structure, like a binary tree. There are many ways to traverse it, and they have different memory-usage profiles. For instance, if we were to simply print the value of each node, using pseudo-code like the following in-order traversal...
visitNode(node) {
if (node == null) return;
visitNode(node.leftChild);
print(node.value);
visitNode(node.rightChild);
}
...our memory usage would be constant, but due to the recursive calls, we would increase the size of the call stack. On very large trees, this could potentially overflow it.
Let's say that we decided to optimize for call-stack size; assuming that this language is capable of proper tailcalls, we could rewrite this as the following pre-order traversal...
visitNode(node, nodes = []) {
if (node != null) {
print(node.value);
visitNode(nodes.head, nodes.tail + [node.left, node.right]);
} else if (node == null && nodes.length != 0 ) {
visitNode(nodes.head, nodes.tail);
} else return;
}
While we would never blow the stack, we would now see heap usage increase linearly with respect to the size of the tree.
Let's say we were then to attempt to lazily traverse the tree - here is where my reasoning gets fuzzy. I think that even using a basic lazy evaluation strategy, we would grow memory at the same rate as the tailcall optimized version. Here is a concrete example using Scala's Stream class, which provides lazy evaluation:
sealed abstract class Node[A] {
def toStream: Stream[Node[A]]
def value: A
}
case class Fork[A](value: A, left: Node[A], right: Node[A]) extends Node[A] {
def toStream: Stream[Node[A]] = this #:: left.toStream.append(right.toStream)
}
case class Leaf[A](value: A) extends Node[A] {
def toStream: Stream[Node[A]] = this #:: Stream.empty
}
Although only the head of the stream is strictly evaluated, anytime the left.toStream.append(right.toStream) is evaluated, I think this would actually evaluate the head of both the left and right streams. Even if it doesn't (due to append cleverness), I think that recursively building this thunk (to borrow a term from Haskell) would essentially grow memory at the same rate. Rather than saying, "put this node in the list of nodes to traverse", we're basically saying, "here's another value to evaluate that will tell you what to traverse next", but the outcome is the same; linear memory growth.
The only strategy I can think of that would avoid this is having mutable state in each node declaring which paths have been traversed. This would allow us to have a referentially transparent function that says, "Given a node, I will tell you which single node you should traverse next", and we could use that to build an O(1) iterator.
Is there another way to accomplish O(1), tailcall optimized traversal of a binary tree, possibly without mutable state?
Is there another way to accomplish O(1), tailcall optimized traversal of a binary tree, possibly without mutable state?
As I stated in my comment, you can do this if the tree need not survive the traversal. Here's a Haskell example:
data T = Leaf | Node T Int T
inOrder :: T -> [Int]
inOrder Leaf = []
inOrder (Node Leaf x r) = x : inOrder r
inOrder (Node (Node l x m) y r) = inOrder $ Node l x (Node m y r)
This takes O(1) auxiliary space if we assume the garbage collector will clean up any Node that we just processed, so we effectively replace it by a right-rotated version. However, if the nodes we process cannot immediately be garbage-collected, then the final clause may build up an O(n) number of nodes before it hits a leaf.
If you have parent pointers, then it's also doable. Parent pointers require mutable state, though, and prevent sharing of subtrees, so they're not really functional. If you represent an iterator by a pair (cur, prev) that is initially (root, nil), then you can perform iteration as outlined here. You need a language with pointer comparisons to make this work, though.
Without parent pointers and mutable state, you need to maintain some data structure that at least tracks where the root of the tree is and how to get there, since you'll need such a structure at some point during in-order or post-order traversal. Such a structure necessarily takes Ω(d) space where d is the depth of the tree.
A fancy answer.
We can use free monads to get efficient memory utilization bound.
{-# LANGUAGE RankNTypes
, MultiParamTypeClasses
, FlexibleInstances
, UndecidableInstances #-}
class Algebra f x where
phi :: f x -> x
A algebra of a functor f is a function phi from f x to x for some x. For example, any monad has a algebra for any object m x:
instance (Monad m) => Algebra m (m x) where
phi = join
A free monad for any functor f can be constructed (possibly, some sort of functors only, like omega-cocomplete, or some such; but all Haskell types are polynomial functors, which are omega-cocomplete, so the statement is certainly true for all Haskell functors):
data Free f a = Free (forall x. Algebra f x => (a -> x) -> x)
runFree g (Free m) = m g
instance Functor (Free f) where
fmap f m = Free $ \g -> runFree (g . f) m
wrap :: (Functor f) => f (Free f a) -> Free f a
wrap f = Free $ \g -> phi $ fmap (runFree g) f
instance (Functor f) => Algebra f (Free f a) where
phi = wrap
instance (Functor f) => Monad (Free f) where
return a = Free ($ a)
m >>= f = fjoin $ fmap f m
fjoin :: (Functor f) => Free f (Free f a) -> Free f a
fjoin mma = Free $ \g -> runFree (runFree g) mma
Now we can use Free to construct free monad for functor T a:
data T a b = T a b b
instance Functor (T a) where
fmap f (T a l r) = T a (f l) (f r)
For this functor we can define a algebra for object [a]
instance Algebra (T a) [a] where
phi (T a l r) = l++(a:r)
A tree is a free monad over functor T a:
type Tree a = Free (T a) ()
It can be constructed using the following functions (if defined as ADT, they'd be constructor names, so nothing extraordinary):
tree :: a -> Tree a -> Tree a -> Tree a
tree a l r = phi $ T a l r -- phi here is for Algebra f (Free f a)
-- and translates T a (Tree a) into Tree a
leaf :: Tree a
leaf = return ()
To demonstrate how this works:
bar = tree 'a' (tree 'b' leaf leaf) $ tree 'r' leaf leaf
buz = tree 'b' leaf $ tree 'u' leaf $ tree 'z' leaf leaf
foo = tree 'f' leaf $ tree 'o' (tree 'o' leaf leaf) leaf
toString = runFree (\_ -> [] :: String)
main = print $ map toString [bar, buz, foo]
As runFree traverses the tree to replace leaf () with [], the algebra for T a [a] in all contexts is the algebra that constructs a string representing in-order traversal of the tree. Because functor T a b constructs a new tree as it goes, it must have the same memory consumption characteristics as the solution quoted by larsmans - if the tree is not kept in memory, the nodes are discarded as soon as they are replaced by the string representing the whole subtree.
Given that you have references to nodes' parents, there's a nice solution posted here. Replace the while loop with a tail-recursive call (passing in last and current and that should do it.
The built-in back-references allow you to keep track of traversal ordering. Without these, I can't think of a way to do it on a (balanced) tree with less than O(log(n)) auxiliary space.
I was not able to find an answer but I got some pointers. Go have a look at http://www.ics.uci.edu/~dan/pub.html, scroll down to
[33] D.S. Hirschberg and S.S. Seiden, A bounded-space tree traversal algorithm, Information Processing Letters 47 (1993)
Download the postscript file, you may need to convert it to PDF (my ps viewer was unable to present it correctly). It mentions on page 2 (Table 1) a number of algorithms and additional literature.