According to the Learning Clojure wikibook backticks are expanded as follows
`(x1 x2 x3 ... xn)
is interpreted to mean
(clojure.core/seq (clojure.core/concat |x1| |x2| |x3| ... |xn|))
Why wrap concat with seq? What difference does it make?
Regardless of how it arose
concat returns a sequence, and
seq returns a sequence with the same content as its sequence argument,
... so seq is effectively an identity-op on a concat... except in one circumstance:
When s is an empty sequence, (seq s) is nil.
I doubt that the expansion is correct, since
`()
... evaluates to
()
... with type
clojure.lang.PersistentList$EmptyList
Whereas
(seq (concat))
... evaluates to
nil
This suggests that the wrapping call to seq is not there.
Strictly speaking, it expands to:
(macroexpand '`(x1 x2 x3))
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote user/x1)) (clojure.core/list (quote user/x2)) (clojure.core/list (quote user/x3))))
(macroexpand `(x1 x2 x3))
(user/x1 user/x2 user/x3)
Why the call seq ? Because sequences are corner stones in Clojure philosophy. I recommend you read Clojure Sequences. Otherwise, I would duplicate it here.
Related
I am reading FP in Scala.
Exercise 3.10 says that foldRight overflows (See images below).
As far as I know , however foldr in Haskell does not.
http://www.haskell.org/haskellwiki/
-- if the list is empty, the result is the initial value z; else
-- apply f to the first element and the result of folding the rest
foldr f z [] = z
foldr f z (x:xs) = f x (foldr f z xs)
-- if the list is empty, the result is the initial value; else
-- we recurse immediately, making the new initial value the result
-- of combining the old initial value with the first element.
foldl f z [] = z
foldl f z (x:xs) = foldl f (f z x) xs
How is this different behaviour possible?
What is the difference between the two languages/compilers that cause this different behaviour?
Where does this difference come from ? The platform ? The language? The compiler?
Is it possible to write a stack-safe foldRight in Scala? If yes, how?
Haskell is lazy. The definition
foldr f z (x:xs) = f x (foldr f z xs)
tells us that the behaviour of foldr f z xs with a non-empty list xs is determined by the laziness of the combining function f.
In particular the call foldr f z (x:xs) allocates just one thunk on the heap, {foldr f z xs} (writing {...} for a thunk holding an expression ...), and calls f with two arguments - x and the thunk. What happens next, is f's responsibility.
In particular, if it's a lazy data constructor (like e.g. (:)), it will immediately be returned to the caller of the foldr call (with the constructor's two slots filled by (references to) the two values).
And if f does demand its value on the right, with minimal compiler optimizations no thunks should be created at all (or one, at the most - the current one), as the value of foldr f z xs is immediately needed and the usual stack-based evaluation can used:
foldr f z [a,b,c,....,n] ==
a `f` (b `f` (c `f` (... (n `f` z)...)))
So foldr can indeed cause SO, when used with strict combining function on extremely long input lists. But if the combining function doesn't demand right away its value on the right, or only demands a part of it, the evaluation will be suspended in a thunk, and the partial result as created by f will be immediately returned. Same with the argument on the left, but they already come as thunks, potentially, in the input list.
Haskell is lazy. So foldr allocates on the heap, not the stack. Depending on the strictness of the argument function, it may allocate a single (small) result, or a large structure.
You're still losing space, compared to a strict, tail-recursive implementation, but it doesn't look as obvious, since you've traded stack for heap.
Note that the authors here are not referring to any foldRight definition in the scala standard library, such as the one defined on List. They are referring to the definition of foldRight they gave above in section 3.4.
The scala standard library defines the foldRight in terms of foldLeft by reversing the list (which can be done in constant stack space) then calling foldLeft with the the arguments of the passed function reversed. This works for lists, but won't work for a structure which cannot be safely reversed, for example:
scala> Stream.continually(false)
res0: scala.collection.immutable.Stream[Boolean] = Stream(false, ?)
scala> res0.reverse
java.lang.OutOfMemoryError: GC overhead limit exceeded
Now lets think about what should be the result of this operation:
Stream.continually(false).foldRight(true)(_ && _)
The answer should be false, it doesn't matter how many false values are in the stream or if it is infinite, if we are going to combine them with a conjunction, the result will be false.
haskell of course gets this with no problem:
Prelude> foldr (&&) True (repeat False)
False
And that is because of two important things: haskell's foldr will traverse the stream from left to right, not right to left, and haskell is lazy by default. The first item here, that foldr actually traverses the list from left to right might surprise or confuse some people who think of a right fold as starting from the right, but the important feature of a right fold is not which end of a structure it starts on, but in which direction the associativity is. So give a list [1,2,3,4] and an op named op, a left fold is
((1 op 2) op 3) op 4)
and a right fold is
(1 op (2 op (3 op 4)))
But the order of evaluation shouldn't matter. So what the authors have done here in chapter 3 is to give you a fold which traverses the list from left to right, but because scala is by default strict, we still will not be able to traverse our stream of infinite falses, but have some patience, they will get to that in chapter 5 :) I'll give you a sneak peek, lets look at the difference between foldRight as it is defined in the standard library and as it is defined in the Foldable typeclass in scalaz:
Here's the implementation from the scala standard library:
def foldRight[B](z: B)(op: (A, B) => B): B
Here's the definition from scalaz's Foldable:
def foldRight[B](z: => B)(f: (A, => B) => B): B
The difference is that the Bs are all lazy, and now we get to fold our infinite stream again, as long as we give a function which is sufficiently lazy in its second parameter:
scala> Foldable[Stream].foldRight(Stream.continually(false),true)(_ && _)
res0: Boolean = false
One easy way to demonstrate this in Haskell is to use equational reasoning to demonstrate lazy evaluation. Let's write the find function in terms of foldr:
-- Return the first element of the list that satisfies the predicate, or `Nothing`.
find :: (a -> Bool) -> [a] -> Maybe a
find p = foldr (step p) Nothing
where step pred x next = if pred x then Just x else next
foldr :: (a -> b -> b) -> b -> [a] -> b
foldr f z [] = z
foldr f z (x:xs) = f x (foldr f z xs)
In an eager language, if you wrote find with foldr it would traverse the whole list and use O(n) space. With lazy evaluation, it stops at the first element that satisfies the predicate, and uses only O(1) space (modulo garbage collection):
find odd [0..]
== foldr (step odd) Nothing [0..]
== step odd 0 (foldr (step odd) Nothing [1..])
== if odd 0 then Just 0 else (foldr (step odd) Nothing [1..])
== if False then Just 0 else (foldr (step odd) Nothing [1..])
== foldr (step odd) Nothing [1..]
== step odd 1 (foldr (step odd) Nothing [2..])
== if odd 1 then Just 1 else (foldr (step odd) Nothing [2..])
== if True then Just 1 else (foldr (step odd) Nothing [2..])
== Just 1
This evaluation stops in a finite number of steps, in spite of the fact that the list [0..] is infinite, so we know that we're not traversing the whole list. In addition, there is an upper bound on the complexity of the expressions at each step, which translates into a constant upper bound on the memory required to evaluate this.
The key here is that the step function that we're folding with has this property: no matter what the values of x and next are, it will either:
Evaluate to Just x, without invoking the next thunk, or
Tail-call the next thunk (in effect, if not literally).
I'm trying to solve a problem: I need to create a map from passed-in values, but while the symbol names for the values are consistent, the keys they map to are not. For instance: I might be passed a value that is a user ID. In the code, I can always use the symbol user-id -- but depending on other factors, I might need to make a map {"userId" user-id} or {"user_id" user-id} or {:user-id user-id} or -- well, you get the picture.
I can write a macro that gets me part-way there:
(defmacro user1 [user-id] `{"userId" ~user-id}
(defmacro user2 [user-id] `{"user_id" ~user-id}
But what I'd much rather do is define a set of maps, then combine them with a given set of symbols:
(def user-id-map-1 `{"userId" `user-id}
(defn combiner [m user-id] m) ;; <-- Around here, a miracle occurs.
I can't figure out how to get this evaluation to occur. It seems like I should be able to make a map containing un-evaluated symbols, then look up those symbols in the lexical scope of a function or macro that binds those symbols as locals -- but how?
Instead of standardizing your symbolic names, use maps with standard keyword keys. You don't need to go near macros, and you can turn your maps into records if need be without much trouble.
What you know as
(def user1 {:id 3124, :surname "Adabolo", :forenames ["Julia" "Frances"]})
... can be transformed by mapping the keys with whatever function you choose:
(defn map-keys [keymap m]
(zipmap (map keymap (keys m)) (vals m)))
For example,
(map-keys name user1)
;{"id" 3124, "surname" "Adabolo", "forenames" ["Julia" "Frances"]}
or
(map-keys {:id :user-id, :surname :family-name} user1)
;{:user-id 3124, :family-name "Adabolo", nil ["Julia" "Frances"]}
If you want rid of the nil entry, wrap the expression in (dissoc ... nil):
(defn map-keys [keymap m]
(dissoc
(zipmap (map keymap (keys m)) (vals m))
nil))
Then
(map-keys {:id :user-id, :surname :family-name} user1)
;{:user-id 3124, :family-name "Adabolo"}
I see from Michał Marczyk's answer, which has priority, that the above essentially rewrites clojure.set/rename-keys, which, however ...
leaves missing keys untouched:
For example,
(clojure.set/rename-keys user1 {:id :user-id, :surname :family-name})
;{:user-id 3124, :forenames ["Julia" "Frances"], :family-name "Adabolo"}
doesn't work with normal functions:
For example,
(clojure.set/rename-keys user1 name)
;IllegalArgumentException Don't know how to create ISeq from: clojure.core$name ...
If you forego the use of false and nil as keys, you can leave missing keys untouched and still use normal functions:
(defn map-keys [keymap m]
(zipmap (map #(or (keymap %) %) (keys m)) (vals m)))
Then
(map-keys {:id :user-id, :surname :family-name} user1)
;{:user-id 3124, :family-name "Adabolo", :forenames ["Julia" "Frances"]}
How about putting your passed-in values in a map keyed by keywords forged from the formal parameter names:
(defmacro zipfn [map-name arglist & body]
`(fn ~arglist
(let [~map-name (zipmap ~(mapv keyword arglist) ~arglist)]
~#body)))
Example of use:
((zipfn argmap [x y z]
argmap)
1 2 3)
;= {:z 3, :y 2, :x 1}
Better yet, don't use macros:
;; could take varargs for ks (though it would then need another name)
(defn curried-zipmap [ks]
#(zipmap ks %))
((curried-zipmap [:x :y :z]) [1 2 3])
;= {:z 3, :y 2, :x 1}
Then you could rekey this map using clojure.set/rename-keys:
(clojure.set/rename-keys {:z 3, :y 2, :x 1} {:z "z" :y "y" :x "x"})
;= {"x" 1, "z" 3, "y" 2}
The second map here is the "translation map" for the keys; you can construct in by merging maps like {:x "x"} describing how the individual keys ought to be renamed.
For the problem you described I can't find a reason to use macros.
I'd recommend something like
(defn assoc-user-id
[m user-id other-factors]
(assoc m (key-for other-factors) user-id))
Where you implement key-for so that it selects the key based on other-factors.
Looking at code with foldl it is hard to understand its syntax, for example:
def lstToMap(lst:List[(String,Int)], map: Map[String, Int] ):Map[String, Int] = {
(map /: lst) (addToMap)
}
Is /: infix operator? What does (map /: lst) mean, partial application? Why I can not call like this:
`/: map lst addToMap`
Method names that end in a : character can be used on the left hand side of the instance they're bound to (ie, they associate to the right). In this case, /: is a method on List. As per the Scaladoc:
Note: /: is alternate syntax for foldLeft; z /: xs is the same as xs foldLeft z.
An alternative to what you wrote would be:
lst./:(map)(addToMap)
Edit: and another alternative with foldLeft:
lst.foldLeft(map)(addToMap)
Yes, /: can be used as an infix operator. However, the fold operation takes three arguments:
The sequence to fold across
The initial value for the reduction
The function used for folding
Using infix you can only specify two of these three arguments: the sequence (which is the receiver) and the initial value. The fact that (map /: lst) is a partial application reflects the fact that you're still missing an argument. Here's an example of a product of a sequence of numbers, starting with an initial value of 1:
(1 /: xs)(_*_)
Since Scala supports curly braces for function literals, you can also use that to make the function argument look more like a function body:
(1 /: xs) { (x, y) =>
x * y
}
I am going through the Scala by Example document and I am having trouble with exercise 9.4.2. Here is the text:
Exercise 9.4.2 Consider the problem of writing a function flatten, which takes a list of element lists as arguments. The result of flatten should be the concatenation of all element lists into a single list. Here is an implementation of this method in terms of :\.
def flatten[A](xs: List[List[A]]): List[A] =
(xs :\ (Nil: List[A])) {(x, xs) => x ::: xs}
Consider replacing the body of flatten by
((Nil: List[A]) /: xs) ((xs, x) => xs ::: x)
What would be the difference in asymptotic complexity between the two versions of flatten?
In fact flatten is predefined together with a set of other userful function in an object
called List in the standatd Scala library. It can be accessed from user program by calling List.flatten. Note that flatten is not a method of class List – it would not make sense there, since it applies only to lists of lists, not to all lists in general.
I do not see how the asymptotic time of these two function variants are different. I'm sure it's because I am missing something fundamental about the meaning of fold left and fold right.
Here is a pdf of the document I am describing:
http://www.scala-lang.org/docu/files/ScalaByExample.pdf
I am generally finding this an excellent introduction into Scala.
Look at the implementation of concatenation ::: (p.68) (the rest of answer is masked with spoiler-tags, mouse-over to read !)
Witness that it's linear (in ::) in the size of the left argument (the list that ends up being the prefix of the result).
Assume (for the sake of the complexity analysis) that your list of lists contains n equal-sized small lists of size a fixed constant k, k<n. If you use foldLeft, you compute:
f (... (f (f a b1) b2) ...) bn
Where f is the concatenation. If you use foldRight:
f a1 (f a2 (... (f an b) ...))
With again f standing for the prefix notation of concatenation. In the second case it's easy : you add k elements at the head each time, so you do (k*n cons).
For the first case (foldLeft), in the first concatenation, the list (f a b1) is of size k. You add it on the second round to b2 to form (f (f a b1) b2) of size 2k ... You do (k+(k+k)+(3k)+... = k*sum_{i=1}^n(i) = k*n(n+1)/2 cons).
(Followup question : is this the only parameter that should be taken into account while thinking of the efficiency of that function ? Doesn't foldLeft have an advantage -not asymptotic complexity- that foldRight doesn't ?)
I want to make a local instance of a Java Scanner class in a clojure program. Why does this not work:
; gives me: count not supported on this type: Symbol
(let s (new Scanner "a b c"))
but it will let me create a global instance like this:
(def s (new Scanner "a b c"))
I was under the impression that the only difference was scope, but apparently not. What is the difference between let and def?
The problem is that your use of let is wrong.
let works like this:
(let [identifier (expr)])
So your example should be something like this:
(let [s (Scanner. "a b c")]
(exprs))
You can only use the lexical bindings made with let within the scope of let (the opening and closing parens). Let just creates a set of lexical bindings. I use def for making a global binding and lets for binding something I want only in the scope of the let as it keeps things clean. They both have their uses.
NOTE: (Class.) is the same as (new Class), it's just syntactic sugar.
LET is not "make a lexical binding in the current scope", but "make a new lexical scope with the following bindings".
(let [s (foo whatever)]
;; s is bound here
)
;; but not here
(def s (foo whatever))
;; s is bound here
Simplified: def is for global constants, let is for local variables.
Correct syntax:
(let [s (Scanner. "a b c")] ...)
The syntax for them is different, even if the meanings are related.
let takes a list of bindings (name value pairs) followed by expressions to evaluate in the context of those binding.
def just takes one binding, not a list, and adds it to the global context.
You could think of let as syntactic sugar for creating a new lexical scope with fn then applying it immediately:
(let [a 3 b 7] (* a b)) ; 21
; vs.
((fn [a b] (* a b)) 3 7) ; 21
So you could implement let with a simple macro and fn:
(defmacro fnlet [bindings & body]
((fn [pairs]
`((fn [~#(map first pairs)] ~#body) ~#(map last pairs)))
(partition 2 bindings)))
(fnlet [a 3 b 7] (* a b)) ; 21