There's something I must be missing about the threading macro in Clojure.
I have a map with values that are maps as well, and I'd like to a lookup in the result of another lookup. Let the map be a simple {:a {:b 2}} -- first I want to look up the key :a, that's going to yield {:b 2}, then look up b, the result is 2. The key for the second lookup needs to be a result of a function.
((fn [x] (get x :b)) ({:a {:b 2} } :a ))
=> 2
Ok, let's make it more readable with the threading macro.
(-> {:a {:b 2} } :a (fn [x] (get x :b)))
I.e. apply :a as a function on the map then apply another function. Well, this doesn't work:
CompilerException java.lang.IllegalArgumentException: Parameter declaration :a should be a vector
Oddly enough, if the anonymous function is extracted to a named one, then it works fine:
(defn f [x] (get x :b))
(-> {:a {:b 2} } :a f)
=> 2
Or even:
(def f (fn [x] (get x :b)) )
(-> {:a {:b 2} } :a f)
=> 2
Why is there a difference between how named and anonymous functions work?
The threading macro sees, and alters, each subform in the series before that form is evaluated, by recursively inserting the prior form as the first argument to each subform.
you start with:
(-> {:a {:b 2} } :a (fn [x] (get x :b)))
this becomes:
(-> (:a {:a {:b 2}}) (fn [x] (get x :b)))
this becomes:
(fn (:a {:b {:b 2}}) [x] (get x :b)))
Which is clearly not what you wanted at all.
But let's see what happens if you add extra parens around the anonymous function:
(-> {:a {:b 2}} :a ((fn [x] (get x :b))))
(-> (:a {:a {:b 2}}) ((fn [x] (get x :b))))
(-> ((fn [x] (get x :b)) (:a {:a {:b 2}})))
((fn [x] (get x :b)) (:a {:a {:b 2}}))
At the last recursive macroexpansion of the -> form we are now left with valid code that does what you want.
To complement noisesmith's response, in this particular case you don't need the threading macro. The idiomatic way to get a value from a nested map is get-in. e.g:
(get-in {:a {:b 2}} [:a :b])
=>
2
Related
I encountered a strange problem relating to defmacro in Clojure, I have code like
(defmacro ttt
([] (ttt 1))
([a] (ttt a 2))
([a b] (ttt a b 3))
([a b c] `(println ~a ~b ~c)))
and I run with (ttt), it suppose to become (println 1 2 3), and print "1 2 3", but what I got is
ArityException Wrong number of args (-1) passed to: t1$ttt clojure.lang.Compiler.macroexpand1 (Compiler.java:6473)
after some investigation, I understand I should write
(defmacro ttt
([] `(ttt 1))
([a] `(ttt ~a 2))
([a b] `(ttt ~a ~b 3))
([a b c] `(println ~a ~b ~c)))
but why the first version failed? and args is too strange to understand, where -1 comes from?
Macros have two hidden arguments
Macros have two hidden arguments &form and &env that provide additional information about invocation and bindings that are the cause of the arity exception here. To refer to other arity versions within the same macro, use quasi-quote expansion.
(defmacro baz
([] `(baz 1))
([a] `(baz ~a 2))
([a b] `(baz ~a ~b 3))
([a b c] `(println ~a ~b ~c)))
user=> (macroexpand-1 '(baz))
(clojure.core/println 1 2 3)
user=> (baz)
1 2 3
nil
Arity exception messages subtract the hidden arguments from the count
The reason you get the (-1) arity exception is because the compiler subtracts these two hidden arguments when generating the error message for the general macro usage. The true message here for your first version of ttt would be "Wrong number of args (1)" because you supplied one argument a but the two additional hidden arguments were not provided by self-invocation.
Multi-arity macros not common in the wild
In practice, I suggest avoiding multi-arity macros altogether. Instead, consider a helper function to do most of the work on behalf of the macro. Indeed, this is often a good practice for other macros as well.
(defn- bar
([] (bar 1))
([a] (bar a 2))
([a b] (bar a b 3))
([a b c] `(println ~a ~b ~c)))
(defmacro foo [& args] (apply bar args))
user=> (macroexpand-1 '(foo))
(clojure.core/println 1 2 3)
user=> (foo)
1 2 3
nil
Macro expansion is recursive
Your second ttt version works as well due to the recursive nature of macro-expansion
user=> (macroexpand-1 '(ttt))
(user/ttt 1)
user=> (macroexpand-1 *1)
(user/ttt 1 2)
user=> (macroexpand-1 *1)
(usr/ttt 1 2 3)
user=> (macroexpand-1 *1)
(clojure.core/println 1 2 3)
So,
user=> (macroexpand '(ttt))
(clojure.core/println 1 2 3)
When Clojure processes definition of ttt macro, it isn't created yet and cannot be used for source code transformation inside the macrodefinition. For compiler, your macro is something like (well, not really, but it is a good example):
(defmacro ttt0 [] (ttt1 1))
(defmacro ttt1 [a] (ttt2 a 2))
(defmacro ttt2 [a b] (ttt3 a b 3))
(defmacro ttt3 [a b c] `(println ~a ~b ~c))
Try to evaluate denition of ttt0, you will get:
CompilerException java.lang.RuntimeException: Unable to resolve symbol: ttt1 in this context
So, when Clojure processes definition of a macro, it must expand macros in unquoted parts of the definition, as for any other part of code. It fails with ttt1, and must fail in your case. My guess is that it is something like a bug. It is difficult to say why you get -1, I think it has to do with internal machinery of the language realization.
Here we can see difference between macros and functions: macros work on any input code to transform it immediately, while a function must be called and everything is always defined and ready for it:
user> (defn ttt
([] (ttt 1))
([a] (ttt a 2))
([a b] (ttt a b 3))
([a b c] :works!))
;; => #'user/ttt
user> (ttt)
;; => :works!
Here calls of ttt are just instructions, they will be executed when ttt is called.
Most of my application state is stored in a large complex map. For the purposes of this question, I will use a simple structure:
(def data
{:a 1
:b {:c {:d 3}}})
I have a large number of functions which all follow the same pattern:
(defn update-map
[my-map val]
(let [a (:a my-map)
d (-> my-map :b :c :d)]
(assoc-in
(assoc my-map :a (+ a val))
[:b :c :d] (+ d val))))
I retrieve one or more values from the map, perform some calculations, and create a new map with updated values. There are two problems with this approach:
I have a lot of repetitive let bindings across different function definitions
If the schema of the map changes, I will have a lot of code to refactor
I've written a macro to reduce the boilerplate code required to define these functions. It works by looking up predefined getter and setter functions, and automatically generating a let block:
(def getters
{'a #(:a %)
'd #(-> % :b :c :d)})
(def setters
{'a #(assoc % :a %2)
'd #(assoc-in % [:b :c :d] %2)})
(defmacro def-map-fn
[name [& args] [& fields] & code]
(let [my-map 'my-map
lookup #(reduce % [] fields)
getter-funcs (lookup #(conj % %2 (list (getters %2) my-map)))
setter-funcs (lookup #(conj % (symbol (str "update-" %2)) (setters %2)))]
`(defn ~name [~my-map ~#args]
(let [~#getter-funcs ~#setter-funcs]
~#code))))
I can now define my functions more elegantly:
(def-map-fn update-map
[val] ; normal function parameters
[a d] ; fields from the map I will be using
(update-d
(update-a my-map (+ a val))
(+ d val)))
When expanded, it will produce a function definition looking something like this:
(defn update-map
[my-map val]
(let [a (#(:a %) my-map)
d (#(-> % :b :c :d) my-map)
update-a #(assoc % :a %2)
update-d #(assoc-in % [:b :c :d] %2)]
(update-d
(update-a my-map (+ a val))
(+ d val))))
One thing that is nagging me about my macro is that it is not intuitive to the programmer that the my-map function parameter is available for use within the function body.
Is this a good use of macros, or should I be using a different approach entirely (like dynamic var bindings)?
You could perhaps use lenses; the getters and setters then become composable functions. Have a look here or here.
Following the first link you can set up the lens as follows:
; We only need three fns that know the structure of a lens.
(defn lens [focus fmap] {:focus focus :fmap fmap})
(defn view [x {:keys [focus]}] (focus x))
(defn update [x {:keys [fmap]} f] (fmap f x))
; The identity lens.
(defn fapply [f x] (f x))
(def id (lens identity fapply))
; Setting can be easily defined in terms of update.
(defn put [x l value] (update x l (constantly value)))
(-> 3 (view id))
; 3
(-> 3 (update id inc))
; 4
(-> 3 (put id 7))
; 7
; in makes it easy to define lenses based on paths.
(defn in [path]
(lens
(fn [x] (get-in x path))
(fn [f x] (update-in x path f))))
(-> {:value 3} (view (in [:value])))
; 3
(-> {:value 3} (update (in [:value]) inc))
; {:value 4}
(-> {:value 3} (put (in [:value]) 7))
; {:value 7}
You can see form the above the the lens can be adapted to use get/set methods (such as get-in/update-in) based on the data structure you are working with. The real power of lenses which seems to also be what you are after is that you can compose them. In the same example the composition function can be defined as follows:
(defn combine [outer inner]
(lens
(fn [x] (-> x (view outer) (view inner)))
(fn [f x] (update x outer #(update % inner f)))))
(defn => [& lenses] (reduce combine lenses))
The => function can now be used to combine any arbitrary lenses such as:
(-> {:new {:value 3}} (view (=> (in [:new]) (in [:value]))))
; 3
(-> {:new {:value 3}} (update (=> (in [:new]) (in [:value])) inc))
; {:new {:value 4}}
(-> {:new {:value 3}} (put (=> (in [:new]) (in [:value])) 7))
; {:new {:value 7}}
The fact that (in [:new]) is just a function means that you could, for example, store it and manipulate it in various ways. For example, it would be possible to walk your nested map structure and create the lens functions which correspond to accessing the value at each level in the nested map and then at the end compose these functions together to create your getter/setter api. With this set up, your lenses could automatically adapt to any changes in your schema.
The ability to compose lenses can also make it easy to interact with the nodes of your nested map. For example, if you were to ever change the node from an atom to a list, you could simply add a new lens to work with it as follows:
(def each (lens seq map))
(-> {:values [3 4 5]} (view (=> (in [:values]) each)))
; (3 4 5)
(-> {:values [3 4 5]} (update (=> (in [:values]) each) inc))
; {:values (4 5 6)}
(-> {:values [3 4 5]} (put (=> (in [:values]) each) 7))
; {:values (7 7 7)}
I highly recommend looking at the full gist to see more examples of what you can do with lenses.
In this situation, my preference is to avoid the use of macros. They often obfuscate code, but more importantly they aren't composable. An ideal solution here would allow you to use the getter and setter functions outside of functions being defined in def-map-fn. I'd stick with regular functions and data as much as possible.
To begin with, you're concerned about having to rewrite a bunch of code if your schema changes. Fair enough. To address that, I'd start with a data representation of your map's schema. See Prismatic schema for a full-featured schema library for Clojure, but for now something along these lines should do:
(def my-schema
{:a :int
:b {:c {:d :int}}})
From this, you can compute the paths for all the properties in your schema:
(defn paths [m]
(mapcat (fn [[k v]]
(conj (if (map? v)
(map (partial apply vector k) (paths v)))
[k]))
m))
(def property-paths
(into {} (for [path (paths my-schema)] [(last path) path])))
Now, to get or set a property, you can look up its path and use that in conjunction with get-in, update-in, etc. as appropriate:
(let [d (get-in my-map (property-paths :d))]
;; Do something with d.
)
If you get tired of always calling get-in, assoc-in, etc., then you can pretty easily generate a bunch of getter functions:
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "get-" (name p)))
[m#] (get-in m# ~path))))
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "set-" (name p)))
[m# v#] (assoc-in m# ~path v#))))
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "update-" (name p)))
[m# tail#] (apply update-in m# ~path #tail))))
Now you have your get-a, set-a, update-a functions available everywhere in your code, without having to call into some uber-macro to set up the bindings for you. For instance:
(let [a (get-a my-map)]
(-> my-map
(set-a 42)
(update-d + a)))
If you really find setting up the above let binding tedious, you could even write a with-properties macro that accepts a map and a list of property names and executes the body in a context that binds values for those names. But I probably wouldn't bother.
Advantages of this approach include:
It's schema-driven, so the schema is defined in one central place and used to generate other code as needed.
It prefers pure functions over macros, so code is more re-usable and composable.
It's an incremental approach that allows your application to grow more naturally. Rather than starting with an uber-macro that tries to anticipate all possible features you might want, you start with data and functions and sprinkle in macros to alleviate some of the repetitiveness as you see usage patterns emerge.
Why don't you just use update-in?
(defn update-map [my-map val]
(-> my-map
(update-in [:a] + val)
(update-in [:b :c :d] + val)))
I have the following Clojure code:
(defn mul [a b]
(* a b))
(defmacro create-my-macro [macroname]
`(defmacro ~macroname [a# b#]
(mul a# b#)))
(create-my-macro my-mul)
(my-mul 1 2)
;; => 2
(my-mul (+ 1 1) 2)
;; => ClassCastException clojure.lang.PersistentList cannot be cast to java.lang.Number
I get the answer I want when I wrap the arguments in the invocation of mul with eval:
(mul (eval #a) (eval #b))
But I don't understand why it is necessary to do that: if the my-mul macro had been defined directly (and not through another macro), it would work. For example, the following works fine:
(defmacro my-mul [a b] `(mul ~a ~b))
(my-mul (+ 1 1) 2)
;; => 4
Why am I seeing this behavior?
Edit: In response to a comment, below are the macroexpands for the failing case (i.e., without using eval):
(macroexpand '(create-my-macro my-mul))
;; => (do
;; (clojure.core/defn my-mul
;; ([&form &env a__58__auto__ b__59__auto__]
;; (foo/mul a__58__auto__ b__59__auto__)))
;; (. (var my-mul) (setMacro)) (var my-mul))
(macroexpand '(my-mul (+ 1 1) 2))
;; => ClassCastException clojure.lang.PersistentList cannot be cast to java.lang.Number clojure.lang.Numbers.multiply (Numbers.java:146)
As you show, the code you'd like to emit looks like:
(defmacro my-mul [a b] `(mul ~a ~b))
So you need to syntax-quote it and suffix all locals with #:
`(defmacro my-mul [a# b#] `(mul ~a# ~b#))
Thus you macro-emitting macros should be:
(defmacro create-my-macro [macroname]
`(defmacro ~macroname [a# b#]
`(mul ~a# ~b#)))
(defmacro create-my-macro [macroname]
`(defmacro ~macroname [a# b#]
`(mul ~a# ~b#)))
I have this macro (from the clojure-koans) that should allow the use of infix operators:
(defmacro infix-better [form]
`(~(second form)
(first form)
(last form) ))
It does what it is supposed to, but doesn't exactly expand to the same expression. For example:
user=> (= '(* 10 2) (macroexpand '(infix-better (10 * 2))))
false
user=> '(* 10 2)
(* 10 2)
user=> (macroexpand '(infix-better (10 * 2)))
(* (clojure.core/first user/form) (clojure.core/last user/form))
The last output would end up being (* 10 2) when the inner expressions are evaluated but the equality test returns false since (clojure.core/first user/form), strictly speaking, isn't 10.
How can I have the expanded macro equal the equivalent hard-coded Clojure?
You have to unquote the second and third forms, just like you did the first, with ~ - "The last output would end up being (* 10 2) when the inner expressions are evaluated" is not in any way true - the inner expressions have already passed their chance to be evaluated.
(defmacro infix-better [form]
`(~(second form)
~(first form)
~(last form)))
Would fix it and keep your code structured the same, but really destructuring gives you a much nicer result:
(defmacro infix-better [[x op y]]
(list op x y)) ;; or `(~op ~x ~y)
I'm trying to write a clojure macro that lets me call a function and retrieve the arguments from a map/struct using the provided key values such as:
(with-params {:apple 2 :banana 3 :cherry 7} + :apple :banana)
;; 5
but when I try to use the macro that I wrote:
(defmacro with-params [s f & symbols]
`(~f ~#(map ~s ~symbols)))
calling
(with-params {:apple 2 :banana 3 :cherry 7} + :apple :banana)
gives me
#<CompilerException java.lang.IllegalStateException: Var clojure.core/unquote is unbound. (NO_SOURCE_FILE:0)>
Could someone help me understand how syntax-quoting works here?
For what it's worth, this shouldn't be a macro anyway. A function is plenty powerful, and the macro version will only work if both the map and the keywords are given as compile-time literals.
(defmacro with-params-macro [s f & symbols]
`(~f ~#(map s symbols)))
(defn with-params-fn [s f & symbols]
(apply f (map s symbols)))
user> (with-params-macro {:x 1} (fn [z] z) :x)
1
user> (let [params {:x 1}]
(with-params-macro params (fn [z] z) :x))
nil
user> (let [params {:x 1}]
(with-params-fn params (fn [z] z) :x))
1
The reason that `(~f ~#(map ~s ~symbols)) doesn't work is that the compiler chokes on the unnecessary unquote (~) inside the unquote-splicing (~#). The unquote-splicing unquotes the outer syntax-quote (`), so the inner two unquotes don't have any matching syntax-quote, which is why you were getting the "unbound" error.
What you want to do is evaluate (map s symbols) first to get the sequence of operands, then pass the flattened result to the function (~f); therefore the correct version is:
(defmacro with-params [s f & symbols] `(~f ~#(map s symbols)))
You can easily verify this with:
(macroexpand '(with-params {:a 1 :b 2 :c 5} * :a :b :c)) ;; (* 1 2 5)
(with-params {:a 1 :b 2 :c 5} * :a :b :c) ;; 10