I'm trying (as a self-learning exercise) to create a Clojure macro that will generate code to apply a function to a sequence of integers and sum the result, e.g.
f(0) + f(1) + f(2) + f(3)
This is my attempt:
(defmacro testsum [func n]
`(fn [x#] (+ ~#( map (fn [i] `(~func x#)) (range n)))))
However something seems to go wrong with the x# gensym and I end up with two different versions of x and hence the function doesn't work:
(macroexpand '(testsum inc 3))
gives:
(fn* ([x__809__auto__]
(clojure.core/+
(inc x__808__auto__)
(inc x__808__auto__)
(inc x__808__auto__))))
This is pretty much exactly what I want apart from the different 809 and 808 versions of x.....
What am I doing wrong? I thought that the auto gensym was meant to create a single unique symbol for exactly this kind of purpose? Is there a better way of doing this?
foo#-style gensyms are valid only inside the syntax-quote where they were created. In your code, the two x#s are created in different syntax-quote blocks:
(defmacro testsum [func n]
`(fn [x#] (+ ~#( map (fn [i] `(~func x#)) (range n)))))
^- s-q1 ^-unquote ^- s-q2
To fix this, use an explicit (gensym) call:
(defmacro testsum [func n]
(let [x (gensym "x")]
`(fn [~x] (+ ~#(map (fn [i] `(~func ~x)) (range n))))))
And the macro expansion ((macroexpand '(testsum inc 3))):
(fn* ([x4966] (clojure.core/+ (inc x4966) (inc x4966) (inc x4966))))
Related
I'm trying to write a macro in Common Lisp that takes any number of expressions and builds a list containing each expression followed by its evaluation in a single line. For example, if I name my macro as
(defmacro list-builder (&rest exp)
...)
and I run
(let ((X 1) (Y 2)) (list-builder (+ X Y) (- Y X) X))
I want it to return:
'((+ X Y) 3 (- Y X) 1 X 1)
The best I've been able to do so far is get a list of the expressions using the code
(defmacro list-builder (&rest exp)
`',#`(',exp ,exp))
INPUT: (let ((X 1) (Y 2)) (list-builder (+ X Y) (+ Y X) X))
'((+ X Y) (+ Y X) X)
Strictly speaking, the macro itself cannot do that; what the macro must do is generate code in which the argument expressions are embedded in such a way that they are evaluated, and also in such a way that they are quoted.
Given (list-builder (+ x y) (+ y x) x) we would like to generate this code: (list '(+ x y) (+ x y) '(+ y x) (+ y x) 'x x).
We can split the macro into an top-level wrapper defined with defmacro and an expander function that does the bulk of the work of producing the list arguments; The macro's body just sticks the list symbol on it and returns it.
Macro helper functions have to be wrapped with a little eval-when dance in Common Lisp to make sure they are available in all conceivable situations that the macro might be processed:
(eval-when (:compile-toplevel :load-toplevel :execute)
(defun list-builder-expander (exprs)
(cond
((null exprs) nil)
((atom exprs) (error "list-builder: dotted syntax unsupported":))
(t (list* `',(car exprs) (car exprs)
(list-builder-expander (cdr exprs)))))))
(defmacro list-builder (&rest exprs)
(cons 'list (list-builder-expander exprs)))
A "slick" implementation, all in one defmacro, inside a single backquote expression, might go like this:
(defmacro list-builder (&rest exprs)
`(list ,#(mapcan (lambda (expr) (list `',expr expr)) exprs)))
The "dotted syntax unsupported" check we implemented before now becomes an error out of mapcan.
The lambda turns each expression E into the list ((quote E) E). mapcan catenates these lists together to form the arguments for list, which are then spliced into the (list ...) form with ,#.
The form `',expr follows from applying the quote shorthand to `(quote ,expr).
Of course, a lisp macro can do that. Since lisp macros provide full control over evaluation of their arguments.
You have to use macro helper functions only in cases in which you want to use recursion. Since macros have problems to call themselves recursively.
But by looping over the &rest rest argument, you can generate variadic macros (macros with arbitrary number of arguments) and still control the evaluation of each of its arguments.
After some trial and error cycles (macro construction is an incremental procedure, since macros are complex structures), I obtained the
"simpler" solution:
(defmacro list-builder (&rest rest)
`(list ,#(loop for x in `,rest
nconcing (list `',x x))))
Test by:
(let ((X 1)
(Y 2))
(list-builder (+ X Y) (- Y X) X))
;; ((+ X Y) 3 (- Y X) 1 X 1)
Sometimes, in loop constructs, instead of collect/collecting, use nconc/nconcing in combination with (list ...) to have more control over how the elements are consed together.
The
(list `',x x)
ensures, that the second x gets evaluated, while the first
`',x
places the content of x into the expression, while its quoting prevents the evluation of the expression placed for x.
The outer list in combination with the splicing of the loop construct into it,
finally captures (prevents) the intrinsic very final evaluation of the macro body.
(defmacro list-builder (&rest args)
`(let ((lst ',args)
(acc nil))
(dolist (v lst)
(push v acc)
(push (eval v) acc))
(nreverse acc)))
We could create the list builder macro to take rest parameters as you did (I simply renamed them as args for pseudo code). I'd create a quoted list (lst) of the expressions within the list, and an empty list (acc) to store the expressions and whatever they evaluate to later. Then we can use dolist to iterate through our list and push each expression to the list, followed by whatever it evaluates to by running eval on the expression. Then we can finally use nreverse to get the correct order for the list.
We can then call it:
(let ((x 1)
(y 2))
(declare (special x))
(declare (special y))
(list-builder (+ x y) (- y x) x))
The result will be:
((+ X Y) 3 (- Y X) 1 X 1)
CL-USER>
Most of my application state is stored in a large complex map. For the purposes of this question, I will use a simple structure:
(def data
{:a 1
:b {:c {:d 3}}})
I have a large number of functions which all follow the same pattern:
(defn update-map
[my-map val]
(let [a (:a my-map)
d (-> my-map :b :c :d)]
(assoc-in
(assoc my-map :a (+ a val))
[:b :c :d] (+ d val))))
I retrieve one or more values from the map, perform some calculations, and create a new map with updated values. There are two problems with this approach:
I have a lot of repetitive let bindings across different function definitions
If the schema of the map changes, I will have a lot of code to refactor
I've written a macro to reduce the boilerplate code required to define these functions. It works by looking up predefined getter and setter functions, and automatically generating a let block:
(def getters
{'a #(:a %)
'd #(-> % :b :c :d)})
(def setters
{'a #(assoc % :a %2)
'd #(assoc-in % [:b :c :d] %2)})
(defmacro def-map-fn
[name [& args] [& fields] & code]
(let [my-map 'my-map
lookup #(reduce % [] fields)
getter-funcs (lookup #(conj % %2 (list (getters %2) my-map)))
setter-funcs (lookup #(conj % (symbol (str "update-" %2)) (setters %2)))]
`(defn ~name [~my-map ~#args]
(let [~#getter-funcs ~#setter-funcs]
~#code))))
I can now define my functions more elegantly:
(def-map-fn update-map
[val] ; normal function parameters
[a d] ; fields from the map I will be using
(update-d
(update-a my-map (+ a val))
(+ d val)))
When expanded, it will produce a function definition looking something like this:
(defn update-map
[my-map val]
(let [a (#(:a %) my-map)
d (#(-> % :b :c :d) my-map)
update-a #(assoc % :a %2)
update-d #(assoc-in % [:b :c :d] %2)]
(update-d
(update-a my-map (+ a val))
(+ d val))))
One thing that is nagging me about my macro is that it is not intuitive to the programmer that the my-map function parameter is available for use within the function body.
Is this a good use of macros, or should I be using a different approach entirely (like dynamic var bindings)?
You could perhaps use lenses; the getters and setters then become composable functions. Have a look here or here.
Following the first link you can set up the lens as follows:
; We only need three fns that know the structure of a lens.
(defn lens [focus fmap] {:focus focus :fmap fmap})
(defn view [x {:keys [focus]}] (focus x))
(defn update [x {:keys [fmap]} f] (fmap f x))
; The identity lens.
(defn fapply [f x] (f x))
(def id (lens identity fapply))
; Setting can be easily defined in terms of update.
(defn put [x l value] (update x l (constantly value)))
(-> 3 (view id))
; 3
(-> 3 (update id inc))
; 4
(-> 3 (put id 7))
; 7
; in makes it easy to define lenses based on paths.
(defn in [path]
(lens
(fn [x] (get-in x path))
(fn [f x] (update-in x path f))))
(-> {:value 3} (view (in [:value])))
; 3
(-> {:value 3} (update (in [:value]) inc))
; {:value 4}
(-> {:value 3} (put (in [:value]) 7))
; {:value 7}
You can see form the above the the lens can be adapted to use get/set methods (such as get-in/update-in) based on the data structure you are working with. The real power of lenses which seems to also be what you are after is that you can compose them. In the same example the composition function can be defined as follows:
(defn combine [outer inner]
(lens
(fn [x] (-> x (view outer) (view inner)))
(fn [f x] (update x outer #(update % inner f)))))
(defn => [& lenses] (reduce combine lenses))
The => function can now be used to combine any arbitrary lenses such as:
(-> {:new {:value 3}} (view (=> (in [:new]) (in [:value]))))
; 3
(-> {:new {:value 3}} (update (=> (in [:new]) (in [:value])) inc))
; {:new {:value 4}}
(-> {:new {:value 3}} (put (=> (in [:new]) (in [:value])) 7))
; {:new {:value 7}}
The fact that (in [:new]) is just a function means that you could, for example, store it and manipulate it in various ways. For example, it would be possible to walk your nested map structure and create the lens functions which correspond to accessing the value at each level in the nested map and then at the end compose these functions together to create your getter/setter api. With this set up, your lenses could automatically adapt to any changes in your schema.
The ability to compose lenses can also make it easy to interact with the nodes of your nested map. For example, if you were to ever change the node from an atom to a list, you could simply add a new lens to work with it as follows:
(def each (lens seq map))
(-> {:values [3 4 5]} (view (=> (in [:values]) each)))
; (3 4 5)
(-> {:values [3 4 5]} (update (=> (in [:values]) each) inc))
; {:values (4 5 6)}
(-> {:values [3 4 5]} (put (=> (in [:values]) each) 7))
; {:values (7 7 7)}
I highly recommend looking at the full gist to see more examples of what you can do with lenses.
In this situation, my preference is to avoid the use of macros. They often obfuscate code, but more importantly they aren't composable. An ideal solution here would allow you to use the getter and setter functions outside of functions being defined in def-map-fn. I'd stick with regular functions and data as much as possible.
To begin with, you're concerned about having to rewrite a bunch of code if your schema changes. Fair enough. To address that, I'd start with a data representation of your map's schema. See Prismatic schema for a full-featured schema library for Clojure, but for now something along these lines should do:
(def my-schema
{:a :int
:b {:c {:d :int}}})
From this, you can compute the paths for all the properties in your schema:
(defn paths [m]
(mapcat (fn [[k v]]
(conj (if (map? v)
(map (partial apply vector k) (paths v)))
[k]))
m))
(def property-paths
(into {} (for [path (paths my-schema)] [(last path) path])))
Now, to get or set a property, you can look up its path and use that in conjunction with get-in, update-in, etc. as appropriate:
(let [d (get-in my-map (property-paths :d))]
;; Do something with d.
)
If you get tired of always calling get-in, assoc-in, etc., then you can pretty easily generate a bunch of getter functions:
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "get-" (name p)))
[m#] (get-in m# ~path))))
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "set-" (name p)))
[m# v#] (assoc-in m# ~path v#))))
(doseq [[p path] property-paths]
(eval `(defn ~(symbol (str "update-" (name p)))
[m# tail#] (apply update-in m# ~path #tail))))
Now you have your get-a, set-a, update-a functions available everywhere in your code, without having to call into some uber-macro to set up the bindings for you. For instance:
(let [a (get-a my-map)]
(-> my-map
(set-a 42)
(update-d + a)))
If you really find setting up the above let binding tedious, you could even write a with-properties macro that accepts a map and a list of property names and executes the body in a context that binds values for those names. But I probably wouldn't bother.
Advantages of this approach include:
It's schema-driven, so the schema is defined in one central place and used to generate other code as needed.
It prefers pure functions over macros, so code is more re-usable and composable.
It's an incremental approach that allows your application to grow more naturally. Rather than starting with an uber-macro that tries to anticipate all possible features you might want, you start with data and functions and sprinkle in macros to alleviate some of the repetitiveness as you see usage patterns emerge.
Why don't you just use update-in?
(defn update-map [my-map val]
(-> my-map
(update-in [:a] + val)
(update-in [:b :c :d] + val)))
Is it possible to macro returned macro? I would like to simplify the code maximum, and I can do this using a macro that returns the function. However, it is too much overhead and it is too slow. To keep the code more readable, I did not use type hints, but even with them my code is ~ 5x slower.
My English is not very precise, so I write what I have and what I wanted to have.
I have this...
(defmacro series [java-array]
`(fn
([i#]
(aget ~java-array i#))
([start# stop#]
(let [limit# (unchecked-subtract-int stop# start#)
result# (double-array limit#)]
(loop [i# 0]
(if (< i# limit#)
(let [r# (double (aget ~java-array i#))]
(aset result# i# r#)
(recur (inc i#)))
result#))))))
I want something like this...
(defmacro series [java-array]
`(defmacro blabla
([i#]
`(aget ~~java-array i#))
([start# stop#]
`(let [limit# (unchecked-subtract-int stop# start#)
result# (double-array limit#)]
(loop [i# 0]
(if (< i# limit#)
(let [r# (double (aget ~~java-array i#))]
(aset result# i# r#)
(recur (inc i#)))
result#))))))
But when I call this...
Wrong number of args (1) passed to: blabla
A simpler example.
I have a lot of java arrays. I do not want to use aget. I want macro to expand to (aget array-name i). I write a macro that expand to (fn [n] (aget array-name i)), but this is unnecessary overhead.
(defmacro series [arr]
`(fn [i#]
(aget (longs ~arr) (int i#))))
I now declare the series, such as "date", and call it in this way (date i), which will return me "i" element of the array.
I think what you're looking for is a way to declare local macros within functions. The clojure.tools.macro package has a macrolet form, which I think you should be able to wrangle into creating a macro for local array lookups.
Here's a little example I put together:
; project.clj
(defproject testproject "1.0.0-SNAPSHOT"
:description "FIXME: write description"
:dependencies [[org.clojure/clojure "1.5.0"]
[org.clojure/tools.macro "0.1.2"]])
; src/testproject/core.clj
(ns testproject.core
(:require [clojure.tools.macro :refer [macrolet]]))
(defn my-test []
(let [n 1000
arr (int-array n (range 10))]
(macrolet [(lookup [i] (list `aget 'arr i))]
(loop [i 0, acc 0]
(if (< i n)
(recur (inc i) (+ acc (lookup i)))
acc)))))
In the example I use macrolet to declare a macro called lookup that will cause (lookup 5) to expand into (lookup arr 5), which is what I think you're looking for.
Notice how you have to be careful when referencing arr from within the local macro definition. If you just declared the macro as`(aget arr ~i), then it tries to find a fully-qualified symbol arr, which doesn't exist. You could alternatively declare the macro as `(aget ~'arr ~i), but I felt that (list `aget 'arr i) looked a lot nicer.
Now you can take it one step further and use defmacro to create a macro that includes macrolet inside, which you can use to simplify the declaration of arrays with a local lookup macro. We leave this as an exercise for the reader.
Since macros are so subtle, and nesting macros in macros only makes things worse, I decided it would probably be more helpful to just give an example and let you figure out how it works:
(defmacro lookup-binding [[fn-sym value] & body]
(let [array-sym (symbol (str fn-sym "-raw"))]
`(let [~array-sym ~value]
(macrolet [(~fn-sym [i#] (list aget '~array-sym i#))]
~#body))))
(defn my-test2 []
(let [n 1000]
(lookup-binding [arr (int-array n (range 10))]
(loop [i 0, acc 0]
(if (< i n)
(recur (inc i) (+ acc (arr i)))
acc)))))
Disclaimer: I'm only trying to show that this is possible, not that it's a good idea. I personally don't think all this extra complexity is worth avoiding aget. I'd suggest just using aget since it's only a few extra characters, and it will make your code more clear/readable.
In my answer to Clojure For Comprehension example, there is some duplication that I would like to remove:
(def all-letters (map char (range 65 90)))
(defn kw [& args] (keyword (apply str args)))
(concat
(for [l all-letters] (kw l))
(for [l all-letters l2 all-letters] (kw l l2))
(for [l all-letters l2 all-letters l3 all-letters] (kw l l2 l3)))
I would like to remove the "for" duplication. I have written the following macro:
(defmacro combine [times]
(let [symbols (repeatedly times gensym)
for-params (vec (interleave symbols (repeat 'all-letters)))]
`(for ~for-params (kw ~#symbols))))
Which works with:
(concat (combine 1) (combine 2) (combine 3))
But if I try:
(for [i (range 1 4)] (combine i))
I get:
CompilerException java.lang.ClassCastException: clojure.lang.Symbol cannot be cast to java.lang.Number, compiling:(NO_SOURCE_PATH:177)
What is going on? Is there a better way of removing the duplication?
Your problem is that combine is a macro that gets expanded at compile time. When you try to expand on a symbol i it fails, because it is designed to take a number (times). i is just a symbol at compile time, it only evaluates to numeric values at runtime.
I'd suggest rewriting combine to be a function rather than a macro: you don't really need macros here and functions are frequently more convenient (as in this case!).
Here's a recursive combine that probably does roughly what you want:
(defn combine
([times] (combine times nil))
([times letters]
(if (<= times 0)
(keyword (apply str letters))
(flatten (for [l all-letters] (combine (dec times) (cons l letters)))))))
You can modify your macro such that the concat becomes part of the macro, such shown below. But I agree with Mikera that it is better to have a function.
(defmacro combine [start end]
`(concat
~#(for [i (range start end)]
(let [symbols (repeatedly i gensym)
for-params (vec (interleave symbols (repeat 'all-letters)))]
`(for ~for-params (kw ~#symbols))))))
user=> (combine 1 2)
(:A :B :C :D :E :F :G :H :I :J :K :L :M :N :O :P :Q :R :S :T :U :V :W :X :Y)
I'm currently working on a LISP exercise for a small project and need severe help. This may be more or less of a beginner's question but I'm absolutely lost on writing a certain function that takes in two unevaluated functions and spits out the result dependent on if the variables were given an assignment or not.
An example would be
(setq p1 '(+ x (* x (- y (/ z 2)))))
Where
(evalexp p1 '( (x 2) (z 8) ))
returns (+ 2 (* 2 (- y 4)))
My goal is to write the evalexp function but I can't even think of where to start.
So far I have
(defun evalexp (e b) )
.. not very much. If anyone could please help or lead me in a good direction I'd be more than appreciative.
Here's a full solution. It's pretty straightforward, so I'll leave out a full explanation. Ask me in the comments if there's anything you can't figure out yourself.
(Using eval to do the actual evaluation might not be what you want in your exercise/project. Look up "meta-circular interpreter" for another way.)
(defun apply-env (exp env)
(reduce (lambda (exp bdg) (subst (cadr bdg) (car bdg) exp))
env :initial-value exp))
(defun try-eval (exp)
(if (atom exp)
exp
(let ((exp (mapcar #'try-eval exp)))
(if (every #'numberp (cdr exp))
(eval exp)
exp))))
(defun evalexp (exp env)
(try-eval (apply-env exp env)))
Here's a hint, this is how you might do it (in pseudocode):
function replace(vars, list):
for each element of list:
if it's an atom:
if there's an association in vars:
replace atom with value in vars
else:
leave atom alone
else:
recursively apply replace to the sublist
There will certainly be some details to work out as you convert this to Lisp code.