Suppose I've got a bunch of namespaces (apple, banana, orange). In these namespaces I use the eat macro, which calls (not "generates", calls) the peel function. The peel function is different for each fruit, but the macros are identical, and fairly big, so I'd like to create a fruit namespace that contains the eat macro. But when I call the eat macro from the apple namespace, the eat macro should call the apple/peel function.
To illustrate (but this doesn't work):
(ns fruit)
(defmacro eat [] (peel))
(ns apple)
(defn peel [] (prn "peeled apple"))
(fruit/eat)
(ns banana)
(defn peel [] (prn "peeled banana"))
(fruit/eat)
To emphasize, this means that the peel function should be called when, and only when, the macro is expanded, as in this example.
(ns apple)
(defn peel [] (prn "peeled apple"))
(defmacro eat [] (peel))
(macroexpand-1 '(eat))
So, any ideas on how to combine macros and polymorphism?
What you're describing is not polymorphism but what is called local capture. You want the eat macro to "capture" the local definition of peel.
This is considered bad style in most Lisps, especially Clojure, as it can lead to subtle and unpredictable bugs.
A better solution is to pass the correct peel to the eat macro when you call it:
(ns fruit)
(defmacro eat [peeler] `(~peeler))
(ns apple)
(defn peel [] (prn "Peeled an apple"))
(fruit/eat peel)
If you really want to do local capture, you can force it with a ~' (unquote-quote) in the macro:
(ns fruit)
(defmacro eat [] `(~'peel))
As explained in the edited question, this is slightly different from local capture, because you're not using peel in the macroexpansion, but rather in the execution of the macro itself.
That's difficult because macros do not evaluate their arguments. Even if you pass peel as an argument to eat, within the body of the macro it's just a symbol, not a callable function.
The only way to do what you want (without using eval) is to resolve the symbol at compile time:
(defmacro eat []
((var-get (resolve 'peel)))
... return the expansion of "eat" ...)
The resolve function takes a symbol and returns the Var it is mapped to in the current namespace. Once you have the Var, you can retrieve the actual function (the Var's value) with var-get. The extra set of parentheses calls that function.
Needless to say, this is a very unusual design and might warrant rethinking.
(defmacro eat [] ((var-get (resolve 'peel))))
Note that you are abusing namespaces, though.
EDIT: Sorry. I alread posted the following. But you're saying "calls, not generates" the peel function. So what I've written is probably not what you want although it seems it would get the intended result.
Simply quoting (peel) worked for me.
(ns fruit)
(defmacro eat [] '(peel))
(ns apple)
(defn peel [] (prn "peeled apple"))
(fruit/eat)
(ns banana)
(defn peel [] (prn "peeled banana"))
(fruit/eat)
Related
I'm trying to move from Common Lisp to Chicken Scheme, and having plenty of problems.
My current problem is this: How can I write a macro (presumably using define-syntax?) that calls other macros?
For example, in Common Lisp I could do something like this:
(defmacro append-to (var value)
`(setf ,var (append ,var ,value)))
(defmacro something-else ()
(let ((values (list))
(append-to values '(1)))))
Whereas in Scheme, the equivalent code doesn't work:
(define-syntax append-to
(syntax-rules ()
((_ var value)
(set! var (append var value)))))
(define-syntax something-else
(syntax-rules ()
((_)
(let ((values (list)))
(append-to values '(1))))))
The append-to macro cannot be called from the something-else macro. I get an error saying the append-to "variable" is undefined.
According to all the information I've managed to glean from Google and other sources, macros are evaluated in a closed environment without access to other code. Essentially, nothing else exists - except built-in Scheme functions and macros - when the macro is evaluated. I have tried using er-macro-transformer, syntax-case (which is now deprecated in Chicken anyway) and even the procedural-macros module.
Surely the entire purpose of macros is that they are built upon other macros, to avoid repeating code. If macros must be written in isolation, they're pretty much useless, to my mind.
I have investigated other Scheme implementations, and had no more luck. Seems it simply cannot be done.
Can someone help me with this, please?
It looks like you're confusing expansion-time with run-time. The syntax-rules example you give will expand to the let+set, which means the append will happen at runtime.
syntax-rules simply rewrites input to given output, expanding macros until there's nothing more to expand. If you want to actually perform some computation at expansion time, the only way to do that is with a procedural macro (this is also what happens in your defmacro CL example).
In Scheme, evaluation levels are strictly separated (this makes separate compilation possible), so a procedure can use macros, but the macros themselves can't use the procedures (or macros) defined in the same piece of code. You can load procedures and macros from a module for use in procedural macros by using use-for-syntax. There's limited support for defining things to run at syntax expansion time by wrapping them in begin-for-syntax.
See for example this SO question or this discussion on the ikarus-users mailing list. Matthew Flatt's paper composable and compilable macros explains the theory behind this in more detail.
The "phase separation" thinking is relatively new in the Scheme world (note that the Flatt paper is from 2002), so you'll find quite a few people in the Scheme community who are still a bit confused about it. The reason it's "new" (even though Scheme has had macros for a long long time) is that procedural macros have only become part of the standard since R6RS (and reverted in R7RS because syntax-case is rather controversial), so the need to rigidly specify them hasn't been an issue until now. For more "traditional" Lispy implementations of Scheme, where compile-time and run-time are all mashed together, this was never an issue; you can just run code whenever.
To get back to your example, it works fine if you separate the phases correctly:
(begin-for-syntax
(define-syntax append-to
(ir-macro-transformer
(lambda (e i c)
(let ((var (cadr e))
(val (caddr e)))
`(set! ,var (append ,var ,val)))))) )
(define-syntax something-else
(ir-macro-transformer
(lambda (e i c)
(let ((vals (list 'print)))
(append-to vals '(1))
vals))))
(something-else) ; Expands to (print 1)
If you put the definition of append-to in a module of its own, and you use-for-syntax it, that should work as well. This will also allow you to use the same module both in the macros you define in a body of code as well as in the procedures, by simply requiring it both in a use and a use-for-syntax expression.
I'd like to build a list of operations to execute in a Redis pipeline. I was using accession since it's much simpler than carmine, but now I need connection pooling (missing from accession) and thus I'm looking at carmine again (it seems the go-to and most complete clojure library for redis)
I managed to get this working:
; require [taoensso.carmine :as redis]
(defn execute1 [request] (redis/wcar {} (eval request)))
(defmacro execute [body] `(redis/wcar {} ~#(eval body)))
(def get1 `(redis/get 1))
(execute1 get1)
(execute [get1])
but given that I'll build vectors of thousands of elements, I'm a bit worried about the possible performance hit of eval (besides the fact that I've been taught to always avoid eval if possible). I reckon that defmacro instead is/can be evaluated at macro-expansion time, which might be earlier (during AOT compilation?) and without using eval.
Is there anything that I can do? should I move to a different library?
(I had a look at carmine's source code: my pain is only due to a small convenience for the author: a *context* used to avoid passing around a couple extra argument: getting rid of it should be simple enough, but I'm not invested enough in it... I might as well decide to move to another data store in the future)
edit: I've been asked to write an example of what I think is the boilerplate that I'd prefer to avoid writing in my actual code, so: (the following is untested, it's just a POC)
(defn hset [id key val]
#(redis/hset id key val))
(defn hsetnx [id key val]
#(redis/hsetnx id key val))
(defn hincrby [id key increment]
#(redis/hincrby id key increment))
(defn hgetall [id key]
#(redis/hgetall id key))
(defn sadd [id el]
#(redis/sadd id el))
(defn scard [id]
#(redis/scard id))
(defn smembers [id]
#(redis/smembers id))
(defmacro execute [forms]
`(redis/wcar {} ~#(map apply forms)))
; end boilerplate
(defn munge-element [[a b c]]
(conj
(mapcat #(hincrby a :whatever %) b)
(sadd c b)
(hsetnx a c))
(defn flush-queue! [queue_]
(execute queue_)
[])
(defn receive [item]
(if (< (count #queue) 2000)
(swap! queue conj (munge-element item))
(swap! queue flush-queue!)))
Obviously, I could write something like this, but if this was truly the inteneded way to use carmine, these curried functions would be provided along (or instead of) the normal ones. Also a lot of lines could be shaved off by constructing up the defs with syntax quoting, but this is accidental complexity, not inherent the original problem.
The following piece of code does not differ very much from yours:
(defmacro execute [& forms]
`(redis/wcar {} ~#forms))
(defn get1 []
(redis/get 1))
(execute (get1) (get1) ...)
This is basically how carmine should be used (following this suggestion in the README). If it doesn't suit your needs, could you clarify why?
After the question was edited to present more clearly what should be accomplished I think I have a solution for you. You're trying to create a list of statements to be executed at some specific time in the future. To achieve this you're wrapping each statement inside a function and I agree, that's a lot of boilerplate.
But unnecessary. You can get the same result by having a macro that automatically creates thunks for you:
(defmacro statements [& forms]
`(vector
~#(for [f forms]
`(fn [] ~f))))
Now, whatever you pass to this macro will result in a vector of zero-parameter functions that can be evaluated e.g. using:
(defn execute-statements [fns]
(redis/wcar {} (mapv #(%) fns))
And your example turns into something along the lines of:
(defn munge-element [[a b c]]
(statements
(mapcat #(redis/hincrby a :whatever %) b)
(redis/sadd c b)
(redis/hsetnx a c))
(defn flush-queue! [queue_]
(mapv execute-statements queue_)
[])
Edit: This executes every batch in its own pipeline. If you want to do it in a single one, use concat instead of conj to build up your queue (in receive) and (execute-statements queue_) instead of (mapv execute-statements queue_).
Note: IIRC correctly, this:
(redis/wcar {} a [b c]])
returns the same result as this:
(redis/wcar {} a b c)
I.e., carmine collects the results somewhere and always returns a vector for all of them. Even if not, I think you can still avoid your dreaded boilerplate by just tweaking the stuff presented here a little bit.
Is there a clean way of implementing dynamic scope that will "reach" into macro calls? Perhaps more importantly, even if there is, should it be avoided?
Here's what I'm seeing in a REPL:
user> (def ^:dynamic *a* nil)
> #'user/*a*
user> (defn f-get-a [] *a*)
> #'user/f-get-a
user> (defmacro m-get-a [] *a*)
> #'user/m-get-a
user> (binding [*a* "boop"] (f-get-a))
> "boop"
user> (binding [*a* "boop"] (m-get-a))
> nil
This m-get-a macro isn't my actual goal, it's just a boiled down version of the problem I have been running into. It took me a while to realize, though, because I kept debugging with macroexpand, which makes everything seem fine:
user> (binding [*a* "boop"] (macroexpand '(m-get-a)))
> "boop"
Doing macroexpand-all (used from clojure.walk) on the outer binding call leads me to believe that the "issue" (or feature, as the case may be) is that (m-get-a) is getting evaluated before the dynamic binding takes:
user> (macroexpand-all '(binding [*a* "boop"] (f-get-a)))
> (let* []
(clojure.core/push-thread-bindings (clojure.core/hash-map #'*a* "boop"))
(try (f-get-a) (finally (clojure.core/pop-thread-bindings))))
user> (macroexpand-all '(binding [*a* "boop"] (m-get-a)))
> (let* []
(clojure.core/push-thread-bindings (clojure.core/hash-map #'*a* "boop"))
(try nil (finally (clojure.core/pop-thread-bindings))))
Here's my crack at a workaround:
(defmacro macro-binding
[binding-vec expr]
(let [binding-map (reduce (fn [m [symb value]]
(assoc m (resolve symb) value))
{}
(partition 2 binding-vec))]
(push-thread-bindings binding-map)
(try (macroexpand expr)
(finally (pop-thread-bindings)))))
It will evaluate a single macro expression with the relevant dynamic bindings. But I don't like using macroexpand in a macro, that just seems wrong. It also seems wrong to resolve symbols in a macro--it feels like a half-assed eval.
Ultimately, I'm writing a relatively lightweight interpreter for a "language" called qgame, and I'd like the ability to define some dynamic rendering function outside of the context of the interpreter execution stuff. The rendering function can perform some visualization of sequential instruction calls and intermediate states. I was using macros to handle the interpreter execution stuff. As of now, I've actually switched to using no macros at all, and also I have the renderer function as an argument to my execution function. It honestly seems way simpler that way, anyways.
But I'm still curious. Is this an intended feature of Clojure, that macros don't have access to dynamic bindings? Is it possible to work around it anyways (without resorting to dark magic)? What are the risks of doing so?
Macro expansion take place during the compilation of you program, so it's imposisble to predict the future value of dynamic variable at that time.
But, probably, you don't need to evaluate *a* during macro expansion and just wants to leave it as is. It this case *a* will be evaluated when the actual code is called. In this case you should quote it with ` symbol:
(defmacro m-get-a [] `*a*)
Your implementation of m-get-a causes clojure to replace (m-get-a) with its value when the code is compiled, which is the core binding of *a*, while my wersion causes it to replace (m-get-a) with variable *a* itself.
You need to quote *a* to make it work:
user=> (def ^:dynamic *a* nil)
#'user/*a*
user=> (defmacro m-get-a [] `*a*)
#'user/m-get-a
user=> (binding [*a* "boop"] (m-get-a))
"boop"
I am trying to implement a huge Java interface with numerous (~50) getter and setter methods (some with irregular names). I thought it would be nice to use a macro to reduce the amount of code. So instead of
(def data (atom {:x nil}))
(reify HugeInterface
(getX [this] (:x #data))
(setX [this v] (swap! data assoc :x v)))
I want to be able to write
(def data (atom {:x nil}))
(reify HugeInterface
(set-and-get getX setX :x))
Is this set-and-get macro (or something similar) possible? I haven't been able to make it work.
(Updated with a second approach -- see below the second horizontal rule -- as well as some explanatory remarks re: the first one.)
I wonder if this might be a step in the right direction:
(defmacro reify-from-maps [iface implicits-map emit-map & ms]
`(reify ~iface
~#(apply concat
(for [[mname & args :as m] ms]
(if-let [emit ((keyword mname) emit-map)]
(apply emit implicits-map args)
[m])))))
(def emit-atom-g&ss
{:set-and-get (fn [implicits-map gname sname k]
[`(~gname [~'this] (~k #~(:atom-name implicits-map)))
`(~sname [~'this ~'v]
(swap! ~(:atom-name implicits-map) assoc ~k ~'v))])})
(defmacro atom-bean [iface a & ms]
`(reify-from-maps ~iface {:atom-name ~a} ~emit-atom-g&ss ~#ms))
NB. that the atom-bean macro passes the actual compile-time value of emit-atom-g&ss on to reify-from-maps. Once a particular atom-bean form is compiled, any subsequent changes to emit-atom-g&ss have no effect on the behaviour of the created object.
An example macroexpansion from the REPL (with some line breaks and indentation added for clarity):
user> (-> '(atom-bean HugeInterface data
(set-and-get setX getX :x))
macroexpand-1
macroexpand-1)
(clojure.core/reify HugeInterface
(setX [this] (:x (clojure.core/deref data)))
(getX [this v] (clojure.core/swap! data clojure.core/assoc :x v)))
Two macroexpand-1s are necessary, because atom-bean is a macro which expands to a further macro call. macroexpand would not be particularly useful, as it would expand this all the way to a call to reify*, the implementation detail behind reify.
The idea here is that you can supply an emit-map like emit-atom-g&ss above, keyed by keywords whose names (in symbolic form) will trigger magic method generation in reify-from-maps calls. The magic is performed by the functions stored as functions in the given emit-map; the arguments to the functions are a map of "implicits" (basically any and all information which should be accessible to all method definitions in a reify-from-maps form, like the name of the atom in this particular case) followed by whichever arguments were given to the "magic method specifier" in the reify-from-maps form. As mentioned above, reify-from-maps needs to see an actual keyword -> function map, not its symbolic name; so, it's only really usable with literal maps, inside other macros or with help of eval.
Normal method definitions can still be included and will be treated as in a regular reify form, provided keys matching their names do not occur in the emit-map. The emit functions must return seqables (e.g. vectors) of method definitions in the format expected by reify: in this way, the case with multiple method definitions returned for one "magic method specifier" is relatively simple. If the iface argument were replaced with ifaces and ~iface with ~#ifaces in reify-from-maps' body, multiple interfaces could be specified for implementation.
Here's another approach, possibly easier to reason about:
(defn compile-atom-bean-converter [ifaces get-set-map]
(eval
(let [asym (gensym)]
`(fn [~asym]
(reify ~#ifaces
~#(apply concat
(for [[k [g s]] get-set-map]
[`(~g [~'this] (~k #~asym))
`(~s [~'this ~'v]
(swap! ~asym assoc ~k ~'v))])))))))
This calls on the compiler at runtime, which is somewhat expensive, but only needs to be done once per set of interfaces to be implemented. The result is a function which takes an atom as an argument and reifies a wrapper around the atom implementing the given interfaces with getters and setters as specified in the get-set-map argument. (Written this way, this is less flexible than the previous approach, but most of the code above could be reused here.)
Here's a sample interface and a getter/setter map:
(definterface IFunky
(getFoo [])
(^void setFoo [v])
(getFunkyBar [])
(^void setWeirdBar [v]))
(def gsm
'{:foo [getFoo setFoo]
:bar [getFunkyBar setWeirdBar]})
And some REPL interactions:
user> (def data {:foo 1 :bar 2})
#'user/data
user> (def atom-bean-converter (compile-atom-bean-converter '[IFunky] gsm))
#'user/atom-bean-converter
user> (def atom-bean (atom-bean-converter data))
#'user/atom-bean
user> (.setFoo data-bean 3)
nil
user> (.getFoo atom-bean)
3
user> (.getFunkyBar data-bean)
2
user> (.setWeirdBar data-bean 5)
nil
user> (.getFunkyBar data-bean)
5
The point is reify being a macro itself which is expanded before your own set-and-get macro - so the set-and-get approach doesn't work. So, instead of an inner macro inside reify, you need a macro on the "outside" that generates the reify, too.
Since the trick is to expand the body before reify sees it, a more general solution could be something along these lines:
(defmacro reify+ [& body]
`(reify ~#(map macroexpand-1 body)))
You can also try to force your macro to expand first:
(ns qqq (:use clojure.walk))
(defmacro expand-first [the-set & code] `(do ~#(prewalk #(if (and (list? %) (contains? the-set (first %))) (macroexpand-all %) %) code)))
(defmacro setter [setterf kw] `(~setterf [~'this ~'v] (swap! ~'data assoc ~kw ~'v)))
(defmacro getter [getterf kw] `(~getterf [~'this] (~kw #~'data)))
(expand-first #{setter getter}
(reify HugeInterface
(getter getX :x)
(setter setX :x)))
I've got a Clojure file with a lot of string constants. I want to collect these strings in a collection by wrapping them in a macro. After several tries I succeeded, but my solution looks fairly hideous.
(ns Memorable)
(def MEMORY (atom []))
(defmacro memorize [s] (swap! MEMORY conj s) s)
(prn (str (memorize "hello") " brave new " (memorize "world"))) ; test
(defmacro make-memories-constant [] `(def MEMORIES ~(deref MEMORY)))
(make-memories-constant)
Are there more elegant solutions for this problem?
As far as code cleanup is concerned, I'd remove the make-memories-constant macro -- you can just do
(def MEMORIES #MEMORY)
or even
(def MEMORY #MEMORY)
so as not to clutter your namespace with another Var. As long as you put this after all calls to memorize, this will save a snapshot of MEMORY with all memorized strings inside.
I'd say this is actually pretty clean, with the actual code executing at runtime no different from what it would have been without any memorize-ing...
Another approach would be to prepare a sort of "memorisation framework" as a separate namespace exporting a macros called setup-memorization, say, which could look something like this (just a rough sketch which won't work without some polishing... updated with a (not so thoroughly) tested version -- this actually works!... still, please tweak it to your needs):
(ns memorization-framework)
(defmacro setup-memorization []
(let [MEMORY (gensym "MEMORY")
MEMORIES (gensym "MEMORIES")]
`(do (def ~MEMORY (atom []))
(defmacro ~'memorize [s#] (swap! ~MEMORY conj s#) s#)
(defmacro ~'get-memory [] #~MEMORY)
(defmacro ~'defmemories [~MEMORIES]
`(do (def ~~MEMORIES #~~MEMORY)
(ns-unmap ~~*ns* '~'~'get-memory)
(ns-unmap ~~*ns* '~'~'memorize)
(ns-unmap ~~*ns* '~'~MEMORY)
~~MEMORIES)))))
Then you'd use the memorization-framework namespace, do (setup-memorization) towards the top of your namespace to set things up, call memorize like you do in your example code to memorise things and finally use end-memorization defmemories to store the string collection somewhere in a Var and remove the temporary Var used to store the atom used for construction-time storage along with the ability to make further calls to memorize from this namespace. (The end-memorization macro is meant to just return the collection to wherever you call it from when called with no arguments or define a new Var to store it if given a single argument, which must be a symbol to be used to name the Var. Update: I've only tested the defmemories version, so I'm leaving it here and removing the "in-place return" variant.)
An example interaction at the REPL while in namespace foo (note I defined setup-memorization in the user namespace):
foo> (user/setup-memorization)
#'foo/defmemories
foo> (get-memory)
[]
foo> (memorize "foo")
"foo"
foo> (memorize "bar")
"bar"
foo> (get-memory)
["foo" "bar"]
foo> (defmemories quux)
["foo" "bar"]
foo> quux
["foo" "bar"]
foo> (get-memory)
; Evaluation aborted.
foo> (memorize)
; Evaluation aborted.
Those "Evaluation aborted." messages indicate that calls to get-memory and memorize throw exceptions after defmemories has been called. This is by design, as mentioned above. Also, get-memory is here mostly to ease testing / debugging.
If you're only going to use all this once, the setup-memorization approach might be overkill, but I guess it'll remove some boilerplate for you if you use it more.