Basically I'm rather new to macros and I'm trying to work out how to write macros in Clojure. The problem is I keep getting exception errors and it's very difficult to figure out where to proceed. So really what I'm wondering is if I can get a list of either methods (or heuristics) for debugging Clojure macros.
Take the current problem I am working on, I have this model:
{:users [:name :email :registered-on]
:post [:title :author]}
and I want to convert it into the form:
(do (def new-form-users (cashew.core/new-form "/users/new" "Create a new Users"
["name" "email" "registered-on"] ("Name" "Email" "Registered-on")))
(def new-form-post (cashew.core/new-form "/post/new" "Create a new Post"
["title" "author"] ("Title" "Author"))))
for which I have written this macro:
(defmacro gen-create-forms [model]
`(do
~#(for [[entity-kw values] model]
(let [entity-sym (-> entity-kw name capitalize)
fields (vec (map name values))]
`(def ~(symbol (str "new-form-" (name entity-kw))) (new-form ~(str "/" (name entity-kw) "/new") ~(str "Create a new " entity-sym) ~fields ~(map capitalize fields)))))))
However when I run the macro I get:
java.lang.String cannot be cast to clojure.lang.IFn
[Thrown class java.lang.ClassCastException]
I've tried calling macroexpand-1, but I get the same error leaving me with little idea on how to solve it.
This tutorial provided by John Lawrence Aspden where he says "When the compiler sees a macro, which is just a function that returns some code, it runs the function, and substitutes the code that is returned into the program." prompted me to write the macro as a function which takes in the values I have and outputs the result that I want them to transform into.
Thus this works:
(defn gen-create-forms [model]
`(do
~#(for [[entity-kw values] model]
(let [entity-sym (-> entity-kw name capitalize)
fields (vec (map name values))]
`(def ~(symbol (str "new-form-" (name entity-kw))) (new-form ~(str "/" (name entity-kw) "/new") ~(str "Create a new " entity-sym) ~fields ~(map capitalize fields)))))))
(gen-create-forms {:users [:name :email :registered-on]
:post [:title :author]})
(do (def new-form-users (cashew.core/new-form "/users/new" "Create a new Users" ["name" "email" "registered-on"] ("Name" "Email" "Registered-on"))) (def new-form-post (cashew.core/new-form "/post/new" "Create a new Post" ["title" "author"] ("Title" "Author"))))
I'm not sure if I am using macros correctly here or if macros are the right strategy for solving this problem. However some ideas about what you do when faced with exceptions when writing a macro or good techniques for debugging them would be much appreciated.
EDIT: It has been brought to my attention by mikera that this is not a good example, however my question still stands. So to reiterate what techniques would you use when faced with an exception if coding a macro in clojure?
Personally I wouldn't use a macro for this - my proposed alternative would be:
Avoid trying to generate new symbol names in the namespace - this can get messy and complicated!
Instead create a single data structure called "new-forms" that contains all the forms in a hashmap. You could use either keywords or strings as keys, personally I'd use keywords like ":users" since you are already using that approach for your model data structure.
Generate "new-forms" with a function that takes the model as a parameter and calls (cashew.core/new-form ... ) for each form in the model as required
When you want to acess a specific form, you can then just do (new-forms :users) or similar to read the appropriate form out of the hashmap.
Related
When I was learning HTML it was very helpful for me to know that ol means ordered list, tr is table row, etc. Some of the lisp primitives/forms are easy: funcall should be function call, defmacro - define macro. Some are in the middle - incf is... increment... f??? But because common lisp is so old, this primitives/special forms/etc... don't seem to ring a bell. Can you guys, help me with figuring them out? And even more importantly: Where can I find an authoritative resource on learning the meaning/history behind each and every one of them? (I will accept an answer based on this second question)
The documentation doesn't help me too:
* (describe #'let)
#<CLOSURE (:SPECIAL LET) {10013DC6AB}>
[compiled closure]
Lambda-list: (&REST ARGS)
Derived type: (FUNCTION (&REST T) NIL)
Documentation:
T
Source file: SYS:SRC;COMPILER;INFO-FUNCTIONS.LISP
* (documentation 'let 'function)
"LET ({(var [value]) | var}*) declaration* form*
During evaluation of the FORMS, bind the VARS to the result of evaluating the
VALUE forms. The variables are bound in parallel after all of the VALUES forms
have been evaluated."
* (inspect 'let)
The object is a SYMBOL.
0. Name: "LET"
1. Package: #<PACKAGE "COMMON-LISP">
2. Value: "unbound"
3. Function: #<CLOSURE (:SPECIAL LET) {10013DC6AB}>
4. Plist: (SB-WALKER::WALKER-TEMPLATE SB-WALKER::WALK-LET)
What do the following lisp primitives/special forms/special operators/functions mean?
let, flet
progn
car
cdr
acc
setq, setf
incf
(write more in the comments so we can make a good list!)
let: LET variable-name be bound to a certain value
flet: LET Function-name be bound to a certain function
progn: execute a PROGram sequence and return the Nth value (the last value)
car: Contents of the Address Register (historic)
cdr: Contents of the Decrement Register (historic)
acc: ACCumulator
setq: SET Quote, a variant of the set function, where in setq the user doesn't quote the variable
setf: SET Function quoted, shorter name of the original name setfq. Here the function/place is not evaluated.
incf: INCrement Function quoted, similar to setf. Increments a place.
Other conventions:
Macros / Special Forms who change a place should have an f at the end: setf, psetf, incf, decf, ...
Macros who are DEFining something should have def or define in front: defun, defmethod, defclass, define-method-combination...
Functions who are destructive should have an n in front, for Non-consing: nreverse, ...
Predicates have a p or -p at the end: adjustable-array-p, alpha-char-p,...
special variables have * at the front and back: *standard-output*, ...
There are other naming conventions.
let: Well, that's a normal word, and is used like in maths ("let x = 3 in ...").
flet: I'd guess "function let", because that's what it does.
progn: This is probably related also to prog1 and prog2. I read is as "program whose nth form dictates the result value". The program has n forms, so it's the last one that forms the result value of the progn form.
car and cdr: "Contents address register" resp. "Contents decrement register". This is related to the IBM 704 which Lisp was originally implemented for.
setq: "set quote", originally an abbreviation for (set (quote *abc*) value).
setf: "set field", came up when lexical variables appeared. This is a good read on the set functions.
Where can I find an authoritative resource on learning the meaning/history behind each and every one of them?
The HyperSpec is a good place to start, and ultimately the ANSI standard. Though "Common Lisp The Language" could also shine some light on the history of some names.
(Oh, and you got defmacro wrong, that's "Definition for Mac, Read Only." ;) )
I have a code-base for a graphics program in cljx that gets compiled to Clojure and ClojureScript.
I now want to introduce my first macro.
(defmacro optional-styled-primitive [args body]
(let [extra (conj args 'style)]
`(fn (~extra (->SShape ~'style ~body))
(~args (->SShape {} ~body))
)
)
)
The purpose of this macro is to take a list of arguments, and an expression that uses those arguments to generate a geometry. And to return a function with two arities : one of which takes an optional style parameter. This macro is then to be used within the file where it's defined, to make a number of other functions that optionally take styles. For example :
(def square (optional-styled-primitive [n] [[0 0] [0 n] [n n] [n 0]]))
But introducing this macro, obviously, breaks the ClojureScript stage of the compilation.
What I can't figure out is what to do about it. The online discussions talk about ClojureScript needing to use :require-macros but I never actually export or require this macro anywhere. I just want to use it where it's defined. So how can I, in the middle of a file, tell the compiler to use Clojure to expand this macro, before it gets to the ClojureScript compiler?
OK.
I've made some progress with this.
Here's what I did.
1) I refactored my macro definition out into a separate file called macros.cljx
2) In the file where I was using the macros, I did this. (A different require for clj and cljs)
(#+clj :require #+cljs :require-macros
[myapp.macros :refer [optional-styled-primitive]])
3) I updated my leiningen project.clj file :
:cljsbuild {:builds [{
:source-paths ["target/classes" "src-cljs" ] ...
The important thing here I added "target/classes", which is the output-path where cljx puts the clj files it creates, to the cljsbuild source-paths. This is where the cljsbuild process can find the clj file with the macro definition.
I'm not sure if this is the right or principled way to solve the problem. But it now seems to be working (unless I'm confused by something).
The question is not about using keywords, but actually about keyword implementation. For example, when I create some function with keyword parameters and make a call:
(defun fun (&key key-param) (print key-param)) => FUN
(find-symbol "KEY-PARAM" 'keyword) => NIL, NIL ;;keyword is not still registered
(fun :key-param 1) => 1
(find-symbol "KEY-PARAM" 'keyword) => :KEY-PARAM, :EXTERNAL
How a keyword is used to pass an argument? Keywords are symbols whos values are themselves, so how a corresponding parameter can be bound using a keyword?
Another question about keywords — keywords are used to define packages. We can define a package named with already existing keyword:
(defpackage :KEY-PARAM) => #<The KEY-PARAMETER package, 0/16 ...
(in-package :KEY-PARAM) => #<The KEY-PARAMETER package, 0/16 ...
(defun fun (&key key-param) (print key-param)) => FUN
(fun :KEY-PARAM 1) => 1
How does system distinguish the usage of :KEY-PARAM between package name and function parameter name?
Also we can make something more complicated, if we define function KEY-PARAM and export it (actually not function, but name):
(in-package :KEY-PARAM)
(defun KEY-PARAM (&key KEY-PARAM) KEY-PARAM) => KEY-PARAM
(defpackage :KEY-PARAM (:export :KEY-PARAM))
;;exporting function KEY-PARAM, :KEY-PARAM keyword is used for it
(in-package :CL-USER) => #<The COMMON-LISP-USER package, ...
(KEY-PARAM:KEY-PARAM :KEY-PARAM 1) => 1
;;calling a function KEY-PARAM from :KEY-PARAM package with :KEY-PARAM parameter...
The question is the same, how Common Lisp does distinguish the usage of keyword :KEY-PARAM here?
If there is some manual about keywords in Common Lisp with explanation of their mechanics, I would be grateful if you posted a link here, because I could find just some short articles only about usage of the keywords.
See Common Lisp Hyperspec for full details of keyword parameters. Notice that
(defun fun (&key key-param) ...)
is actually short for:
(defun fun (&key ((:key-param key-param)) ) ...)
The full syntax of a keyword parameter is:
((keyword-name var) default-value supplied-p-var)
default-value and supplied-p-var are optional. While it's conventional to use a keyword symbol as the keyword-name, it's not required; if you just specify a var instead of (keyword-name var), it defaults keyword-name to being a symbol in the keyword package with the same name as var.
So, for example, you could do:
(defun fun2 (&key ((myoption var))) (print var))
and then call it as:
(fun 'myoption 3)
The way it works internally is when the function is being called, it steps through the argument list, collecting pairs of arguments <label, value>. For each label, it looks in the parameter list for a parameter with that keyword-name, and binds the corresponding var to value.
The reason we normally use keywords is because the : prefix stands out. And these variables have been made self-evaluating so that we don't have to quote them as well, i.e. you can write :key-param instead of ':key-param (FYI, this latter notation was necessary in earlier Lisp systems, but the CL designers decided it was ugly and redundant). And we don't normally use the ability to specify a keyword with a different name from the variable because it would be confusing. It was done this way for full generality. Also, allowing regular symbols in place of keywords is useful for facilities like CLOS, where argument lists get merged and you want to avoid conflicts -- if you're extending a generic function you can add parameters whose keywords are in your own package and there won't be collisions.
The use of keyword arguments when defining packages and exporting variables is again just a convention. DEFPACKAGE, IN-PACKAGE, EXPORT, etc. only care about the names they're given, not what package it's in. You could write
(defpackage key-param)
and it would usually work just as well. The reason many programmers don't do this is because it interns a symbol in their own package, and this can sometimes cause package conflicts if this happens to have the same name as a symbol they're trying to import from another package. Using keywords divorces these parameters from the application's package, avoiding potential problems like this.
The bottom line is: when you're using a symbol and you only care about its name, not its identity, it's often safest to use a keyword.
Finally, about distinguishing keywords when they're used in different ways. A keyword is just a symbol. If it's used in a place where the function or macro just expects an ordinary parameter, the value of that parameter will be the symbol. If you're calling a function that has &key arguments, that's the only time they're used as labels to associate arguments with parameters.
The good manual is Chapter 21 of PCL.
Answering your questions briefly:
keywords are exported symbols in keyword package, so you can refer to them not only as :a, but also as keyword:a
keywords in function argument lists (called lambda-lists) are, probably, implemented in the following way. In presence of &key modifier the lambda form is expanded into something similar to this:
(let ((key-param (getf args :key-param)))
body)
when you use a keyword to name a package it is actually used as a string-designator. This is a Lisp concept that allows to pass to a certain function dealing with strings, that are going to be used as symbols later (for different names: of packages, classes, functions, etc.) not only strings, but also keywords and symbols. So, the basic way to define/use package is actually this:
(defpackage "KEY-PARAM" ...)
But you can as well use:
(defpackage :key-param ...)
and
(defpackage #:key-param ...)
(here #: is a reader macro to create uninterned symbols; and this way is the preferred one, because you don't create unneeded keywords in the process).
The latter two forms will be converted to upper-case strings. So a keyword stays a keyword, and a package gets its named as string, converted from that keyword.
To sum up, keywords have the value of themselves, as well as any other symbols. The difference is that keywords don't require explicit qualification with keyword package or its explicit usage. And as other symbols they can serve as names for objects. Like, for example, you can name a function with a keyword and it will be "magically" accessible in every package :) See #Xach's blogpost for details.
There is no need for "the system" to distinguish different uses of keywords. They are just used as names. For example, imagine two plists:
(defparameter *language-scores* '(:basic 0 :common-lisp 5 :python 3))
(defparameter *price* '(:basic 100 :fancy 500))
A function yielding the score of a language:
(defun language-score (language &optional (language-scores *language-scores*))
(getf language-scores language))
The keywords, when used with language-score, designate different programming languages:
CL-USER> (language-score :common-lisp)
5
Now, what does the system do to distinguish the keywords in *language-scores* from those in *price*? Absolutely nothing. The keywords are just names, designating different things in different data structures. They are no more distinguished than homophones are in natural language – their use determines what they mean in a given context.
In the above example, nothing prevents us from using the function with a wrong context:
(language-score :basic *prices*)
100
The language did nothing to prevent us from doing this, and the keywords for the not-so-fancy programming language and the not-so-fancy product are just the same.
There are many possibilities to prevent this: Not allowing the optional argument for language-score in the first place, putting *prices* in another package without externing it, closing over a lexical binding instead of using the globally special *language-scores* while only exposing means to add and retrieve entries. Maybe just our understanding of the code base or convention are enough to prevent us from doing that. The point is: The system's distinguishing the keywords themselves is not necessary to achieve what we want to implement.
The specific keyword uses you ask about are not different: The implementation might store the bindings of keyword-arguments in an alist, a plist, a hash-table or whatever. In the case of package names, the keywords are just used as package designators and the package name as a string (in uppercase) might just be used instead. Whether the implementation converts strings to keywords, keywords to strings, or something entirely different internally doesn't really matter. What matters is just the name, and in which context it is used.
I have a problem constructing a DSL in Clojure. This is the concrete problem I have isolated from everything else.
Let's say we hava a simple macro:
user> (defmacro m1 [x] `'~x)
#'user/m1
it just returns the literal supplied
user> (m1 toUpperCase)
toUpperCase
if we call java method for object everything works as expected
user> (. "a" toUpperCase)
"A"
but if we substitute method name for macro call that returns the methodname
user> (. "a" (m1 toUpperCase))
; Evaluation aborted.
Unable to resolve symbol: toUpperCase in this context
I want to use some java library that has fluent interface like a().b().c().
This maps to Clojure as:
(.. obj method1 method2 method3....etc)
I want to create macros that substitute some parts of this chain so my code should be like:
(.. obj method1 macro1)
and that should expand to
(.. obj method1 method2 method3)
definline also doesn't help. I tried that also
The reason you're running into this problem is that the . special form does not evaluate its second argument (the symbol specifying the method or field) in the way you expect: it sees it as a call of the METHOD m1, with the ARGUMENT toUppercase. Because of that, you cannot generate the symbol for the method dynamically just as an argument to . (dot) - even if you use a macro to specify that argument.
A way to work around that is to include the . in your macro:
(defmacro m1 [x y] `(. ~x (~y)))
(m1 "a" toUppercase)
user> "A"
Note that you need to wrap parentheses around ~y to indicate you want to call a method instead of reading a field.
I realize that the first rule of Macro Club is Don't Use Macros, so the following question is intended more as an exercise in learning Clojure than anything else (I realize this isn't necessarily the best use of macros).
I want to write a simple macro which acts as a wrapper around a regular (defn) macro and winds up adding some metadata to the defined function. So I'd like to have something like this:
(defn-plus f [x] (inc x))
...expand out to something like this:
(defn #^{:special-metadata :fixed-value} f [x] (inc x))
In principle this doesn't seem that hard to me, but I'm having trouble nailing down the specifics of getting the [args] and other forms in the defined function to be parsed out correctly.
As a bonus, if possible I'd like the macro to be able to handle all of the disparate forms of defn (ie, with or without docstrings, multiple arity definitions, etc). I saw some things in the clojure-contrib/def package that looked possibly helpful, but it was difficult to find sample code which used them.
Updated:
The previous version of my answer was not very robust. This seems like a simpler and more proper way of doing it, stolen from clojure.contrib.def:
(defmacro defn-plus [name & syms]
`(defn ~(vary-meta name assoc :some-key :some-value) ~#syms))
user> (defn-plus ^Integer f "Docstring goes here" [x] (inc x))
#'user/f
user> (meta #'f)
{:ns #<Namespace user>, :name f, :file "NO_SOURCE_PATH", :line 1, :arglists ([x]), :doc "Docstring goes here", :some-key :some-value, :tag java.lang.Integer}
#^{} and with-meta are not the same thing. For an explanation of the difference between them, see Rich's discussion on the Clojure mailing list. It's all a bit confusing and it's come up a bunch of times on the mailing list; see also here for example.
Note that def is a special form and it handles metadata a bit oddly compared with some other parts of the language. It sets the metadata of the var you're deffing to the metadata of the symbol that names the var; that's the only reason the above works, I think. See the DefExpr class in Compiler.java in the Clojure source if you want to see the guts of it all.
Finally, page 216 of Programming Clojure says:
You should generally avoid reader macros in macro expansions, since reader macros are evaluated at read time, before macro expansion begins.