I recently discovered that all of my implementations of Scheme throw an error when I try to use (cadaddr (list 1 3 (list 5 7) 9)). Apparently, by default Scheme does not allow any car and cdr combinations in the single-function form that use more than four abbreviated car and cdrcalls. I originally blamed this on Scheme's minimalism, but then I discovered that Common Lisp also shares this defect.
Can this be solved with a macro? Can we write a macro that allows an arbitrary amount of a and d in its c[...]r calls and returns the expected result, while also having the Common Lisp-like compatibility with macros like setf? If not, why not? And if so, has a reason ever been given for this is not a default feature in any lisp that I've seen?
Such a macro is described in Let Over Lambda for common lisp. You must wrap your code with (with-cxrs ...) to bring them all into scope, but it walks your code to see which combinators you need. I wrote a Clojure port of it years ago for fun, though of course nobody (including me) has ever wanted to use it for real. You could port it to Scheme if you liked.
(defn cxr-impl [name]
(when-let [op (second (re-matches #"c([ad]+)r" name))]
`(comp ~#(map {\a `first \d `rest} op))))
(defmacro with-cxrs [& body]
(let [symbols (remove coll? (tree-seq coll? seq body))]
`(let [~#(for [sym symbols
:let [impl (cxr-impl (name sym))]
:when impl
thing [sym impl]]
thing)]
~#body)))
user> (macroexpand-1 '(with-cxrs (inc (caadaaddadr x))))
(let [caadaaddadr (comp first first rest first first rest rest first rest)]
(inc (caadaaddadr x)))
https://groups.google.com/g/clojure/c/CanBrJPJ4aI/m/S7wMNqmj_Q0J
As noted in the mailing list thread, there are some bugs you'd have to work out if you wanted to use this for real.
In my ongoing quest to recreate lodash in lisp as a way of getting familiar with the language I am trying to write a concat-list function that takes an initial list and an arbitrary number of additional lists and concatenates them.
I'm sure that this is just a measure of getting familiar with lisp convention, but right now my loop is just returning the second list in the argument list, which makes sense since it is the first item of other-lists.
Here's my non-working code (edit: refactored):
(defun concat-list (input-list &rest other-lists)
;; takes an arbitrary number of lists and merges them
(loop
for list in other-lists
append list into input-list
return input-list
)
)
Trying to run (concat-list '(this is list one) '(this is list two) '(this is list three)) and have it return (this is list one this is list two this is list three).
How can I spruce this up to return the final, merged list?
The signature of your function is a bit unfortunate, it becomes easier if you don't treat the first list specially.
The easy way:
(defun concat-lists (&rest lists)
(apply #'concatenate 'list lists))
A bit more lower level, using loop:
(defun concat-lists (&rest lists)
(loop :for list :in lists
:append list))
Going lower, using dolist:
(defun concat-lists (&rest lists)
(let ((result ()))
(dolist (list lists (reverse result))
(setf result (revappend list result)))))
Going even lower would maybe entail implementing revappend yourself.
It's actually good style in Lisp not to use LABELS based iteration, since a) it's basically a go-to like low-level iteration style and it's not everywhere supported. For example the ABCL implementation of Common Lisp on the JVM does not support TCO last I looked. Lisp has wonderful iteration facilities, which make the iteration intention clear:
CL-USER 217 > (defun my-append (&rest lists &aux result)
(dolist (list lists (nreverse result))
(dolist (item list)
(push item result))))
MY-APPEND
CL-USER 218 > (my-append '(1 2 3) '(4 5 6) '(7 8 9))
(1 2 3 4 5 6 7 8 9)
Some pedagogical solutions to this problem
If you just want to do this, then use append, or nconc (destructive), which are the functions which do it.
If you want to learn how do to it, then learning about loop is not how to do that, assuming you want to learn Lisp: (loop for list in ... append list) really teaches you nothing but how to write a crappy version of append using arguably the least-lispy part of CL (note I have nothing against loop & use it a lot, but if you want to learn lisp, learning loop is not how to do that).
Instead why not think about how you would write this if you did not have the tools to do it, in a Lispy way.
Well, here's how you might do that:
(defun append-lists (list &rest more-lists)
(labels ((append-loop (this more results)
(if (null this)
(if (null more)
(nreverse results)
(append-loop (first more) (rest more) results))
(append-loop (rest this) more (cons (first this) results)))))
(append-loop list more-lists '())))
There's a dirty trick here: I know that results is completely fresh so I am using nreverse to reverse it, which does so destructively. Can we write nreverse? Well, it's easy to write reverse, the non-destructive variant:
(defun reverse-nondestructively (list)
(labels ((r-loop (tail reversed)
(if (null tail)
reversed
(r-loop (rest tail) (cons (first tail) reversed)))))
(r-loop list '())))
And it turns out that a destructive reversing function is only a little harder:
(defun reverse-destructively (list)
(labels ((rd-loop (tail reversed)
(if (null tail)
reversed
(let ((rtail (rest tail)))
(setf (rest tail) reversed)
(rd-loop rtail tail)))))
(rd-loop list '())))
And you can check it works:
> (let ((l (make-list 1000 :initial-element 1)))
(time (reverse-destructively l))
(values))
Timing the evaluation of (reverse-destructively l)
User time = 0.000
System time = 0.000
Elapsed time = 0.000
Allocation = 0 bytes
0 Page faults
Why I think this is a good approach to learning Lisp
[This is a response to a couple of comments which I thought was worth adding to the answer: it is, of course, my opinion.]
I think that there are at least three different reasons for wanting to solve a particular problem in a particular language, and the approach you might want to take depends very much on what your reason is.
The first reason is because you want to get something done. In that case you want first of all to find out if it has been done already: if you want to do x and the language a built-in mechanism for doing x then use that. If x is more complicated but there is some standard or optional library which does it then use that. If there's another language you could use easily which does x then use that. Writing a program to solve the problem should be something you do only as a last resort.
The second reason is because you've fallen out of the end of the first reason, and you now find yourself needing to write a program. In that case what you want to do is use all of the tools the language provides in the best way to solve the problem, bearing in mind things like maintainability, performance and so on. In the case of CL, then if you have some problem which naturally involves looping, then, well, use loop if you want to. It doesn't matter whether loop is 'not lispy' or 'impure' or 'hacky': just do what you need to do to get the job done and make the code maintainable. If you want to print some list of objects, then by all means write (format t "~&~{~A~^, ~}~%" things).
The third reason is because you want to learn the language. Well, assuming you can program in some other language there are two approaches to doing this.
the first is to say 'I know how to do this thing (write loops, say) in languages I know – how do I do it in Lisp?', and then iterate this for all the thing you already know how to do in some other language;
the second is to say 'what is it that makes Lisp distinctive?' and try and understand those things.
These approaches result in very approaches to learning. In particular I think the first approach is often terrible: if the language you know is, say, Fortran, then you'll end up writing Fortran dressed up as Lisp. And, well, there are perfectly adequate Fortran compilers out there: why not use them? Even worse, you might completely miss important aspects of the language and end up writing horrors like
(defun sum-list (l)
(loop for i below (length l)
summing (nth i l)))
And you will end up thinking that Lisp is slow and pointless and return to the ranks of the heathen where you will spread such vile calumnies until, come the great day, the golden Lisp horde sweeps it all away. This has happened.
The second approach is to ask, well, what are the things that are interesting about Lisp? If you can program already, I think this is a much better approach to the first, because learning the interesting and distinctive features of a language first will help you understand, as quickly as possible, whether its a language you might actually want to know.
Well, there will inevitably be argument about what the interesting & distinctive features of Lisp are, but here's a possible, partial, set.
The language has a recursively-defined data structure (S expressions or sexprs) at its heart, which is used among other things to represent the source code of the language itself. This representation of the source is extremely low-commitment: there's nothing in the syntax of the language which says 'here's a block' or 'this is a conditiona' or 'this is a loop'. This low-commitment can make the language hard to read, but it has huge advantages.
Recursive processes are therefore inherently important and the language is good at expressing them. Some variants of the language take this to the extreme by noticing that iteration is simply a special case of recursion and have no iterative constructs at all (CL does not do this).
There are symbols, which are used as names for things both in the language itself and in programs written in the language (some variants take this more seriously than others: CL takes it very seriously).
There are macros. This really follows from the source code of the language being represented as sexprs and this structure having a very low commitment to what it means. Macros, in particular, are source-to-source transformations, with the source being represented as sexprs, written in the language itself: the macro language of Lisp is Lisp, without restriction. Macros allow the language itself to be seamlessly extended: solving problems in Lisp is done by designing a language in which the problem can be easily expressed and solved.
The end result of this is, I think two things:
recursion, in addition to and sometimes instead of iteration is an unusually important technique in Lisp;
in Lisp, programming means building a programming language.
So, in the answer above I've tried to give you examples of how you might think about solving problems involving a recursive data structure recursively: by defining a local function (append-loop) which then recursively calls itself to process the lists. As Rainer pointed out that's probably not a good way of solving this problem in Common Lisp as it tends to be hard to read and it also relies on the implementation to turn tail calls into iteration which is not garuanteed in CL. But, if your aim is to learn to think the way Lisp wants you to think, I think it is useful: there's a difference between code you might want to write for production use, and code you might want to read and write for pedagogical purposes: this is pedagogical code.
Indeed, it's worth looking at the other half of how Lisp might want you to think to solve problems like this: by extending the language. Let's say that you were programming in 1960, in a flavour of Lisp which has no iterative constructs other than GO TO. And let's say you wanted to process some list iteratively. Well, you might write this (this is in CL, so it is not very like programming in an ancient Lisp would be: in CL tagbody establishes a lexical environment in the body of which you can have tags – symbols – and then go will go to those tags):
(defun print-list-elements (l)
;; print the elements of a list, in order, using GO
(let* ((tail l)
(current (first tail)))
(tagbody
next
(if (null tail)
(go done)
(progn
(print current)
(setf tail (rest tail)
current (first tail))
(go next)))
done)))
And now:
> (print-list-elements '(1 2 3))
1
2
3
nil
Let's program like it's 1956!
So, well, let's say you don't like writing this sort of horror. Instead you'd like to be able to write something like this:
(defun print-list-elements (l)
;; print the elements of a list, in order, using GO
(do-list (e l)
(print e)))
Now if you were using most other languages you need to spend several weeks mucking around with the compiler to do this. But in Lisp you spend a few minutes writing this:
(defmacro do-list ((v l &optional (result-form nil)) &body forms)
;; Iterate over a list. May be buggy.
(let ((tailn (make-symbol "TAIL"))
(nextn (make-symbol "NEXT"))
(donen (make-symbol "DONE")))
`(let* ((,tailn ,l)
(,v (first ,tailn)))
(tagbody
,nextn
(if (null ,tailn)
(go ,donen)
(progn
,#forms
(setf ,tailn (rest ,tailn)
,v (first ,tailn))
(go ,nextn)))
,donen
,result-form))))
And now your language has an iteration construct which it previously did not have. (In real life this macro is called dolist).
And you can go further: given our do-list macro, let's see how we can collect things into a list:
(defun collect (thing)
;; global version: just signal an error
(declare (ignorable thing))
(error "not collecting"))
(defmacro collecting (&body forms)
;; Within the body of this macro, (collect x) will collect x into a
;; list, which is returned from the macro.
(let ((resultn (make-symbol "RESULT"))
(rtailn (make-symbol "RTAIL")))
`(let ((,resultn '())
(,rtailn nil))
(flet ((collect (thing)
(if ,rtailn
(setf (rest ,rtailn) (list thing)
,rtailn (rest ,rtailn))
(setf ,resultn (list thing)
,rtailn ,resultn))
thing))
,#forms)
,resultn)))
And now we can write the original append-lists function entirely in terms of constructs we've invented:
(defun append-lists (list &rest more-lists)
(collecting
(do-list (e list) (collect e))
(do-list (l more-lists)
(do-list (e l)
(collect e)))))
If that's not cool then nothing is.
In fact we can get even more carried away. My original answer above used labels to do iteration As Rainer has pointed out, this is not safe in CL since CL does not mandate TCO. I don't particularly care about that (I am happy to use only CL implementations which mandate TCO), but I do care about the problem that using labels this way is hard to read. Well, you can, of course, hide this in a macro:
(defmacro looping ((&rest bindings) &body forms)
;; A sort-of special-purpose named-let.
(multiple-value-bind (vars inits)
(loop for b in bindings
for var = (typecase b
(symbol b)
(cons (car b))
(t (error "~A is hopeless" b)))
for init = (etypecase b
(symbol nil)
(cons (unless (null (cddr b))
(error "malformed binding ~A" b))
(second b))
(t
(error "~A is hopeless" b)))
collect var into vars
collect init into inits
finally (return (values vars inits)))
`(labels ((next ,vars
,#forms))
(next ,#inits))))
And now:
(defun append-lists (list &rest more-lists)
(collecting
(looping ((tail list) (more more-lists))
(if (null tail)
(unless (null more)
(next (first more) (rest more)))
(progn
(collect (first tail))
(next (rest tail) more))))))
And, well, I just think it is astonishing that I get to use a programming language where you can do things like this.
Note that both collecting and looping are intentionally 'unhygenic': they introduce a binding (for collect and next respectively) which is visible to code in their bodies and which would shadow any other function definition of that name. That's fine, in fact, since that's their purpose.
This kind of iteration-as-recursion is certainly cool to think about, and as I've said I think it really helps you to think about how the language can work, which is my purpose here. Whether it leads to better code is a completely different question. Indeed there is a famous quote by Guy Steele from one of the 'lambda the ultimate ...' papers:
procedure calls may be usefully thought of as GOTO statements which also pass parameters
And that's a lovely quote, except that it cuts both ways: procedure calls, in a language which optimizes tail calls, are pretty much GOTO, and you can do almost all the horrors with them that you can do with GOTO. But GOTO is a problem, right? Well, it turns out so are procedure calls, for most of the same reasons.
So, pragmatically, even in a language (or implementation) where procedure calls do have all these nice characteristics, you end up wanting constructs which can express iteration and not recursion rather than both. So, for instance, Racket which, being a Scheme-family language, does mandate tail-call elimination, has a whole bunch of macros with names like for which do iteration.
And in Common Lisp, which does not mandate tail-call elimination but which does have GOTO, you also need to build macros to do iteration, in the spirit of my do-list above. And, of course, a bunch of people then get hopelessly carried away and the end point is a macro called loop: loop didn't exist (in its current form) in the first version of CL, and it was common at that time to simply obtain a copy of it from somewhere, and make sure it got loaded into the image. In other words, loop, with all its vast complexity, is just a macro which you can define in a CL which does not have it already.
OK, sorry, this is too long.
(loop for list in (cons '(1 2 3)
'((4 5 6) (7 8 9)))
append list)
I'm new in learning and working with clojure so I've got a basic question on macros in clojure. I didn't find a case where you really need macros so I'm wondering if there is a real case where only a macro and no normal function or multimethod solves your problem.
Can someone show a simple example for this?
I think I didn't understand the concept of macros in clojure.
Clojure macros take literal code whereas functions take evaluated code. In turn, macros are useful when you need to manipulate literal code. Literal code and evaluated code are equivalent except for two (very important) instances: symbols and expressions (maps, vectors, sets, strings, keywords, numbers, booleans, etcetera, will all "evalute to themselves").
user=> 1 ;evaluates to itself
1
user=> "abc" ;evaluates to itself
"abc"
user=> :xyz ;evaluates to itself
:xyz
user=> [1 "abc" :xyz] ;evaluates to itself
[1 "abc" :xyz]
As opposed to:
user=> (+ 1 2) ;an expression evaluates to not itself
3
user=> Math/PI ;a symbol evaluates to not itself
3.141592653589793
user=> + ;another example, a little weirder
#<core$_PLUS_ clojure.core$_PLUS_#417ffb28>
Let's say you wanted to create some-fn-or-macro to behave like this:
user=> (some-fn-or-macro (get {:a 10 :b 20} :a))
"(get {:a 10 :b 20} :a)"
user=> (some-fn-or-macro +)
"+"
You will not be able to do this with a function. Try it:
user=> (defn some-fn-or-macro [expr] (str expr))
#'user/some-fn-or-macro
user=> (some-fn-or-macro (get {:a 10 :b 20} :a))
"10"
What happened here? The argument to some-fn-or-macro (namely expr) got evaluated prior to being string-ized. However, if all we do is change the definition from a function to a macro, everything will be great:
user=> (defmacro some-fn-or-macro [expr] (str expr))
#'user/some-fn-or-macro
user=> (some-fn-or-macro (get {:a 10 :b 20} :a))
"(get {:a 10 :b 20} :a)"
That being said, if we take the original function definition again, and simply quote the argument on invocation, that also works:
user=> (defn some-fn-or-macro [expr] (str expr))
#'user/some-fn-or-macro
user=> (some-fn-or-macro '(get {:a 10 :b 20} :a))
"(get {:a 10 :b 20} :a)"
So you only ever need to write a macro if your use-case demands that arguments remain literal/unevaluated. If you have control over how your tool is used (which I'm guessing is always marginally true), you can decide to develop a function, and instruct users to quote arguments as necessary.
***Note: How I've used macros above might leave you in the dark about one extremely important fact of macros: their output gets evaluated. For example:
user=> (defmacro example-macro [] '(+ 1 2))
#'user/example-macro
user=> (example-macro)
3
You might think this is odd. There are a couple ways to make sense of it. Macros expect to take source code as input, so it's only natural that they'd give source code as output--and source code demands evaluation at some point. Actually, I tend to think of the difference between macros and functions as "shifted evaluation"--evaluation happens either "before" invocation, on the arguments (for functions); or "after" invocation, on the output (for macros).
Important thing here is that macro does not evaluate its arguments and can be used for arbitrary transformation of source code.
Most basic examples of macros would be when and when-not macros:
(defmacro when
"Evaluates test. If logical true, evaluates body in an implicit do."
[test & body]
`(if ~test (do ~#body)))
(defmacro when-not
"Evaluates test. If logical false, evaluates body in an implicit do."
[test & body]
`(if test nil (do ~#body)))
Function won't work here, as it has to evaluate all its arguments before execution.
P.S. If you're interested in the topic and want to know more, see also this my answer. It is about Common Lisp, but it could be useful for you too. I also give a link to a cool Paul Graham's article at the end of the answer.
P.S.S If you want an example of a new useful macro, I would like to comment something of Paul Graham here:
It would be convenient here if I could give an example of a powerful macro, and say there! how about that? But if I did, it would just look like gibberish to someone who didn't know Lisp; there isn't room here to explain everything you'd need to know to understand what it meant.
Imagine the following code to dynamically create a macro:
(def a (list '+ 1 2))
(def b (list '- 10 5))
(def c (list '/ 22 2))
(defmacro gg [h]
(let [k# `~h]
k#))
The intent is to pass a vector of symbols to a macro, do some evaluation on each element of the vector such that it returns a nice macro-esque form, then have the macro combine them into a nice macro and evaluate it. The above example works all except the actual evaluation.
When I run it I get:
(gg [a b c])
=> [(+ 1 2) (- 10 5) (/ 22 2)]
What is the secret to passing a symbol that is a list of symbols and getting a macro to evaluate them? I have tried lots of combinations of quoting and have yet to hit the right one.
The real purpose of this question is to build an Archimedes Ogre query based on a definition of a path through the graph. If someone has an example of that, I would be grateful.
EDIT:
(defmacro gg2 [h]
`(do ~#(map identity h)))
(macroexpand '(gg2 [a b c]))
=> (do a b c)
(gg2 [a b c])
=> (/ 22 2)
I was hoping to get 11 rather than the form.
You don't need a macro. Macros don't do what you're looking for here. What you are looking for is eval.
(def a '/)
(def b 22)
(def c 2)
(eval (list* [a b c]))
=> 11
Of course, you can write a macro which expands into (eval (list* ...)) if you want. It could just as well be a function though.
This is a very common mistake when starting out with macros; trying to write a macro which depends on the run-time value of its arguments. Macros run at compile-time, and generally the values of the symbols which you pass to a macro are not yet available when the macro is expanded.
About the use of eval, some cautions are in order. No one said it better than Paul Graham:
Generally it is not a good idea to call eval at runtime, for two reasons:
It’s inefficient: eval is handed a raw list, and either has to compile it on the spot, or evaluate it in an interpreter. Either way is slower than compiling the code beforehand, and just calling it.
It’s less powerful, because the expression is evaluated with no lexical context. Among other things, this means that you can’t refer to ordinary variables visible outside the expression being evaluated.
Usually, calling eval explicitly is like buying something in an airport gift-shop. Having waited till the last moment, you have to pay high prices for a limited selection of second-rate goods.
It seems like in order to use multiple return values in Racket, I have to either use define-values or collect them into a list with (call-with-values (thunk (values-expr)) list). In the latter case, why would someone to choose to return multiple values instead of a list, if just have to collect them into a list anyway? Additionally, both of these are very wordy and awkward to work into most code. I feel like I must be misunderstanding something very basic about multiple-return-values. For that matter, how do I write a procedure accepting multiple return values?
Although I may be missing some of the Scheme history and other nuances, I'll give you my practical answer.
First, one rule of thumb is if you need to return more than 2 or 3 values, don't use multiple values and don't use a list. Use a struct. That will usually be easier to read and maintain.
Racket's match forms make it much easier to destructure a list return value -- as easy as define-values:
(define (f)
(list 1 2))
(match-define (list a b) (f))
(do-something-with a b)
;; or
(match (f)
[(list a b) (do-something-with a b)])
If you have some other function, g, that takes a (list/c a b), and you want to compose it with f, it's simpler if f returns a list. It's also simpler if both use a two-element struct. Whereas call-with-values is kind of an awkward hot mess, I think.
Allowing multiple return value is an elegant idea, because it makes return values symmetric with arguments. Using multiple values is also faster than lists or structs (in the current implementation of Racket, although it could work otherwise).
However when readability is a higher priority than performance, then in modern Racket it can be more practical to use a list or a struct, IMHO. Having said that I do use multiple values for one-off private helper functions.
Finally, there's a long, interesting discussion on the Racket mailing list.
Racket doc gives us the quintessential example why, in disguise:
> (let-values ([(q r) (quotient/remainder 10 3)])
(if (zero? r)
q
"3 does *not* divide 10 evenly"))
"3 does *not* divide 10 evenly"
We get two values directly, and use them separately in a computation that follows.
update: In Common Lisp, with its decidedly practical, down-to-the-metal, non-functional approach (where they concern themselves with each extra cons cell allocation), it makes much more sense, especially as it allows one to call such procedures in a "normal" way as well, automatically ignoring the "extra" results, kind of like
(let ([q (quotient/remainder 10 3)])
(list q))
But in Racket this is invalid code. So yeah, it looks like an extraneous feature, better to be avoided altogether.
Using list as the consumer defeats the purpose of multiple values so in that case you could just have used lists to begin with. Multiple values is actually a way of optimization.
Semanticly returning a list and several values are similar, but where you return many values in a list effort goes into creation of cons cells to make the list and destructuring accessors to get the values at the other end. In many cases however, you wouldn't notice the difference in performance.
With multiple values the values are on the stack and (call-with-values (lambda () ... (values x y z)) (lambda (x y z) ...) only checks the number to see if it's correct.. If it's ok you just apply the next procedure since the stack has it's arguments all set from the previous call.
You can make syntactic sugar around this and some popular ones are let-values and SRFI-8 receive is a slightly simpler one. Both uses call-with-values as primitive.
values is handy because it
checks that the number of elements returned is correct
destructures
For example, using
(define (out a b) (printf "a=~a b=~a\n" a b))
then
(let ((lst (list 1 2 3)))
(let ((a (first lst)) (b (second lst))) ; destructure
(out a b)))
will work even though lst has 3 elements, but
(let-values (((a b) (values 1 2 3)))
(out a b))
will not.
If you want the same control and destructuring with a list, you can however use match:
(let ((lst (list 1 2)))
(match lst ((list a b) (out a b))))
Note that he creation of the structure, e.g. (list 1 2) vs (values 1 2) is equivalent.