EDIT I'm not just asking about 'indenting' every single line of code but about "pretty printing" and/or a wider definition of "indenting", where lines would be grouped/split depending on what they contain.
Here's a function (which I wrote to solve an Euler problem but that is not the point: I could have written it differently) which I'm trying to pretty print:
Version manually indented:
(apply max
(flatten
(for [x (range 100 1000)]
(map
#(if (= (str (* x %)) (apply str (reverse (str (* x %)))))
(* x %)
0)
(range x 1000)))))
Apparently this isn't the one true Lisp indentation/printing style and, as I'd like to get good habits from the start, I tried to "pretty print" it, doing the following from Emacs' slime-repl clojure REPL:
user> (doc pprint) ;; just to show which version of pprint I'm using
clojure.pprint/pprint
...
So I tried to do:
user> (pprint '(apply max
(flatten
(for [x (range 100 1000)]
(map
#(if (= (str (* x %)) (apply str (reverse (str (* x %)))))
(* x %)
0)
(range x 1000))))))
And I got the following:
(apply
max
(flatten
(for
[x (range 100 1000)]
(map
(fn*
[p1__13958#]
(if
(=
(str (* x p1__13958#))
(apply str (reverse (str (* x p1__13958#)))))
(* x p1__13958#)
0))
(range x 1000)))))
As I understand it the #(...) notation for the lambda is a "reader macro". So how can I pretty print without triggering the expansion of macros? More generally: how can I pretty print "what I see in my text editor"?
Also, is it normal that nearly everything goes on a new line? For example:
(apply
max
(flatten
(for
[x (range 100 1000)]
(map
...
seems to be a waste of quite some vertical screen real estate. Is this the way Lisp code should be indented?
And a last related question: can Emacs be configured to "pretty print" a selected region or an entire .clj buffer?
There's no "official" way to indent lisp code, but I like the way clojure-mode indents clojure code (see below).
I don't know why pprint prints code the way it does but according to this page, it's an early release that isn't feature-complete yet.
For the emacs indentation part, you might want to take a look at clojure-mode.
The closest I have seen to a consensus is to "indent it like Emacs does", I suppose to be pollitically correct I should include the corollary "indent it like vim does". the other editors like eclipse+CCW seem to match this fairly closely. though the default pretty printer adds more newlines.
Here's what I'm currently doing:
(defn pprint-code [code]
(with-pprint-dispatch code-dispatch
(binding [*print-suppress-namespaces* true]
(pprint code))))
NOTE: Look in clojure.pprint to resolve the free-vars.
It's certainly an improvement. I still don't exactly know how to tweak *print-miser-width* and/or *print-right-margin* to get better output. YMMV. My current beef is with (auto-)gensyms - I want them demangled whilst pprinting in certain contexts.
Ideally, forms/functions could define their own formatting rules, e.g. let-bindings should always be on separate lines. I'm not aware of being able to do this with clojure.core.
For reference, see this overview.
Related
I am taking a programming language principle class where the professor talks about macro using Lisp (Precisly, Elisp). However, he didn't teach us how to write this language. As a result, I am trying to learn myself.
However, there are something that I just can't understand regarding the use of "tick" (') in Lisp. I do, however, understand the basic use. For example, if we have (cdr '(a b c)) it will give us a list (b c). Without the tick symbol, (a b c) would be evaluated as a function.
In the code written by my professor, I have noticed a strange use of "tick" symbol that I can't really understand the usage:
; not sure why there is a "'" in front of if and it is being added to a list ??
(defmacro myand(x y) (list 'if x y nil))
; Another example. Tick symbol before "greater than" symbol ? and made into a list?
(list 'if (list '> x y))
The uses of list and tick symbol just doesn't really make sense to me. Can anyone explain what is going on here? I suppose this is something special to Lisp
The 'tick symbol' is ': '<x> is syntactic sugar for (quote <x>) for any <x>. And (quote <x>) is simply <x> for any object <x>.
quote is needed in Lisp because programs in Lisp often want to reason about things which would have meaning as parts of Lisp programs, and need to say 'this isn't part of this program: it's data'. So, for instance, if I have a list of the form
((a . 1) (b . 2) ... (x . 26)) and I want to look up a symbol in it, I might write a function like this:
(defun ass (key alist)
(if (null? alist)
'()
(if (eql (car (car alist)) key)
(car alist)
(ass key (cdr alist)))))
Then when I wanted to look up, say x, I would have to say (ass (quote x) ...) because if I said (ass x ...) then x would be treated as a variable, and I don't want that: I want the symbol x.
In your example you are looking at programs which write Lisp programs – macros in other words – and these programs need to spend a lot of their time manipulating what will be Lisp source code as data. So in your myand macro, what you want is that
(myand x y)
should be rewritten as
(if x y nil)
Which is the same thing. So to construct this you need a list of four elements: the symbol if, the two variables and the symbol nil. Well you can make a list with list:
(list 'if x y 'nil)
Except that it turns out that the value of the symbol nil is just itself (nil is rather special in many Lisps in fact), so you can not bother quoting that: (list 'if x y nil) (I personally would have quoted it in this case, to make it clear what was literal and what was being evaluated, but that's probably not common).
I would like to generate code: (nth 0 x) (nth 1 x) ... (nth n x)
where x is just some variable name, and n is some number.
I'm trying do this in a following way:
(defmacro gen(n)
(loop for i from 1 to n do
`(nth i x))
)
Checking how it expands by:
(print (macroexpand-1 '(gen 5)))
Console output is: NIL. How to do it properly?
You need to replace do with collect in your loop.
Note however that your macro captures the variable x from the calling environment.
Generally speaking, "macros are advanced material", if you are not comfortable with loop, you probably should not be writing them.
Consider what the value of the following code is:
(loop for i from 1 to 5
do `(nth ,i x))
Since there's no collection happening, the return value of the loop is nil. If we change do to collect:
(loop for i from 1 to 5
collect `(nth ,i x))
We see that we are getting somewhere. However, the resulting list is not actually valid Common Lisp code (and relies on there being a variable x in the environment where the macro is used).
It is not clear what you want to do with these (just run them? they're side-effect free, so just wrapping this in a progn feels somewhat useless), but you need to either cons a progn, list or similar to the front of the list of lists to make it into valid code.
(defmacro gen (n &key (var 'x) (accumulator 'list))
(cons accumulator
(loop for i from 1 to n
collect `(nth ,i ,var))))
This eventually gives us this macro that seems to actually do something approaching "valid".
If Racket's match macro were a function I could do this:
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(apply match expr my-clauses)))
I think there are two very different ways to do this. One is to move my-clauses into macro world, and make a macro something like this (doesn't work):
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(define-syntax-rule (match-clauses expr)
(match expr my-clauses)) ; this is not the way it's done.
; "Macros that work together" discusses this ideas, right? I'll be reading that today.
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(match-clauses expr)))
The alternative, which might be better in the end because it would allow me to change my-clauses at runtime, would be to somehow perform the pattern matching at runtime. Is there any way I can use match on runtime values?
In this question Ryan Culpepper says
It's not possible to create a function where the formal parameters and body are given as run-time values (S-expressions) without using eval.
So I guess I'd have to use eval, but the naive way won't work because match is a macro
(eval `(match ,expr ,#my-clauses) (current-namespace))
I got the desired result with the following voodoo from the guide
(define my-clauses '([(list'+ x y) (list '+ y x)]
[_ 42]))
(define-namespace-anchor a)
(define ns (namespace-anchor->namespace a))
(eval `(match '(+ 1 2) ,#my-clauses) ns) ; '(+ 2 1)
Is the pattern matching happening at runtime now? Is it a bad idea?
To answer the first part of your question (assuming you don't necessarily need the match clauses to be supplied at runtime):
The key is to:
Define my-clauses for compile time ("for syntax").
Reference that correctly in the macro template.
So:
(begin-for-syntax
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42])))
(define-syntax (match-clauses stx)
(syntax-case stx ()
[(_ expr) #`(match expr #,#my-clauses)]))
The pattern matching is happening at runtime in the last example.
One way to check is to look at the expansion:
> (syntax->datum
(expand '(eval `(match '(+ 1 2) ,#my-clauses) ns)))
'(#%app eval (#%app list* 'match ''(+ 1 2) my-clauses) ns)
Whether is a good idea...
Using eval is rather slow, so if you call it often it might be better to find another solution. If you haven't seen it already you might want to read "On eval in dynamic languages generally and in Racket specifically." on the Racket blog.
Thank you both very much, your answers gave me much food for thought. What I am trying to do is still not very well defined, but I seem to be learning a lot in the process, so that's good.
The original idea was to make an equation editor that is a hybrid between paredit and a computer algebra system. You enter an initial math s-expression, e.g. (+ x (* 2 y) (^ (- y x) 2). After that the program presents you with a list of step transformations that you would normally make by hand: substitute a variable, distribute, factor, etc. Like a CAS, but one step at a time. Performing a transformation would happen when the user presses the corresponding key combination, although one possibility is to just show a bunch of possible results, and let the user choose the new state of the expression amongst them. For UI charterm will do for now.
At first I thought I would have the transformations be clauses in a match expression, but now I think I'll make them functions that take and return s-expressions. The trouble with choosing compile time vs runtime is that I want the user to be able to add more transformations, and choose his own keybindings. That could mean that they write some code which I require, or they require mine, before the application is compiled, so it doesn't force me to use eval. But it may be best if I give the user a REPL so he has programmatic control of the expression and his interactions with it as well.
Anyway, today I got caught up reading about macros, evaluation contexts and phases. I'm liking racket more and more and I'm still to investigate about making languages... I will switch to tinkering mode now and see if I get some basic form of what I'm describing to work before my head explodes with new ideas.
I've been reading an article by Olin Shivers titled Stylish Lisp programming techniques and found the second example there (labeled "Technique n-1") a bit puzzling. It describes a self-modifying macro that looks like this:
(defun gen-counter macro (x)
(let ((ans (cadr x)))
(rplaca (cdr x)
(+ 1 ans))
ans))
It's supposed to get its calling form as argument x (i.e. (gen-counter <some-number>)). The purpose of this is to be able to do something like this:
> ;this prints out the numbers from 0 to 9.
(do ((n 0 (gen-counter 1)))
((= n 10) t)
(princ n))
0.1.2.3.4.5.6.7.8.9.T
>
The problem is that this syntax with the macro symbol after the function name is not valid in Common Lisp. I've been unsuccessfully trying to obtain similar behavior in Common Lisp. Can someone please provide a working example of analogous macro in CL?
Why the code works
First, it's useful to consider the first example in the paper:
> (defun element-generator ()
(let ((state '(() . (list of elements to be generated)))) ;() sentinel.
(let ((ans (cadr state))) ;pick off the first element
(rplacd state (cddr state)) ;smash the cons
ans)))
ELEMENT-GENERATOR
> (element-generator)
LIST
> (element-generator)
OF
> (element-generator)
This works because there's one literal list
(() . (list of elements to be generated)
and it's being modified. Note that this is actually undefined behavior in Common Lisp, but you'll get the same behavior in some Common Lisp implementations. See Unexpected persistence of data and some of the other linked questions for a discussion of what's happening here.
Approximating it in Common Lisp
Now, the paper and code you're citing actually has some useful comments about what this code is doing:
(defun gen-counter macro (x) ;X is the entire form (GEN-COUNTER n)
(let ((ans (cadr x))) ;pick the ans out of (gen-counter ans)
(rplaca (cdr x) ;increment the (gen-counter ans) form
(+ 1 ans))
ans)) ;return the answer
The way that this is working is not quite like an &rest argument, as in Rainer Joswig's answer, but actually a &whole argument, where the the entire form can be bound to a variable. This is using the source of the program as the literal value that gets destructively modified! Now, in the paper, this is used in this example:
> ;this prints out the numbers from 0 to 9.
(do ((n 0 (gen-counter 1)))
((= n 10) t)
(princ n))
0.1.2.3.4.5.6.7.8.9.T
However, in Common Lisp, we'd expect the macro to be expanded just once. That is, we expect (gen-counter 1) to be replaced by some piece of code. We can still generate a piece of code like this, though:
(defmacro make-counter (&whole form initial-value)
(declare (ignore initial-value))
(let ((text (gensym (string 'text-))))
`(let ((,text ',form))
(incf (second ,text)))))
CL-USER> (macroexpand '(make-counter 3))
(LET ((#:TEXT-1002 '(MAKE-COUNTER 3)))
(INCF (SECOND #:TEXT-1002)))
Then we can recreate the example with do
CL-USER> (do ((n 0 (make-counter 1)))
((= n 10) t)
(princ n))
023456789
Of course, this is undefined behavior, since it's modifying literal data. It won't work in all Lisps (the run above is from CCL; it didn't work in SBCL).
But don't miss the point
The whole article is sort of interesting, but recognize that it's sort of a joke, too. It's pointing out that you can do some funny things in an evaluator that doesn't compile code. It's mostly satire that's pointing out the inconsistencies of Lisp systems that have different behaviors under evaluation and compilation. Note the last paragraph:
Some short-sighted individuals will point out that these programming
techniques, while certainly laudable for their increased clarity and
efficiency, would fail on compiled code. Sadly, this is true. At least
two of the above techniques will send most compilers into an infinite
loop. But it is already known that most lisp compilers do not
implement full lisp semantics -- dynamic scoping, for instance. This
is but another case of the compiler failing to preserve semantic
correctness. It remains the task of the compiler implementor to
adjust his system to correctly implement the source language, rather
than the user to resort to ugly, dangerous, non-portable, non-robust
``hacks'' in order to program around a buggy compiler.
I hope this provides some insight into the nature of clean, elegant
Lisp programming techniques.
—Olin Shivers
Common Lisp:
(defmacro gen-counter (&rest x)
(let ((ans (car x)))
(rplaca x (+ 1 ans))
ans))
But above only works in the Interpreter, not with a compiler.
With compiled code, the macro call is gone - it is expanded away - and there is nothing to modify.
Note to unsuspecting readers: you might want to read the paper by Olin Shivers very careful and try to find out what he actually means...
For the life of me, I can't understand continuations. I think the problem stems from the fact that I don't understand is what they are for. All the examples that I've found in books or online are very trivial. They make me wonder, why anyone would even want continuations?
Here's a typical impractical example, from TSPL, which I believe is quite recognized book on the subject. In english, they describe the continuation as "what to do" with the result of a computation. OK, that's sort of understandable.
Then, the second example given:
(call/cc
(lambda (k)
(* 5 (k 4)))) => 4
How does this make any sense?? k isn't even defined! How can this code be evaluated, when (k 4) can't even be computed? Not to mention, how does call/cc know to rip out the argument 4 to the inner most expression and return it? What happens to (* 5 .. ?? If this outermost expression is discarded, why even write it?
Then, a "less" trivial example stated is how to use call/cc to provide a nonlocal exit from a recursion. That sounds like flow control directive, ie like break/return in an imperative language, and not a computation.
And what is the purpose of going through these motions? If somebody needs the result of computation, why not just store it and recall later, as needed.
Forget about call/cc for a moment. Every expression/statement, in any programming language, has a continuation - which is, what you do with the result. In C, for example,
x = (1 + (2 * 3));
printf ("Done");
has the continuation of the math assignment being printf(...); the continuation of (2 * 3) is 'add 1; assign to x; printf(...)'. Conceptually the continuation is there whether or not you have access to it. Think for a moment what information you need for the continuation - the information is 1) the heap memory state (in general), 2) the stack, 3) any registers and 4) the program counter.
So continuations exist but usually they are only implicit and can't be accessed.
In Scheme, and a few other languages, you have access to the continuation. Essentially, behind your back, the compiler+runtime bundles up all the information needed for a continuation, stores it (generally in the heap) and gives you a handle to it. The handle you get is the function 'k' - if you call that function you will continue exactly after the call/cc point. Importantly, you can call that function multiple times and you will always continue after the call/cc point.
Let's look at some examples:
> (+ 2 (call/cc (lambda (cont) 3)))
5
In the above, the result of call/cc is the result of the lambda which is 3. The continuation wasn't invoked.
Now let's invoke the continuation:
> (+ 2 (call/cc (lambda (cont) (cont 10) 3)))
12
By invoking the continuation we skip anything after the invocation and continue right at the call/cc point. With (cont 10) the continuation returns 10 which is added to 2 for 12.
Now let's save the continuation.
> (define add-2 #f)
> (+ 2 (call/cc (lambda (cont) (set! add-2 cont) 3)))
5
> (add-2 10)
12
> (add-2 100)
102
By saving the continuation we can use it as we please to 'jump back to' whatever computation followed the call/cc point.
Often continuations are used for a non-local exit. Think of a function that is going to return a list unless there is some problem at which point '() will be returned.
(define (hairy-list-function list)
(call/cc
(lambda (cont)
;; process the list ...
(when (a-problem-arises? ...)
(cont '()))
;; continue processing the list ...
value-to-return)))
Here is text from my class notes: http://tmp.barzilay.org/cont.txt. It is based on a number of sources, and is much extended. It has motivations, basic explanations, more advanced explanations for how it's done, and a good number of examples that go from simple to advanced, and even some quick discussion of delimited continuations.
(I tried to play with putting the whole text here, but as I expected, 120k of text is not something that makes SO happy.
TL;DR: continuations are just captured GOTOs, with values, more or less.
The exampe you ask about,
(call/cc
(lambda (k)
;;;;;;;;;;;;;;;;
(* 5 (k 4)) ;; body of code
;;;;;;;;;;;;;;;;
)) => 4
can be approximately translated into e.g. Common Lisp, as
(prog (k retval)
(setq k (lambda (x) ;; capture the current continuation:
(setq retval x) ;; set! the return value
(go EXIT))) ;; and jump to exit point
(setq retval ;; get the value of the last expression,
(progn ;; as usual, in the
;;;;;;;;;;;;;;;;
(* 5 (funcall k 4)) ;; body of code
;;;;;;;;;;;;;;;;
))
EXIT ;; the goto label
(return retval))
This is just an illustration; in Common Lisp we can't jump back into the PROG tagbody after we've exited it the first time. But in Scheme, with real continuations, we can. If we set some global variable inside the body of function called by call/cc, say (setq qq k), in Scheme we can call it at any later time, from anywhere, re-entering into the same context (e.g. (qq 42)).
The point is, the body of call/cc form may contain an if or a condexpression. It can call the continuation only in some cases, and in others return normally, evaluating all expressions in the body of code and returning the last one's value, as usual. There can be deep recursion going on there. By calling the captured continuation an immediate exit is achieved.
So we see here that k is defined. It is defined by the call/cc call. When (call/cc g) is called, it calls its argument with the current continuation: (g the-current-continuation). the current-continuation is an "escape procedure" pointing at the return point of the call/cc form. To call it means to supply a value as if it were returned by the call/cc form itself.
So the above results in
((lambda(k) (* 5 (k 4))) the-current-continuation) ==>
(* 5 (the-current-continuation 4)) ==>
; to call the-current-continuation means to return the value from
; the call/cc form, so, jump to the return point, and return the value:
4
I won't try to explain all the places where continuations can be useful, but I hope that I can give brief examples of main place where I have found continuations useful in my own experience. Rather than speaking about Scheme's call/cc, I'd focus attention on continuation passing style. In some programming languages, variables can be dynamically scoped, and in languages without dynamically scoped, boilerplate with global variables (assuming that there are no issues of multi-threaded code, etc.) can be used. For instance, suppose there is a list of currently active logging streams, *logging-streams*, and that we want to call function in a dynamic environment where *logging-streams* is augmented with logging-stream-x. In Common Lisp we can do
(let ((*logging-streams* (cons logging-stream-x *logging-streams*)))
(function))
If we don't have dynamically scoped variables, as in Scheme, we can still do
(let ((old-streams *logging-streams*))
(set! *logging-streams* (cons logging-stream-x *logging-streams*)
(let ((result (function)))
(set! *logging-streams* old-streams)
result))
Now lets assume that we're actually given a cons-tree whose non-nil leaves are logging-streams, all of which should be in *logging-streams* when function is called. We've got two options:
We can flatten the tree, collect all the logging streams, extend *logging-streams*, and then call function.
We can, using continuation passing style, traverse the tree, gradually extending *logging-streams*, finally calling function when there is no more tree to traverse.
Option 2 looks something like
(defparameter *logging-streams* '())
(defun extend-streams (stream-tree continuation)
(cond
;; a null leaf
((null stream-tree)
(funcall continuation))
;; a non-null leaf
((atom stream-tree)
(let ((*logging-streams* (cons stream-tree *logging-streams*)))
(funcall continuation)))
;; a cons cell
(t
(extend-streams (car stream-tree)
#'(lambda ()
(extend-streams (cdr stream-tree)
continuation))))))
With this definition, we have
CL-USER> (extend-streams
'((a b) (c (d e)))
#'(lambda ()
(print *logging-streams*)))
=> (E D C B A)
Now, was there anything useful about this? In this case, probably not. Some minor benefits might be that extend-streams is tail-recursive, so we don't have a lot of stack usage, though the intermediate closures make up for it in heap space. We do have the fact that the eventual continuation is executed in the dynamic scope of any intermediate stuff that extend-streams set up. In this case, that's not all that important, but in other cases it can be.
Being able to abstract away some of the control flow, and to have non-local exits, or to be able to pick up a computation somewhere from a while back, can be very handy. This can be useful in backtracking search, for instance. Here's a continuation passing style propositional calculus solver for formulas where a formula is a symbol (a propositional literal), or a list of the form (not formula), (and left right), or (or left right).
(defun fail ()
'(() () fail))
(defun satisfy (formula
&optional
(positives '())
(negatives '())
(succeed #'(lambda (ps ns retry) `(,ps ,ns ,retry)))
(retry 'fail))
;; succeed is a function of three arguments: a list of positive literals,
;; a list of negative literals. retry is a function of zero
;; arguments, and is used to `try again` from the last place that a
;; choice was made.
(if (symbolp formula)
(if (member formula negatives)
(funcall retry)
(funcall succeed (adjoin formula positives) negatives retry))
(destructuring-bind (op left &optional right) formula
(case op
((not)
(satisfy left negatives positives
#'(lambda (negatives positives retry)
(funcall succeed positives negatives retry))
retry))
((and)
(satisfy left positives negatives
#'(lambda (positives negatives retry)
(satisfy right positives negatives succeed retry))
retry))
((or)
(satisfy left positives negatives
succeed
#'(lambda ()
(satisfy right positives negatives
succeed retry))))))))
If a satisfying assignment is found, then succeed is called with three arguments: the list of positive literals, the list of negative literals, and function that can retry the search (i.e., attempt to find another solution). For instance:
CL-USER> (satisfy '(and p (not p)))
(NIL NIL FAIL)
CL-USER> (satisfy '(or p q))
((P) NIL #<CLOSURE (LAMBDA #) {1002B99469}>)
CL-USER> (satisfy '(and (or p q) (and (not p) r)))
((R Q) (P) FAIL)
The second case is interesting, in that the third result is not FAIL, but some callable function that will try to find another solution. In this case, we can see that (or p q) is satisfiable by making either p or q true:
CL-USER> (destructuring-bind (ps ns retry) (satisfy '(or p q))
(declare (ignore ps ns))
(funcall retry))
((Q) NIL FAIL)
That would have been very difficult to do if we weren't using a continuation passing style where we can save the alternative flow and come back to it later. Using this, we can do some clever things, like collect all the satisfying assignments:
(defun satisfy-all (formula &aux (assignments '()) retry)
(setf retry #'(lambda ()
(satisfy formula '() '()
#'(lambda (ps ns new-retry)
(push (list ps ns) assignments)
(setf retry new-retry))
'fail)))
(loop while (not (eq retry 'fail))
do (funcall retry)
finally (return assignments)))
CL-USER> (satisfy-all '(or p (or (and q (not r)) (or r s))))
(((S) NIL) ; make S true
((R) NIL) ; make R true
((Q) (R)) ; make Q true and R false
((P) NIL)) ; make P true
We could change the loop a bit and get just n assignments, up to some n, or variations on that theme. Often times continuation passing style is not needed, or can make code hard to maintain and understand, but in the cases where it is useful, it can make some otherwise very difficult things fairly easy.