The reason I don't understand why Emacs 24's new lexical scoping features are that great is that I can't think of any new functionality that couldn't have been implemented without them. For example, the following closure:
(setq lexical-binding t)
(defun f1 (num1)
(lambda (num2)
(setq num1 (* num1 num2))))
(fset 'f2 (f1 5))
==> (closure ((num1 . 5) t) (num2) (setq num1 (* num1 num2)))
(f2 5)
==> 25
(f2 2)
==> 50
Can be implemented with regular dynamic scoping like so:
(defun f1 (num)
(let ((tmpvar (make-symbol "num")))
(set tmpvar num)
`(lambda (num2)
(set ',tmpvar (* (eval ,tmpvar) num2)))))
(fset 'f2 (f1 5))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f2 5)
==> 25
(f2 2)
==> 50
(fset 'f3 (f1 9))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f3 3)
==> 27
(f3 2)
==> 54
(f2 10)
==> 500
Okay, so not all languages have something analogous to elisp's uninterned symbols, so I understand why lexical scoping is so great in their case. But what about elisp? Can you think of anything that I can do now (as of Emacs 24) that I couldn't do before, thanks to lexical scoping?
You don't need uninterned symbols, use cons instead of make-symbol, car instead of eval, and setcar instead of set and your example will work just as well (and be more efficient).
Note also that the progression from machine-language to ever higher-level languages has been mostly based on making more and more things impossible (or at least much harder). Of course those facilities taken away from the programmers were rarely used and/or considered too dangerous. Think of using uninitialized variables (possible in C, but impossible in Java and many other languages), or jumping into the middle of an instruction.
As for some downside of your example code: not only it's less readable, but the compiler basically won't be able to know that you're constructing code, so it won't be allowed to look inside that "`(lambda ...)" to compile it, expand its macro calls, give you warnings about suspicious elements, ...
There have always been workarounds to simulate lexical binding in Emacs, so it's not so much about being able to do new things.
The manual says:
Lexical binding opens up a lot more opportunities for optimization, so
Emacs Lisp code that makes use of lexical binding is likely to run
faster in future Emacs versions. Such code is also much more friendly
to concurrency, which we want to add to Emacs in the near future.
I think that is the primary benefit.
The flip side to this is that dynamic binding was purposely chosen when Emacs was written for good reasons which still hold true today, and so lexical binding should certainly not be considered the New Way of doing things.
Global variables are generally considered a bad idea in programming, but Emacs is an unusual case where this doesn't really apply, because much of its immense flexibility -- one of the key things that makes Emacs great -- derives directly from dynamic binding. Without dynamic binding, it would not be possible to bend the application to the requirements of the individual user to anything like the extent that Emacs allows.
In my opinion, lexical binding should be used cautiously and only for variables for which another user could not conceivably find a reason to override. By default variables should be defvar'd so that the ability to customize the behaviour (even in ways that the author did not anticipate) is preserved.
I think implementations of OO programming tend to fit well with lexical scope. An object with state maps rather directly to a lexical closure.
I'm sure that the CLOS implementation in common lisp leverages lexical scope heavily. It's hard for me to imagine how that spec could be implemented with dynamic scope only, but I'm sure it's possible.
Related
AND and OR are macros and since macros aren't first class in scheme/racket they cannot be passed as arguments to other functions. A partial solution is to use and-map or or-map. Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function? Are there any languages that have first class macros?
In general, no. Consider that let is (or could be) implemented as a macro on top of lambda:
(let ((x 1))
(foo x))
could be a macro that expands to
((lambda (x) (foo x)) 1)
Now, what would it look like to convert let to a function? Clearly it is nonsense. What would its inputs be? Its return value?
Many macros will be like this. In fact, any macro that could be routinely turned into a function without losing any functionality is a bad macro! Such a macro should have been a function to begin with.
I agree with #amalloy. If something is written as a macro, it probably does something that functions can't do (e.g., introduce bindings, change evaluation order). So automatically converting arbitrary macro into a function is a really bad idea even if it is possible.
Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function?
No, but it is somewhat doable to write a macro that would take some macro and turn it into a function.
#lang racket
(require (for-syntax racket/list))
(define-syntax (->proc stx)
(syntax-case stx ()
[(_ mac #:arity arity)
(with-syntax ([(args ...) (generate-temporaries (range (syntax-e #'arity)))])
#'(λ (args ...) (mac args ...)))]))
((->proc and #:arity 2) 42 12)
(apply (->proc and #:arity 2) '(#f 12))
((->proc and #:arity 2) #f (error 'not-short-circuit))
You might also be interested in identifier macro, which allows us to use an identifier as a macro in some context and function in another context. This could be used to create a first class and/or which short-circuits when it's used as a macro, but could be passed as a function value in non-transformer position.
On the topic of first class macro, take a look at https://en.wikipedia.org/wiki/Fexpr. It's known to be a bad idea.
Not in the way you probably expect
To see why, here is a way of thinking about macros: A macro is a function which takes a bit of source code and turns it into another bit of source code: the expansion of the macro. In other words a macro is a function whose domain and range are source code.
Once the source code is fully expanded, then it's fed to either an evaluator or a compiler. Let's assume it's fed to a compiler because it makes the question easier to answer: a compiler itself is simply a function whose domain is source code and whose range is some sequence of instructions for a machine (which may or may not be a real machine) to execute. Those instructions might include things like 'call this function on these arguments'.
So, what you are asking is: can the 'this function' in 'call this function on these arguments' be some kind of macro? Well, yes, it could be, but whatever source code it is going to transform certainly can not be the source code of the program you are executing, because that is gone: all that's left is the sequence of instructions that was the return value of the compiler.
So you might say: OK, let's say we disallow compilers: can we do it now? Well, leaving aside that 'disallowing compilers' is kind of a serious limitation, this was, in fact, something that very old dialects of Lisp sort-of did, using a construct called a FEXPR, as mentioned in another answer. It's important to realise that FEXPRs existed because people had not yet invented macros. Pretty soon, people did invent macros, and although FEXPRs and macros coexisted for a while – mostly because people had written code which used FEXPRs which they wanted to keep running, and because writing macros was a serious pain before things like backquote existed – FEXPRs died out. And they died out because they were semantically horrible: even by the standards of 1960s Lisps they were semantically horrible.
Here's one small example of why FEXPRs are so horrible: Let's say I write this function in a language with FEXPRs:
(define (foo f g x)
(apply f (g x)))
Now: what happens when I call foo? In particular, what happens if f might be a FEXPR?. Well, the answer is that I can't compile foo at all: I have to wait until run-time and make some on-the-fly decision about what to do.
Of course this isn't what these old Lisps with FEXPRs probably did: they would just silently have assumed that f was a normal function (which they would have called an EXPR) and compiled accordingly (and yes, even very old Lisps had compilers). If you passed something which was a FEXPR you just lost: either the thing detected that, or more likely it fall over horribly or gave you some junk answer.
And this kind of horribleness is why macros were invented: macros provide a semantically sane approach to processing Lisp code which allows (eventually, this took a long time to actually happen) minor details like compilation being possible at all, code having reasonable semantics and compiled code having the same semantics as interpreted code. These are features people like in their languages, it turns out.
Incidentally, in both Racket and Common Lisp, macros are explicitly functions. In Racket they are functions which operate on special 'syntax' objects because that's how you get hygiene, but in Common Lisp, which is much less hygienic, they're just functions which operate on CL source code, where the source code is simply made up of lists, symbols &c.
Here's an example of this in Racket:
> (define foo (syntax-rules ()
[(_ x) x]))
> foo
#<procedure:foo>
OK, foo is now just an ordinary function. But it's a function whose domain & range are Racket source code: it expects a syntax object as an argument and returns another one:
> (foo 1)
; ?: bad syntax
; in: 1
; [,bt for context]
This is because 1 is not a syntax object.
> (foo #'(x 1))
#<syntax:readline-input:5:10 1>
> (syntax-e (foo #'(x 1)))
1
And in CL this is even easier to see: Here's a macro definition:
(defmacro foo (form) form)
And now I can get hold of the macro's function and call it on some CL source code:
> (macro-function 'foo)
#<Function foo 4060000B6C>
> (funcall (macro-function 'foo) '(x 1) nil)
1
In both Racket and CL, macros are, in fact, first-class (or, in the case of Racket: almost first-class, I think): they are functions which operate on source code, which itself is first-class: you can write Racket and CL programs which construct and manipulate source code in arbitrary ways: that's what macros are in these languages.
In the case of Racket I have said 'almost first-class', because I can't see a way, in Racket, to retrieve the function which sits behind a macro defined with define-syntax &c.
I've created something like this in Scheme, it's macro that return lambda that use eval to execute the macro:
(define-macro (macron m)
(let ((x (gensym)))
`(lambda (,x)
(eval `(,',m ,#,x)))))
Example usage:
;; normal eval
(define x (map (lambda (x)
(eval `(lambda ,#x)))
'(((x) (display x)) ((y) (+ y y)))))
;; using macron macro
(define x (map (macron lambda)
'(((x) (display x)) ((y) (+ y y)))))
and x in both cases is list of two functions.
another example:
(define-macro (+++ . args)
`(+ ,#args))
((macron +++) '(1 2 3))
This question already has answers here:
setq and defvar in Lisp
(4 answers)
Closed 6 years ago.
I was following a tutorial on lisp and they did the following code
(set 'x 11)
(incf x 10)
and interpreter gave the following error:
; in: INCF X
; (SETQ X #:NEW671)
;
; caught WARNING:
; undefined variable: X
;
; compilation unit finished
; Undefined variable:
; X
; caught 1 WARNING condition
21
what is the proper way to increment x ?
This is indeed how you are meant to increment x, or at least one way of doing so. However it is not how you are meant to bind x. In CL you need to establish a binding for a name before you use it, and you don't do that by just assigning to it. So, for instance, this code (in a fresh CL image) is not legal CL:
(defun bad ()
(setf y 2))
Typically this will cause a compile-time warning and a run-time error, although it may do something else: its behaviour is not defined.
What you have done, in particular, is actually worse than this: you have rammed a value into the symbol-value of x (with set, which does this), and then assumed that something like (incf x) will work, which it is extremely unlikely to do. For instance consider something like this:
(defun worse ()
(let ((x 2))
(set 'x 4)
(incf x)
(values x (symbol-value 'x))))
This is (unlike bad) legal code, but it probably does not do what you want it to do.
Many CL implementations do allow assignment to previously unbound variables at the top-level, because in a conversational environment it is convenient. But the exact meaning of such assignments is outwith the language standard.
CMUCL and its derivatives, including SBCL, have historically been rather more serious about this than other implementations were at the time. I think the reason for this is that the interpreter was a bunch more serious than most others and/or they secretly compiled everything anyway and the compiler picked things up.
A further problem is that CL has slightly awkward semantics for top-level variables: if you go to the effort to establish a toplevel binding in the normal way, with defvar & friends, then you also cause the variable to be special -- dynamically scoped -- and this is a pervasive effect: it makes all bindings of that name special. That is often a quite undesirable consequence. CL, as a language, has no notion of a top-level lexical variable.
What many implementations did, therefore, was to have some kind of informal notion of a top-level binding of something which did not imply a special declaration: if you just said (setf x 3) at the toplevel then this would not contage the entire environment. But then there were all sorts of awkward questions: after doing that, what is the result of (symbol-value 'x) for instance?
Fortunately CL is a powerful language, and it is quite possible to define top-level lexical variables within the language. Here is a very hacky implementation called deflexical. Note that there are better implementations out there (including at least one by me, which I can't find right now): this is not meant to be a bullet-proof solution.
(defmacro deflexical (var &optional value)
;; Define a cheap-and-nasty global lexical variable. In this
;; implementation, global lexicals are not boundp and the global
;; lexical value is not stored in the symbol-value of the symbol.
;;
;; This implementation is *not* properly thought-through and is
;; without question problematic
`(progn
(define-symbol-macro ,var (get ',var 'lexical-value))
(let ((flag (cons nil nil)))
;; assign a value only if there is not one already, like DEFVAR
(when (eq (get ',var 'lexical-value flag) flag)
(setf (get ',var 'lexical-value) ,value))
;; Return the symbol
',var)))
I'm currently reading the book Land of LISP, and I'm just working through the first chapter. In there, there is a little program written where the computer guesses numbers between 1 and 100. Its code is as follows:
(defparameter *small* 1)
(defparameter *big* 100)
(defun guess-my-number ()
(ash (+ *small* *big*) -1))
(defun smaller ()
(setf *big* (1- (guess-my-number)))
(guess-my-number))
(defun bigger ()
(setf *small* (1+ (guess-my-number)))
(guess-my-number))
(defun start-over ()
(defparameter *small* 1)
(defparameter *big* 100)
(guess-my-number))
So far, I understand what happens, and Using 'ash' in LISP to perform a binary search? helped me a lot in this. Nevertheless there's one thing left that puzzles me: As far as I have learned, you use setf to assign values to variables, and defparameter to initially define variables. I also have understood the difference between defparameter and defvar(at least I believe I do ;-)).
So now my question is: If I should use setf to assign a value to a variable once it had been initialized, why does the start-over function use defparameter and not setf? Is there a special reason for this, or is this just sloppiness?
The function is just:
(defun start-over ()
(setf *small* 1)
(setf *big* 100)
(guess-my-number))
It is already declared to be a special global variable. No need to do it inside the function again and again.
You CAN use DEFPARAMETER inside a function, but it is bad style.
DEFPARAMETER is for declaring global special variables and optional documentation for them. Once. If you need to do it several times, it's mostly done when a whole file or system gets reloaded. The file compiler also recognizes it in top-level position as a special declaration for a dynamically bound variable.
Example:
File 1:
(defparameter *foo* 10)
(defun foo ()
(let ((*foo* ...))
...))
File 2:
(defun foo-start ()
(defparameter *foo* 10))
(defun foo ()
(let ((*foo* ...))
...))
If Lisp compiles File 1 fresh with compile-file, the compiler recognizes the defparameter and in the following let we have a dynamic binding.
If Lisp compiles File 2 fresh with compile-file, the compiler doesn't recognize the defparameter and in the following let we have a lexical binding. If we compile it again, from this state, we have a dynamic binding.
So here version 1 is better, because it is easier to control and understand.
In your example DEFPARAMETER appears multiple times, which is not useful. I might ask, where is the variable defined and the answer would point to multiple source locations...
So: make sure that your program elements get mostly defined ONCE - unless you have a good reason not to do so.
So you have global variables. Those can be defined by defconstant (for really non-chainging stuff), defparameter (a constant that you can change) and defvar (a variable that does not overwrite if you load.
You use setf to alter the state of lexical as well and global variables. start-over could have used setf since the code doesn't really define it but change it. If you would have replaced defparameter with defvar start-over would stop working.
(defparameter *par* 5)
(setf *par* 6)
(defparameter *par* 5)
*par* ; ==> 5
(defvar *var* 5)
(setf *var* 6)
(defvar *var* 5)
*var* ; ==> 6
defconstant is like defparameter except once defined the CL implementation is free to inline it in code. Thus if you redefine a constant you need to redefine all functions that uses it or else it might use the old value.
(defconstant +c+ 5)
(defun test (x)
(+ x +c+))
(test 1) ; ==> 6
(defconstant +c+ 6)
(test 1) ; ==> 6 or 7
(defun test (x)
(+ x +c+))
(test 1) ; ==> 7
Normally, one would use defvar to initialy define global variables. The difference between defvar and defparameter is subtle, cf. the section in the CLHS and this plays a role here: defparameter (in contrast to defvar) assigns the value anew, whereas defvar would leave the old binding in place.
To address what to use: In general, defvar and friends are used as top-level forms, not inside some function (closures being the most notable exception in the context of defun). I would use setf, not defparameter.
For a beginner symbols, variables, etc. can be a bit surprising. Symbols are surprisingly featureful. Just to mention a few things you can ask a symbol for it's symbol-value, symbol-package, symbol-name, symbol-function etc. In addition symbols can have a varying amount of information declared about them, for example a type, that provides advice the compile might leverage to create better code. This is true of all symbols, for example *, the symbol you use for multiplication has a symbol-function that does that multiplication. It also has a symbol-value, i.e. the last value returned in the current REPL.
One critical bit of declarative information about symbols is if they are "special." (Yeah, it's a dumb name.) For good reasons it's good practice to declare all global symbols to be special. defvar, defparameter, and defconstant do that for you. We have a convention that all special variables are spelled with * on the front and back, *standard-output* for example. This convention is so common that some compilers will warn you if you neglect to follow it.
One benefit of declaring a symbol as special is that it will suppress the warning you get when you misspell a variable in your functions. For example (defun faster (how-much) (incf *speed* hw-much)) will generate a warning about hw-much.
The coolest feature of a symbol that is special is that it is managed with what we call dynamic scoping, in contrast to lexical scope. Recall how * has the value of the last result in the REPL. Well in some implementations you can have multiple REPLs (each running in it's own thread) each will want to have it's own *, it's own *standard-output*, etc. etc. This is easy in Common Lisp; the threads establish a "dynamic extent" and bind the specials that should be local to that REPL.
So yes, you could just setf *small* in you example; but if you never declare it to be special then you are going to get warnings about how the compiler thinks you misspelled it.
This question already has answers here:
What can you do with Lisp macros that you can't do with first-class functions?
(8 answers)
Closed 5 years ago.
In my quest to fully understand the so powerful lisp macros a question came to my mind. I know that a golden rule about macros is the one saying "Never use a macro when a function will do the work".
However reading Chapter 9 - Practical: Building a Unit Test Framework - from the book Practical Common Lisp I was introduced to the below macro whose purpose was to get rid of the duplication of the test case expression, with its attendant risk of mislabeling of results.
;; Function defintion.
(defun report-result (result form)
(format t "~:[FAIL~;pass~] ... ~a~%" result form))
;; Macro Definition
(defmacro check (form)
`(report-result ,form ',form))
OK, I understand its purpose but I could have done it using a function instead of a macro, for instance:
(setf unevaluated.form '(= 2 (+ 2 3)))
(defun my-func (unevaluated.form)
(report-result (eval unevaluated.form) unevaluated.form))
Is this only possible because the given macro is too simple ?
Furthermore, is Lisp Macro System so powerful relatively its opponents due to the code itself - like control structures, functions, etc - is represented as a LIST ?
But if it were a macro you, could have done:
(check (= 2 (+ 2 3)))
With a function, you have to do:
(check '(= 2 (+ 2 3)))
Also, with the macro the (= 2 (+ 2 3)) is actually compiled by the compiler, whereas with the function it's evaluated by the eval function, not necessarily the same thing.
Addenda:
Yes, it's just evaluating the function. Now what that means is dependent upon the implementation. Some can interpret it, others can compile and execute it. But the simple matter is that you don't know from system to system.
The null lexical environment that others are mentioning is also a big deal.
Consider:
(defun add3f (form)
(eval `(+ 3 ,form)))
(demacro add3m (form)
`(+ 3 ,form))
Then observe:
[28]> (add3m (+ 2 3))
8
[29]> (add3f '(+ 2 3))
8
[30]> (let ((x 2)) (add3m (+ x 3)))
8
[31]> (let ((x 2)) (add3f '(+ x 3)))
*** - EVAL: variable X has no value
The following restarts are available:
USE-VALUE :R1 Input a value to be used instead of X.
STORE-VALUE :R2 Input a new value for X.
ABORT :R3 Abort main loop
Break 1 [32]> :a
That's really quite damning for most use cases. Since the eval has no lexical environment, it can not "see" the x from the enclosing let.
The better substitution would be not with eval, which won't perform as expected for all cases (for example, it doesn't have access to the lexical environment), and is also overkill (see here: https://stackoverflow.com/a/2571549/977052), but something using anonymous functions, like this:
(defun check (fn)
(report-result (funcall fn) (function-body fn)))
CL-USER> (check (lambda () (= 2 (+ 2 3))))
By the way, this is how such things are accomplished in Ruby (anonymous functions are called procs there).
But, as you see, it becomes somewhat less elegant (unless you add syntax sugar) and, there's actually a bigger problem: ther's no function-body function in Lisp (although there may be non-standard ways to get at it). Overall, as you see, for this particular task the alternative solutions are substantially worse, although in some cases such approach could work.
In general, though, if you want to do something with the source code of the expressions passed into the macro (and usually this is the primary reason of using macros), functions would not be sufficient.
The report-result function needs both the source code and the result of the execution.
The macro CHECK provides both from a single source form.
If you put a bunch of check forms into the file, they are easily compiled using the usual process of compiling Lisp files. You'll get a compiled version of the checking code.
Using a function and EVAL (better use COMPILE) you would have deferred the source evaluation to a later time. It would also not be clear if it is interpreted or compiled. In case of compilation, you would then later get the compiler's checks.
Also, even if I can use Common Lisp, should I? Is Scheme better?
You have several answers here, but none is really comprehensive (and I'm not talking about having enough details or being long enough). First of all, the bottom line: you should not use Common Lisp if you want to have a good experience with SICP.
If you don't know much Common Lisp, then just take it as that. (Obviously you can disregard this advice as anything else, some people only learn the hard way.)
If you already know Common Lisp, then you might pull it off, but at considerable effort, and at a considerable damage to your overall learning experience. There are some fundamental issues that separate Common Lisp and Scheme, which make trying to use the former with SICP a pretty bad idea. In fact, if you have the knowledge level to make it work, then you're likely above the level of SICP anyway. I'm not saying that it's not possible -- it is of course possible to implement the whole book in Common Lisp (for example, see Bendersky's pages) just as you can do so in C or Perl or whatever. It's just going to harder with languages that are further apart from Scheme. (For example, ML is likely to be easier to use than Common Lisp, even when its syntax is very different.)
Here are some of these major issues, in increasing order of importance. (I'm not saying that this list is exhaustive in any way, I'm sure that there are a whole bunch of additional issues that I'm omitting here.)
NIL and related issues, and different names.
Dynamic scope.
Tail call optimization.
Separate namespace for functions and values.
I'll expand now on each of these points:
The first point is the most technical. In Common Lisp, NIL is used both as the empty list and as the false value. In itself, this is not a big issue, and in fact the first edition of SICP had a similar assumption -- where the empty list and false were the same value. However, Common Lisp's NIL is still different: it is also a symbol. So, in Scheme you have a clear separation: something is either a list, or one of the primitive types of values -- but in Common Lisp, NIL is not only false and the empty list: it is also a symbol. In addition to this, you get a host of slightly different behavior -- for example, in Common Lisp the head and the tail (the car and cdr) of the empty list is itself the empty list, while in Scheme you'll get a runtime error if you try that. To top it off, you have different names and naming convention, for example -- predicates in Common Lisp end by convention with P (eg, listp) while predicates in Scheme end in a question mark (eg, list?); mutators in Common Lisp have no specific convention (some have an N prefix), while in Scheme they almost always have a suffix of !. Also, plain assignment in Common Lisp is usually setf and it can operate on combinations too (eg, (setf (car foo) 1)), while in Scheme it is set! and limited to setting bound variables only. (Note that Common Lisp has the limited version too, it's called setq. Almost nobody uses it though.)
The second point is a much deeper one, and possibly one that will lead to completely incomprehensible behavior of your code. The thing is that in Common Lisp, function arguments are lexically scoped, but variables that are declared with defvar are dynamically scoped. There is a whole range of solutions that rely on lexically scoped bindings -- and in Common Lisp they just won't work. Of course, the fact that Common Lisp has lexical scope means that you can get around this by being very careful about new bindings, and possibly using macros to get around the default dynamic scope -- but again, this requires a much more extensive knowledge than a typical newbie has. Things get even worse than that: if you declare a specific name with a defvar, then that name will be bound dynamically even if they're arguments to functions. This can lead to some extremely difficult to track bugs which manifest themselves in an extremely confusing way (you basically get the wrong value, and you'll have no clue why that happens). Experienced Common Lispers know about it (especially those that have been burnt by it), and will always follow the convention of using stars around dynamically scoped names (eg, *foo*). (And by the way, in Common Lisp jargon, these dynamically scoped variables are called just "special variables" -- which is another source of confusion for newbies.)
The third point was also discussed in some of the previous comments. In fact, Rainer had a pretty good summary of the different options that you have, but he didn't explain just how hard it can make things. The thing is that proper tail-call-optimization (TCO) is one of the fundamental concepts in Scheme. It is important enough that it is a language feature rather than merely an optimization. A typical loop in Scheme is expressed as a tail-calling function (for example, (define (loop) (loop))) and proper Scheme implementations are required to implement TCO which will guarantee that this is, in fact, an infinite loop rather than running for a short while until you blow up the stack space. This is all the essence of Rainer's first non solution, and the reason he labeled it as "BAD".
His third option -- rewriting functional loops (expressed as recursive functions) as Common Lisp loops (dotimes, dolist, and the infamous loop) can work for a few simple cases, but at a very high cost: the fact that Scheme is a language that does proper TCO is not only fundamental to the language -- it is also one of the major themes in the book, so by doing so, you will have lost that point completely. In addition, there are some cases that you just cannot translate Scheme code into a Common Lisp loop construct -- for example, as you work your way through the book, you'll get to implement a meta-circular-interpreter which is an implementation of a mini-Scheme language. It takes a certain click to realize that this meta evaluator implements a language that is itself doing TCO if the language that you implement this evaluator in is itself doing TCO. (Note that I'm talking about the "simple" interpreters -- later in the book you implement this evaluator as something close to a register machine, where you kind of explicitly make it do TCO.) The bottom line to all of this, is that this evaluator -- when implemented in Common Lisp -- will result in a language that is itself not doing TCO. People who are familiar with all of this should not be surprised: after all, the "circularity" of the evaluator means that you're implementing a language with semantics that are very close to the host language -- so in this case you "inherit" the Common Lisp semantics rather than the Scheme TCO semantics. However, this means that your mini-evaluator is now crippled: it has no TCO, so it has no way of doing loops! To get loops in, you will need to implement new constructs in your interpreter, which will usually use the iteration constructs in Common Lisp. But now you're going further away from what's in the book, and you're investing considerable effort in approximately implementing the ideas in SICP to the different language. Note also that all of this is related to the previous point I raised: if you follow the book, then the language that you implement will be lexically scoped, taking it further away from the Common Lisp host language. So overall, you completely lose the "circular" property in what the book calls "meta circular evaluator". (Again, this is something that might not bother you, but it will damage the overall learning experience.) All in all, very few languages get close to Scheme in being able to implement the semantics of the language inside the language as a non-trivial (eg, not using eval) evaluator that easily.
In fact, if you do go with a Common Lisp, then in my opinion, Rainer's second suggestion -- use a Common Lisp implementation that supports TCO -- is the best way to go. However, in Common Lisp this is fundamentally a compiler optimization: so you will likely need to (a) know about the knobs in the implementation that you need to turn to make TCO happen, (b) you will need to make sure that the Common Lisp implementation is actually doing proper TCO, and not just optimization of self calls (which is the much simpler case that is not nearly as important), (c) you would hope that the Common Lisp implementation that does TCO can do so without damaging debugging options (again, since this is considered an optimization in Common Lisp, then turning this knob on, might also be taken by the compiler as saying "I don't care much for debuggability").
Finally, my last point is not too hard to overcome, but it is conceptually the most important one. In Scheme, you have a uniform rule: identifiers have a value, which is determined lexically -- and that's it. It's a very simple language. In Common Lisp, in addition to the historical baggage of sometimes using dynamic scope and sometimes using lexical scope, you have symbols that have two different value -- there's the function value that is used whenever a variable appears at the head of an expression, and there is a different value that is used otherwise. For example, in (foo foo), each of the two instances of foo are interpreted differently -- the first is the function value of foo and the second is its variable value. Again, this is not hard to overcome -- there are a number of constructs that you need to know about to deal with all of this. For example, instead of writing (lambda (x) (x x)) you need to write (lambda (x) (funcall x x)), which makes the function that is being called appear in a variable position, therefore the same value will be used there; another example is (map car something) which you will need to translate to (map #'car something) (or more accurately, you will need to use mapcar which is Common Lisp's equivalent of the car function); yet another thing that you'll need to know is that let binds the value slot of the name, and labels binds the function slot (and has a very different syntax, just like defun and defvar.)
But the conceptual result of all of this is that Common Lispers tend to use higher-order code much less than Schemers, and that goes all the way from the idioms that are common in each language, to what implementations will do with it. (For example, many Common Lisp compilers will never optimize this call: (funcall foo bar), while Scheme compilers will optimize (foo bar) like any function call expression, because there is no other way to call functions.)
Finally, I'll note that much of the above is very good flamewar material: throw any of these issues into a public Lisp or Scheme forum (in particular comp.lang.lisp and comp.lang.scheme), and you'll most likely see a long thread where people explain why their choice is far better than the other, or why some "so called feature" is actually an idiotic decision that was made by language designers that were clearly very drunk at the time, etc etc. But the thing is that these are just differences between the two languages, and eventually people can get their job done in either one. It just happens that if the job is "doing SICP" then Scheme will be much easier considering how it hits each of these issues from the Scheme perspective. If you want to learn Common Lisp, then going with a Common Lisp textbook will leave you much less frustrated.
Using SICP with Common Lisp is possible and fun
You can use Common Lisp for learning with SICP without much problems. The Scheme subset that is used in the book is not very sophisticated. SICP does not use macros and it uses no continuations. There are DELAY and FORCE, which can be written in Common Lisp in a few lines.
Also for a beginner using (function foo) and (funcall foo 1 2 3) is actually better (IMHO !), because the code gets clearer when learning the functional programming parts. You can see where variables and lambda functions are being called/passed.
Tail call optimization in Common Lisp
There is only one big area where using Common Lisp has a drawback: tail call optimization (TCO). Common Lisp does not support TCO in its standard (because of unclear interaction with the rest of the language, not all computer architectures support it directly (think JVM), not all compilers support it (some Lisp Machine), it makes some debugging/tracing/stepping harder, ...).
There are three ways to live with that:
Hope that the stack does not blow out. BAD.
Use a Common Lisp implementation that supports TCO. There are some. See below.
Rewrite the functional loops (and similar constructs) into loops (and similar constructs) using DOTIMES, DO, LOOP, ...
Personally I would recommend 2 or 3.
Common Lisp has excellent and easy to use compilers with TCO support (SBCL, LispWorks, Allegro CL, Clozure CL, ...) and as a development environment use either the built-in ones or GNU Emacs/SLIME.
For use with SICP I would recommend SBCL, since it compiles always by default, has TCO support by default and the compiler catches a lot of coding problems (undeclared variables, wrong argument lists, a bunch of type errors, ...). This helps a lot during learning. Generally make sure the code is compiled, since Common Lisp interpreters will usually not support TCO.
Sometimes it might also helpful to write one or two macros and provide some Scheme function names to make code look a bit more like Scheme. For example you could have a DEFINE macro in Common Lisp.
For the more advanced users, there is an old Scheme implementation written in Common Lisp (called Pseudo Scheme), that should run most of the code in SICP.
My recommendation: if you want to go the extra mile and use Common Lisp, do it.
To make it easier to understand the necessary changes, I've added a few examples - remember, it needs a Common Lisp compiler with support for tail call optimization:
Example
Let's look at this simple code from SICP:
(define (factorial n)
(fact-iter 1 1 n))
(define (fact-iter product counter max-count)
(if (> counter max-count)
product
(fact-iter (* counter product)
(+ counter 1)
max-count)))
We can use it directly in Common Lisp with a DEFINE macro:
(defmacro define ((name &rest args) &body body)
`(defun ,name ,args ,#body))
Now you should use SBCL, CCL, Allegro CL or LispWorks. These compilers support TCO by default.
Let's use SBCL:
* (define (factorial n)
(fact-iter 1 1 n))
; in: DEFINE (FACTORIAL N)
; (FACT-ITER 1 1 N)
;
; caught STYLE-WARNING:
; undefined function: FACT-ITER
;
; compilation unit finished
; Undefined function:
; FACT-ITER
; caught 1 STYLE-WARNING condition
FACTORIAL
* (define (fact-iter product counter max-count)
(if (> counter max-count)
product
(fact-iter (* counter product)
(+ counter 1)
max-count)))
FACT-ITER
* (factorial 1000)
40238726007709....
Another Example: symbolic differentiation
SICP has a Scheme example for differentiation:
(define (deriv exp var)
(cond ((number? exp) 0)
((variable? exp)
(if (same-variable? exp var) 1 0))
((sum? exp)
(make-sum (deriv (addend exp) var)
(deriv (augend exp) var)))
((product? exp)
(make-sum
(make-product (multiplier exp)
(deriv (multiplicand exp) var))
(make-product (deriv (multiplier exp) var)
(multiplicand exp))))
(else
(error "unknown expression type -- DERIV" exp))))
Making this code run in Common Lisp is easy:
some functions have different names, number? is numberp in CL
CL:COND uses T instead of else
CL:ERROR uses CL format strings
Let's define Scheme names for some functions. Common Lisp code:
(loop for (scheme-symbol fn) in
'((number? numberp)
(symbol? symbolp)
(pair? consp)
(eq? eq)
(display-line print))
do (setf (symbol-function scheme-symbol)
(symbol-function fn)))
Our define macro from above:
(defmacro define ((name &rest args) &body body)
`(defun ,name ,args ,#body))
The Common Lisp code:
(define (variable? x) (symbol? x))
(define (same-variable? v1 v2)
(and (variable? v1) (variable? v2) (eq? v1 v2)))
(define (make-sum a1 a2) (list '+ a1 a2))
(define (make-product m1 m2) (list '* m1 m2))
(define (sum? x)
(and (pair? x) (eq? (car x) '+)))
(define (addend s) (cadr s))
(define (augend s) (caddr s))
(define (product? x)
(and (pair? x) (eq? (car x) '*)))
(define (multiplier p) (cadr p))
(define (multiplicand p) (caddr p))
(define (deriv exp var)
(cond ((number? exp) 0)
((variable? exp)
(if (same-variable? exp var) 1 0))
((sum? exp)
(make-sum (deriv (addend exp) var)
(deriv (augend exp) var)))
((product? exp)
(make-sum
(make-product (multiplier exp)
(deriv (multiplicand exp) var))
(make-product (deriv (multiplier exp) var)
(multiplicand exp))))
(t
(error "unknown expression type -- DERIV: ~a" exp))))
Let's try it in LispWorks:
CL-USER 19 > (deriv '(* (* x y) (+ x 3)) 'x)
(+ (* (* X Y) (+ 1 0)) (* (+ (* X 0) (* 1 Y)) (+ X 3)))
Streams example from SICP in Common Lisp
See the book code in chapter 3.5 in SICP. We use the additions to CL from above.
SICP mentions delay, the-empty-stream and cons-stream, but does not implement it. We provide here an implementation in Common Lisp:
(defmacro delay (expression)
`(lambda () ,expression))
(defmacro cons-stream (a b)
`(cons ,a (delay ,b)))
(define (force delayed-object)
(funcall delayed-object))
(defparameter the-empty-stream (make-symbol "THE-EMPTY-STREAM"))
Now comes portable code from the book:
(define (stream-null? stream)
(eq? stream the-empty-stream))
(define (stream-car stream) (car stream))
(define (stream-cdr stream) (force (cdr stream)))
(define (stream-enumerate-interval low high)
(if (> low high)
the-empty-stream
(cons-stream
low
(stream-enumerate-interval (+ low 1) high))))
Now Common Lisp differs in stream-for-each:
we need to use cl:progn instead of begin
function parameters need to be called with cl:funcall
Here is a version:
(defmacro begin (&body body) `(progn ,#body))
(define (stream-for-each proc s)
(if (stream-null? s)
'done
(begin (funcall proc (stream-car s))
(stream-for-each proc (stream-cdr s)))))
We also need to pass functions using cl:function:
(define (display-stream s)
(stream-for-each (function display-line) s))
But then the example works:
CL-USER 20 > (stream-enumerate-interval 10 20)
(10 . #<Closure 1 subfunction of STREAM-ENUMERATE-INTERVAL 40600010FC>)
CL-USER 21 > (display-stream (stream-enumerate-interval 10 1000))
10
11
12
...
997
998
999
1000
DONE
Do you already know some Common Lisp? I assume that is what you mean by 'Lisp'. In that case you might want to use it instead of Scheme. If you don't know either, and you are working through SICP solely for the learning experience, then probably you are better off with Scheme. It has much better support for new learners, and you won't have to translate from Scheme to Common Lisp.
There are differences; specifically, SICP's highly functional style is wordier in Common Lisp because you have to quote functions when passing them around and use funcall to call a function bound to a variable.
However, if you want to use Common Lisp, you can try using Eli Bendersky's Common Lisp translations of the SICP code under the tag SICP.
They are similar but not the same.
I believe If you go with Scheme it would be easier.
Edit: Nathan Sanders' comment is correct. It's clearly been a while since I last read the book, but I just checked and it does not use call/cc directly. I've upvoted Nathan's answer.
Whatever you use needs to implement continuations, which SICP uses a lot. Not even all Scheme interpreters implement them, and I'm not aware of any Common Lisp that does.