If one types (+ 2 3), lisp returns 5 but how could one tell lisp compiler to return it unevaluated(returning (+ 2 3)) ? Is there some flag that could turn off automatic application of operator on operands ? in read-eval-print, eval should be suspended.
Since this has to be done as a function that takes in expression and returns it unevaluated, how do I quote it inside function ?
For common lisp, there is no way receive the "quoted" original expression as argument to a plain function. The evaluation rules are fixed. You can do, what you want, with a macro, however
(defmacro my-quote (expr) (list 'quote expr))
==> MY-QUOTE
(my-quote (+ 2 3))
==> (+ 2 3)
FWIW in Racket (I know it isn't Common Lisp) an application like (+ 2 3) is expanded to (#%app + 2 3). If you rebind #%app to your own macro, you can change the result of evaluating (+ 2 3).
A complete example:
#lang racket
(module my-app racket
(define-syntax (my-app stx)
(syntax-case stx ()
[(my-app procedure argument ...)
#''(procedure argument ...)]))
(provide (rename-out [my-app #%app])))
(require (submod "." my-app))
(+ 2 x)
The result of running this program is '(+ 2 x)
Related
I am using syntax transformers to define macros in Racket. I want to create some helper functions to help me manipulate the syntax. However, the functions I defined outside the syntax transformer are not available inside the syntax transformer. For example, in the following code
(define (my-function x) (+ x 1))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
I got the error "my-function: reference to an unbound identifier at phase: 1; the transformer environment".
After some searching, I am able to write the following code so that my-function is available inside the syntax transformer.
(begin-for-syntax
(define (my-function x) (+ x 1)))
(provide (for-syntax my-function))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
But the problem is, my-function is not available outside the syntax transformer this time. Sometimes I want to check those helper functions in ordinary code, so I need to be able to call it from both inside and outside the syntax transformer, just like the function cadr. How can I achieve that?
I know my question has something to do with Racket's syntax model, in particular the concept of "phase level", but I never really understand it. If you could provide some easy-to-follow tutorials explaining it I would even be more grateful.
A common way is to define your function that you want to share across phases in another (sub)module. Then, require it twice.
#lang racket
(module common racket
(provide my-function)
(define (my-function x) (+ x 1)))
(require 'common
(for-syntax 'common))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
(my-function 1)
(my-macro 123)
I am hoping someone can explain why tests 1-5 work but test 6 does not. I thought that quoting a lambda with ' and using #' in front of a lambda both returned pointers to the function with the only difference being that the #' will compile it first.
(defun test-1 (y)
(mapcar (lambda (x) (expt x 2))
'(1 2 3)))
(defun test-2 (y)
(mapcar (lambda (x) (expt x y))
'(1 2 3)))
(defun test-3 (y)
(mapcar #'(lambda (x) (expt x 2))
'(1 2 3)))
(defun test-4 (y)
(mapcar #'(lambda (x) (expt x y))
'(1 2 3)))
(defun test-5 (y)
(mapcar '(lambda (x) (expt x 2))
'(1 2 3)))
(defun test-6 (y)
(mapcar '(lambda (x) (expt x y))
'(1 2 3)))
I am using the free version of Franz Industries Allegro Common Lisp. The following are the outputs:
(test-1 2) ; --> (1 4 9)
(test-2 2) ; --> (1 4 9)
(test-3 2) ; --> (1 4 9)
(test-4 2) ; --> (1 4 9)
(test-5 2) ; --> (1 4 9)
(test-6 2) ; --> Error: Attempt to take the value of the unbound variable `Y'. [condition type: UNBOUND-VARIABLE]
For a start, you should be aware that your tests 1-4 are conforming Common Lisp, while your tests 5 and 6 are not. I believe Allegro is perfectly well allowed to do what it does for 5 and 6, but what it is doing is outside the standard. The bit of the standard that talks about this is the definition of functions like mapcar, which take function designators as argument, and the definition of a function designator:
function designator n. a designator for a function; that is, an object that denotes a function and that is one of: a symbol (denoting the function named by that symbol in the global environment), or a function (denoting itself). The consequences are undefined if a symbol is used as a function designator but it does not have a global definition as a function, or it has a global definition as a macro or a special form. [...]
From this it is clear that a list like (lambda (...) ...) is not a function designator: it's just a list whose car happens to be lambda. What Allegro is doing is noticing that this list is in fact something that can be turned into a function and doing that.
Well, let's just write a version of mapcar which does what Allegro's does:
(defun mapcar/coercing (maybe-f &rest lists)
(apply #'mapcar (coerce maybe-f 'function) lists))
This just uses coerce which is a function which knows how to turn lists like this into functions, among other things. If its argument is already a function, coerce just returns it.
Now we can write the two tests using this function:
(defun test-5/coercing (y)
(mapcar/coercing '(lambda (x) (expt x 2))
'(1 2 3)))
(defun test-6/coercing (y)
(mapcar/coercing '(lambda (x) (expt x y))
'(1 2 3)))
So, after that preamble, why can't test-6/explicit work? Well the answer is that Common Lisp is (except for for special variables) lexically scoped. Lexical scope is just a fancy way of saying that the bindings (variables) that are available are exactly and only the bindings you can see by looking at the source of the program. (Except, in the case of CL for special bindings, which I'll ignore, since there are none here.)
So, given this, think about test-6/coercing, and in particular the call to mapcar/coercing: in that call, coerce has to turn the list (lambda (x) (expt z y)) into a function. So it does that. But the function it returns doesn't bind y and there is no binding for y visible in it: the function uses y 'free'.
The only way that this could work is if the function that coerce constructs for us were to dynamically look for a binding for y. Well, that's what dynamically-scoped languages do, but CL is not dynamically-scoped.
Perhaps a way of making this even clearer is to realise that we can lift the function creation right out of the function:
(defun test-7 (y f)
(mapcar f '(1 2 3)))
> (test-7 1 (coerce '(lambda (x) (expt x y)) 'function))
It's clear that this can't work in a lexically-scoped language.
So, then, how do tests 1-4 work?
Well, firstly there are only actually two tests here. In CL, lambda is a macro and (lambda (...) ...) is entirely equivalent to (function (lambda (...) ...)). And of course #'(lambda (...) ...) is also the same as (function (lambda (...) ...)): it's just a read-macro for it.
And (function ...) is a magic thing (a special form) which says 'this is a function'. The important thing about function is that it's not a function: it's a deeply magic thing which tells the evaluator (or the compiler) that its argument is the description of a function in the current lexical context, so, for instance in
(let ((x 1))
(function (lambda (y) (+ x y))))
The x referred to by the function this creates is the x bound by let. So in your tests 2 and 4 (which are the same):
(defun test-4 (y)
(mapcar (function (lambda (x) (expt x y)))
'(1 2 3)))
The binding of y which the function created refers to is the binding of y which is lexically visible, which is the argument of test-4 itself.
Let's add a y parameter to avoid closing over variables and see what kind of values we are manipulating:
USER> (type-of #'(lambda (x y) (expt x y)))
FUNCTION
USER> (type-of (lambda (x y) (expt x y)))
FUNCTION
USER> (type-of '(lambda (x y) (expt x y)))
CONS
As you can see, the two first lambda-like forms are evaluated as functions, while the third is evaluated as a cons-cell. As far as Lisp is concerned, the third argument is just a tree of symbols with no meaning.
Reader macros
I thought that quoting a lambda with ' and using #' in front of a lambda both returned pointers to the function with the only difference being that the #' will compile it first.
Let's go back to the definitions, ' and #' are reader macros, respectively Single-Quote and Sharpsign Single-Quote. They are found in front of other forms, for example 'f is read as (quote f) and #'f is read as (function f). At read-time, f and the resulting forms are just unevaluated data.
We will see below how both special operators are interpreted, but what matters really is the lexical scope, so let's open a parenthesis.
Lexical environment
Lexical environments are the set of bindings in effect at some point of your code. When you evaluate a let or an flet it enriches the current environment with new bindings. When you call EVAL on an expression, you start evaluating from a null lexical environment, even if the call to eval itself is in a non-null environment.
Here x is just unbound during eval:
(let ((x 3)) (eval '(list x))) ;; ERROR
Here we build a let to be evaluated by eval:
(eval '(let ((x 3)) (list x)))
=> (3)
That's all for the crash course on lexical environments.
Special operators
FUNCTION
Special operator FUNCTION takes an argument that is either the name of a function (symbol or setf), or a lambda expression; in particular:
The value of function is the functional value of name in the current lexical environment.
Here the lambda expression is evaluated in the current lexical environment, which means it can refer to variable outside the lambda expression. That's the definition of closures, they capture the surrounding bindings.
NB. you do not need to prefix lambda with #', because there is a macro named (lambda ...) that expands into (function (lambda ...)). It looks like this could expand recursively forever, but this is not the case: at first the macro is expanded so that (lambda ...) becomes (function (lambda ...)), then the special operator function knows how to evaluate the lambda expression itself.
This means that (lambda ...) and #'(lambda ...) are equivalent. Note in particular that there is nothing about whether one form is compiled or not at this point, the compiler will see the same expression after macroexpansion.
QUOTE
Special operator QUOTE evaluates (quote f) as f, where f itself is unevaluated. In test-5 and test-6, there is no function, just an unevaluated structured expression that can be interpreted as code.
Type coercion
Now, certain functions like MAPCAR are used to apply functions. Notice how the specification says that the function parameter is a function designator:
function --- a designator for a function that must take as many arguments as there are lists.
A designator for a type is not necessarily a value of that type, but can be a value that can be coerced to that type. Sometimes a user wants to specify a pathname, and enters a string, but a string is not a value of type pathname: the system has to converts the string into a pathname.
Common Lisp defines a COERCE function with rules regarding how values can be converted to other values. In you case, mapcar first does (coerce (lambda ...) 'function). This is defined as follows:
If the result-type is function, and object is a lambda expression, then the result is a closure of object in the null lexical environment.
The value is thus evaluated in a null lexical environment, so it does not have access to the surrounding bindings; y is a free variable in your lambda expression, and since it is evaluated in a null environment, it is unbound. That's why test-5 pass but test-6 fails.
Name resolution, compilers and late binding
There is a difference whether you write #'f or 'f when referring to a function f where f is a symbol: in the first case, the expression evaluated to an object of type function, and in the second case, you only evaluate a symbol.
Name resolution for this function can change depending and how the compiler works. With a symbol as a function designator, the function does not even need to be defined, the name is resolved when the symbols has to be coerced as a function.
When you write #'f, some compilers may remove one level of indirection and directly make your code jump to the code associated with the function, without having to resolve the name at runtime.
However, this also means that with such compilers (e.g. SBCL), you need to recompile some call sites on function redefinition, as-if the function was declared inline, otherwise some old code will still reference the previous definition of #'f. This is something that is not necessarily important to consider at the beginning, but it can be a source of confusion to keep in mind when you are live coding.
I want to know if these two definitions of nth are equal:
I. is defined as macro:
(defmacro -nth (n lst)
(defun f (n1 lst1)
(cond ((eql n1 0) lst1)
(t `(cdr ,(f (- n1 1) lst1)))))
`(car ,(f n lst)))
II. is defined as a bunch of functions:
(defun f (n lst)
(cond ((eql n 0) lst)
(t `(cdr ,(f (- n 1) lst)))))
(defun f1 (n lst)
`(car ,(f n `',lst)))
(defun --nth (n lst)
(eval (f1 n lst)))
Am i get the right idea? Is macro definition is evaluating of expression, constructed in its body?
OK, let start from the beginning.
Macro is used to create new forms that usually depend on macro's input. Before code is complied or evaluated, macro has to be expanded. Expansion of a macro is a process that takes place before evaluation of form where it is used. Result of such expansion is usually a lisp form.
So inside a macro here are a several levels of code.
Not quoted code will be evaluated during macroexpansion (not at run-time!), in your example you define function f when macro is expanded (for what?);
Next here is quoted (with usual quote or backquote or even nested backquotes) code that will become part of macroexpansion result (in its literal form); you can control what part of code will be evaluated during macroexpansion and what will stay intact (quoted, partially or completely). This allows one to construct anything before it will be executed.
Another feature of macro is that it does not evaluate its parameters before expansion, while function does. To give you picture of what is a macro, see this (just first thing that came to mind):
(defmacro aif (test then &optional else)
`(let ((it ,test))
(if it ,then ,else)))
You can use it like this:
CL-USER> (defparameter *x* '((a . 1) (b . 2) (c . 3) (d . 4)))
*X*
CL-USER> (aif (find 'c *x* :key #'car) (1+ (cdr it)) 0)
4
This macro creates useful lexical binding, capturing variable it. After checking of a condition, you don't have to recalculate result, it's accessible in forms 'then' and 'else'. It's impossible to do with just a function, it has introduced new control construction in language. But macro is not just about creating lexical environments.
Macro is a powerful tool. It's impossible to fully describe what you can do with it, because you can do everything. But nth is not something you need a macro for. To construct a clone of nth you can try to write a recursive function.
It's important to note that LISP macro is most powerful thing in the programming world and LISP is the only language that has this power ;-)
To inspire you, I would recommend this article: http://www.paulgraham.com/avg.html
To master macro, begin with something like this:
http://www.gigamonkeys.com/book/macros-defining-your-own.html
Then may be Paul Graham's "On Lisp", then "Let Over Lambda".
There is no need for either a macro nor eval to make abstractions to get the nth element of a list. Your macro -nth doesn't even work unless the index is literal number. try this:
(defparameter test-list '(9 8 7 6 5 4 3 2 1 0))
(defparameter index 3)
(nth index test-list) ; ==> 6 (this is the LISP provided nth)
(-nth index test-list) ; ==> ERROR: index is not a number
A typical recursive solution of nth:
(defun nth2 (index list)
(if (<= index 0)
(car list)
(nth2 (1- index) (cdr list))))
(nth2 index test-list) ; ==> 6
A typical loop version
(defun nth3 (index list)
(loop :for e :in list
:for i :from index :downto 0
:when (= i 0) :return e))
(nth3 index test-list) ; ==> 6
Usually a macro is something you use when you see your are repeating yourself too much and there is no way to abstract your code further with functions. You may make a macro that saves you the time to write boilerplate code. Of course there is a trade off of not being standard code so you usually write the macro after a couple of times have written the boilerplate.
eval should never be used unless you really have to. Usually you can get by with funcall and apply. eval works only in the global scope so you loose closure variables.
Background:
I intend to generate debug messages for the code I'm developing. I wrote a macro to avoid writing the logging calls in each function. I know this limits the ability to generate more customized debug messages, but in return it isolates logging from code. And that is what I'm aiming for. This macro approach has other drawbacks too, for example it limits creation of function bindings to this macro only, but I think I can live with that.
Following is the definition of the macro and an example demonstrating its usage.
(define-syntax (define-func stx)
(syntax-case stx ()
[(define-func (func-name args ...) body1 body2 ...)
(if (and (identifier? #'func-name)
(andmap symbol? (syntax->datum #'(args ...))))
(syntax (define (func-name args ...)
(log-debug (format "Function-name ~a:" (syntax-e #'func-name)) (list args ...))
body1
body2 ...))
(raise-syntax-error 'define-func "not an identifier" stx))]
[else (raise-syntax-error 'define-func "bad syntax" stx)]))
(define-func (last l)
(cond [(null? l) null]
[(null? (rest l)) (first l)]
[else (last (rest l))]))
(define-func (main)
(last (list 1 2 3 4 5 6 7 8 9))
(logger))
log-debug and logger are defined in separate module
The output produced is somewhat like following:
Function-name last:
args:
:-> (7 8 9)
Function-name last:
args:
:-> (8 9)
Function-name last:
args:
:-> (9)
Now I want to make it more readable. By readability I mean to provide some kind of indentation so that the person reading the log could make sense of call flow. For example something like following:
Function-name last:
args:
:-> (7 8 9)
Function-name last:
args:
:-> (8 9)
Function-name last:
args:
:-> (9)
It is easier to figure out who called whom and so forth. I have an idea that can do this. It involves a variable that keeps track of indentation then after logging the function name I will increase the indent and after evaluation of body and before returning the value decrements the value. Something like following:
(define indent 0)
(define-syntax (define-func stx)
(syntax-case stx ()
[ (... ...)
(...
(log-debug ...)
(increment indent)
(let [(retval (body1 body2 ...)]
(decrease indent)
retval))]))
increment and decrease increases and decreases indentation respectively.
Problem:
It works even for function that returns void. I'm not sure whether its the correct behavior. In racket void is a special value, but I'm not sure that creating a binding to void is right way to go.
Is there any better way to achieve the same? If not are there any problems in this design? I'm open to any idea/change as long as they keep the logging and code separate.
thanks for the help!
I have several suggestions for you:
It's probably better to use a parameter instead of a variable, for "global" stuff like your indentation level, since the original value is restored for you at the end of the parameterize expression.
All those raise-syntax-error checks you have in your macro are totally superfluous: syntax-case already provides guards (also known as fenders) that allow you to do any validation of macro "arguments" necessary:
(define-syntax (define-func stx)
(syntax-case stx ()
[(_ (func-name args ...) body1 body2 ...)
(andmap identifier? (syntax->list #'(func-name args ...)))
#'(define (func-name args ...)
(log-debug (format "Function-name ~a:" 'func-name)
(list args ...))
body1
body2 ...)]))
I've also fixed up your code in several places, as you can see above:
I used (_ ...) instead of (define-func ...), since in syntax-case (unlike syntax-rules), the latter will actually bind a pattern variable called define-func, which will affect any recursive macro calls you may want to do (I'll grant that you don't have one here, but it's a good habit to have anyway).
Rather than completely flatten the #'(args ...) in the guard, I just turned it into a list of syntax objects so you can test using identifier?. This is more intention-revealing than testing using symbol?, and allows us to also test func-name in the same expression.
You don't need to use (syntax-e #'func-name) inside the expanded code! Just quote it.
I have an s-expression bound to a variable in Common Lisp:
(defvar x '(+ a 2))
Now I want to create a function that when called, evaluates the expression in the scope in which it was defined. I've tried this:
(let ((a 4))
(lambda () (eval x)))
and
(let ((a 4))
(eval `(lambda () ,x)))
But both of these create a problem: EVAL will evaluate the code at the top level, so I can't capture variables contained in the expression. Note that I cannot put the LET form in the EVAL. Is there any solution?
EDIT: So if there is not solution to the EVAL problem, how else can it be done?
EDIT: There was a question about what exactly I am try to do. I am writing a compiler. I want to accept an s-expression with variables closed in the lexical environment where the expression is defined. It may indeed be better to write it as a macro.
You need to create code that has the necessary bindings. Wrap a LET around your code and bind every variable you want to make available in your code:
(defvar *x* '(+ a 2))
(let ((a 4))
(eval `(let ((a ,a))
,*x*)))
CLISP implements an extension to evaluate a form in the lexical environment. From the fact that it is an extension, I suspect you can't do that in a standard-compliant way.
(ext:eval-env x (ext:the-environment))
See http://clisp.cons.org/impnotes.html#eval-environ.
What is the actual problem that you want to solve? Most likely, you're trying to tackle it the wrong way. Lexical bindings are for things that appear lexically within their scope, not for random stuff you get from outside.
Maybe you want a dynamic closure? Such a thing doesn't exist in Common Lisp, although it does in some Lisp dialects (like Pico Lisp, as far as I understand).
Note that you can do the following, which is similar:
(defvar *a*)
(defvar *x* '(+ *a* 2)) ;'
(let ((a 10))
;; ...
(let ((*a* a))
(eval *x*)))
I advise you to think hard about whether you really want this, though.
In Common Lisp you can define *evalhook* Which allows you to pass an environment to (eval ...). *evalhook* is platform independent.
It is possible to use COMPILE to compile the expression into function and then use PROGV to FUNCALL the compiled function in the environment where variables are dynamically set. Or, better, use COMPILE to compile the expression into function that accepts variables.
Compile accepts the function definition as a list and turns it into function. In case of SBCL, this function is compiled into machine code and will execute efficiently.
First option (using compile and progv):
(defvar *fn* (compile nil '(lambda () (+ a 2)))
(progv '(a) '(4) (funcall *fn*))
=>
6
Second option:
(defvar *fn* (compile nil '(lambda (a) (+ a 2))))
(funcall *fn* 4)
=>
6