In Guix there's a layer made of macros beautifying the creation and manipulation of srfi-9 records
The code is in guix/records.scm
The code is large and articulated
I thought to expand those macros to see their input and their output in order to have a feel of what they do
The thing is that even the standard vanilla srfi-9 records are macros around structures, in their own turn
So the macro expansion gives me a completely expanded code creating and manipulating structures.
I'd prefer to see the result of a single pass of macro expansion, to see what srfi-9 code the guix macros have produced
In Clojure there are macroexpand and macroexpand-1
macroexpand-1 does a single pass of macro expansion and macroexpand calls macroexpand-1 repeatedly until there are no more macors to expand in the forms being processed
You can see this here
Is there a similar option in Guile scheme ?
Am I missing any workflow trick, any tool, any library function or macro for dealing with this ?
Here is a simple version of macroexpand1:
#lang racket
(define-syntax (expand1 stx)
(syntax-case stx ()
[(_expand1 form)
(syntax-case #'form ()
[(id . more)
(identifier? #'id)
(let ([transformer (syntax-local-value #'id)])
(with-syntax ([expansion (transformer #'form)])
#''expansion))]
[_
#''form])]))
(expand1 (or 1 2 3))
The output is:
'(let ((or-part 1)) (if or-part or-part (or 2 3)))
Note that Clojure's macro expansion algorithm is more simplistic than what's used in most Scheme implementation.
For an explanation psyntax aka portable syntax-case look in the book "Beautiful Code" for the chapter by Dybvig.
https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf
Ok, in the NEWS file there's this excerpt
** Removed function: `macroexpand-1'
It is unclear how to implement `macroexpand-1' with syntax-case,
though PLT Scheme does prove that it is possible.
I don' t know what the problem is between macroexpand-1 and syntax-case and I don't know what Racket is doing about it
But at least now I know that macroexpand-1 has been explicitly removed
Related
AND and OR are macros and since macros aren't first class in scheme/racket they cannot be passed as arguments to other functions. A partial solution is to use and-map or or-map. Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function? Are there any languages that have first class macros?
In general, no. Consider that let is (or could be) implemented as a macro on top of lambda:
(let ((x 1))
(foo x))
could be a macro that expands to
((lambda (x) (foo x)) 1)
Now, what would it look like to convert let to a function? Clearly it is nonsense. What would its inputs be? Its return value?
Many macros will be like this. In fact, any macro that could be routinely turned into a function without losing any functionality is a bad macro! Such a macro should have been a function to begin with.
I agree with #amalloy. If something is written as a macro, it probably does something that functions can't do (e.g., introduce bindings, change evaluation order). So automatically converting arbitrary macro into a function is a really bad idea even if it is possible.
Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function?
No, but it is somewhat doable to write a macro that would take some macro and turn it into a function.
#lang racket
(require (for-syntax racket/list))
(define-syntax (->proc stx)
(syntax-case stx ()
[(_ mac #:arity arity)
(with-syntax ([(args ...) (generate-temporaries (range (syntax-e #'arity)))])
#'(λ (args ...) (mac args ...)))]))
((->proc and #:arity 2) 42 12)
(apply (->proc and #:arity 2) '(#f 12))
((->proc and #:arity 2) #f (error 'not-short-circuit))
You might also be interested in identifier macro, which allows us to use an identifier as a macro in some context and function in another context. This could be used to create a first class and/or which short-circuits when it's used as a macro, but could be passed as a function value in non-transformer position.
On the topic of first class macro, take a look at https://en.wikipedia.org/wiki/Fexpr. It's known to be a bad idea.
Not in the way you probably expect
To see why, here is a way of thinking about macros: A macro is a function which takes a bit of source code and turns it into another bit of source code: the expansion of the macro. In other words a macro is a function whose domain and range are source code.
Once the source code is fully expanded, then it's fed to either an evaluator or a compiler. Let's assume it's fed to a compiler because it makes the question easier to answer: a compiler itself is simply a function whose domain is source code and whose range is some sequence of instructions for a machine (which may or may not be a real machine) to execute. Those instructions might include things like 'call this function on these arguments'.
So, what you are asking is: can the 'this function' in 'call this function on these arguments' be some kind of macro? Well, yes, it could be, but whatever source code it is going to transform certainly can not be the source code of the program you are executing, because that is gone: all that's left is the sequence of instructions that was the return value of the compiler.
So you might say: OK, let's say we disallow compilers: can we do it now? Well, leaving aside that 'disallowing compilers' is kind of a serious limitation, this was, in fact, something that very old dialects of Lisp sort-of did, using a construct called a FEXPR, as mentioned in another answer. It's important to realise that FEXPRs existed because people had not yet invented macros. Pretty soon, people did invent macros, and although FEXPRs and macros coexisted for a while – mostly because people had written code which used FEXPRs which they wanted to keep running, and because writing macros was a serious pain before things like backquote existed – FEXPRs died out. And they died out because they were semantically horrible: even by the standards of 1960s Lisps they were semantically horrible.
Here's one small example of why FEXPRs are so horrible: Let's say I write this function in a language with FEXPRs:
(define (foo f g x)
(apply f (g x)))
Now: what happens when I call foo? In particular, what happens if f might be a FEXPR?. Well, the answer is that I can't compile foo at all: I have to wait until run-time and make some on-the-fly decision about what to do.
Of course this isn't what these old Lisps with FEXPRs probably did: they would just silently have assumed that f was a normal function (which they would have called an EXPR) and compiled accordingly (and yes, even very old Lisps had compilers). If you passed something which was a FEXPR you just lost: either the thing detected that, or more likely it fall over horribly or gave you some junk answer.
And this kind of horribleness is why macros were invented: macros provide a semantically sane approach to processing Lisp code which allows (eventually, this took a long time to actually happen) minor details like compilation being possible at all, code having reasonable semantics and compiled code having the same semantics as interpreted code. These are features people like in their languages, it turns out.
Incidentally, in both Racket and Common Lisp, macros are explicitly functions. In Racket they are functions which operate on special 'syntax' objects because that's how you get hygiene, but in Common Lisp, which is much less hygienic, they're just functions which operate on CL source code, where the source code is simply made up of lists, symbols &c.
Here's an example of this in Racket:
> (define foo (syntax-rules ()
[(_ x) x]))
> foo
#<procedure:foo>
OK, foo is now just an ordinary function. But it's a function whose domain & range are Racket source code: it expects a syntax object as an argument and returns another one:
> (foo 1)
; ?: bad syntax
; in: 1
; [,bt for context]
This is because 1 is not a syntax object.
> (foo #'(x 1))
#<syntax:readline-input:5:10 1>
> (syntax-e (foo #'(x 1)))
1
And in CL this is even easier to see: Here's a macro definition:
(defmacro foo (form) form)
And now I can get hold of the macro's function and call it on some CL source code:
> (macro-function 'foo)
#<Function foo 4060000B6C>
> (funcall (macro-function 'foo) '(x 1) nil)
1
In both Racket and CL, macros are, in fact, first-class (or, in the case of Racket: almost first-class, I think): they are functions which operate on source code, which itself is first-class: you can write Racket and CL programs which construct and manipulate source code in arbitrary ways: that's what macros are in these languages.
In the case of Racket I have said 'almost first-class', because I can't see a way, in Racket, to retrieve the function which sits behind a macro defined with define-syntax &c.
I've created something like this in Scheme, it's macro that return lambda that use eval to execute the macro:
(define-macro (macron m)
(let ((x (gensym)))
`(lambda (,x)
(eval `(,',m ,#,x)))))
Example usage:
;; normal eval
(define x (map (lambda (x)
(eval `(lambda ,#x)))
'(((x) (display x)) ((y) (+ y y)))))
;; using macron macro
(define x (map (macron lambda)
'(((x) (display x)) ((y) (+ y y)))))
and x in both cases is list of two functions.
another example:
(define-macro (+++ . args)
`(+ ,#args))
((macron +++) '(1 2 3))
While reading through Paul Graham's Essays, I've become more and more curious about Lisp.
In this article, he mentions that one of the most powerful features is that you can write programs that write other programs.
I couldn't find an intuitive explanation on his site or elsewhere. Is there some minimal Lisp program that shows an example of how this is done? Or, can you explain in words what this means exactly?
Lisp is homoiconic. Here is a function which build an s-expression representing a sum.
(defun makes(x) (list '+ x 2))
so (makes 5) evaluates to (+ 5 2) which is a valid s-expression. You could pass that to eval
There are more complex examples with Lisp macros. See also this. Read the section on Evaluation and Compilation of Common Lisp HyperSpec (also notice its compile, defmacro, eval forms). Be aware of multi-staged programming.
I strongly recommend reading SICP (it is freely downloadable) then Lisp In Small Pieces. You could also enjoy reading Gödel, Escher, Bach.... and J.Pitrat's blog on Bootstrapping Artificial Intelligence.
BTW, with C on POSIX, you might also code programs generating C code (or use GCCJIT or LLVM), compiling that generated code as a plugin, and dlopen-ing it.
While homoiconicity is the fundamental property that makes this easy, a good example of this in practice is the macro facility present in many lisps. Homoiconicity allows you to write lisp functions that take lisp source (represented as lists of lists) and do list manipulation operations on it to produce other lisp source. A macro is a plain lisp function for doing this which is installed into the compiler/evaluator of your lisp as an extension of the language's syntax. The macro gets called like a normal function, but instead of waiting until runtime the compiler passes the raw code of the macro's arguments to it. The macro is then responsible for returning some alternative code for the compiler to process in its place.
A simple example is the built-in when macro, used like so (assuming some variable x):
(when (evenp x)
(print "It's even!")
(* 5 x))
when is similar to the more fundamental if, but where if takes 3 sub-expressions (test, then-case, else-case) when takes the test and then an arbitrary number of expressions to run in the "then" case (it returns nil in the else case). To write this using if you need an explicit block (a progn in Common Lisp):
(if (evenp x)
(progn
(print "It's even!")
(* 5 x))
nil)
Translating the when version to the if version is some very simple list-manipluation:
(defun when->if (when-expression)
(list 'if
(second when-expression)
(append (list 'progn)
(rest (rest when-expression)))))
Although I'd probably use the list templating syntax and some shorter functions to get this:
(defun when->if (when-expression)
`(if ,(second when-expression) (progn ,#(cddr when-expression)) nil))
This gets called like so: (when->if (list 'when (list 'evenp 'x) ...)).
Now all we need to do is inform the compiler that when it sees an expression like (when ...) (actually I'm writing one for (my-when ...) to avoid clashing with the built-in version) it should use something like our when->if to turn it into code it understands. The actual macro syntax for this actually lets you take apart the expression/list ("destructure" it) as part of the arguments of the macro, so it ends up looking like this:
(defmacro my-when (test &body then-case-expressions)
`(if ,test (progn ,#then-case-expressions) nil))
Looks sorta like a regular function, except it's taking code and outputting other code. Now we can write (my-when (evenp x) ...) and everything works.
The lisp macro facility forms a major component of the expressive power of lisps- they allow you to mold the language to better suit your project and abstract away nearly any boilerplate. Macros can be as simple as when or complex enough to make a third-party OOP library feel like a first-class part of the language (in fact many lisps still implement OOP as a pure lisp library as opposed to a special component of the core compiler, not that you can tell from using them).
A good example are Lisp macros. They aren't evaluated, but instead they transform to the expressions within them. That is what makes them essentially programs that write program. They transform the expressions within them between compile-time and runtime. This means that you can essentially create your own syntax since a macro isn't actually evaluated. A good example would be this invalid common lisp form:
(backwards ("Hello world" nil format))
Clearly the syntax for the format function is backwards. BUT... we are passing it to a macro which isn't evaluated, so we will not get a backtrace error, because the macro isn't actually evaluated. Here is what our macro looks like:
(defmacro backwards (expr)
(reverse expr))
As you can see, we reverse the expression within the macro, which is why it becomes a standard Lisp form between compile-time and runtime. We have essentially altered the syntax of Lisp with a simple example. The call to the macro isn't evaluated, but is translated. A more complex example would be creating a web page in html:
(defmacro standard-page ((&key title href)&body body)
`(with-html-output-to-string (*standard-output* nil :prologue t :indent t)
(:html :lang "en"
(:head
(:meta :charset "utf-8")
(:title ,title)
(:link :rel "stylesheet"
:type "text/css"
:href ,href))
,#body)))
We can essentially create a macro, and the call to that macro will not be evaluated, but it will expand to valid lisp syntax, and that will be evaluated. If we look at the macro expansion we can see that the expansion is what is evaluated:
(pprint (macroexpand-1 '(standard-page (:title "Hello"
:href "my-styles.css")
(:h1 "Hello world"))))
Which expands to:
(WITH-HTML-OUTPUT-TO-STRING (*STANDARD-OUTPUT* NIL :PROLOGUE T :INDENT T)
(:HTML :LANG "en"
(:HEAD (:META :CHARSET "utf-8") (:TITLE "Hello")
(:LINK :REL "stylesheet" :TYPE "text/css" :HREF "my-styles.css"))
(:H1 "Hello world")))
This is why Paul Graham mentions that you can essentially write programs that write programs, and ViaWeb was essentially one big macro. A bunch of macros like this writing code that could write code that could write code...
I'm trying to move from Common Lisp to Chicken Scheme, and having plenty of problems.
My current problem is this: How can I write a macro (presumably using define-syntax?) that calls other macros?
For example, in Common Lisp I could do something like this:
(defmacro append-to (var value)
`(setf ,var (append ,var ,value)))
(defmacro something-else ()
(let ((values (list))
(append-to values '(1)))))
Whereas in Scheme, the equivalent code doesn't work:
(define-syntax append-to
(syntax-rules ()
((_ var value)
(set! var (append var value)))))
(define-syntax something-else
(syntax-rules ()
((_)
(let ((values (list)))
(append-to values '(1))))))
The append-to macro cannot be called from the something-else macro. I get an error saying the append-to "variable" is undefined.
According to all the information I've managed to glean from Google and other sources, macros are evaluated in a closed environment without access to other code. Essentially, nothing else exists - except built-in Scheme functions and macros - when the macro is evaluated. I have tried using er-macro-transformer, syntax-case (which is now deprecated in Chicken anyway) and even the procedural-macros module.
Surely the entire purpose of macros is that they are built upon other macros, to avoid repeating code. If macros must be written in isolation, they're pretty much useless, to my mind.
I have investigated other Scheme implementations, and had no more luck. Seems it simply cannot be done.
Can someone help me with this, please?
It looks like you're confusing expansion-time with run-time. The syntax-rules example you give will expand to the let+set, which means the append will happen at runtime.
syntax-rules simply rewrites input to given output, expanding macros until there's nothing more to expand. If you want to actually perform some computation at expansion time, the only way to do that is with a procedural macro (this is also what happens in your defmacro CL example).
In Scheme, evaluation levels are strictly separated (this makes separate compilation possible), so a procedure can use macros, but the macros themselves can't use the procedures (or macros) defined in the same piece of code. You can load procedures and macros from a module for use in procedural macros by using use-for-syntax. There's limited support for defining things to run at syntax expansion time by wrapping them in begin-for-syntax.
See for example this SO question or this discussion on the ikarus-users mailing list. Matthew Flatt's paper composable and compilable macros explains the theory behind this in more detail.
The "phase separation" thinking is relatively new in the Scheme world (note that the Flatt paper is from 2002), so you'll find quite a few people in the Scheme community who are still a bit confused about it. The reason it's "new" (even though Scheme has had macros for a long long time) is that procedural macros have only become part of the standard since R6RS (and reverted in R7RS because syntax-case is rather controversial), so the need to rigidly specify them hasn't been an issue until now. For more "traditional" Lispy implementations of Scheme, where compile-time and run-time are all mashed together, this was never an issue; you can just run code whenever.
To get back to your example, it works fine if you separate the phases correctly:
(begin-for-syntax
(define-syntax append-to
(ir-macro-transformer
(lambda (e i c)
(let ((var (cadr e))
(val (caddr e)))
`(set! ,var (append ,var ,val)))))) )
(define-syntax something-else
(ir-macro-transformer
(lambda (e i c)
(let ((vals (list 'print)))
(append-to vals '(1))
vals))))
(something-else) ; Expands to (print 1)
If you put the definition of append-to in a module of its own, and you use-for-syntax it, that should work as well. This will also allow you to use the same module both in the macros you define in a body of code as well as in the procedures, by simply requiring it both in a use and a use-for-syntax expression.
I'm exploring Scheme macros, but I've been unable to find a portable way of writing anaphoric macros.
I'm trying to write an each-it macro, such that this code:
(each-it (list 1 2 3)
(display it))
Expands to this:
(for-each (lambda (it)
(display it))
(list 1 2 3))
I've written a macro with syntax-rules, but this gives me an error about an undefined identifier when I try to use it.
(define-syntax each-it
(syntax-rules ()
((each-it lst body)
(for-each (lambda (it) body)
lst))))
This SO question mentions define-syntax-parameter, which seems to be Racket only. This blog post gives some Scheme code samples, but the code samples don't run in DrRacket in R5RS mode (I think it's the square brackets?).
R4RS has an interesting macro appendix but it is not present in R5RS and I don't know if I can depend on it.
Can I write my each-it macro in a completely portable way? If not, what are the most widely available macro system features for writing my macro?
This should be portable, at least in R6RS:
(define-syntax each-it
(lambda (x)
(syntax-case x ()
((_ lst body)
(with-syntax ((it (datum->syntax x 'it)))
#'(for-each (lambda (it) body) lst))))))
Yes, you can write it in a portable way assuming that R6RS is portable enough for you. (The same cannot be said on R7RS, which currently has nothing more than just syntax-rules, and it's unclear what will be included in the large language, or when it will happen.) See uselpa's for how to do that.
So why am I writing another answer? Because actually doing that is going to be a bad idea. A bad idea not in some vague academic sense that doesn't matter for most real world code -- bad in a sense that is likely to bite you later on. I know that "paper" makes it look intimidating, but read at least the first two sections of the paper mentioned in the other SO question you've seen. Specifically, Section 1.2 shows the problem you'll be running against. Then, Section 2 shows how to do it "properly", in a way that makes it tedious to write macros that expand to uses of your macro. At this point, it will be appealing to take the "just keep it hygienic", but at the end of Section 2 you'll see why that's not working either.
The bottom line, IMO, is to just not do it unless you have syntax parameters or something similar. Maybe the only exception to that (which might be your case) is when the macro is something that you intend to use yourself, and you will never provide it to others.
What is the best route for an experienced imperative style programmer only familiar with C macros to learn the Racket macro system. Not just the mechanics of it (the how?) but also the where and why to use it and examples that illustrate this.
Should one learn Scheme (or Lisp) macros first? I've heard that the book "On Lisp" has a good explanation of Lisp macros with excellent examples of their use. Would this be useful or not?
I believe that the best tutorial on macros is still Kent Dybvig's "Writing Hygenic macros in Scheme with syntax-case". It's not specifically about Racket, but basically everything will carry over, and reading the guide that Chris mentioned above will cover any remaining differences. Despite the name, it covers both "hygenic" and "unhygenic" macros.
Edit July 2014:
Since I wrote the above, Greg Hendershott has written a fabulous tutorial on macros in Racket, entitled Fear of Macros. That's now the best place to start learning about Racket macros.
Racket's documentation has an excellent tutorial on using macros.
I would definitely recommend Scheme macros over CL macros if you haven't deal with any sort of Lisp macros before. That's because Scheme macros use pattern matching, and it's much easier to read.
Example (using Racket's define-syntax-rule):
(define-syntax-rule (let ((var val) ...)
expr ...)
((lambda (var ...)
expr ...)
val ...))
This is a very simple macro that defines let in terms of creating a corresponding lambda, then applying it. It's easy to read, and easy to reason about what it does.
Slightly more complicated macro:
(define-syntax let*
(syntax-rules ()
((let* ()
expr ...)
(let ()
expr ...))
((let* ((var val) next ...)
expr ...)
(let ((var val))
(let* (next ...)
expr ...)))))
This defines let* in terms of nested lets, so that the bindings are done sequentially. It includes a base case (with no bindings), as well as a recursive case.