I'm working with the Racket macro extension syntax-id-rules, that some other Scheme implementations provide under the name identifier-syntax. These let you specify macro expansions that will happen even when the defined identifier isn't in head position. So for example:
(define hidden #f)
(define-syntax proxy
(syntax-id-rules (set!)
[(set! proxy v) (set! hidden v)]
[proxy hidden]))
will set up the identifier proxy to be a proxy for hidden. This is a useless example, but it illustrates the usage.
I find myself in a situation where I want a global ordinary macro, let's call it foo, that I want to override in some cases where I'm using an identifier macro like proxy. That is, I want to be able to do something like this:
(define-syntax foo
(syntax-rules ()
[(foo arg ...) 'default]))
(define hidden #f)
(define-syntax proxy
(syntax-id-rules (foo set!)
[(foo proxy arg ...) 'special]
[(set! proxy v) (set! hidden v)]
[proxy hidden]))
(foo proxy) ; should return 'special
But in fact the last line returns 'default, because the foo macro gets expanded before the proxy one.
Any ideas how I might achieve something along these lines, but with the proxy identifier macro overriding the default macro definition for foo? I'm not committed to the above architecture specifically.
Added: This isn't for any real-world usage, but part of a demonstration of a theoretical point in formal semantics.
#soegaard explained it perfectly. You can't do what you want directly without modifying the macro expander.
To extend #soegaard's answer, here is a way to simulate what you are asking for. It essentially does a "double-dispatch" macro expansion. As soegaard noted though, there's probably a more idiomatic way to achieve what you want, depending on your goals.
#lang racket
(require (for-syntax syntax/parse))
(begin-for-syntax
(define (special-condition? id)
(and (identifier? id)
(regexp-match #rx"^p" ; starts with "p"
(symbol->string (syntax->datum id))))))
(define-syntax foo
(syntax-parser
[(_ special-case arg ...)
#:when (special-condition? #'special-case)
#'(special-case 'hidden-special-case-tag arg ...)]
; else
[(_ arg ...) #''default]))
(define hidden #f)
(define-syntax proxy
(syntax-id-rules (quote set!)
[(proxy (quote hidden-special-case-tag) arg ...) 'special]
[(set! proxy v) (set! hidden v)]
[(proxy arg ...) 'other]
[proxy hidden]))
(foo non-proxy) ; => 'default
(foo proxy) ; => 'special
(proxy) ; => 'other
proxy ; => #f
(set! proxy #t)
proxy ; => #t
It seems to me that you need to find an alternative strategy. Maybe we can find a solution if you provide more details of the situation you want to use this in.
Anyways, here is why your strategy don't work. When you write
(define-syntax proxy ...)
you associate a syntax-transformer with the identifier proxy. That transformer is called by the expander when it sees either (proxy ...), (set! proxy ...), or proxy.
In order to control what (foo proxy arg ...) expands to you need to specify it in the syntax-transformer associated with foo.
Now depending on the situation there might be tricks that can be played.
For example one could imagine wrapping your program in a new form, that rewrites (foo proxy arg ...) into (proxy 'was-a-foo-originally arg ...) and then let the syntax-transformer for proxy handle the rest.
The easy solution is to move the handling of (foo proxy arg ...) into the transformer for foo, but you specifically ask for a solution where foo isn't changed.
Related
I would like to get the name of the file where a particular function or macro definition was last written, for various documentation & testing purposes?
I have posted an answer that works well from me, after the help of PJB on #CommonLisp (IRC.LIBERA.CHAT), but will accept any answers which provide a portable solution that does not depend on SWANK or provides additional detail on how to achieve the same in other IDEs like LispWorks, ACL, etc.
If what you seek is a portable solution – one that is written in portable CL – then the answer to that is to define wrappers for defining forms and then use the wrappers.
(defvar *flocs* (make-hash-table :test #'equal))
(defgeneric function-location (f/name)
(:method ((name t))
(values (gethash name *flocs* nil) t))
(:method ((f function))
(multiple-value-bind (le cp nm) (function-lambda-expression f)
(declare (ignore le cp))
(if nm
(function-location nm)
(values nil nil)))))
(defmacro define-function (f args &body doc/decls/forms)
(when (or *load-pathname* *compile-file-pathname*)
;; Prefer *load-pathname*
(setf (gethash f *flocs*) (or *load-pathname* *compile-file-pathname*)))
`(defun ,f ,args ,#doc/decls/forms))
In real life you'd call define-function defun of course, and similarly with define-variable etc, and then construct a conduit package for CL which exported all the CL symbols while replacing the defining forms with these ones.
If what you seek is a portable solution in the sense that it exports some standard interface but has varying implementation-dependent backends, then probably looking at what SLY or SWANK do is a good start. In the case of LW you would want the backend to use DSPECs which are how it deals with location information:
> (dspec:dspec-definition-locations '(defun foo))
(((defun foo) :listener))
> (dspec:dspec-definition-locations '(defun needs))
(((defmacro needs)
#P"..."))
> (defclass foo () ())
#<standard-class foo 402000B763>
> (dspec:name-definition-locations dspec:*dspec-classes* 'foo)
(((defclass foo) :listener) ((defun foo) :listener))
PJB on #CommonLisp (IRC.LIBERA.CHAT) greatly assisted and provided the below answer (which I have paraphrased so any mistakes are on my end):
Firstly, CL is a lisp-2 (actually, lisp-∞) and thus there may not be a single definition attached to a symbol. For example, which foo is being referred to in the below?
(deftype foo () (member foo))
(defvar foo 'foo)
(defun foo () foo)
In addition, as an example, symbols may be defined in the REPL (no associated filename), so this is not a straightforward question that can be answered without making assumptions on the intent.
Now, when the behaviour of M-. is satisfactory for your requirements, you could look in slime/swank what the implementation specific API is to do it.
This should point you to swank:find-definition-for-emacs, which may be what you are after:
(swank:find-definitions-for-emacs "foo") #| -->
(("#'foo"
(:location (:file "/private/tmp/foo.lisp")
(:position 50)
(:snippet "(defun foo () foo)")))
("(type foo)"
(:location (:file "/private/tmp/foo.lisp")
(:position 1)
(:snippet "(deftype foo () '(member foo))")))
("(variable foo)"
(:location (:file "/private/tmp/foo.lisp")
(:position 32)
(:snippet "(defvar foo 'foo)"))))
Make sure to load swank as a dependency in your .asd files if you want to rely on the above.
EDIT: I also found the below very useful (and swank has a similar file for most implementations, so just go through each to see their equivalents):
https://github.com/slime/slime/blob/68c58c0194ff03cd147fcec99f0ee90ba9178875/swank/sbcl.lisp#L811
The function call (sb-introspect:find-definition-sources-by-name name type) (name is a symbol, type is a keyword, e.g. :function - refer above link) returns the file in which a definition is stored, assuming you are using SBCL. More (SBCL) details also in:
https://github.com/sbcl/sbcl/blob/master/contrib/sb-introspect/introspect.lisp
If I have a function that I want to be available outside of the current module, I can do the following...
(provide my-function)
Can I do this for a list of functions?
I tried the following...
(define f1 ...) ; body omitted for clarity
(define f2 ...) ; ditto
(define my-funs '(f1 f2))
(provide my-funs)
...but this gave "Unbound identifier in: f1" when I tried it.
Can I do this? Thanks
Update: Just to clarify what I'm trying to do here, I am working my way through Beautiful Racket, and am doing the first tutorial. At the stage where he defines the expander, he adds a handle function to handle the operators...
(define (handle [arg #f])
(cond
[(number? arg) (push-stack! arg)]
[(or (equal? * arg) (equal? + arg))
(define op-result (arg (pop-stack!) (pop-stack!)))
(push-stack! op-result)]))
But then, in order to make this work, he provides both + and *...
(provide + *)
This means that these two operators are hard-coded twice. When adding support for other operators, you'd need to modify the handle function and the provide call. I am trying to work out if we can define a list of operators, and use that in both, so you'd only need to make one modification to support new operators.
No, you can't do this.
You can export a list of functions by using filtered-out and begin-for-syntax (as seen below), but this prevents you from using the list within your code.
Exporting a list
#lang racket
(module fns racket
(require racket/provide)
(define (f1 a) (+ a 1))
(define (f2 a) (+ a 2))
(begin-for-syntax
(define my-funs '(f1 f2)))
(provide
(filtered-out
(λ (name) (and (member (string->symbol name) my-funs) name))
(all-defined-out))))
(require 'fns)
(display (f1 2))
How this works
provide can take any number of provide-spec forms and specifying multiple provide-specs is equivalent to writing multiple provide forms. One of the available provide-spec forms is all-defined-out, which will export all defined symbols in the module (or file if a module isn't explicitly specified).
By requiring racket/provide, we get access to helper functions that can transform and operate on provide-spec forms; filtered-out in particular allows us to run arbitrary code over a provide-spec and returns a valid provide-spec. (The required proc-expr is a function that takes a string (the string value of the exported identifiers) and returns a string or a falsy value. That's why when using member, we wrap it in an and and return the raw name itself. This could also be accomplished with findf: (λ (name) (findf (λ (n) (equal? (string->symbol name) n)) my-funs)))
However, this isn't quite enough, as provide is executed at "compile time", meaning that our list my-funs isn't available yet. To handle that, we need to wrap that definition in begin-for-syntax, which makes the binding available at "compile time" as well. But, by moving my-funs to "compile time", you lose the ability to use my-funs in non-"compile time" code. This means, for instance, you couldn't say (cond ... [(member arg my-funs) ...]):
(define (handle [arg #f])
(cond
[(number? arg) (push-stack! arg)]
[(member arg my-funs)
;; ^--- Error here with "my-funs: unbound identifier"
(define op-result (arg (pop-stack!) (pop-stack!)))
(push-stack! op-result)]))
make-variable-transformer (or make-set!-transformer, as it is called in Racket) can work with identifiers hardcoded in the literal list of the macro definition. The following example with set! invariably comes up when variable transformers are mentioned:
(make-variable-transformer
(λ (stx)
(syntax-case stx (set!)
((set! id _) ...)
(id ...))))
This is nice and all, useful for transparently integrating foreign structures with primitive operations known ahead of time, and it's a bonus that it can work through identifier syntax and rename transformers.
But what I'm wondering is if it's possible to work with syntax dynamically like:
(let-syntax ((# (make-variable-transformer
(λ (stx)
(syntax-case stx ()
((v # i) (vector? #'v) #'(vector-ref v i)))))))
(#(0 1 2) # 1))
=> 1
This doesn't work because the macro call doesn't match the template as syntax-case expects # to be in the initial position since there's no v in the literal list (and it probably assigns # to v pattern variable).
In short: Is it possible to write a syntax transformer that can accomplish this without reader extensions or overriding application, perhaps through a metamacro that rewrites the literal token list of an inner syntax-case (à la Petrofsky extraction)?
NB: The vector example itself is unimportant and I'm not interested in alternative solutions to this exact use-case.
since there's no v in the literal list (and it probably assigns # to v pattern variable).
Not really. set! is a special case that the macro expander handles specifically to make it cooperate with make-variable-transformer. But for other literals, they will fail. E.g.,
(let-syntax ((# (make-variable-transformer
(λ (stx)
(syntax-case stx (v)
((v # i) #'1))))))
(v # 1))
fails with v: unbound identifier.
The second issue with your above code is the side condition (vector? #'v). #'v is a syntax object, so (vector? #'v) will always result in #f. It's unclear what is the right behavior. For example, do you intend for:
(define v (vector 1 2 3))
(v # 1)
to work? If so, a compile-time side condition would be inappropriate, because it's not known if v is a vector at compile-time.
For your main question, the answer is no. It's not possible under the constraints that you imposed. The expansion steps are detailed here, and none of the steps looks beyond the head of the list.
But if we don't care about the constraints. I.e., overriding #%app is OK. It could work.
An issue that you need to think about is, suppose you have (a b c) where b is your kind of macro and a is a regular macro. Who should get the control first? If a should get the control first, you can override #%app to implement this kind of macro. Here's my quick implementation.
#lang racket
(require syntax/parse/define
(only-in racket [#%app racket:#%app])
(for-syntax syntax/apply-transformer))
(begin-for-syntax
(struct my-transformer (t)))
(define-syntax-parser #%app
[(_ x ...)
(define transformer
(for/first ([operand (attribute x)]
#:when (and (identifier? operand)
(my-transformer?
(syntax-local-value operand (λ () #f)))))
(syntax-local-value operand)))
(cond
[transformer (local-apply-transformer
(my-transformer-t transformer)
#'(x ...)
'expression)]
[else #'(racket:#%app x ...)])])
(define-syntax #
(my-transformer
(syntax-parser
[(v _ i) #'(vector-ref v i)])))
(define v (vector 42 1337 1729))
(v # 1) ;=> 1337
Finally, you can always override #%module-begin and simulate the macro expander. It's an overkill solution, but could be appropriate if you want more advanced features, like allowing users to customize precedence so that b is expanded before a.
I'm trying to define a new language in racket, let's call it wibble. Wibble will allow modules to be loaded so it has to translate it's forms to Racket require forms. But I'm having trouble getting require to work when used in a language extension. I eventually tracked down my problems to the following strange behaviour.
Here's my reader which redefines read and read-syntax
=== wibble/lang/reader.rkt ===
#lang racket/base
(provide (rename-out (wibble-read read) (wibble-read-syntax read-syntax)))
(define (wibble-read in)
(wibble-read-syntax #f in))
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in)))
(define (module-name src)
(if (path? src)
(let-values (((base name dir?) (split-path src)))
(string->symbol (path->string (path-replace-suffix name #""))))
'anonymous-module))
(define (read-all src in)
(let loop ((all '()))
(let ((obj (read-syntax src in)))
(if (eof-object? obj)
(reverse all)
(loop (cons obj all))))))
and here's my much simplified language module, this introduces (require racket/base) into each wibble module
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin)) #%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin (require #,(datum->syntax stx 'racket/base)) x ...)))))
With the above code then this wibble code 'works', i.e. there are no errors
#lang wibble
(cons 1 2)
(cons 3 4)
but the following
#lang wibble
(cons 1 2)
gives error message cons: unbound identifier in module in: cons
Really I'm just looking for an explanation as to what going on. I'm sure the difference is related to this from the racket docs (Racket Reference 3.1)
If a single form is provided, then it is partially expanded in a
module-begin context. If the expansion leads to #%plain-module-begin,
then the body of the #%plain-module-begin is the body of the module.
If partial expansion leads to any other primitive form, then the form
is wrapped with #%module-begin using the lexical context of the module
body; this identifier must be bound by the initial module-path import,
and its expansion must produce a #%plain-module-begin to supply the
module body. Finally, if multiple forms are provided, they are wrapped
with #%module-begin, as in the case where a single form does not
expand to #%plain-module-begin.
but even with that I don't understand why having a single form makes any difference, it's seems to be somthing to do with the timing of partial expansion but I'm not really sure. Nor do I understand why Racket treats a single form as a special case.
Incidentally I can fix the problem with a slight modification to my reader
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in) (void)))
Hard-coding a (void) form means I always have more than one form and eveything works.
Sorry for the long post, I'm just looking for some understanding of how this stuff works.
Alright, I think that I've figured it out.
Your intuition is correct in that the problem lies within the timing of the partial expansion of the single-form module body. Inside of your reader.rkt file, you produce a (module ...) form. As the quoted excerpt from your question states, the forms ... portion of this is then treated specially, since there is only one. Let's take a look at an excerpt from the documentation on partial expansion:
As a special case, when expansion would otherwise add an #%app, #%datum, or #%top identifier to an expression, and when the binding turns out to be the primitive #%app, #%datum, or #%top form, then expansion stops without adding the identifier.
I am almost certain that the partial expansion which occurs at this point does something to the cons identifier. This is the one part that I remain unsure of... my gut tells me that what's happening is that the partial expansion is attempting to find the binding for the cons identifier (since it is the first part of the parentheses, the identifier could be bound to a macro which should be expanded, so that needs to be checked) but is unable to, so it throws a tantrum. Note that even if cons has no phase 1 (syntax-expansion time) binding, the macro expander still expects there to be a phase 0 (runtime) binding for the identifier (among other things, this helps the expander remain hygienic). Because all of this partial expansion happens to the body of your (module ...) form (which is done before your (#%module-begin ...) form where you inject the (#%require ...) form), cons has no binding during the expansion, so the expansion, I believe, fails.
Nevertheless, a naive fix for your problem is to rewrite wibble-read-syntax as follows:
(define (wibble-read-syntax src in)
(let* ((read-in (read-all src in))
(in-stx (and (pair? read-in) (car read-in))))
#`(module #,(module-name src) wibble/lang
(require #,(datum->syntax in-stx 'racket/base))
#,#read-in))
You can then remove the (#%require ...) form from your (#%module-begin ...) macro.
That's not, in my opinion, the best way to fix the issue, however. As a matter of cleanliness, hard-coding in a require form like you've done in wibble/lang.rkt would make Eli Barzilay and co. cry. A much simpler way to do what you are trying to do is by updating your lang.rkt file to something like so:
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin))
(except-out (all-from-out racket/base) #%module-begin #%app #%datum #%top)
#%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin x ...)))))
Writing in this convention removes the need for any hard-coded (require ...) forms and prevents subtle bugs like the one you've unearthed from occuring. If you are confused why this works, remember that you've already provided the #%module-begin identifier using this file, which is subsequently bound in all #lang wibble files. In principle, there is no limit on what identifiers you can bind in this fashion. If you would like some further reading, here's a shameless self-advertisement for a blog post I wrote a little while back on the subject.
I hope I've helped.
The problem is with the require (though I'm not sure I 100% understand all the behavior).
(require X) imports bindings from X with the lexical context of #'X. #'X here has the context of stx, which is the entire #'(module-begin x ...), which is not the context you want. You want the context of one of the cons expressions, i.e., one of the #'xs.
Something like this should work:
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
[(_) #'(#%module-begin)]
[(m x y ...)
#`(#%module-begin
(require #,(datum->syntax #'x 'racket/base))
x y ...)])))
Though, as #belph warned, there's probably a more idiomatic way to accomplish what you want.
The behavior of your original program, and as you intuited, likely has to do with module's different treatment of single and multi sub-forms, but I think the "working" case might be an accident and could be a bug in the racket compiler.
Background:
I intend to generate debug messages for the code I'm developing. I wrote a macro to avoid writing the logging calls in each function. I know this limits the ability to generate more customized debug messages, but in return it isolates logging from code. And that is what I'm aiming for. This macro approach has other drawbacks too, for example it limits creation of function bindings to this macro only, but I think I can live with that.
Following is the definition of the macro and an example demonstrating its usage.
(define-syntax (define-func stx)
(syntax-case stx ()
[(define-func (func-name args ...) body1 body2 ...)
(if (and (identifier? #'func-name)
(andmap symbol? (syntax->datum #'(args ...))))
(syntax (define (func-name args ...)
(log-debug (format "Function-name ~a:" (syntax-e #'func-name)) (list args ...))
body1
body2 ...))
(raise-syntax-error 'define-func "not an identifier" stx))]
[else (raise-syntax-error 'define-func "bad syntax" stx)]))
(define-func (last l)
(cond [(null? l) null]
[(null? (rest l)) (first l)]
[else (last (rest l))]))
(define-func (main)
(last (list 1 2 3 4 5 6 7 8 9))
(logger))
log-debug and logger are defined in separate module
The output produced is somewhat like following:
Function-name last:
args:
:-> (7 8 9)
Function-name last:
args:
:-> (8 9)
Function-name last:
args:
:-> (9)
Now I want to make it more readable. By readability I mean to provide some kind of indentation so that the person reading the log could make sense of call flow. For example something like following:
Function-name last:
args:
:-> (7 8 9)
Function-name last:
args:
:-> (8 9)
Function-name last:
args:
:-> (9)
It is easier to figure out who called whom and so forth. I have an idea that can do this. It involves a variable that keeps track of indentation then after logging the function name I will increase the indent and after evaluation of body and before returning the value decrements the value. Something like following:
(define indent 0)
(define-syntax (define-func stx)
(syntax-case stx ()
[ (... ...)
(...
(log-debug ...)
(increment indent)
(let [(retval (body1 body2 ...)]
(decrease indent)
retval))]))
increment and decrease increases and decreases indentation respectively.
Problem:
It works even for function that returns void. I'm not sure whether its the correct behavior. In racket void is a special value, but I'm not sure that creating a binding to void is right way to go.
Is there any better way to achieve the same? If not are there any problems in this design? I'm open to any idea/change as long as they keep the logging and code separate.
thanks for the help!
I have several suggestions for you:
It's probably better to use a parameter instead of a variable, for "global" stuff like your indentation level, since the original value is restored for you at the end of the parameterize expression.
All those raise-syntax-error checks you have in your macro are totally superfluous: syntax-case already provides guards (also known as fenders) that allow you to do any validation of macro "arguments" necessary:
(define-syntax (define-func stx)
(syntax-case stx ()
[(_ (func-name args ...) body1 body2 ...)
(andmap identifier? (syntax->list #'(func-name args ...)))
#'(define (func-name args ...)
(log-debug (format "Function-name ~a:" 'func-name)
(list args ...))
body1
body2 ...)]))
I've also fixed up your code in several places, as you can see above:
I used (_ ...) instead of (define-func ...), since in syntax-case (unlike syntax-rules), the latter will actually bind a pattern variable called define-func, which will affect any recursive macro calls you may want to do (I'll grant that you don't have one here, but it's a good habit to have anyway).
Rather than completely flatten the #'(args ...) in the guard, I just turned it into a list of syntax objects so you can test using identifier?. This is more intention-revealing than testing using symbol?, and allows us to also test func-name in the same expression.
You don't need to use (syntax-e #'func-name) inside the expanded code! Just quote it.