I am facing a strange behaviour. When using SBCL (Common Lisp) within Emacs via Slime, after ASDF is used, slime-autodoc does not propose arguments list for functions/macros any more.
I am working with SBCL 2.2.6 + ASDF 3.3.6, within Emacs 28.1 with Slime 2.27.
End goal: I would like that, when I write code, I am proposed the arguments of the version/macro.
For instance, when I type "(elt ", the mini-buffer should show: "(elt SEQUENCE INDEX)"
For this purpose, I use slime-autodoc, which works fine... except after ASDF is used.
Exemple...
(1) I open SBCL REPL via M-x slime
(2) I define a simple function (defun foo (x) (+ x x))
Autodoc works: if I type "(foo ", it reminds me "(foo X)"
By the way (swank::autodoc '("foo" swank::%cursor-marker%)) returns ("(foo x)" T) and (swank::arglist #'foo) returns (X) NIL
This is OK.
(3) Then I use ASDF to load a personal basic project, with nothing specific (functions working on integers).
(swank:operate-on-system-for-emacs "toolbox" 'load-op)
(4) Then I define a new simple function (defun bar (y) (+ y y))
Autodoc does not work any more:
when I type "(bar ", it shows "(bar)" in mini-buffer instead of "(bar Y")
With no surprise: (swank::autodoc '("bar" swank::%cursor-marker%)) returns ("(bar)" T) instead of ("(bar Y)" T) and (swank::arglist #'bar) returns () NIL, instead of (Y) NIL
This is not OK.
Important note: autodoc still works for 'foo' function previously defined in (2). It still proposes argument 'X' for it.
I cannot interpret it. It is as if foo and bar were not in the same 'space', as if the use of ASDF has changed the 'space' where things are created. However, we are apparently still in CL-USER package.
AND and OR are macros and since macros aren't first class in scheme/racket they cannot be passed as arguments to other functions. A partial solution is to use and-map or or-map. Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function? Are there any languages that have first class macros?
In general, no. Consider that let is (or could be) implemented as a macro on top of lambda:
(let ((x 1))
(foo x))
could be a macro that expands to
((lambda (x) (foo x)) 1)
Now, what would it look like to convert let to a function? Clearly it is nonsense. What would its inputs be? Its return value?
Many macros will be like this. In fact, any macro that could be routinely turned into a function without losing any functionality is a bad macro! Such a macro should have been a function to begin with.
I agree with #amalloy. If something is written as a macro, it probably does something that functions can't do (e.g., introduce bindings, change evaluation order). So automatically converting arbitrary macro into a function is a really bad idea even if it is possible.
Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function?
No, but it is somewhat doable to write a macro that would take some macro and turn it into a function.
#lang racket
(require (for-syntax racket/list))
(define-syntax (->proc stx)
(syntax-case stx ()
[(_ mac #:arity arity)
(with-syntax ([(args ...) (generate-temporaries (range (syntax-e #'arity)))])
#'(λ (args ...) (mac args ...)))]))
((->proc and #:arity 2) 42 12)
(apply (->proc and #:arity 2) '(#f 12))
((->proc and #:arity 2) #f (error 'not-short-circuit))
You might also be interested in identifier macro, which allows us to use an identifier as a macro in some context and function in another context. This could be used to create a first class and/or which short-circuits when it's used as a macro, but could be passed as a function value in non-transformer position.
On the topic of first class macro, take a look at https://en.wikipedia.org/wiki/Fexpr. It's known to be a bad idea.
Not in the way you probably expect
To see why, here is a way of thinking about macros: A macro is a function which takes a bit of source code and turns it into another bit of source code: the expansion of the macro. In other words a macro is a function whose domain and range are source code.
Once the source code is fully expanded, then it's fed to either an evaluator or a compiler. Let's assume it's fed to a compiler because it makes the question easier to answer: a compiler itself is simply a function whose domain is source code and whose range is some sequence of instructions for a machine (which may or may not be a real machine) to execute. Those instructions might include things like 'call this function on these arguments'.
So, what you are asking is: can the 'this function' in 'call this function on these arguments' be some kind of macro? Well, yes, it could be, but whatever source code it is going to transform certainly can not be the source code of the program you are executing, because that is gone: all that's left is the sequence of instructions that was the return value of the compiler.
So you might say: OK, let's say we disallow compilers: can we do it now? Well, leaving aside that 'disallowing compilers' is kind of a serious limitation, this was, in fact, something that very old dialects of Lisp sort-of did, using a construct called a FEXPR, as mentioned in another answer. It's important to realise that FEXPRs existed because people had not yet invented macros. Pretty soon, people did invent macros, and although FEXPRs and macros coexisted for a while – mostly because people had written code which used FEXPRs which they wanted to keep running, and because writing macros was a serious pain before things like backquote existed – FEXPRs died out. And they died out because they were semantically horrible: even by the standards of 1960s Lisps they were semantically horrible.
Here's one small example of why FEXPRs are so horrible: Let's say I write this function in a language with FEXPRs:
(define (foo f g x)
(apply f (g x)))
Now: what happens when I call foo? In particular, what happens if f might be a FEXPR?. Well, the answer is that I can't compile foo at all: I have to wait until run-time and make some on-the-fly decision about what to do.
Of course this isn't what these old Lisps with FEXPRs probably did: they would just silently have assumed that f was a normal function (which they would have called an EXPR) and compiled accordingly (and yes, even very old Lisps had compilers). If you passed something which was a FEXPR you just lost: either the thing detected that, or more likely it fall over horribly or gave you some junk answer.
And this kind of horribleness is why macros were invented: macros provide a semantically sane approach to processing Lisp code which allows (eventually, this took a long time to actually happen) minor details like compilation being possible at all, code having reasonable semantics and compiled code having the same semantics as interpreted code. These are features people like in their languages, it turns out.
Incidentally, in both Racket and Common Lisp, macros are explicitly functions. In Racket they are functions which operate on special 'syntax' objects because that's how you get hygiene, but in Common Lisp, which is much less hygienic, they're just functions which operate on CL source code, where the source code is simply made up of lists, symbols &c.
Here's an example of this in Racket:
> (define foo (syntax-rules ()
[(_ x) x]))
> foo
#<procedure:foo>
OK, foo is now just an ordinary function. But it's a function whose domain & range are Racket source code: it expects a syntax object as an argument and returns another one:
> (foo 1)
; ?: bad syntax
; in: 1
; [,bt for context]
This is because 1 is not a syntax object.
> (foo #'(x 1))
#<syntax:readline-input:5:10 1>
> (syntax-e (foo #'(x 1)))
1
And in CL this is even easier to see: Here's a macro definition:
(defmacro foo (form) form)
And now I can get hold of the macro's function and call it on some CL source code:
> (macro-function 'foo)
#<Function foo 4060000B6C>
> (funcall (macro-function 'foo) '(x 1) nil)
1
In both Racket and CL, macros are, in fact, first-class (or, in the case of Racket: almost first-class, I think): they are functions which operate on source code, which itself is first-class: you can write Racket and CL programs which construct and manipulate source code in arbitrary ways: that's what macros are in these languages.
In the case of Racket I have said 'almost first-class', because I can't see a way, in Racket, to retrieve the function which sits behind a macro defined with define-syntax &c.
I've created something like this in Scheme, it's macro that return lambda that use eval to execute the macro:
(define-macro (macron m)
(let ((x (gensym)))
`(lambda (,x)
(eval `(,',m ,#,x)))))
Example usage:
;; normal eval
(define x (map (lambda (x)
(eval `(lambda ,#x)))
'(((x) (display x)) ((y) (+ y y)))))
;; using macron macro
(define x (map (macron lambda)
'(((x) (display x)) ((y) (+ y y)))))
and x in both cases is list of two functions.
another example:
(define-macro (+++ . args)
`(+ ,#args))
((macron +++) '(1 2 3))
Is it possible to expand a macro at run time in a compiled lisp executable? I would expect it to be impossible as macro expansion can only happen pre-compilation yet when i call macroexpand in compiled code i get output.
A macro is a function that's normally called automatically during compilation or evaluation, and whose return value is then compiled or evaluated in place of the original expression.
But since it's just a function, there's nothing preventing it from being called explicitly during run time as well, and that's what MACROEXPAND and MACROEXPAND-1 do.
It's roughly equivalent to:
(defun macroexpand-1 (form &optional env)
(if (and (listp form) (car form)) ;; list expression
(let ((macfun (macro-function (car form)))
(if macfun
(funcall macfun form env)
form))
form))
(Note that this definition doesn't handle symbol macros or use *MACROEXPAND-HOOK*, to keep it simple.)
It's possible to use EVAL-WHEN when defining the macro to make the macro definition only available in the compilation environment. If you do that, trying to expand at run time will fail.
In Lisp, the terms "run time" and "compile time" are situations in which a particular piece of code is being processed, not absolutes like in some static languages. If we evaluate (compile nil '(lambda ())), this is the compile function's run-time, but the lambda form's compile time: both times are happening at the same time.
The entire language is available in all situations. When you build a self-contained executable, that image contains not only support for expanding macros, but for compiling code. Your Lisp application can call compile-file to compile Lisp source to object form, and load to load the resulting object code.
The process of removing garbage, and unused functionality from a Lisp application image to make it smaller is called "tree shaking". If you don't want the compiler or macro expander in your application, find out whether/how they can be removed with your implementation's tree shaking support.
I want to achieve something like this:
(setq my-global-keybindings
'(([?\C-x ?\C-d] . dired)
([?\C-x ?\C-b] . ibuffer)
([?\C-x b] . ivy-switch-buffer)))
(apply #'bind-keys* my-global-keybindings)
But bind-keys* is a macro here.
In the specific case of binding keys, I think the better option is to look for a function that replaces that bind-keys* macro (I don't see any justification why it should be a macro rather than a function).
But as for the more general question, here's how I'd do it:
(defmacro my-multi-bind-keys (bindings)
(macroexp-progn
(mapcar (lambda (binding)
`(bind-keys* ,#binding))
bindings)))
(my-multi-bind-keys (([?\C-x ?\C-d] dired)
([?\C-x ?\C-b] ibuffer)
([?\C-x b] ivy-switch-buffer)))
Note that using a setq like you did is problematic: macros needs to be expanded during compilation but the setq should not be executed by the compiler (it should be compiled by the compiler to be run when the code is later executed) so there's no sound way for the macro to get access to the var's value (at least until time-travel is made to work).
Given this situation:
(defmacro mac1 ...)
(defun func1 ()
(mac1 ...))
Redefinition of mac1 in the live environment will not affect func1 until it is itself recompiled.
Is there a way to get emacs or lisp to automatically recompile func1 whenever mac1 is recompiled?
Something like this in the source would be acceptable:
(watch
(defmacro mac1 ...))
(on-signal (mac1)
(defun func1 ...))
Not that hard to implement, but I would rather avoid wheel reinvention.
Macros are not necessarily compiled. If you are using SBCL (see manual), there is a variable named sb-ext:*evaluator-mode* which can be set to :interpret so that macro are expanded during evaluation. Other implementations might provide something similar. That allows you to change the definitions of macros without recompiling call sites like you already do with functions.
Alternatively, slime defines a function named slime-who-macroexpands. You'd have to dig a little bit to see how it works and maybe exploit it, either in the Common Lisp environment (swank) or the emacs side.
For example in LispWorks you can do the following. Probably SBCL has a similar facility.
Let's say we have this:
(defmacro foo ()
`(list 1 2 3))
(defun bar () (first (foo)))
(defun baz () (second (foo)))
Now you can ask who calls foo:
CL-USER 11 > (who-calls 'foo)
(BAZ BAR)
This makes it easy to re-compile both functions:
CL-USER 12 > (mapcar 'compile (who-calls 'foo))
;;;*** Warning in BAZ: The definition of BAZ is already compiled.
;;;*** Warning in BAR: The definition of BAR is already compiled.
(BAZ BAR)
Since LispWorks keeps a who-calls database, one can recompile all functions which directly use and depend on other functions/macros.
The editor has the command Edit Callers and Continue Tags Search to find the callers and then to recompile them manually. It should be easy/possible to write an editor command which recompiles all callers.