I have a question concerning dynamically built functions (or something comparable). In Java I can write some Source programmatically into a String, compile this string and execute it like a function several times.
Imagine I have some genetic algorithm to create the best code for taking n input parameters, calculate them according the genome and returning m output parameters. So I wonder if it's possible (and I'm quite sure it is), to create a list of lists of lists .... containing the function, and then call this function some thousand times with different input parameters to calculate the error rate.
What I do need now is an example, how to create such a list programmatically and how to use it. Currently I'm totally stuck.
Any references or examples warmly welcomed.
Lisp code is data: lists, symbols, numbers, ...
(defun foo () 42)
Lisp has functions like list and +. You can use it:
(list 'defun
'foo
'()
(+ 25 17))
How do you compile a function? Eval the code and compile it.
(compile (eval my-function-definition))
So put together:
CL-USER 10 > (compile (eval (list 'defun 'foo '() (+ 25 17))))
FOO
NIL
NIL
CL-USER 11 > (foo)
42
Is it really compiled?
CL-USER 11 > (disassemble 'foo)
424000EA3C:
0: 49396275 cmpq [r10+75], rsp
4: 7720 ja L1
6: 4883F900 cmpq rcx, 0
10: 751A jne L1
12: 4157 push r15
14: 55 push rbp
15: 4889E5 moveq rbp, rsp
18: 4989DF moveq r15, rbx
21: BF50010000 move edi, 150
26: B901000000 move ecx, 1
31: 4889EC moveq rsp, rbp
34: 5D pop rbp
35: 415F pop r15
37: C3 ret
L1: 38: 41FFA6E7020000 jmp [r14+2E7] ; SYSTEM::*%WRONG-NUMBER-OF-ARGUMENTS-STUB
45: 90 nop
46: 90 nop
47: 90 nop
48: 90 nop
49: 90 nop
50: 90 nop
51: 90 nop
Seems so...
As Rainer Joswig said, you should read some Lisp book (particularily the chapters about list manipulation and macros), but here's a very simple example using compile:
(defun test (op number)
(let ((func (compile nil `(lambda (y)
(,op y y)))))
(format t "The function is ~:[not compiled~;compiled~].~%"
(compiled-function-p func))
(funcall func number)))
(test '+ 5) ; => 10
(test '* 5) ; => 25
This constructs the lambda expression using backquote syntax, but you could just build it with regular list manipulation operations (push, cons and such) as well.
Related
I've been solving some challenges at codesignal.com using C-Lisp to learn it and I've been avoiding using loops to make lisp style code.
In this challenge called alternatingSums (which gives you an int array a that can be very large and ask you to return an array/list {sumOfEvenIndexedElements, sumOfOddIndexedElements}) i have been receiving stack overflow error with this code:
(defun alternatingSums(a &optional (index 0) (accumulated '(0 0)))
(cond ((= index (length a))
accumulated)
((evenp index)
(alternatingSums
a
(1+ index)
`(,(+ (svref a index ) (elt accumulated 0)) ,(elt accumulated 1)))
)
((oddp index)
(alternatingSums
a
(1+ index)
`(,(elt accumulated 0) ,(+ (svref a index ) (elt accumulated 1))))
)
)
)
isn't it tail-recursive or can tail-recursive functions still get stack-overflow?
Recursive functions which call themselves from tail position can lead to stack overflow; language implementations must support some form of tail call elimination to avoid the problem.
I've been avoiding using loops to make lisp style code.
Common Lisp does not require that implementations do tail call elimination, but Scheme implementations must do so. It is idiomatic in Scheme to use recursion for iteration, but in Common Lisp it is idiomatic to use other iteration devices unless recursion provides a natural solution for the problem at hand.
Although Common Lisp implementations are not required to do tail call elimination, many do. Clisp does support limited tail call elimination, but only in compiled code, and only for self-recursive tail calls. This is not well-documented, but there is some discussion to be found here #Renzo. OP posted code will be subject to tail call elimination when compiled in Clisp since the function alternatingSums calls itself from tail position. This covers most cases in which you may be interested in tail call elimination, but note that tail call elimination is then not done for mutually recursive function definitions in Clisp. See the end of this answer for an example.
Defining a function from the REPL, or loading a definition from a source file, will result in interpreted code. If you are working in a development environment like SLIME, it is easy to compile: from the source file buffer either do Ctrl-c Ctrl-k to compile the whole file and send it to the REPL, or place the point inside of or immediately after a function definition and do Ctrl-c Ctrl-c to compile a single definition and send it to the REPL.
You could also compile the source file before loading it, e.g. (load (compile-file "my-file.lisp")). Or you could load the source file, and compile a function after that, e.g. (load "my-file.lisp"), then (compile 'my-function).
As already mentioned, it would probably be more likely that idiomatic Common Lisp code would not use recursion for this sort of function anyway. Here is a definition using the loop macro that some would find more clear and concise:
(defun alternating-sums (xs)
(loop for x across xs
and i below (length xs)
if (evenp i) sum x into evens
else sum x into odds
finally (return (list evens odds))))
The Case of Mutually Recursive Functions in Clisp
Here is a simple pair of mutually recursive function definitions:
(defun my-evenp (n)
(cond ((zerop n) t)
((= 1 n) nil)
(t (my-oddp (- n 1)))))
(defun my-oddp (n)
(my-evenp (- n 1)))
Neither function calls itself directly, but my-evenp has a call to my-oddp in tail position, and my-oddp has a call to my-evenp in tail position. One would like for these tail calls to be eliminated to avoid blowing the stack for large inputs, but Clisp does not do this. Here is the disassembly:
CL-USER> (disassemble 'my-evenp)
Disassembly of function MY-EVENP
14 byte-code instructions:
0 (LOAD&PUSH 1)
1 (CALLS2&JMPIF 172 L16) ; ZEROP
4 (CONST&PUSH 0) ; 1
5 (LOAD&PUSH 2)
6 (CALLSR&JMPIF 1 47 L19) ; =
10 (LOAD&DEC&PUSH 1)
12 (CALL1 1) ; MY-ODDP
14 (SKIP&RET 2)
16 L16
16 (T)
17 (SKIP&RET 2)
19 L19
19 (NIL)
20 (SKIP&RET 2)
CL-USER> (disassemble 'my-oddp)
Disassembly of function MY-ODDP
3 byte-code instructions:
0 (LOAD&DEC&PUSH 1)
2 (CALL1 0) ; MY-EVENP
4 (SKIP&RET 2)
Compare with a tail recursive function that calls itself. Here there is no call to factorial in the disassembly, but instead a jump instruction has been inserted: (JMPTAIL 2 5 L0).
(defun factorial (n acc)
(if (zerop n) acc
(factorial (- n 1) (* n acc))))
CL-USER> (disassemble 'factorial)
Disassembly of function FACTORIAL
11 byte-code instructions:
0 L0
0 (LOAD&PUSH 2)
1 (CALLS2&JMPIF 172 L15) ; ZEROP
4 (LOAD&DEC&PUSH 2)
6 (LOAD&PUSH 3)
7 (LOAD&PUSH 3)
8 (CALLSR&PUSH 2 57) ; *
11 (JMPTAIL 2 5 L0)
15 L15
15 (LOAD 1)
16 (SKIP&RET 3)
Some Common Lisp implementations do support tail call elimination for mutually recursive functions. Here is the disassembly of my-oddp from SBCL:
;; SBCL
; disassembly for MY-ODDP
; Size: 40 bytes. Origin: #x52C8F9E4 ; MY-ODDP
; 9E4: 498B4510 MOV RAX, [R13+16] ; thread.binding-stack-pointer
; 9E8: 488945F8 MOV [RBP-8], RAX
; 9EC: BF02000000 MOV EDI, 2
; 9F1: 488BD3 MOV RDX, RBX
; 9F4: E8771B37FF CALL #x52001570 ; GENERIC--
; 9F9: 488B5DF0 MOV RBX, [RBP-16]
; 9FD: B902000000 MOV ECX, 2
; A02: FF7508 PUSH QWORD PTR [RBP+8]
; A05: E9D89977FD JMP #x504093E2 ; #<FDEFN MY-EVENP>
; A0A: CC10 INT3 16 ; Invalid argument count trap
This is a little harder to read than the previous examples because SBCL compiles to assembly language instead of byte code, but you can see that a jump instruction has been substituted for the call to my-evenp:
; A05: E9D89977FD JMP #x504093E2 ; #<FDEFN MY-EVENP>
Common Lisp compilers are not required to optimize tail calls. Many do, but not all implementations compile your code by default; you have to compile the file using compile-file, or else the function individually with (compile 'alternatingsums).
CLISP contains both an interpreter, which processes the nested-list representation of Lisp source code, and a byte code compiler. The compiler supports tail recursion, whereas the interpreter doesn't:
$ clisp -q
[1]> (defun countdown (n) (unless (zerop n) (countdown (1- n))))
COUNTDOWN
[2]> (countdown 10000000)
*** - Program stack overflow. RESET
[3]> (compile 'countdown)
COUNTDOWN ;
NIL ;
NIL
[4]> (countdown 10000000)
NIL
Peeking under the hood a little bit:
[5]> (disassemble 'countdown)
Disassembly of function COUNTDOWN
1 required argument
0 optional arguments
No rest parameter
No keyword parameters
8 byte-code instructions:
0 L0
0 (LOAD&PUSH 1)
1 (CALLS2&JMPIF 172 L10) ; ZEROP
4 (LOAD&DEC&PUSH 1)
6 (JMPTAIL 1 3 L0)
10 L10
10 (NIL)
11 (SKIP&RET 2)
NIL
We can see that the virtual machine has a JMPTAIL primitive.
Another approach to tail calling is via macros. Years ago, I hacked up a macro called tlet which lets you define (what look like) lexical functions using syntax similar to labels. The tlet construct compiles to a tagbody form in which the tail calls among the functions are go forms. It does not analyze calls for being in tail position: all calls are unconditional transfers that do not return regardless of their position in the syntax. The same source file also provides a trampoline-based implementation of tail calling among global functions.
Here is tlet in CLISP; note: the expression has not been compiled, yet it doesn't run out of stack:
$ clisp -q -i tail-recursion.lisp
;; Loading file tail-recursion.lisp ...
;; Loaded file tail-recursion.lisp
[1]> (tlet ((counter (n) (unless (zerop n) (counter (1- n)))))
(counter 100000))
NIL
tlet is not an optimizer. The call to counter is semantically a goto, always; it's not a procedure call that can sometimes turn into a goto under the right circumstances. Watch what happens when we add a print:
[2]> (tlet ((counter (n) (unless (zerop n) (print (counter (1- n))))))
(counter 100000))
NIL
That's right; nothing! (counter (1- n)) never returns, and so print is never called.
I've had this question more than once before.
Generic Question
Is it possible to transparently locally shadow a function f with a wrapper of it with the same name f?
I.e., how to locally have (f wrapped-args...) expand to (f args...)?
Flet seems to let us do so, but has limitations, namely, the resulting wrapper is not setf-able. Is it possible to do so without resorting to flet?
Ideally there would be a macro that lets us write the "wrapped" f calls and it expands the code to the original "non-wrapped" f call.
At first I believed macrolet could be that, for it says in the documentation that it first expands the macro and then applies setf on the expanded form, but I'm not being able to use it (keep reading below).
Motivation
This is useful in contexts where some paremeters are implicit and should not be repeated over and over, for more DRY code.
In my previous question (let-curry) there's a particular example of that. Attempting to "automatically" assign some of the parameters of the functions (let-curry).
Caveats of flet
I got some excellent answers there, however, I hit some limitations. By resorting to flet to accomplish such local "shadowing" of the function name to a wrapper over it, such wrappers are not setf-able, thus, such wrappers cannot be used as flexibly as the original function, only to read values, not write.
Concrete question
With the link above, how can one write the macro flet-curry and have the wrapper functions be setf-able?
Bonus: Can that macro expand the wrapped calls to the original ones with 0 runtime overhead?
I tried taking the selected answer in that post and using macrolet instead of flet to no avail.
Thank you!
UPDATE
I was asked to give a concrete example for this generic question.
Comments of wishes in the code:
(locally (declare (optimize safety))
(defclass scanner ()
((source
:initarg :source
:accessor source
:type string)
(tokens
:initform nil
:accessor tokens
:type list)
(start
:initform 0
:accessor start
:type integer)
(current
:initform 0
:accessor current
:type integer)
(line
:initform 1
:accessor line
:type integer))
(:metaclass checked-class)))
(defun lox-string (scanner)
"Parse string into a token and add it to tokens"
;; Any function / defmethod / accessor can be passed to let-curry
;; 1. I'd like to add the accessor `line` to this list of curried methods:
(let-curry scanner (peek at-end-p advance source start current)
(loop while (and (char/= #\" (peek))
(not (at-end-p)))
do
;; 2. but cannot due to the incf call which calls setf:
(if (char= #\Newline (peek)) (incf (line scanner))
(advance)))
(when (at-end-p)
(lox.error::lox-error (line scanner) "Unterminated string.")
(return-from lox-string nil))
(advance) ;; consume closing \"
(add-token scanner 'STRING (subseq (source)
(1+ (start))
(1- (current))))))
Meaning I'd like let-curry to transform any call of the curried functions in that block from
(f arg1 arg2 ...)
to
(f scanner arg1 arg2 ...)
in place, as if I'd written the latter form and not the former in the source code. If that were the case with some ?macro?, then it would be setf-able by design.
It seems a macro would be the right tool for this but I don't know how.
Thanks again :)
P.S.: If you need access to the full code it's here: https://github.com/AlbertoEAF/cl-lox (scanner.lisp)
Binding with macrolet is not trivial since:
Once you bind f in a macrolet, if it expands as (f ...), you are going to have infinite macroexpansion.
Also, you could expand the macrolet as (apply #'f ...) (which is great, since APPLY can be a SETF place1), but then you have errors because #'f is bound to a local macro, not the original function. If, however, you first evaluate #'f, bind it to a hidden variable, then define a macro that applies the variable's value, SETF APPLY complains (at least in SBCL) that the function must not be a symbol (ie. dynamically computed).
1: For example (let ((x (list 0 1 2))) (prog1 x (setf (apply #'second list ()) 9)))
But you don't need macrolet, since you can bind SETF functions in FLET; here is what you could write manually if you wanted to redefine some functions locally:
(defun lox-string (scanner)
(flet
((peek () (peek scanner))
(at-end-p () (at-end-p scanner))
(advance () (advance scanner))
(line () (line scanner))
((setf line) (n) (setf (line scanner) n))
(source () (source scanner))
(start () (start scanner))
(current () (current scanner)))
(loop
while (and (char/= #\" (peek))
(not (at-end-p)))
do
(if (char= #\Newline (peek))
(incf (line))
(advance)))
(when (at-end-p)
(error "Unterminated string at line ~a" (line)))
(advance)
(add-token scanner 'STRING (subseq (source)
(1+ (start))
(1- (current))))))
Expand as FLET
The following macro expands as inlinable flets and handles SETF functions in a special way, since the first argument is always the value being set:
(defmacro with-curry ((&rest fn-specs) prefix &body body)
(loop
with args = (gensym)
and n = (gensym)
and prefix = (alexandria:ensure-list prefix)
for f in fn-specs
collect (if (and (consp f) (eq 'setf (first f)))
`(,f (,n &rest ,args) (apply #',f ,n ,#prefix ,args))
`(,f (&rest ,args) (apply #',f ,#prefix ,args)))
into flets
finally (return
`(flet ,flets
(declare (inline ,#fn-specs))
,#body))))
For example:
(let ((scanner (make-instance 'scanner)))
(with-curry (start (setf start)) scanner
(setf (start) (+ (start) 10))))
This macroexpands as:
(LET ((SCANNER (MAKE-INSTANCE 'SCANNER)))
(FLET ((START (&REST #:G849)
(APPLY #'START SCANNER #:G849))
((SETF START) (#:G850 &REST #:G849)
(APPLY #'(SETF START) #:G850 SCANNER #:G849)))
(DECLARE (INLINE START (SETF START)))
(LET* ((#:NEW1 (+ (START) 10)))
(FUNCALL #'(SETF START) #:NEW1))))
Inlining FLET
The inline declaration is a request (the compiler may ignore it) to replace each calls to the function by its body (parameters are substituted by the function call arguments; it looks like β-reduction in lambda-calculus).
When the compiler recognizes it, it is as-if you defined the code as a macrolet, removing the need to call a function. When inlining is in effect, apply will see during compilation both the function object to call and all the arguments, so the compiler can emit code as-if you wrote directly all parameters.
Let's test that with SBCL, first with a notinline declaration to explicitly prevent inlining:
(disassemble
(lambda ()
(declare (optimize (debug 0) (safety 0)))
(flet ((p (&rest args) (apply #'print args)))
(declare (notinline p))
(p 0) (p 1))))
The output of the disassembler is a bit long, and I won't claim I understand what happens exactly; there is a first segment that apparently allocates memory (for the local function?):
; disassembly for (LAMBDA ())
; Size: 187 bytes. Origin: #x53F0A5B6 (segment 1 of 2) ; (LAMBDA ())
; 5B6: 49896D28 MOV [R13+40], RBP ; thread.pseudo-atomic-bits
; 5BA: 4D8B5D68 MOV R11, [R13+104] ; thread.alloc-region
; 5BE: 498D4B10 LEA RCX, [R11+16]
; 5C2: 493B4D70 CMP RCX, [R13+112]
; 5C6: 0F878C000000 JNBE L8
; 5CC: 49894D68 MOV [R13+104], RCX ; thread.alloc-region
; 5D0: L0: 498D4B07 LEA RCX, [R11+7]
; 5D4: 49316D28 XOR [R13+40], RBP ; thread.pseudo-atomic-bits
; 5D8: 7402 JEQ L1
; 5DA: CC09 INT3 9 ; pending interrupt trap
; 5DC: L1: C7410117001050 MOV DWORD PTR [RCX+1], #x50100017 ; NIL
; 5E3: 488BDD MOV RBX, RBP
; 5E6: 488D5424F0 LEA RDX, [RSP-16]
; 5EB: 4883EC10 SUB RSP, 16
; 5EF: 48891A MOV [RDX], RBX
; 5F2: 488BEA MOV RBP, RDX
; 5F5: E82F000000 CALL L4
; 5FA: 49896D28 MOV [R13+40], RBP ; thread.pseudo-atomic-bits
; 5FE: 4D8B5D68 MOV R11, [R13+104] ; thread.alloc-region
; 602: 498D4B10 LEA RCX, [R11+16]
; 606: 493B4D70 CMP RCX, [R13+112]
; 60A: 775A JNBE L9
; 60C: 49894D68 MOV [R13+104], RCX ; thread.alloc-region
; 610: L2: 498D4B07 LEA RCX, [R11+7]
; 614: 49316D28 XOR [R13+40], RBP ; thread.pseudo-atomic-bits
; 618: 7402 JEQ L3
; 61A: CC09 INT3 9 ; pending interrupt trap
; 61C: L3: C641F902 MOV BYTE PTR [RCX-7], 2
; 620: C7410117001050 MOV DWORD PTR [RCX+1], #x50100017 ; NIL
; 627: EB03 JMP L5
; 629: L4: 8F4508 POP QWORD PTR [RBP+8]
... followed by a second segment which looks like it actually defines and call the local function (?):
; Origin #x53F0A62C (segment 2 of 2) ; (FLET P)
; 62C: L5: 488BF4 MOV RSI, RSP
; 62F: L6: 4881F917001050 CMP RCX, #x50100017 ; NIL
; 636: 7412 JEQ L7
; 638: FF71F9 PUSH QWORD PTR [RCX-7]
; 63B: 488B4901 MOV RCX, [RCX+1]
; 63F: 8D41F9 LEA EAX, [RCX-7]
; 642: A80F TEST AL, 15
; 644: 74E9 JEQ L6
; 646: CC0A INT3 10 ; cerror trap
; 648: 06 BYTE #X06 ; BOGUS-ARG-TO-VALUES-LIST-ERROR
; 649: 04 BYTE #X04 ; RCX
; 64A: L7: 488B053FFFFFFF MOV RAX, [RIP-193] ; #<FUNCTION PRINT>
; 651: FF2425A8000052 JMP QWORD PTR [#x520000A8] ; TAIL-CALL-VARIABLE
; 658: L8: 6A11 PUSH 17
; 65A: FF142550000052 CALL QWORD PTR [#x52000050] ; CONS->R11
; 661: E96AFFFFFF JMP L0
; 666: L9: 6A11 PUSH 17
; 668: FF142550000052 CALL QWORD PTR [#x52000050] ; CONS->R11
; 66F: EB9F JMP L2
Anyway, it is very different from the disassembly output of the inline case:
(disassemble
(lambda ()
(declare (optimize (debug 0) (safety 0)))
(flet ((p (&rest args) (apply #'print args)))
(declare (inline p))
(p 0) (p 1))))
This prints:
; disassembly for (LAMBDA ())
; Size: 45 bytes. Origin: #x540D3CF6 ; (LAMBDA ())
; CF6: 4883EC10 SUB RSP, 16
; CFA: 31D2 XOR EDX, EDX
; CFC: B902000000 MOV ECX, 2
; D01: 48892C24 MOV [RSP], RBP
; D05: 488BEC MOV RBP, RSP
; D08: B8C2283950 MOV EAX, #x503928C2 ; #<FDEFN PRINT>
; D0D: FFD0 CALL RAX
; D0F: BA02000000 MOV EDX, 2
; D14: B902000000 MOV ECX, 2
; D19: FF7508 PUSH QWORD PTR [RBP+8]
; D1C: B8C2283950 MOV EAX, #x503928C2 ; #<FDEFN PRINT>
; D21: FFE0 JMP RAX
The above is shorter, and directly calls print.
It is equivalent to the disassembly where inlining is done by hand:
(disassemble (lambda ()
(declare (optimize (debug 0) (safety 0)))
(print 0) (print 1)))
; disassembly for (LAMBDA ())
; Size: 45 bytes. Origin: #x540D4066 ; (LAMBDA ())
; 66: 4883EC10 SUB RSP, 16
; 6A: 31D2 XOR EDX, EDX
; 6C: B902000000 MOV ECX, 2
; 71: 48892C24 MOV [RSP], RBP
; 75: 488BEC MOV RBP, RSP
; 78: B8C2283950 MOV EAX, #x503928C2 ; #<FDEFN PRINT>
; 7D: FFD0 CALL RAX
; 7F: BA02000000 MOV EDX, 2
; 84: B902000000 MOV ECX, 2
; 89: FF7508 PUSH QWORD PTR [RBP+8]
; 8C: B8C2283950 MOV EAX, #x503928C2 ; #<FDEFN PRINT>
; 91: FFE0 JMP RAX
While I have not been following this in detail, note that setf does not have to be a problem here.
Consider this:
(defclass grunga-object ()
;; grunga objects have grungas, but they may be unbound
((grunga :accessor object-grunga :initarg :grunga)))
(defgeneric object-has-valid-grunga-p (o)
;; Does some object have a valid grunga?
(:method (o)
nil))
(defmethod object-has-valid-grunga-p ((o grunga-object))
;; grunga object's grungas are valid if they are bound
(slot-boundp o 'grunga))
(defun grunga (object &optional (default 'grunga))
;; get the grunga of a thing
(if (object-has-valid-grunga-p object)
(object-grunga object)
default))
(defun (setf grunga) (new object)
;; set the grunga of a thing
(setf (object-grunga object) new))
Now this will work fine:
(defun foo (o)
(flet ((grunga (object)
(grunga object 3)))
(setf (grunga o) (grunga o))
o))
and (grunga (foo (make-instance 'grunga-object))) will return 3. In this case the local grunga function calls the global one, while (setf grunga) – a different function – is called directly.
If you want to override the (setf grunga) function you can do that as well:
(defun bar (o &optional (exploded-value 'exploded))
(flet ((grunga (object)
(grunga object 3))
((setf grunga) (new object &optional (exploding t))
(setf (grunga object) (if exploding (cons exploded-value new) new))))
(setf (grunga o t) (grunga o))
o))
And now (grunga (bar (make-instance 'grunga-object) 'crunched)) is (cruched . 3). In this case both grunga and (setf grunga) are local functions which call their global counterparts.
Note that this may be more complicated with setf forms defined by define-setf-*: I never use those if I can possibly avoid it.
I am using SBCL Common Lisp. I am not an expert, but I like to think I understand it well enough to muddle along. However, I have recently encountered a strange problem with defmacro.
Why does the following code not compile, and how do I change it to make it compile?
(let ((a nil))
(defmacro testmacro ())
(testmacro))
The error is:
Unhandled UNDEFINED-FUNCTION in thread #<SB-THREAD:THREAD "main thread" RUNNING
{100399C9A3}>:
The function COMMON-LISP-USER::TESTMACRO is undefined.
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {100399C9A3}>
0: ((LAMBDA NIL :IN SB-DEBUG::FUNCALL-WITH-DEBUG-IO-SYNTAX))
1: (SB-IMPL::CALL-WITH-SANE-IO-SYNTAX #<CLOSURE (LAMBDA NIL :IN SB-DEBUG::FUNCALL-WITH-DEBUG-IO-SYNTAX) {1003A0EA8B}>)
2: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<CLOSURE (LAMBDA NIL :IN SB-DEBUG::FUNCALL-WITH-DEBUG-IO-SYNTAX) {1003A0EA5B}>)
3: (PRINT-BACKTRACE :STREAM #<SB-SYS:FD-STREAM for "standard error" {10039A22B3}> :START 0 :FROM :INTERRUPTED-FRAME :COUNT NIL :PRINT-THREAD T :PRINT-FRAME-SOURCE NIL :METHOD-FRAME-STYLE NIL)
4: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<UNDEFINED-FUNCTION TESTMACRO {1003A0C193}> #<unavailable argument>)
5: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<UNDEFINED-FUNCTION TESTMACRO {1003A0C193}>)
6: (INVOKE-DEBUGGER #<UNDEFINED-FUNCTION TESTMACRO {1003A0C193}>)
7: (ERROR UNDEFINED-FUNCTION :NAME TESTMACRO)
8: ((LAMBDA (&REST SB-C::ARGS) :IN SB-C::INSTALL-GUARD-FUNCTION))
9: (SB-INT:SIMPLE-EVAL-IN-LEXENV (LET ((A NIL)) (DEFMACRO TESTMACRO NIL) (TESTMACRO)) #<NULL-LEXENV>)
10: (EVAL-TLF (LET ((A NIL)) (DEFMACRO TESTMACRO NIL) (TESTMACRO)) 0 NIL)
11: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (LET ((A NIL)) (DEFMACRO TESTMACRO NIL) (TESTMACRO)) 0)
12: ((LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (LET ((A NIL)) (DEFMACRO TESTMACRO NIL) (TESTMACRO)) :CURRENT-INDEX 0)
13: (SB-C::%DO-FORMS-FROM-INFO #<CLOSURE (LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {10039B19FB}> #<SB-C::SOURCE-INFO {10039B19B3}> SB-C::INPUT-ERROR-IN-LOAD)
14: (SB-INT:LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /mnt/nas-data/Documents/Projects/SHARED PROJECTS - EVAN/pkcmd-lx/temp.lisp" {10039A5103}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
15: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<SB-SYS:FD-STREAM for "file /mnt/nas-data/Documents/Projects/SHARED PROJECTS - EVAN/pkcmd-lx/temp.lisp" {10039A5103}> NIL)
16: (LOAD #<SB-SYS:FD-STREAM for "file /mnt/nas-data/Documents/Projects/SHARED PROJECTS - EVAN/pkcmd-lx/temp.lisp" {10039A5103}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
17: ((FLET SB-IMPL::LOAD-SCRIPT :IN SB-IMPL::PROCESS-SCRIPT) #<SB-SYS:FD-STREAM for "file /mnt/nas-data/Documents/Projects/SHARED PROJECTS - EVAN/pkcmd-lx/temp.lisp" {10039A5103}>)
18: ((FLET #:WITHOUT-INTERRUPTS-BODY-146 :IN SB-IMPL::PROCESS-SCRIPT))
19: (SB-IMPL::PROCESS-SCRIPT "temp.lisp")
20: (SB-IMPL::TOPLEVEL-INIT)
21: ((FLET #:WITHOUT-INTERRUPTS-BODY-82 :IN SAVE-LISP-AND-DIE))
22: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))
The obvious answer is, of course, to put the defmacro outside of the let binding. But, for technical reasons, this would be very inconvenient for my project. Besides, I know of no valid reason why defining a macro under a let binding should fail.
Is 'let over macro' specifically prohibited in Common Lisp? Or am I missing something?
EDIT: The crux of my requirement is that the macro shares the same level as the function it calls, and subsequent code is not nested within it.
This is because I am attempting to write a macro that generates a function and a macro at the same time. The generated macro would call the function. So we end up with something like the following.
I write a line like:
(generate-stuff function-name)
And this resolves into:
(defun function-name-1 () ...)
(defmacro function-name (&rest args)
`(function-name-1 ,#args)
Therefore the macro and the function it calls must be at the same lexical level, adjacent to each other, and cannot create a new lexical environment (macrolet) for subsequent code to nest within.
This would all work fine, except that I happen to be within a let binding at the time; because the function in question has to refer to variables within this binding.
Note that the macro is accessible from outside the binding:
(let ...)
(defmacro my-macro ...)
(my-macro)
It seems absurd to me that a macro defined inside a let binding should only be accessible after the binding has ended. Nothing else in lisp behaves this way.
Local macros can be defined with MACROLET.
Besides, I know of no valid reason why defining a macro under a let binding should fail.
It doesn't fail. It's just not available during compile time, if you just compile the file or compile the expression. The compiler does not make defmacro definitions available in the compile-time environment, if the definition is not at top-level. Inside a progn it would be at top-level, but not inside a let.
Remember: SBCL compiles Lisp source code to machine code using a compiler. It executes machine code.
Let's look at your example:
(let ((a nil))
(defmacro testmacro ())
(testmacro))
Generally putting a global macro into a LET is bad practice and it's hard to understand what it actually should do. What should the influence of the binding of a be? Remember, a compiler compiles code before it executes.
Let's assume we have SBCL and SBCL loads a file with the above form:
(load "foo.lisp")
SBCL does now: READ the whole form, COMPILE the whole form, EXECUTE the whole form. In that order.
SBCL reads the first form. Which is the equivalent of:
CL-USER 155 > (read-from-string "(let ((a nil))
(defmacro testmacro ())
(testmacro))")
(LET ((A NIL)) (DEFMACRO TESTMACRO NIL) (TESTMACRO))
Thus we have data.
The next step is that SBCL compiles that code.
it sees a LET expression. It creates code for the LET bindings. It does not execute the LET. We are just compiling it.
it then compiles the body of the LET
it sees a DEFMACRO form: the macro gets expanded and it gets compiled to machine code. It does not execute the DEFMACRO for. We are just expanding and compiling it.
it sees a TESTMACRO form: It has no idea what it is. It compiles it as a function call to an undefined function and warns about an undefined function
The next step is that SBCL executes the compiled code
it runs the compiled LET form and creates the binding for a
it then executes the body of the LET
it executes the compiled DEFMACRO form: a global macro named TESTMACRO gets defined
it the executes the compiled TESTMACRO form: the function gets called. It is not defined. An error is signalled.
That's all logical and in no way absurd. SBCL first reads the LET, then compiles the LET to machine code and then runs the machine code.
These are three independent steps: read-time, compile-time, run-time.
Let's try it in the REPL:
First we are READing the form
* (read-from-string "(let ((a nil))
(defmacro testmacro ())
(testmacro))")
(LET ((A NIL))
(DEFMACRO TESTMACRO ())
(TESTMACRO))
129
Then we are compiling the form:
* (compile nil `(lambda () ,*))
; in: LAMBDA ()
; (LET ((A NIL))
; (DEFMACRO TESTMACRO ())
; (TESTMACRO))
;
; caught STYLE-WARNING:
; The variable A is defined but never used.
; in: LAMBDA ()
; (TESTMACRO)
;
; caught STYLE-WARNING:
; undefined function: COMMON-LISP-USER::TESTMACRO
;
; compilation unit finished
; Undefined function:
; TESTMACRO
; caught 2 STYLE-WARNING conditions
#<FUNCTION (LAMBDA ()) {226B025B}>
T
NIL
You can see that SBCL tells us about some problems. TESTMACRO is undefined.
Now we run the code:
* (funcall *)
STYLE-WARNING:
TESTMACRO is being redefined as a macro when it was previously assumed to be a function.
debugger invoked on a UNDEFINED-FUNCTION in thread
#<THREAD "main thread" RUNNING {10004F84C3}>:
The function COMMON-LISP-USER::TESTMACRO is undefined.
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [CONTINUE ] Retry calling TESTMACRO.
1: [USE-VALUE ] Call specified function.
2: [RETURN-VALUE ] Return specified values.
3: [RETURN-NOTHING] Return zero values.
4: [ABORT ] Exit debugger, returning to top level.
("undefined function")
0]
As expected - the compiler did warn us: the function TESTMACRO is undefined.
If you want SBCL to compile a macro form, you have to make sure that the macro form is known at compile-time.
Can someone help understand what is happenning here?
(DEFUN G(L)(+(CAR L)(CADR L)))
(SETQ H`F)(SET H `G)
I want to know what happends when I evaluate (F` (2 3 4 5 6 ))
I've written it in my lisp interpreter but I get the following error:
Undefined function F
Common Lisp has a namespace for values and another namespace for functions.
CL-USER 49 > (DEFUN G(L)(+(CAR L)(CADR L)))
G
CL-USER 50 > (SETQ H 'F)
F
CL-USER 51 > (SET H 'G)
G
CL-USER 52 > F
G
CL-USER 53 > (symbol-value 'F)
G
CL-USER 54 > (symbol-function 'f)
Error: Undefined function F in form (SYMBOL-FUNCTION F).
All you did was setting the value of F, but not the function F.
I want to write a bitboard in common lisp, so I need a 64 bit integer. How do I get a 64 bit integer in common lisp? Also, are there any libraries that could help me accomplish this without writing everything from scratch?
You can declare your variables to be of type (signed-byte 64) or (unsigned-byte 64):
CL-USER> (typexpand '(unsigned-byte 64))
(INTEGER 0 18446744073709551615)
T
CL-USER> (typexpand '(signed-byte 64))
(INTEGER -9223372036854775808 9223372036854775807)
T
It depends upon your implementation if it is actually clever enough to really stuff this in 8 consecutive bytes or if it will use a bignum for this. Appropriate optimize-declarations might help.
Here's a (very simple) example of such type declarations, and handling integers in binary:
(let* ((x #b01)
(y #b10)
(z (logior x y)))
(declare ((signed-byte 64) x y z))
(format t "~a~%" (logbitp 1 x))
(format t "~a~%" (logbitp 1 (logior x (ash 1 1))))
(format t "~b~%" z))
Output:
NIL
T
11
Here's a setf-expander definition to get a simple setter for bits in integers, and a corresponding getter:
(define-setf-expander logbit (index place &environment env)
(multiple-value-bind (temps vals stores store-form access-form)
(get-setf-expansion place env)
(let ((i (gensym))
(store (gensym))
(stemp (first stores)))
(values `(,i ,#temps)
`(,index ,#vals)
`(,store)
`(let ((,stemp (dpb ,store (byte 1 ,i) ,access-form))
,#(cdr stores))
,store-form
,store)
`(logbit ,i ,access-form)))))
(defun logbit (index integer)
(ldb (byte 1 index) integer))
These can be used like this:
(let ((x 1))
(setf (logbit 3 x) 1)
x)
==> 9
(let ((x 9))
(setf (logbit 3 x) 0)
x)
==> 1
(logbit 3 1)
==> 0
(logbit 3 9)
==> 1
In portable Common Lisp 'Integers' are as large as you like. There is a more efficient subset of integers called 'fixnums'. The exact range of fixnums is implementation depended. But it is typically not the full 64 bit (on a 64bit architecture) which can be used, since most Common Lisp implementations need type tag bits. For the user there is not much of a difference. Fixnums are a subset of integers and one can add two fixnums and get a not-fixnum integer result. The only differences that may be observable is that computation with non-fixnum integers is slower, needs more storage, ... Generally, if you want to do computation with integers, you don't need to declare that you want to calculate with 64bit. You just use Integers and the usual operations for those.
If you want real 64bit large integers (represented in only 64bits, without tags, etc.) and computation with those, you'll leave the portable ANSI CL capabilities. If and how CLISP supports that, is best asked on the CLISP mailing list.
Documentation
Type FIXNUM
Type INTEGER
Example usage of bit vectors/arrays to implement a 8x8 bit-board
(starting with brutally and prematurely optimized code just to show a
way to get tight assembler code):
(defun make-bitboard ()
(make-array '(8 8) :element-type '(mod 2) :initial-element 0))
MAKE-BITBOARD will create a 8x8 bitboard as an array of bits. When
using SBCL, this is internally represented as 1 bit per element (so
you have 64 bits + array instance overhead). If you ask for
optimizations when accessing the board, you'll get fast code.
(declaim (inline get-bitboard))
(defun get-bitboard (bit-board x y)
(declare (optimize speed (safety 0) (debug 0))
(type (simple-array (mod 2) (8 8)) bit-board)
(type fixnum x y))
(aref bit-board x y))
(declaim (notinline get-bitboard))
The DECLAIMs are there to allow local
inlining requests for
GET-BITBOARD.
An example of using GET-BITBOARD:
(defun use-bitboard (bit-board)
(declare (optimize speed (safety 0) (debug 0))
(type (simple-array (mod 2) (8 8)) bit-board)
(inline get-bitboard))
(let ((sum 0))
(declare (type fixnum sum))
(dotimes (i 8)
(declare (type fixnum i))
(dotimes (j 8)
(declare (type fixnum j))
(incf sum (the (mod 2) (get-bitboard bit-board i j)))))
sum))
Since there is no SET-BITBOARD yet, an example of using USE-BITBOARD is:
(use-bitboard (make-bitboard))
Disassembling USE-BITBOARD (SBCL again, Linux x64) shows that the
compiler inlined GET-BITBOARD:
; disassembly for USE-BITBOARD
; 030F96A2: 31F6 XOR ESI, ESI ; no-arg-parsing entry point
; 6A4: 31D2 XOR EDX, EDX
; 6A6: EB54 JMP L3
; 6A8: 90 NOP
; 6A9: 90 NOP
; 6AA: 90 NOP
; 6AB: 90 NOP
; 6AC: 90 NOP
; 6AD: 90 NOP
; 6AE: 90 NOP
; 6AF: 90 NOP
; 6B0: L0: 31DB XOR EBX, EBX
; 6B2: EB3E JMP L2
; 6B4: 90 NOP
; 6B5: 90 NOP
; 6B6: 90 NOP
; 6B7: 90 NOP
; 6B8: 90 NOP
; 6B9: 90 NOP
; 6BA: 90 NOP
; 6BB: 90 NOP
; 6BC: 90 NOP
; 6BD: 90 NOP
; 6BE: 90 NOP
; 6BF: 90 NOP
; 6C0: L1: 488D04D500000000 LEA RAX, [RDX*8]
; 6C8: 4801D8 ADD RAX, RBX
; 6CB: 4C8B4711 MOV R8, [RDI+17]
; 6CF: 48D1F8 SAR RAX, 1
; 6D2: 488BC8 MOV RCX, RAX
; 6D5: 48C1E906 SHR RCX, 6
; 6D9: 4D8B44C801 MOV R8, [R8+RCX*8+1]
; 6DE: 488BC8 MOV RCX, RAX
; 6E1: 49D3E8 SHR R8, CL
; 6E4: 4983E001 AND R8, 1
; 6E8: 49D1E0 SHL R8, 1
; 6EB: 4C01C6 ADD RSI, R8
; 6EE: 4883C302 ADD RBX, 2
; 6F2: L2: 4883FB10 CMP RBX, 16
; 6F6: 7CC8 JL L1
; 6F8: 4883C202 ADD RDX, 2
; 6FC: L3: 4883FA10 CMP RDX, 16
; 700: 7CAE JL L0
; 702: 488BD6 MOV RDX, RSI
; 705: 488BE5 MOV RSP, RBP
; 708: F8 CLC
; 709: 5D POP RBP
; 70A: C3 RET
Not sure why the compiler put in all those NOPs (leaving space for
instrumentation later? alignments?) but if you look at the code at the
end it's pretty compact (not as compact as hand-crafted assembler, of
course).
Now this is an obvious case of premature optimization. The correct way
to start here would be to simply write:
(defun get-bitboard (bit-board x y)
(aref bit-board x y))
(defun use-bitboard (bit-board)
(let ((sum 0))
(dotimes (i 8)
(dotimes (j 8)
(incf sum (get-bitboard bit-board i j))))
sum))
... and then use a profiler when running the game code that uses the
bit-board to see where the CPU bottlenecks are. SBCL includes a nice
statistical profiler.
Starting with the simpler and slower code, with no declarations for
speed, is best. Just compare the size of the code - I started with the
code with plenty of declarations to make the simple code at the end
look even simpler by comparison :-). The advantage here is that you
can treat Common Lisp as a scripting/prototyping language when trying
out ideas, then squeeze more performance out of the code that the
profiler suggests.
The assembly code is obviously not as tight as loading the whole board in
one 64 bit register and then accessing individual bits. But if you
suddenly decide that you want more than 1 bit per square, it's much
easier to change the CL code than to change assembler code (just
change the array type everywhere from '(mod 2) to '(mod 16), for
instance).
You want to use bit vectors, which are arbitrary sized arrays of bits, rather than something like a 64 bit integer. The implementation will deal with the internal representations for you.