I know Racket doesn't have "docstrings" in the same way that many other languages do, but given how convenient documenting things at the source is, I'd like to approximate something like it in Racket.
When I first learned about Scribble and #langs, I thought it would be possible to do something like:
#lang racket
#lang scribble
... and then write code in Racket with docstrings in Scribble. But this doesn't work, probably because "languages don't compose."
#lang racket
(require scribble/manual)
#racket['hi]
which results in:
my-source.rkt:4:0: #racket: unbound identifier
I came across scribble/srcdoc which seems compelling because it sounds like it allows you to piggyback docs on top of contracts which already serve as a kind of minimal (typically module-level) documentation, in addition to providing runtime checks of course. I haven't been able to get it to work so far, but instead of thrashing around for another several hours I thought it would be more useful to ask about it here. For what it's worth, here's what I'm seeing at the moment:
#lang racket
(require scribble/srcdoc
(for-doc scribble/base scribble/manual))
(provide
(proc-doc/names fib
(-> integer? integer?)
(n)
#{Computes the #racket[n]th Fibonacci number}))
(define (fib n)
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2)))))
which results in:
my-source.rkt:6:1: proc-doc/names: bad syntax
in: (proc-doc/names fib (-> integer? integer?) (n) # (Computes the #racket (n) th Fibonacci number))
Since reference docs like these are better at explaining how things work than how to use it, I'm looking for an answer that is more of a how-to on writing "docstrings" in Racket. It needn't be long, just sufficient to help the reader employ this "contract + docstring" pattern in their code (and, possibly, describing other alternatives).
You want the at-exp 'meta-language'. This lets you program in another language (in this case racket, with the modification of using at-expressions.
So taking your example above, you get:
#lang at-exp racket
(require scribble/srcdoc
(for-doc scribble/base scribble/manual))
(provide
(proc-doc/names fib
(-> integer? integer?)
(n)
#{Computes the #racket[n]th Fibonacci number}))
(define (fib n)
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2)))))
Note the at-exp in the first line.
You can also do:
#lang at-exp racket
(require scribble/manual)
#racket['hi]
and get:
(sized-element #f (list (cached-element #0=(style "RktVal" (list 'tt-chars (css-addition '(collects #"scribble" #"racket.css")) (tex-addition '(collects #"scribble" #"racket.tex")))) "'" ...) (cached-element #0# "hi" ...)) ...)
Related
In one variant of Common Lisp (I think it was CMUCL, but I might be wrong—I can't find it any more) there was a function that was (I think) called function-lambda-expression. If it got a procedure, it would print out the lambda expression that had generated it. Example:
(let ((my-thunk (lambda () (+ 1 2))))
(write my-thunk)
(write (function-lambda-expression my-thunk)))
This would print out something like:
#<PROCEDURE>
(LAMBDA () (+ 1 2))
It was terribly useful for debugging and exploring the language.
I'm looking for a function like this in Racket. I've looked through the Racket Documentation but I can't find anything like it. (I wouldn't be surprised if I overlooked it, however.) Is there an equivalent in Racket?
No. Racket's lambda produces a closure that does not remember its S-expression (or syntax object) form. It does typically remember its name (or its abbreviated source location, if no name can be inferred), and that's often enough to help with debugging. (See object-name.)
You can build your own variant of lambda that has this feature, using Racket's applicable structures and a simple macro. Here's a basic example:
#lang racket
(struct exp-closure (f exp)
#:property prop:procedure (struct-field-index f))
(define-syntax-rule (exp-lambda formals . body)
(exp-closure (lambda formals . body)
(quote (exp-lambda formals . body))))
(let ([my-thunk (exp-lambda () (+ 1 2))])
(printf "fun is ~v\n" my-thunk)
(printf "exp is ~v\n" (exp-closure-exp my-thunk))
(printf "result is ~v\n" (my-thunk)))
This produces
fun is #<procedure:...tmp/lambda.rkt:11:19>
exp is '(exp-lambda () (+ 1 2))
result is 3
A better version of this macro might propagate the source location of the macro use to the lambda expression it creates, or the inferred name (see syntax-local-infer-name), or both.
I need to read a Racket source file and run it through macro expansion. I have a simple test file that Racket itself happily accepts:
C:\ayane>type factorial.rkt
#lang racket
(provide factorial)
(define (factorial n)
(if (<= n 1)
1
(* n (factorial (sub1 n)))))
Now I try from the REPL:
C:\ayane>racket
Welcome to Racket v6.5.
> (read-accept-reader #t)
> (expand (with-input-from-file "factorial.rkt" (lambda () (read-syntax "factorial.rkt"))))
#<syntax::1 (module factorial racket (#%m...>
So far so good. Now the same thing from a test program:
C:\ayane>type test.rkt
#lang racket
(read-accept-reader #t)
(expand (with-input-from-file "factorial.rkt"
(lambda ()
(read-syntax "factorial.rkt"))))
C:\ayane>racket test.rkt
factorial.rkt::1: module: unbound identifier;
also, no #%app syntax transformer is bound
at: module
in: (module factorial racket (#%module-begin (provide factorial) (define (factorial n) (if (<= n 1) 1 (* n (factorial (sub1 n)))))))
context...:
C:\ayane\test.rkt: [running body]
So it looks like the same code works interactively but not in a program. What am I missing?
You need to specify which namespace expand should use to lookup top-level variables (i.e. variables not bound in the program).
For example:
(parameterize ([current-namespace (make-base-namespace)])
(expand ...))
For more information see the comments in the file below in which I attempt to explain the relationship between namespaces and expand:
https://github.com/soegaard/meta/blob/master/runtime/racket-eval.rkt#L122
The answer from #soegaard addresses the immediate issue, but if you want a comprehensive program that reimplements expansion from primitives, you can look at
https://github.com/samth/pycket/blob/master/pycket/pycket-lang/expand.rkt
I want to know if these two definitions of nth are equal:
I. is defined as macro:
(defmacro -nth (n lst)
(defun f (n1 lst1)
(cond ((eql n1 0) lst1)
(t `(cdr ,(f (- n1 1) lst1)))))
`(car ,(f n lst)))
II. is defined as a bunch of functions:
(defun f (n lst)
(cond ((eql n 0) lst)
(t `(cdr ,(f (- n 1) lst)))))
(defun f1 (n lst)
`(car ,(f n `',lst)))
(defun --nth (n lst)
(eval (f1 n lst)))
Am i get the right idea? Is macro definition is evaluating of expression, constructed in its body?
OK, let start from the beginning.
Macro is used to create new forms that usually depend on macro's input. Before code is complied or evaluated, macro has to be expanded. Expansion of a macro is a process that takes place before evaluation of form where it is used. Result of such expansion is usually a lisp form.
So inside a macro here are a several levels of code.
Not quoted code will be evaluated during macroexpansion (not at run-time!), in your example you define function f when macro is expanded (for what?);
Next here is quoted (with usual quote or backquote or even nested backquotes) code that will become part of macroexpansion result (in its literal form); you can control what part of code will be evaluated during macroexpansion and what will stay intact (quoted, partially or completely). This allows one to construct anything before it will be executed.
Another feature of macro is that it does not evaluate its parameters before expansion, while function does. To give you picture of what is a macro, see this (just first thing that came to mind):
(defmacro aif (test then &optional else)
`(let ((it ,test))
(if it ,then ,else)))
You can use it like this:
CL-USER> (defparameter *x* '((a . 1) (b . 2) (c . 3) (d . 4)))
*X*
CL-USER> (aif (find 'c *x* :key #'car) (1+ (cdr it)) 0)
4
This macro creates useful lexical binding, capturing variable it. After checking of a condition, you don't have to recalculate result, it's accessible in forms 'then' and 'else'. It's impossible to do with just a function, it has introduced new control construction in language. But macro is not just about creating lexical environments.
Macro is a powerful tool. It's impossible to fully describe what you can do with it, because you can do everything. But nth is not something you need a macro for. To construct a clone of nth you can try to write a recursive function.
It's important to note that LISP macro is most powerful thing in the programming world and LISP is the only language that has this power ;-)
To inspire you, I would recommend this article: http://www.paulgraham.com/avg.html
To master macro, begin with something like this:
http://www.gigamonkeys.com/book/macros-defining-your-own.html
Then may be Paul Graham's "On Lisp", then "Let Over Lambda".
There is no need for either a macro nor eval to make abstractions to get the nth element of a list. Your macro -nth doesn't even work unless the index is literal number. try this:
(defparameter test-list '(9 8 7 6 5 4 3 2 1 0))
(defparameter index 3)
(nth index test-list) ; ==> 6 (this is the LISP provided nth)
(-nth index test-list) ; ==> ERROR: index is not a number
A typical recursive solution of nth:
(defun nth2 (index list)
(if (<= index 0)
(car list)
(nth2 (1- index) (cdr list))))
(nth2 index test-list) ; ==> 6
A typical loop version
(defun nth3 (index list)
(loop :for e :in list
:for i :from index :downto 0
:when (= i 0) :return e))
(nth3 index test-list) ; ==> 6
Usually a macro is something you use when you see your are repeating yourself too much and there is no way to abstract your code further with functions. You may make a macro that saves you the time to write boilerplate code. Of course there is a trade off of not being standard code so you usually write the macro after a couple of times have written the boilerplate.
eval should never be used unless you really have to. Usually you can get by with funcall and apply. eval works only in the global scope so you loose closure variables.
I've been reading an article by Olin Shivers titled Stylish Lisp programming techniques and found the second example there (labeled "Technique n-1") a bit puzzling. It describes a self-modifying macro that looks like this:
(defun gen-counter macro (x)
(let ((ans (cadr x)))
(rplaca (cdr x)
(+ 1 ans))
ans))
It's supposed to get its calling form as argument x (i.e. (gen-counter <some-number>)). The purpose of this is to be able to do something like this:
> ;this prints out the numbers from 0 to 9.
(do ((n 0 (gen-counter 1)))
((= n 10) t)
(princ n))
0.1.2.3.4.5.6.7.8.9.T
>
The problem is that this syntax with the macro symbol after the function name is not valid in Common Lisp. I've been unsuccessfully trying to obtain similar behavior in Common Lisp. Can someone please provide a working example of analogous macro in CL?
Why the code works
First, it's useful to consider the first example in the paper:
> (defun element-generator ()
(let ((state '(() . (list of elements to be generated)))) ;() sentinel.
(let ((ans (cadr state))) ;pick off the first element
(rplacd state (cddr state)) ;smash the cons
ans)))
ELEMENT-GENERATOR
> (element-generator)
LIST
> (element-generator)
OF
> (element-generator)
This works because there's one literal list
(() . (list of elements to be generated)
and it's being modified. Note that this is actually undefined behavior in Common Lisp, but you'll get the same behavior in some Common Lisp implementations. See Unexpected persistence of data and some of the other linked questions for a discussion of what's happening here.
Approximating it in Common Lisp
Now, the paper and code you're citing actually has some useful comments about what this code is doing:
(defun gen-counter macro (x) ;X is the entire form (GEN-COUNTER n)
(let ((ans (cadr x))) ;pick the ans out of (gen-counter ans)
(rplaca (cdr x) ;increment the (gen-counter ans) form
(+ 1 ans))
ans)) ;return the answer
The way that this is working is not quite like an &rest argument, as in Rainer Joswig's answer, but actually a &whole argument, where the the entire form can be bound to a variable. This is using the source of the program as the literal value that gets destructively modified! Now, in the paper, this is used in this example:
> ;this prints out the numbers from 0 to 9.
(do ((n 0 (gen-counter 1)))
((= n 10) t)
(princ n))
0.1.2.3.4.5.6.7.8.9.T
However, in Common Lisp, we'd expect the macro to be expanded just once. That is, we expect (gen-counter 1) to be replaced by some piece of code. We can still generate a piece of code like this, though:
(defmacro make-counter (&whole form initial-value)
(declare (ignore initial-value))
(let ((text (gensym (string 'text-))))
`(let ((,text ',form))
(incf (second ,text)))))
CL-USER> (macroexpand '(make-counter 3))
(LET ((#:TEXT-1002 '(MAKE-COUNTER 3)))
(INCF (SECOND #:TEXT-1002)))
Then we can recreate the example with do
CL-USER> (do ((n 0 (make-counter 1)))
((= n 10) t)
(princ n))
023456789
Of course, this is undefined behavior, since it's modifying literal data. It won't work in all Lisps (the run above is from CCL; it didn't work in SBCL).
But don't miss the point
The whole article is sort of interesting, but recognize that it's sort of a joke, too. It's pointing out that you can do some funny things in an evaluator that doesn't compile code. It's mostly satire that's pointing out the inconsistencies of Lisp systems that have different behaviors under evaluation and compilation. Note the last paragraph:
Some short-sighted individuals will point out that these programming
techniques, while certainly laudable for their increased clarity and
efficiency, would fail on compiled code. Sadly, this is true. At least
two of the above techniques will send most compilers into an infinite
loop. But it is already known that most lisp compilers do not
implement full lisp semantics -- dynamic scoping, for instance. This
is but another case of the compiler failing to preserve semantic
correctness. It remains the task of the compiler implementor to
adjust his system to correctly implement the source language, rather
than the user to resort to ugly, dangerous, non-portable, non-robust
``hacks'' in order to program around a buggy compiler.
I hope this provides some insight into the nature of clean, elegant
Lisp programming techniques.
—Olin Shivers
Common Lisp:
(defmacro gen-counter (&rest x)
(let ((ans (car x)))
(rplaca x (+ 1 ans))
ans))
But above only works in the Interpreter, not with a compiler.
With compiled code, the macro call is gone - it is expanded away - and there is nothing to modify.
Note to unsuspecting readers: you might want to read the paper by Olin Shivers very careful and try to find out what he actually means...
Suppose I want to trigger a Scheme macro on something other than the first item in an s-expression. For example, suppose that I wanted to replace define with an infix-style :=, so that:
(a := 5) -> (define a 5)
((square x) := (* x x)) -> (define (square x) (* x x))
The actual transformation seems to be quite straightforward. The trick will be getting Scheme to find the := expressions and macro-expand them. I've thought about surrounding large sections of code that use the infix syntax with a standard macro, maybe: (with-infix-define expr1 expr2 ...), and having the standard macro walk through the expressions in its body and perform any necessary transformations. I know that if I take this approach, I'll have to be careful to avoid transforming lists that are actually supposed to be data, such as quoted lists, and certain sections of quasiquoted lists. An example of what I envision:
(with-infix-define
((make-adder n) := (lambda (m) (+ n m)))
((foo) :=
(add-3 := (make-adder 3))
(add-6 := (make-adder 6))
(let ((a 5) (b 6))
(+ (add-3 a) (add-6 b))))
(display (foo))
(display '(This := should not be transformed))
So, my question is two-fold:
If I take the with-infix-define route, do I have to watch out for any stumbling blocks other than quote and quasiquote?
I feel a bit like I'm reinventing the wheel. This type of code walk seems like exactly what standard macro expanding systems would have to do - the only difference is that they only look at the first item in a list when deciding whether or not to do any code transformation. Is there any way I can just piggyback on existing systems?
Before you continue with this, it's best to think things over thoroughly -- IME you'd often find that what you really want a reader-level handling of := as an infix syntax. That will of course mean that it's also infix in quotations etc, so it would seem bad for now, but again, my experience is that you end up realizing that it's better to do things consistently.
For completeness, I'll mention that in Racket there's a read-syntax hack for infix-like expressions: (x . define . 1) is read as (define x 1). (And as above, it works everywhere.)
Otherwise, your idea of a wrapping macro is pretty much the only thing you can do. This doesn't make it completely hopeless though, you might have a hook into your implementation's expander that can allow you to do such things -- for example, Racket has a special macro called #%module-begin that wraps a complete module body and #%top-interaction that wraps toplevel expressions on the REPL. (Both of these are added implicitly in the two contexts.) Here's an example (I'm using Racket's define-syntax-rule for simplicity):
#lang racket/base
(provide (except-out (all-from-out racket/base)
#%module-begin #%top-interaction)
(rename-out [my-module-begin #%module-begin]
[my-top-interaction #%top-interaction]))
(define-syntax infix-def
(syntax-rules (:= begin)
[(_ (begin E ...)) (begin (infix-def E) ...)]
[(_ (x := E ...)) (define x (infix-def E) ...)]
[(_ E) E]))
(define-syntax-rule (my-module-begin E ...)
(#%module-begin (infix-def E) ...))
(define-syntax-rule (my-top-interaction . E)
(#%top-interaction . (infix-def E)))
If I put this in a file called my-lang.rkt, I can now use it as follows:
#lang s-exp "my-lang.rkt"
(x := 10)
((fib n) :=
(done? := (<= n 1))
(if done? n (+ (fib (- n 1)) (fib (- n 2)))))
(fib x)
Yes, you need to deal with a bunch of things. Two examples in the above are handling begin expressions and handling function bodies. This is obviously a very partial list -- you'll also want bodies of lambda, let, etc. But this is still better than some blind massaging, since that's just not practical as you can't really tell in advance how some random piece of code will end up. As an easy example, consider this simple macro:
(define-syntax-rule (track E)
(begin (eprintf "Evaluating: ~s\n" 'E)
E))
(x := 1)
The upshot of this is that for a proper solution, you need some way to pre-expand the code, so that you can then scan it and deal with the few known core forms in your implmenetation.
Yes, all of this is repeating work that macro expanders do, but since you're changing how expansion works, there's no way around this. (To see why it's a fundamental change, consider something like (if := 1) -- is this a conditional expression or a definition? How do you decide which one takes precedence?) For this reason, for languages with such "cute syntax", a more popular approach is to read and parse the code into plain S-expressions, and then let the actual language implementation use plain functions and macros.
Redefining define is a little complicated. See #Eli's excellent explanation.
If on the other hand, you are content with := to use set! things are a little simpler.
Here is a small example:
#lang racket
(module assignment racket
(provide (rename-out [app #%app]))
(define-syntax (app stx)
(syntax-case stx (:=)
[(_ id := expr)
(identifier? #'id)
(syntax/loc stx (set! id expr))]
[(_ . more)
(syntax/loc stx (#%app . more))])))
(require 'assignment)
(define x 41)
(x := (+ x 1))
(displayln x)
To keep the example to a single file, I used submodules (available in the prerelease version of Racket).