building a hash table with gensym and macrolet - lisp

I'm trying to build a hash table (among other actions) while reading. I don't want the hash table to have global scope (yet), so I'm doing this with a macro and gensym. Inside the macro x, I'm defining a macro s which is similar to setf, but defines an entry in a hash table instead of defining a symbol somewhere. It blows up. I think I understand the error message, but how do I make it work?
The code:
#!/usr/bin/clisp -repl
(defmacro x (&rest statements)
(let ((config-variables (gensym)))
`(macrolet ((s (place value)
(setf (gethash 'place ,config-variables) value)))
(let ((,config-variables (make-hash-table :test #'eq)))
(progn ,#statements)
,config-variables))))
(defun load-config ()
(let ((config-file-tree (read *standard-input*)))
(eval config-file-tree)))
(defun load-test-config ()
(with-input-from-string (*standard-input* "(x (s fred 3) (s barney 5))")
(load-config)))
(load-test-config)
The output:
*** - LET*: variable #:G12655 has no value
The following restarts are available:
USE-VALUE :R1 Input a value to be used instead of #:G12655.
STORE-VALUE :R2 Input a new value for #:G12655.
SKIP :R3 skip (LOAD-TEST-CONFIG)
STOP :R4 stop loading file /u/asterisk/semicolon/build.l/stackoverflow-semi

Just guessing what Bill might really want.
Let's say he wants a mapping from some keys to some values as a configuration in a file.
Here is the procedural way.
open a stream to the data
read it as an s-expression
walk the data and fill a hash-table
Example code:
(defun read-mapping (&optional (stream *standard-input*))
(destructuring-bind (type &rest mappings) (read stream)
(assert (eq type 'mapping))
(let ((table (make-hash-table)))
(loop for (key value) in mappings
do (setf (gethash key table) value))
table)))
(defun load-config ()
(read-mapping))
(defun load-test-config ()
(with-input-from-string (*standard-input* "(mapping (fred 3) (barney 5))")
(load-config)))
(load-test-config)
Use:
CL-USER 57 > (load-test-config)
#<EQL Hash Table{2} 402000151B>
CL-USER 58 > (describe *)
#<EQL Hash Table{2} 402000151B> is a HASH-TABLE
BARNEY 5
FRED 3
Advantages:
no macros
data does not get encoded in source code and generated source code
no evaluation (security!) via EVAL needed
no object code bloat via macros which are expanding to larger code
functional abstraction
much easier to understand and debug
Alternatively I would write a read-macro for { such that {(fred 3) (barney 5)} would be directly read as an hash-table.
If you want to have computed values:
(defun make-table (mappings &aux (table (make-hash-table)))
(loop for (key value) in mappings
do (setf (gethash key table) (eval value)))
table)
CL-USER 66> (describe (make-table '((fred (- 10 7)) (barney (- 10 5)))))
#<EQL Hash Table{2} 4020000A4B> is a HASH-TABLE
BARNEY 5
FRED 3
Turning that into a macro:
(defmacro defmapping (&body mappings)
`(make-table ',mappings))
(defmapping
(fred 3)
(barney 5))

In a macrolet you are as well defining a macro, so the usual rules apply, i.e. you have to backquote expressions, that are to be evaluated at run-time. Like this:
(defmacro x (&rest statements)
(let ((config-variables (gensym)))
`(macrolet ((s (place value)
`(setf (gethash ',place ,',config-variables) ,value)))
(let ((,config-variables (make-hash-table :test #'eq)))
(progn ,#statements)
,config-variables))))

Related

SIMPLE-READER-ERROR , illegal sharp macro character, Subcharacter #\< not defined for dispatch char #\#

In chapter 4 of the book, Successful Lisp, by David B. Lamkins,
there is a simple application to keep track of bank checks.
https://dept-info.labri.fr/~strandh/Teaching/MTP/Common/David-Lamkins/chapter04.html
At the end, we write a macro that will save and restore functions.
The problem occurs when I execute the reader function and the value to read is the hash table.
The macro to save and restore functions is:
(defmacro def-i/o (writer-name reader-name (&rest vars))
(let ((file-name (gensym))
(var (gensym))
(stream (gensym)))
`(progn
(defun ,writer-name (,file-name)
(with-open-file (,stream ,file-name
:direction :output :if-exists :supersede)
(dolist (,var (list ,#vars))
(declare (special ,#vars))
(print ,var ,stream))))
(defun ,reader-name (,file-name)
(with-open-file (,stream ,file-name
:direction :input :if-does-not-exist :error)
(dolist (,var ',vars)
(set ,var (read ,stream)))))
t)))
Here is my hash table and what is going on:
(defvar *payees* (make-hash-table :test #'equal))
(check-check 100.00 "Acme" "Rocket booster T-1000")
CL-USER> *payees*
#<HASH-TABLE :TEST EQUAL :COUNT 0 {25520F91}>
CL-USER> (check-check 100.00 "Acme" "T-1000 rocket booster")
#S(CHECK
:NUMBER 100
:DATE "2020-4-1"
:AMOUNT 100.0
:PAID "Acme"
:MEMO "T-1000 rocket booster")
CL-USER> (def-i/o save-checks charge-checks (*payees*))
T
CL-USER> (save-checks "/home/checks.dat")
NIL
CL-USER> (makunbound '*payees*)
*PAYEES*
CL-USER> (load-checks "/home/checks.dat")
; Evaluation aborted on #<SB-INT:SIMPLE-READER-ERROR "illegal sharp macro character: ~S" {258A8541}>.
In Lispworks, the error message is:
Error: subcharacter #\< not defined for dispatch char #\#.
To simplify, I get the same error if I execute:
CL-USER> (defvar *payees* (make-hash-table :test #'equal))
*PAYEES*
CL-USER> (with-open-file (in "/home/checks.dat"
:direction :input)
(set *payees* (read in)))
; Evaluation aborted on #<SB-INT:SIMPLE-READER-ERROR "illegal sharp macro character: ~S" {23E83B91}>.
Can someone explain to me where the problem is coming from, and what I need to fix in my code for this to work.
Thank you in advance for the explanations you can give me and the help you can give me.
Values which cannot be read back are printed with #<, see Sharpsign Less-Than-Sign. Common Lisp does not define how hash-tables should be printed readably (there is no reader syntax for hash tables):
* (make-hash-table)
#<HASH-TABLE :TEST EQL :COUNT 0 {1006556E53}>
The example in the book is only suitable to be used with the bank example that was previously shown, and is not intended to be a general-purpose serialization mechanism.
There are many ways to print a hash-table and read them back, depending on your needs, but there is no default representation. See cl-store for a library that store arbitrary objects.
Custom dump function
Let's write a form that evaluate to an equivalent hash-table; let's define a generic function named dump, and a default method that simply returns the object as-is:
(defgeneric dump (object)
(:method (object) object))
Given a hash-table, we can serialize it as a plist (sequence of key/value elements in a list), while also calling dump on keys and values, in case our hash-tables contain hash-tables:
(defun hash-table-plist-dump (hash &aux plist)
(with-hash-table-iterator (next hash)
(loop
(multiple-value-bind (some key value) (next)
(unless some
(return plist))
(push (dump value) plist)
(push (dump key) plist)))))
Instead of reinventing the wheel, we could also have used hash-table-plist from the alexandria system.
(ql:quickload :alexandria)
The above is equivalent to:
(defun hash-table-plist-dump (hash)
(mapcar #'dump (alexandria:hash-table-plist hash)))
Then, we can specialize dump for hash-tables:
(defmethod dump ((hash hash-table))
(loop
for (initarg accessor)
in '((:test hash-table-test)
(:size hash-table-size)
(:rehash-size hash-table-rehash-size)
(:rehash-threshold hash-table-rehash-threshold))
collect initarg into args
collect `(quote ,(funcall accessor hash)) into args
finally (return
(alexandria:with-gensyms (h k v)
`(loop
:with ,h := (make-hash-table ,#args)
:for (,k ,v)
:on (list ,#(hash-table-plist-dump hash))
:by #'cddr
:do (setf (gethash ,k ,h) ,v)
:finally (return ,h))))))
The first part of the loop computes all the hash-table properties, like the kind of hash function to use or the rehash-size, and builds args, the argument list for a call to make-hash-table with the same values.
In the finally clause, we build a loop expression (see backquotes) that first allocates the hash-table, then populates it according to the current values of the hash, and finally return the new hash. Note that the generated code does not depend on alexandria, it could be read back from another Lisp system that does not have this dependency.
Example
CL-USER> (alexandria:plist-hash-table '("abc" 0 "def" 1 "ghi" 2 "jkl" 3)
:test #'equal)
#<HASH-TABLE :TEST EQUAL :COUNT 4 {100C91F8C3}>
Dump it:
CL-USER> (dump *)
(LOOP :WITH #:H603 := (MAKE-HASH-TABLE :TEST 'EQUAL :SIZE '14 :REHASH-SIZE '1.5
:REHASH-THRESHOLD '1.0)
:FOR (#:K604 #:V605) :ON (LIST "jkl" 3 "ghi" 2 "def" 1 "abc"
0) :BY #'CDDR
:DO (SETF (GETHASH #:K604 #:H603) #:V605)
:FINALLY (RETURN #:H603))
The generated data is also valid Lisp code, evaluate it:
CL-USER> (eval *)
#<HASH-TABLE :TEST EQUAL :COUNT 4 {100CD5CE93}>
The resulting hash is equalp to the original one:
CL-USER> (equalp * ***)
T

What does gensym do in Lisp?

contextualization: I've been doing a university project in which I have to write a parser for regular expressions and build the corresponding epsilon-NFA. I have to do this in Prolog and Lisp.
I don't know if questions like this are allowed, if not I apologize.
I heard some of my classmates talking about how they used the function gensym for that, I asked them what it did and even checked up online but I literally can't understand what this function does neither why or when is best to use it.
In particular, I'm more intrested in what it does in Lisp.
Thank you all.
GENSYM creates unique symbols. Each call creates a new symbol. The symbol usually has a name which includes a number, which is counted up. The name is also unique (the symbol itself is already unique) with a number, so that a human reader can identify different uninterned symbols in the source code.
CL-USER 39 > (gensym)
#:G1083
CL-USER 40 > (gensym)
#:G1084
CL-USER 41 > (gensym)
#:G1085
CL-USER 42 > (gensym)
#:G1086
gensym is often used in Lisp macros for code generation, when the macro needs to create new identifiers, which then don't clash with existing identifiers.
Example: we are going to double the result of a Lisp form and we are making sure that the Lisp form itself will be computed only once. We do that by saving the value in a local variable. The identifier for the local variable will be computed by gensym.
CL-USER 43 > (defmacro double-it (it)
(let ((new-identifier (gensym)))
`(let ((,new-identifier ,it))
(+ ,new-identifier ,new-identifier))))
DOUBLE-IT
CL-USER 44 > (macroexpand-1 '(double-it (cos 1.4)))
(LET ((#:G1091 (COS 1.4)))
(+ #:G1091 #:G1091))
T
CL-USER 45 > (double-it (cos 1.4))
0.33993432
a little clarification of the existing answers (as the op is not yet aware of the typical common lisp macros workflow):
consider the macro double-it, proposed by mr. Joswig. Why would we bother creating this whole bunch of let? when it can be simply:
(defmacro double-it (it)
`(+ ,it ,it))
and ok, it seems to be working:
CL-USER> (double-it 1)
;;=> 2
but look at this, we want to increment x and double it
CL-USER> (let ((x 1))
(double-it (incf x)))
;;=> 5
;; WHAT? it should be 4!
the reason can be seen in macro expansion:
(let ((x 1))
(+ (setq x (+ 1 x)) (setq x (+ 1 x))))
you see, as the macro doesn't evaluate form, just splices it into generated code, it leads to incf being executed twice.
the simple solution is to bind it somewhere, and then double the result:
(defmacro double-it (it)
`(let ((x ,it))
(+ x x)))
CL-USER> (let ((x 1))
(double-it (incf x)))
;;=> 4
;; NICE!
it seems to be ok now. really it expands like this:
(let ((x 1))
(let ((x (setq x (+ 1 x))))
(+ x x)))
ok, so what about the gensym thing?
let's say, you want to print some message, before doubling your value:
(defmacro double-it (it)
`(let* ((v "DOUBLING IT")
(val ,it))
(princ v)
(+ val val)))
CL-USER> (let ((x 1))
(double-it (incf x)))
;;=> DOUBLING IT
;;=> 4
;; still ok!
but what if you accidentally name value v instead of x:
CL-USER> (let ((v 1))
(double-it (incf v)))
;;Value of V in (+ 1 V) is "DOUBLING IT", not a NUMBER.
;; [Condition of type SIMPLE-TYPE-ERROR]
It throws this weird error! Look at the expansion:
(let ((v 1))
(let* ((v "DOUBLING IT") (val (setq v (+ 1 v))))
(princ v)
(+ val val)))
it shadows the v from the outer scope with string, and when you are trying to add 1, well it obviously can't. Too bad.
another example, say you want to call the function twice, and return 2 results as a list:
(defmacro two-funcalls (f v)
`(let ((x ,f))
(list (funcall x ,v) (funcall x ,v))))
CL-USER> (let ((y 10))
(two-funcalls (lambda (z) z) y))
;;=> (10 10)
;; OK
CL-USER> (let ((x 10))
(two-funcalls (lambda (z) z) x))
;; (#<FUNCTION (LAMBDA (Z)) {52D2D4AB}> #<FUNCTION (LAMBDA (Z)) {52D2D4AB}>)
;; NOT OK!
this class of bugs is very nasty, since you can't easily say what's happened.
What is the solution? Obviously not to name the value v inside macro. You need to generate some sophisticated name that no one would reproduce in their code, like my-super-unique-value-identifier-2019-12-27. This would probably save you, but still you can't really be sure. That's why gensym is there:
(defmacro two-funcalls (f v)
(let ((fname (gensym)))
`(let ((,fname ,f))
(list (funcall ,fname ,v) (funcall ,fname ,v)))))
expanding to:
(let ((y 10))
(let ((#:g654 (lambda (z) z)))
(list (funcall #:g654 y) (funcall #:g654 y))))
you just generate the var name for the generated code, it is guaranteed to be unique (meaning no two gensym calls would generate the same name for the runtime session),
(loop repeat 3 collect (gensym))
;;=> (#:G645 #:G646 #:G647)
it still can potentially be clashed with user var somehow, but everybody knows about the naming and doesn't call the var #:GXXXX, so you can consider it to be impossible. You can further secure it, adding prefix
(loop repeat 3 collect (gensym "MY_GUID"))
;;=> (#:MY_GUID651 #:MY_GUID652 #:MY_GUID653)
GENSYM will generate a new symbol at each call. It will be garanteed, that the symbol did not exist before it will be generated and that it will never be generated again. You may specify a symbols prefix, if you like:
CL-USER> (gensym)
#:G736
CL-USER> (gensym "SOMETHING")
#:SOMETHING737
The most common use of GENSYM is generating names for items to avoid name clashes in macro expansion.
Another common purpose is the generaton of symbols for the construction of graphs, if the only thing demand you have is to attach a property list to them, while the name of the node is not of interest.
I think, the task of NFA-generation could make good use of the second purpose.
This is a note to some of the other answers, which I think are fine. While gensym is the traditional way of making new symbols, in fact there is another way which works perfectly well and is often better I find: make-symbol:
make-symbol creates and returns a fresh, uninterned symbol whose name is the given name. The new-symbol is neither bound nor fbound and has a null property list.
So, the nice thing about make-symbol is it makes a symbol with the name you asked for, exactly, without any weird numerical suffix. This can be helpful when writing macros because it makes the macroexpansion more readable. Consider this simple list-collection macro:
(defmacro collecting (&body forms)
(let ((resultsn (make-symbol "RESULTS"))
(rtailn (make-symbol "RTAIL")))
`(let ((,resultsn '())
(,rtailn nil))
(flet ((collect (it)
(let ((new (list it)))
(if (null ,rtailn)
(setf ,resultsn new
,rtailn new)
(setf (cdr ,rtailn) new
,rtailn new)))
it))
,#forms
,resultsn))))
This needs two bindings which the body can't refer to, for the results, and the last cons of the results. It also introduces a function in a way which is intentionally 'unhygienic': inside collecting, collect means 'collect something'.
So now
> (collecting (collect 1) (collect 2) 3)
(1 2)
as we want, and we can look at the macroexpansion to see that the introduced bindings have names which make some kind of sense:
> (macroexpand '(collecting (collect 1)))
(let ((#:results 'nil) (#:rtail nil))
(flet ((collect (it)
(let ((new (list it)))
(if (null #:rtail)
(setf #:results new #:rtail new)
(setf (cdr #:rtail) new #:rtail new)))
it))
(collect 1)
#:results))
t
And we can persuade the Lisp printer to tell us that in fact all these uninterned symbols are the same:
> (let ((*print-circle* t))
(pprint (macroexpand '(collecting (collect 1)))))
(let ((#2=#:results 'nil) (#1=#:rtail nil))
(flet ((collect (it)
(let ((new (list it)))
(if (null #1#)
(setf #2# new #1# new)
(setf (cdr #1#) new #1# new)))
it))
(collect 1)
#2#))
So, for writing macros I generally find make-symbol more useful than gensym. For writing things where I just need a symbol as an object, such as naming a node in some structure, then gensym is probably more useful. Finally note that gensym can be implemented in terms of make-symbol:
(defun my-gensym (&optional (thing "G"))
;; I think this is GENSYM
(check-type thing (or string (integer 0)))
(let ((prefix (typecase thing
(string thing)
(t "G")))
(count (typecase thing
((integer 0) thing)
(t (prog1 *gensym-counter*
(incf *gensym-counter*))))))
(make-symbol (format nil "~A~D" prefix count))))
(This may be buggy.)

Lisp - Passing unquoted list to macro

I'm currently experimenting with macro's in Lisp and I would like to write a macro which can handle syntax as follows:
(my-macro (args1) (args2))
The macro should take two lists which would then be available within my macro to do further processing. The catch, however, is that the lists are unquoted to mimic the syntax of some real Lisp/CLOS functions. Is this possible?
Currently I get the following error when attempting to do something like this:
Undefined function ARGS1 called with arguments ().
Thanks in advance!
I think you need to show what you have tried to do. Here is an example of a (silly) macro which has an argument pattern pretty much what yours is:
(defmacro stupid-let ((&rest vars) (&rest values) &body forms)
;; Like LET but with a terrible syntax
(unless (= (length vars) (length values))
(error "need exactly one value for each variable"))
(unless (every #'symbolp vars)
(error "not every variable is a symbol"))
`(let ,(mapcar #'list vars values) ,#forms))
Then
> (macroexpand '(stupid-let (a b c) (1 2 3) (+ a b c)))
(let ((a 1) (b 2) (c 3)) (+ a b c))
The above macro depends on defmacro's arglist-destructuring, but you don't have to do that:
(defun proper-list-p (l)
;; elaborate version with an occurs check, quadratic.
(labels ((plp (tail tails)
(if (member tail tails)
nil
(typecase tail
(null t)
(cons (plp (rest tail) (cons tail tails)))
(t nil)))))
(plp l '())))
(defmacro stupid-let (vars values &body forms)
;; Like LET but with a terrible syntax
(unless (and (proper-list-p vars) (proper-list-p values))
(error "need lists of variables and values"))
(unless (= (length vars) (length values))
(error "need exactly one value for each variable"))
(unless (every #'symbolp vars)
(error "not every variable is a symbol"))
`(let ,(mapcar #'list vars values) ,#forms))
As a slightly more useful example, here is a macro which is a bit like the CLOS with-slots / with-accessors macros:
(defmacro with-mindless-accessors ((&rest accessor-specifications) thing
&body forms)
"Use SYMBOL-MACROLET to define mindless accessors for THING.
Each accessor specification is either a symbol which names the symbol
macro and the accessor, or a list (macroname accessorname) which binds
macroname to a symbol macro which calls accessornam. THING is
evaluated once only."
(multiple-value-bind (accessors functions)
(loop for accessor-specification in accessor-specifications
if (symbolp accessor-specification)
collect accessor-specification into acs
and collect accessor-specification into fns
else if (and (proper-list-p accessor-specification)
(= (length accessor-specification) 2)
(every #'symbolp accessor-specification))
collect (first accessor-specification) into acs
and collect (second accessor-specification) into fns
else do (error "bad accessor specification ~A" accessor-specification)
end
finally (return (values acs fns)))
(let ((thingn (make-symbol "THING")))
`(let ((,thingn ,thing))
(symbol-macrolet ,(loop for accessor in accessors
for function in functions
collect `(,accessor (,function ,thingn)))
,#forms)))))
So now we can write this somewhat useless code:
> (with-mindless-accessors (car cdr) (cons 1 2)
(setf cdr 3)
(+ car cdr))
4
And this:
> (let ((l (list 1 2)))
(with-mindless-accessors (second) l
(setf second 4)
l))
(1 4)

How can macro variable capture happen with a gensym symbol?

I'm learning common lisp. I have written a version of the once-only macro, which suffers from an unusual variable capture problem.
My macro is this:
(defmacro my-once-only (names &body body)
(let ((syms (mapcar #'(lambda (x) (gensym))
names)))
``(let (,,#(mapcar #'(lambda (sym name) ``(,',sym ,,name))
syms names))
,(let (,#(mapcar #'(lambda (name sym) `(,name ',sym))
names syms))
,#body))))
The canonical version of only-once is this:
(defmacro once-only ((&rest names) &body body)
(let ((gensyms (loop for n in names collect (gensym))))
`(let (,#(loop for g in gensyms collect `(,g (gensym))))
`(let (,,#(loop for g in gensyms for n in names collect ``(,,g ,,n)))
,(let (,#(loop for n in names for g in gensyms collect `(,n ,g)))
,#body)))))
The difference, as far as I can tell, is that the canonical version generates new symbols for every expansion of the macro using only-once. For example:
CL-USER> (macroexpand-1 '(once-only (foo) foo))
(LET ((#:G824 (GENSYM)))
`(LET (,`(,#:G824 ,FOO))
,(LET ((FOO #:G824))
FOO)))
T
CL-USER> (macroexpand-1 '(my-once-only (foo) foo))
`(LET (,`(,'#:G825 ,FOO))
,(LET ((FOO '#:G825))
FOO))
T
The variable my macro uses to store the value of foo is the same for every expansion of this form, in this case it would be #:G825. This is akin to defining a macro like the following:
(defmacro identity-except-for-bar (foo)
`(let ((bar 2))
,foo))
This macro captures bar, and this capture manifests when bar is passed to it, like so:
CL-USER> (let ((bar 1))
(identity-except-for-bar bar))
2
However, I cannot think of any way to pass #:G825 to a macro that uses my-only-once so that it breaks like this, because the symbols gensym returns are unique, and I cannot create a second copy of it outside of the macro. I assume that capturing it is unwanted, otherwise the canonical version wouldn't bother adding the additional layer of gensym. How could capturing a symbol like #:G826 be a problem? Please provide an example where this capture manifests.
We can demonstrate a behavioral difference between my-once-only and once-only:
Let's store our test form in a variable.
(defvar *form* '(lexalias a 0 (lexalias b (1+ a) (list a b))))
This test form exercises a macro called lexalias, which we will define in two ways. First with once-only:
(defmacro lexalias (var value &body body)
(once-only (value)
`(symbol-macrolet ((,var ,value))
,#body)))
(eval *form*) -> (0 1)
Then with my-once-only:
(defmacro lexalias (var value &body body)
(my-once-only (value)
`(symbol-macrolet ((,var ,value))
,#body)))
(eval *form*) -> (1 1)
Oops! The problem is that under my-once-only, both a and b end up being symbol-macrolet aliases for exactly the same gensym; the returned expression (list a b) ends up being something like (list #:g0025 #:g0025).
If you're writing a macro-writing helper that implements once-only evaluation, you have no idea how the symbol is going to be used by the code which calls the macro, whose author uses your once-only tool. There are two big unknowns: the nature of the macro and of its use.
As you can see, if you don't make fresh gensyms, it will not work correctly in all conceivable scenarios.

Is there an existing lisp macro for building up a list?

In Python, I am able to use yield to build up a list without having to define a temporary variable:
def get_chars_skipping_bar(word):
while word:
# Imperative logic which can't be
# replaced with a for loop.
if word[:3] == 'bar':
word = word[3:]
else:
yield foo[0]
foo = foo[1:]
In elisp, I can't see any way of doing this, either built-in or using any pre-existing libraries. I'm forced to manually build a up a list and call nreverse on it. Since this is a common pattern, I've written my own macro:
(require 'dash)
(require 'cl)
(defun replace-calls (form x func)
"Replace all calls to X (a symbol) in FORM,
calling FUNC to generate the replacement."
(--map
(cond
((consp it)
(if (eq (car it) x)
(funcall func it)
(replace-calls it x func)))
(:else it))
form))
(defmacro with-results (&rest body)
"Execute BODY, which may contain forms (yield foo).
Return a list built up from all the values passed to yield."
(let ((results (gensym "results")))
`(let ((,results (list)))
,#(replace-calls body 'yield
(lambda (form) `(push ,(second form) ,results)))
(nreverse ,results))))
Example usage:
(setq foo "barbazbarbarbiz")
(with-results
(while (not (s-equals? "" foo))
;; Imperative logic which can't be replaced with cl-loop's across.
(if (s-starts-with? "bar" foo)
(setq foo (substring foo 3))
(progn
(yield (substring foo 0 1))
(setq foo (substring foo 1))))))
There must be a better way of doing this, or an existing solution, somewhere in elisp, cl.el, or a library.
The Python function is actually a generator. In ANSI Common Lisp, we would usually reach for a lexical closure to simulate a generator, or else us a library to define generators directly, like Pygen. Maybe these approaches can be ported to Emacs Lisp.
AFAIK, people just use push+nreverse like you do. If you want to define your macro in a more robust way (e.g. so it doesn't misfire on something like (memq sym '(yield stop next))) you could do it as:
(defmacro with-results (&rest body)
"Execute BODY, which may contain forms (yield EXP).
Return a list built up from all the values passed to `yield'."
(let ((results (gensym "results")))
`(let ((,results '()))
(cl-macrolet ((yield (exp) `(push ,exp ,results)))
,#body)
(nreverse ,results))))
Maybe something like this:
(setq foo "barbaz")
(cl-loop for i from 0 to (1- (length foo))
collect (string (aref foo i)))
In any case, there's nothing wrong with push and nreverse.
Lisp is different from Python. yield is not used. I also see the use of coroutine-like constructs for this as a mistake. It's the equivalent of the come-from construct. Suddenly routines have multiple context dependent entry points.
In Lisp use functions/closures instead.
In Common Lisp, the LOOP macro allows efficient mappings over vectors. The following code can be abstracted to some mapping function, if preferred:
CL-USER 17 > (defun chars-without-substring (string substring)
(loop with i = 0
while (< i (length string))
when (and (>= (- (length string) i) (length substring))
(string= substring string
:start2 i
:end2 (+ i (length substring))))
do (incf i (length substring))
else
collect (prog1 (char string i) (incf i))))
CHARS-WITHOUT-SUBSTRING
CL-USER 18 > (chars-without-substring "barbazbarbarbiz" "bar")
(#\b #\a #\z #\b #\i #\z)