Is there a way to push-back to a list in elisp?
The closest thing I found was
(add-to-list 'myList myValue t) ;; t tells it to put to back of the list
The problem, however, is that add-to-list enforces uniqueness. The other alternative is (push 'myList myVal) but that can only push to the front of a list. I tried using (cons myList myVal) but IIRC that returns something other than a list.
The only thing that has worked is
(setq myList (append myList (list myVal)))
but that syntax is hideous and feels like a lot of extra work to do something simple.
Is there a faster way to push to the back of a list. It's clearly possible as seen in (add-to-list), but is there a way to do it without enforcing uniqueness?
In other words, a good old push-back function like with C++ and the <List> class
Lisp lists vs "lists" in other languages
Lisp lists are chains of cons cells ("linked lists"), not specialized sequential containers like in C, and not a weird blend of lists and vectors like in Perl and Python.
This beautiful approach allows the same methodology to be applied to code and data, creating a programmable programming language.
The reasons Lisp does not have a "push-back" function are that
it does not need it :-)
it is not very well defined
No need
Adding to the end is linear in the length of the list, so, for
accumulation, the standard pattern is to combine
push while iterating and
nreverse when done.
Not well defined
There is a reason why add-to-list takes a symbol as the argument (which makes it useless for programming).
What happens when you are adding to an empty list?
You need to modify the place where the list is stored.
What happens when the list shares structure with another object?
If you use
(setq my-list (nconc my-list (list new-value)))
all the other objects are modified too.
If you write, like you suggested,
(setq my-list (append my-list (list new-value)))
you will be allocating (length my-list) cells on each addition.
Try this:
(defun prueba ()
(interactive)
(let ((mylist '(1 2 3)))
(message "mylist -> %s" mylist)
(add-to-list 'mylist 1 t)
(message "mylist -> %s" mylist)
(setq mylist '(1 2 3))
(add-to-list 'mylist 1 t '(lambda (a1 a2) nil))
(message "mylist -> %s" mylist)
))
adding a compare function that always returns nil as the fourth argument to add-to-list allows you
to add duplicates.
Related
In my ongoing quest to recreate lodash in lisp as a way of getting familiar with the language I am trying to write a concat-list function that takes an initial list and an arbitrary number of additional lists and concatenates them.
I'm sure that this is just a measure of getting familiar with lisp convention, but right now my loop is just returning the second list in the argument list, which makes sense since it is the first item of other-lists.
Here's my non-working code (edit: refactored):
(defun concat-list (input-list &rest other-lists)
;; takes an arbitrary number of lists and merges them
(loop
for list in other-lists
append list into input-list
return input-list
)
)
Trying to run (concat-list '(this is list one) '(this is list two) '(this is list three)) and have it return (this is list one this is list two this is list three).
How can I spruce this up to return the final, merged list?
The signature of your function is a bit unfortunate, it becomes easier if you don't treat the first list specially.
The easy way:
(defun concat-lists (&rest lists)
(apply #'concatenate 'list lists))
A bit more lower level, using loop:
(defun concat-lists (&rest lists)
(loop :for list :in lists
:append list))
Going lower, using dolist:
(defun concat-lists (&rest lists)
(let ((result ()))
(dolist (list lists (reverse result))
(setf result (revappend list result)))))
Going even lower would maybe entail implementing revappend yourself.
It's actually good style in Lisp not to use LABELS based iteration, since a) it's basically a go-to like low-level iteration style and it's not everywhere supported. For example the ABCL implementation of Common Lisp on the JVM does not support TCO last I looked. Lisp has wonderful iteration facilities, which make the iteration intention clear:
CL-USER 217 > (defun my-append (&rest lists &aux result)
(dolist (list lists (nreverse result))
(dolist (item list)
(push item result))))
MY-APPEND
CL-USER 218 > (my-append '(1 2 3) '(4 5 6) '(7 8 9))
(1 2 3 4 5 6 7 8 9)
Some pedagogical solutions to this problem
If you just want to do this, then use append, or nconc (destructive), which are the functions which do it.
If you want to learn how do to it, then learning about loop is not how to do that, assuming you want to learn Lisp: (loop for list in ... append list) really teaches you nothing but how to write a crappy version of append using arguably the least-lispy part of CL (note I have nothing against loop & use it a lot, but if you want to learn lisp, learning loop is not how to do that).
Instead why not think about how you would write this if you did not have the tools to do it, in a Lispy way.
Well, here's how you might do that:
(defun append-lists (list &rest more-lists)
(labels ((append-loop (this more results)
(if (null this)
(if (null more)
(nreverse results)
(append-loop (first more) (rest more) results))
(append-loop (rest this) more (cons (first this) results)))))
(append-loop list more-lists '())))
There's a dirty trick here: I know that results is completely fresh so I am using nreverse to reverse it, which does so destructively. Can we write nreverse? Well, it's easy to write reverse, the non-destructive variant:
(defun reverse-nondestructively (list)
(labels ((r-loop (tail reversed)
(if (null tail)
reversed
(r-loop (rest tail) (cons (first tail) reversed)))))
(r-loop list '())))
And it turns out that a destructive reversing function is only a little harder:
(defun reverse-destructively (list)
(labels ((rd-loop (tail reversed)
(if (null tail)
reversed
(let ((rtail (rest tail)))
(setf (rest tail) reversed)
(rd-loop rtail tail)))))
(rd-loop list '())))
And you can check it works:
> (let ((l (make-list 1000 :initial-element 1)))
(time (reverse-destructively l))
(values))
Timing the evaluation of (reverse-destructively l)
User time = 0.000
System time = 0.000
Elapsed time = 0.000
Allocation = 0 bytes
0 Page faults
Why I think this is a good approach to learning Lisp
[This is a response to a couple of comments which I thought was worth adding to the answer: it is, of course, my opinion.]
I think that there are at least three different reasons for wanting to solve a particular problem in a particular language, and the approach you might want to take depends very much on what your reason is.
The first reason is because you want to get something done. In that case you want first of all to find out if it has been done already: if you want to do x and the language a built-in mechanism for doing x then use that. If x is more complicated but there is some standard or optional library which does it then use that. If there's another language you could use easily which does x then use that. Writing a program to solve the problem should be something you do only as a last resort.
The second reason is because you've fallen out of the end of the first reason, and you now find yourself needing to write a program. In that case what you want to do is use all of the tools the language provides in the best way to solve the problem, bearing in mind things like maintainability, performance and so on. In the case of CL, then if you have some problem which naturally involves looping, then, well, use loop if you want to. It doesn't matter whether loop is 'not lispy' or 'impure' or 'hacky': just do what you need to do to get the job done and make the code maintainable. If you want to print some list of objects, then by all means write (format t "~&~{~A~^, ~}~%" things).
The third reason is because you want to learn the language. Well, assuming you can program in some other language there are two approaches to doing this.
the first is to say 'I know how to do this thing (write loops, say) in languages I know – how do I do it in Lisp?', and then iterate this for all the thing you already know how to do in some other language;
the second is to say 'what is it that makes Lisp distinctive?' and try and understand those things.
These approaches result in very approaches to learning. In particular I think the first approach is often terrible: if the language you know is, say, Fortran, then you'll end up writing Fortran dressed up as Lisp. And, well, there are perfectly adequate Fortran compilers out there: why not use them? Even worse, you might completely miss important aspects of the language and end up writing horrors like
(defun sum-list (l)
(loop for i below (length l)
summing (nth i l)))
And you will end up thinking that Lisp is slow and pointless and return to the ranks of the heathen where you will spread such vile calumnies until, come the great day, the golden Lisp horde sweeps it all away. This has happened.
The second approach is to ask, well, what are the things that are interesting about Lisp? If you can program already, I think this is a much better approach to the first, because learning the interesting and distinctive features of a language first will help you understand, as quickly as possible, whether its a language you might actually want to know.
Well, there will inevitably be argument about what the interesting & distinctive features of Lisp are, but here's a possible, partial, set.
The language has a recursively-defined data structure (S expressions or sexprs) at its heart, which is used among other things to represent the source code of the language itself. This representation of the source is extremely low-commitment: there's nothing in the syntax of the language which says 'here's a block' or 'this is a conditiona' or 'this is a loop'. This low-commitment can make the language hard to read, but it has huge advantages.
Recursive processes are therefore inherently important and the language is good at expressing them. Some variants of the language take this to the extreme by noticing that iteration is simply a special case of recursion and have no iterative constructs at all (CL does not do this).
There are symbols, which are used as names for things both in the language itself and in programs written in the language (some variants take this more seriously than others: CL takes it very seriously).
There are macros. This really follows from the source code of the language being represented as sexprs and this structure having a very low commitment to what it means. Macros, in particular, are source-to-source transformations, with the source being represented as sexprs, written in the language itself: the macro language of Lisp is Lisp, without restriction. Macros allow the language itself to be seamlessly extended: solving problems in Lisp is done by designing a language in which the problem can be easily expressed and solved.
The end result of this is, I think two things:
recursion, in addition to and sometimes instead of iteration is an unusually important technique in Lisp;
in Lisp, programming means building a programming language.
So, in the answer above I've tried to give you examples of how you might think about solving problems involving a recursive data structure recursively: by defining a local function (append-loop) which then recursively calls itself to process the lists. As Rainer pointed out that's probably not a good way of solving this problem in Common Lisp as it tends to be hard to read and it also relies on the implementation to turn tail calls into iteration which is not garuanteed in CL. But, if your aim is to learn to think the way Lisp wants you to think, I think it is useful: there's a difference between code you might want to write for production use, and code you might want to read and write for pedagogical purposes: this is pedagogical code.
Indeed, it's worth looking at the other half of how Lisp might want you to think to solve problems like this: by extending the language. Let's say that you were programming in 1960, in a flavour of Lisp which has no iterative constructs other than GO TO. And let's say you wanted to process some list iteratively. Well, you might write this (this is in CL, so it is not very like programming in an ancient Lisp would be: in CL tagbody establishes a lexical environment in the body of which you can have tags – symbols – and then go will go to those tags):
(defun print-list-elements (l)
;; print the elements of a list, in order, using GO
(let* ((tail l)
(current (first tail)))
(tagbody
next
(if (null tail)
(go done)
(progn
(print current)
(setf tail (rest tail)
current (first tail))
(go next)))
done)))
And now:
> (print-list-elements '(1 2 3))
1
2
3
nil
Let's program like it's 1956!
So, well, let's say you don't like writing this sort of horror. Instead you'd like to be able to write something like this:
(defun print-list-elements (l)
;; print the elements of a list, in order, using GO
(do-list (e l)
(print e)))
Now if you were using most other languages you need to spend several weeks mucking around with the compiler to do this. But in Lisp you spend a few minutes writing this:
(defmacro do-list ((v l &optional (result-form nil)) &body forms)
;; Iterate over a list. May be buggy.
(let ((tailn (make-symbol "TAIL"))
(nextn (make-symbol "NEXT"))
(donen (make-symbol "DONE")))
`(let* ((,tailn ,l)
(,v (first ,tailn)))
(tagbody
,nextn
(if (null ,tailn)
(go ,donen)
(progn
,#forms
(setf ,tailn (rest ,tailn)
,v (first ,tailn))
(go ,nextn)))
,donen
,result-form))))
And now your language has an iteration construct which it previously did not have. (In real life this macro is called dolist).
And you can go further: given our do-list macro, let's see how we can collect things into a list:
(defun collect (thing)
;; global version: just signal an error
(declare (ignorable thing))
(error "not collecting"))
(defmacro collecting (&body forms)
;; Within the body of this macro, (collect x) will collect x into a
;; list, which is returned from the macro.
(let ((resultn (make-symbol "RESULT"))
(rtailn (make-symbol "RTAIL")))
`(let ((,resultn '())
(,rtailn nil))
(flet ((collect (thing)
(if ,rtailn
(setf (rest ,rtailn) (list thing)
,rtailn (rest ,rtailn))
(setf ,resultn (list thing)
,rtailn ,resultn))
thing))
,#forms)
,resultn)))
And now we can write the original append-lists function entirely in terms of constructs we've invented:
(defun append-lists (list &rest more-lists)
(collecting
(do-list (e list) (collect e))
(do-list (l more-lists)
(do-list (e l)
(collect e)))))
If that's not cool then nothing is.
In fact we can get even more carried away. My original answer above used labels to do iteration As Rainer has pointed out, this is not safe in CL since CL does not mandate TCO. I don't particularly care about that (I am happy to use only CL implementations which mandate TCO), but I do care about the problem that using labels this way is hard to read. Well, you can, of course, hide this in a macro:
(defmacro looping ((&rest bindings) &body forms)
;; A sort-of special-purpose named-let.
(multiple-value-bind (vars inits)
(loop for b in bindings
for var = (typecase b
(symbol b)
(cons (car b))
(t (error "~A is hopeless" b)))
for init = (etypecase b
(symbol nil)
(cons (unless (null (cddr b))
(error "malformed binding ~A" b))
(second b))
(t
(error "~A is hopeless" b)))
collect var into vars
collect init into inits
finally (return (values vars inits)))
`(labels ((next ,vars
,#forms))
(next ,#inits))))
And now:
(defun append-lists (list &rest more-lists)
(collecting
(looping ((tail list) (more more-lists))
(if (null tail)
(unless (null more)
(next (first more) (rest more)))
(progn
(collect (first tail))
(next (rest tail) more))))))
And, well, I just think it is astonishing that I get to use a programming language where you can do things like this.
Note that both collecting and looping are intentionally 'unhygenic': they introduce a binding (for collect and next respectively) which is visible to code in their bodies and which would shadow any other function definition of that name. That's fine, in fact, since that's their purpose.
This kind of iteration-as-recursion is certainly cool to think about, and as I've said I think it really helps you to think about how the language can work, which is my purpose here. Whether it leads to better code is a completely different question. Indeed there is a famous quote by Guy Steele from one of the 'lambda the ultimate ...' papers:
procedure calls may be usefully thought of as GOTO statements which also pass parameters
And that's a lovely quote, except that it cuts both ways: procedure calls, in a language which optimizes tail calls, are pretty much GOTO, and you can do almost all the horrors with them that you can do with GOTO. But GOTO is a problem, right? Well, it turns out so are procedure calls, for most of the same reasons.
So, pragmatically, even in a language (or implementation) where procedure calls do have all these nice characteristics, you end up wanting constructs which can express iteration and not recursion rather than both. So, for instance, Racket which, being a Scheme-family language, does mandate tail-call elimination, has a whole bunch of macros with names like for which do iteration.
And in Common Lisp, which does not mandate tail-call elimination but which does have GOTO, you also need to build macros to do iteration, in the spirit of my do-list above. And, of course, a bunch of people then get hopelessly carried away and the end point is a macro called loop: loop didn't exist (in its current form) in the first version of CL, and it was common at that time to simply obtain a copy of it from somewhere, and make sure it got loaded into the image. In other words, loop, with all its vast complexity, is just a macro which you can define in a CL which does not have it already.
OK, sorry, this is too long.
(loop for list in (cons '(1 2 3)
'((4 5 6) (7 8 9)))
append list)
I have a list of elements following
("(aviyon" "213" "flyingman" "no))") as list
What i want is that I want to split this list containing strings using parentheses as splitter but also want to include these parentheses in a new list without breaking the order
My desired output of new list(or same list modified)
("(" "aviyon" "213" "flyingman" "no" ")" ")")
I am coming from imperative languages and this would be 15 minute job in Java or C++. But here i'm stuck what to do. I know i have to
1- Get a element from list in a loop
I think this is done with (nth 1 '(listname) )
2- separate without removing delimiter put in to a new list
I found functions such as SPLIT-SEQUENCE but i can't do without removing it and without breaking original order.
Any help would be appreciated.
You can use cl-ppcre library to do the job.
For example:
CL-USER> (ql:quickload :cl-ppcre)
CL-USER> (cl-ppcre:split "([\\(\\)])" "(aviyon" :with-registers-p t)
("" "(" "aviyon")
CL-USER> (cl-ppcre:split "([\\(\\)])" "no))" :with-registers-p t)
("no" ")" "" ")")
CL-USER>
However, it makes empty-strings in a list. Use remove-if function to get rid of them:
CL-USER> (defun empty-string-p (s) (string= s ""))
EMPTY-STRING-P
CL-USER> (remove-if 'empty-string-p
(list "no" ")" "" ")"))
("no" ")" ")")
Finally, you can construct a function which does both, and run it in an imperative loop (yes, Common Lisp is not functional as many think):
CL-USER> (defun remove-empty-strings (l)
(remove-if 'empty-string-p l))
REMOVE-EMPTY-STRINGS
CL-USER> (defun split (s)
(cl-ppcre:split "([\\(\\)])"
s
:with-registers-p t))
SPLIT
CL-USER> (defparameter *the-list* '("(aviyon" "213" "flyingman" "no))"))
*THE-LIST*
CL-USER> (loop for item in *the-list*
for splitted = (split item)
for cleaned = (remove-empty-strings splitted)
append cleaned)
("(" "aviyon" "213" "flyingman" "no" ")" ")")
Let's have another answer, without external libraries.
Like you already did, we can split in the problem into smaller parts:
define a function which builds a list of tokens from a string, all-tokens
apply this function on all strings in your input list, and concatenate the result:
(mapcan #'all-tokens strings)
The first part, taking a state and building a list from it, looks like an unfold operation (anamorphism).
Fold (catamorphism), called reduce in Lisp, builds a value from a list of values and a function (and optionally an initial value).
The dual operation, unfold, takes a value (the state), a function, and generate a list of values.
In the case of unfold, the step function accepts a state and returns new state along with the resulting list.
Here, let's define a state as 3 values: a string, a starting position in the string, and a stack of tokens parsed so far.
Our step function next-token returns the next state.
;; definition follows below
(declare (ftype function next-token))
The main function which gets all tokens from a string just computes a fixpoint:
(defun all-tokens (string)
(do (;; initial start value is 0
(start 0)
;; initial token stack is nil
(tokens))
;; loop until start is nil, then return the reverse of tokens
((not start) (nreverse tokens))
;; advance state
(multiple-value-setq (string start tokens)
(next-token string start tokens))))
We need an auxiliary function:
(defun parenthesisp (c)
(find c "()"))
The step function is defined as follows:
(defun next-token (string start token-stack)
(let ((search (position-if #'parenthesisp string :start start)))
(typecase search
(number
;; token from start to parenthesis
(when (> search start)
(push (subseq string start search) token-stack))
;; parenthesis
(push (subseq string search (1+ search)) token-stack)
;; next state
(values string (1+ search) token-stack))
(null
;; token from start to end of string
(when (< start (1- (length string)))
(push (subseq string start) token-stack))
;; next-state
(values string nil token-stack)))))
You can try with a single string:
(next-token "(aviyon" 0 nil)
"(aviyon"
1
("(")
If you take the resulting state values and reuse them, you have:
(next-token "(aviyon" 1 '("("))
"(aviyon"
NIL
("aviyon" "(")
And here, the second return value is NIL, which ends the generation process.
Finally, you can do:
(mapcan #'all-tokens '("(aviyon" "213" "flyingman" "no))"))
Which gives:
("(" "aviyon" "213" "flyingman" "no" ")" ")")
The above code is not fully generic in the sense that all-tokens knows too much about next-token: you could rewrite it to take any kind of state.
You could also handle sequences of strings using the same mechanism, by keeping more information in your state variable.
Also, in a real lexer you would not want to reverse the whole list of tokens, you would use a queue to feed a parser.
solution
Since you didn't understood Alexander's solution and since I anyway wrote my solution:
;; load two essential libraries for any common lisper
(ql:quickload :cl-ppcre)
(ql:quickload :alexandria)
;; see below to see how to install quicklisp for `ql:quickload` command
;; it is kind of pythons `import` and if not install `pip install`
;; in one command for common-lisp
(defun remove-empty-string (string-list)
(remove-if #'(lambda (x) (string= x "")) string-list))
(defun split-parantheses-and-preserve-them (strings-list)
(remove-empty-string
(alexandria:flatten
(mapcar #'(lambda (el) (cl-ppcre:split "(\\(|\\))"
el
:with-registers-p t))
strings-list))))
;; so now your example
(defparameter *list* '("(aviyon" "213" "flyingman" "no))"))
(split-parantheses-and-preserve-them *list*)
;; returns:
;; ("(" "aviyon" "213" "flyingman" "no" ")" ")")
how this works
(cl-ppcre:split "(\\(|\\))" a-string)
splits the string by ( or ). Because in regex pattern ( or ) are used for capturing the match - like here too (the outer parantheses captures) - you have to escape them. \\( or \\).
So with cl-ppcre:split you can split any string in common lisp by regex-pattern. Super cool and super efficient package written by Edi Weitz. He wrote several super sophisticated packages for common lisp - they are also called ediware or edicls in the community.
By the way - cl-ppcre is even more efficient and faster than gold-standard for regex: the perl regex engine!
:with-regiesters-p t option then preserves the matched delimiter - which has to be captured by parentheses like this: (<pattern>) in the pattern.
mapcar this over the list to apply it on each string element in your string list.
However, what you got after that is a list of lists.
(Each inner list containing the splitted result for each string-element of the list).
Flatten the list by alexandria:flatten.
For many functions not in the standard of lisp, but which you think they are basic - like flatten a list - look always first in alexandria - mostly it has a function you desire - it is a huge library. That is why you need it anyway as a common lisper ;) .
But still, there will be empty strings around to be removed.
That is why I wrote remove-empty-string which uses remove-if - which together with remove-if-not is the standard filtering function for lists.
It takes a predicate function - here (lambda (x) (string= x "")) which gives T if string is an empty string and NIL if not.
It removes all elements in the resulting flattened list in our function, which are empty strings.
In other languages it will be named filter but yeah - sometimes function names in common-lisp are not very well chosen. Sometimes I think we should create alias names and move over to them and keep the old names for backward-compatibility. Clojure has nicer names for functions ... Maybe cl people should overtake clojure function names ...
quicklisp
#Alexander Artemenko wrote exactly my solution - he came first. I will add:
If you are so new to common lisp, maybe you don't know how to use quicklisp.
Do in terminal (linux or macos):
wget https://beta.quicklisp.org/quicklisp.lisp
Otherwise manually download in windows from the address.
I put it into ~/quicklisp folder.
Then in clisp or sbcl do:
(load "~/quicklisp/quicklisp.lisp") ;; just path to where downloaded
;; quicklisp.lisp file is!
;; then install quicklisp:
(quicklisp-quickstart:install)
;; then search for cl-ppcre
(ql:system-apropos "cl-ppcre")
;; then install cl-ppcre
(ql:quickload "cl-ppcre")
;; and to autoload everytime you start sbcl or clisp
;; in linux or mac - sorry I don't now windows that well
;; I have the opinion every programmer should us unix
;; as their OS
;; you have to let quicklisp be loaded when they start
;; by an entry into the init file
;; mostly located in ~/.sbclrc or ~/.clisprc.slip or such ...
;; respectively.
;; quicklisp does an entry automatically if you do:
(ql:add-to-init-file)
;; after installation do:
(quit)
;; If you then restart sbcl or clisp and try:
(ql:quickload :cl-ppcre)
;; it should work, - if not, you have to manually load
;; quicklisp first
(load "~/quicklisp/setup.lisp") ;; or wherever quicklisp's
;; setup.lisp file has been stored in your system!
;; and then you can do
(ql:quickload :cl-ppcre)
;; to install alexandria package then, do
(ql:quickload :alexandria) ;; or "alexandria"
;; ql:quickload installs the package from quicklisp repository,
;; if it cannot find package on your system.
;; learn more about quicklisp, since this is the package
;; manager of common lisp - like pip for python
I'm trying to add an item to an array if this item is not already in the array. Essentially what I want to achieve is to iterate through a list remove any duplicates by checking if they already exist.
Here's how I've started:
(defun reduce(mylist)
(setq newList (list ()) )
(loop for item in mylist
do (
(if (not (member item newList))
)
))
)
(reduce '(3 4 5 4 5))
The error I'm getting is (IF (NOT (MEMBER 'ITEM 'NEWLIST))) should be a lambda expression. I know this is something to do with how item and newList are being accessed and can't figure out how to correct it.
The error is caused by wrapping the body of the do form with parentheses. Parens have meaning in lisp, so 'extra' wrapping like this will break the code.
There are some other mistakes. setq is used to assign to an unbound variable. You should use let to establish a binding instead. The initial value of that variable is a one-length list containing (), while it should probably just be ().
reduce is already a Common Lisp function, so a difference choice of name would be a good idea.
Finally, the formatting isn't either idiomatic or consistent - you have mylist and newList. A lisp programmer would spell these names my-list and new-list.
Here's how it might look when cleaned up a bit. I've left some important parts out for you to fill in.
(defun unique-elements (list)
(let ((result ()))
(loop for item in list
do (when (not (member item result))
...))
...))
(It would be cleaner to use loop's collection machinery for this job, but I decided an example of how to use do would be more helpful.)
Can someone help me understand how push can be implemented as a macro? The naive version below evaluates the place form twice, and does so before evaluating the element form:
(defmacro my-push (element place)
`(setf ,place (cons ,element ,place)))
But if I try to fix this as below then I'm setf-ing the wrong place:
(defmacro my-push (element place)
(let ((el-sym (gensym))
(place-sym (gensym)))
`(let ((,el-sym ,element)
(,place-sym ,place))
(setf ,place-sym (cons ,el-sym ,place-sym)))))
CL-USER> (defparameter *list* '(0 1 2 3))
*LIST*
CL-USER> (my-push 'hi *list*)
(HI 0 1 2 3)
CL-USER> *list*
(0 1 2 3)
How can I setf the correct place without evaluating twice?
Doing this right seems to be a little more complicated. For instance, the code for push in SBCL 1.0.58 is:
(defmacro-mundanely push (obj place &environment env)
#!+sb-doc
"Takes an object and a location holding a list. Conses the object onto
the list, returning the modified list. OBJ is evaluated before PLACE."
(multiple-value-bind (dummies vals newval setter getter)
(sb!xc:get-setf-expansion place env)
(let ((g (gensym)))
`(let* ((,g ,obj)
,#(mapcar #'list dummies vals)
(,(car newval) (cons ,g ,getter))
,#(cdr newval))
,setter))))
So reading the documentation on get-setf-expansion seems to be useful.
For the record, the generated code looks quite nice:
Pushing into a symbol:
(push 1 symbol)
expands into
(LET* ((#:G906 1) (#:NEW905 (CONS #:G906 SYMBOL)))
(SETQ SYMBOL #:NEW905))
Pushing into a SETF-able function (assuming symbol points to a list of lists):
(push 1 (first symbol))
expands into
(LET* ((#:G909 1)
(#:SYMBOL908 SYMBOL)
(#:NEW907 (CONS #:G909 (FIRST #:SYMBOL908))))
(SB-KERNEL:%RPLACA #:SYMBOL908 #:NEW907))
So unless you take some time to study setf, setf expansions and company, this looks rather arcane (it may still look so even after studying them). The 'Generalized Variables' chapter in OnLisp may be useful too.
Hint: if you compile your own SBCL (not that hard), pass the --fancy argument to make.sh. This way you'll be able to quickly see the definitions of functions/macros inside SBCL (for instance, with M-. inside Emacs+SLIME). Obviously, don't delete those sources (you can run clean.sh after install.sh, to save 90% of the space).
Taking a look at how the existing one (in SBCL, at least) does things, I see:
* (macroexpand-1 '(push 1 *foo*))
(LET* ((#:G823 1) (#:NEW822 (CONS #:G823 *FOO*)))
(SETQ *FOO* #:NEW822))
T
So, I imagine, mixing in a combination of your version and what this generates, one might do:
(defmacro my-push (element place)
(let ((el-sym (gensym))
(new-sym (gensym "NEW")))
`(let* ((,el-sym ,element)
(,new-sym (cons ,el-sym ,place)))
(setq ,place ,new-sym)))))
A few observations:
This seems to work with either setq or setf. Depending on what problem you're actually trying to solve (I presume re-writing push isn't the actual end goal), you may favor one or the other.
Note that place does still get evaluated twice... though it does at least do so only after evaluating element. Is the double evaluation something you actually need to avoid? (Given that the built-in push doesn't, I'm left wondering if/how you'd be able to... though I'm writing this up before spending terribly much time thinking about it.) Given that it's something that needs to evaluate as a "place", perhaps this is normal?
Using let* instead of let allows us to use ,el-sym in the setting of ,new-sym. This moves where the cons happens, such that it's evaluated in the first evaluation of ,place, and after the evaluation of ,element. Perhaps this gets you what you need, with respect to evaluation ordering?
I think the biggest problem with your second version is that your setf really does need to operate on the symbol passed in, not on a gensym symbol.
Hopefully this helps... (I'm still somewhat new to all this myself, so I'm making some guesses here.)
I am working on a genetic programming hobby project.
I have a function/macro setup that, when evaluated in a setq/setf form, will generate a list that will look something like this.
(setq trees (make-trees 2))
==> (+ x (abs x))
Then it will get bound out to a lambda function #<FUNCTION :LAMBDA (X) ... > via strategic use of functions/macros
However, I want to get a bit more effective with this than manually assigning to variables, so I wrote something like this:
(setq sample
(let* ((trees (make-trees 2))
(tree-bindings (bind-trees trees))
(evaluated-trees (eval-fitness tree-bindings))))
(list (trees tree-bindings evaluated-trees)))
However, I get EVAL: trees has no value when I place this in a let form. My suspicion is that the macro expansions don't get fully performed in a LET as compared to a SETF, but that doesn't make sense to me.
What is the cause of this issue?
--- edit: yanked my code and put the whole file in a pastebin ---
Supposing that I decide that a setq isn't going to do it for me and I write a simple function to do it:
(defun generate-sample ()
(let ((twiggs (make-trees 2)))
(let ((tree-bindings (bind-trees twiggs)))
(let ((evaluated-trees (eval-fitness tree-bindings)))
(list twiggs tree-bindings evaluated-trees)))))
This yields an explosion of ...help file error messages (??!?)... and "eval: variable twiggs has no value", which stems from the bind-trees definition on SLIME inspection.
I am reasonably sure that I've completely hosed my macros. http://pastebin.org/673619
(Setq make-trees 2) sets the value of the variable make-trees to 2, then returns 2.
I do not see a reason for a macro in what you describe. Is it true that your make-trees creates a single random tree, which can be interpreted as a program? Just define this as a function with defun. I am thinking of something like this:
(defun make-tree (node-number)
(if (= node-number 1)
(make-leaf)
(cons (get-random-operator)
(mapcar #'make-tree
(random-partition (- node-number 1))))))
Let and setq do totally different things. Setq assigns a value to an existing variable, while let creates a new lexical scope with a number of lexical bindings.
I think that you should present more of your code; currently, your question does not make a lot of sense.
Update:
I will fix your snippet's indentation to make things clearer:
(setq sample
(let* ((trees (make-trees 2))
(tree-bindings (bind-trees trees))
(evaluated-trees (eval-fitness tree-bindings))))
(list (trees tree-bindings evaluated-trees)))
Now, as written before, let* establishes lexical bindings. These
are only in scope within its body:
(setq sample
(let* ((trees (make-trees 2))
(tree-bindings (bind-trees trees))
(evaluated-trees (eval-fitness tree-bindings)))
;; here trees, tree-bindings, and evaluated-trees are bound
) ; end of let* body
;; here trees, tree-bindings, and evaluated trees are not in scope anymore
(list (trees tree-bindings evaluated-trees)))
That last line is spurious, too. If those names were bound, it would
return a list of one element, which would be the result of evaluating
the function trees with tree-bindings and evaluated-trees as
arguments.
You might get what you want like this:
(setq sample
(let* ((trees (make-trees 2))
(tree-bindings (bind-trees trees))
(evaluated-trees (eval-fitness tree-bindings)))
(list trees tree-bindings evaluated-trees)))
Another update:
The purpose of macros is to eliminate repeated code when that elimination is not possible with functions. One frequent application is when dealing with places, and you also need them to define new control constructs. As long as you do not see that something cannot work as a function, do not use a macro for it.
Here is some code that might help you:
(defun make-tree-lambda (depth)
(list 'lambda '(x)
(new-tree depth)))
(defun make-tree-function (lambda-tree)
(eval lambda-tree))
(defun eval-fitness (lambda-form-list input-output-list)
"Determines how well the lambda forms approach the wanted function
by comparing their output with the wanted output in the supplied test
cases. Returns a list of mean quadratic error sums."
(mapcar (lambda (lambda-form)
(let* ((actual-results (mapcar (make-tree-function lambda-form)
(mapcar #'first input-output-list)))
(differences (mapcar #'-
actual-results
(mapcar #'second input-output-list)))
(squared-differences (mapcar #'square
differences)))
(/ (reduce #'+ squared-differences)
(length squared-differences))))
lambda-form-list))
(defun tree-fitness (tree-list input-output-list)
"Creates a list of lists, each inner list is (tree fitness). Input
is a list of trees, and a list of test cases."
(mapcar (lambda (tree fitness)
(list tree fitness))
tree-list
(eval-fitness (mapcar #'make-tree-lambda tree-list)
input-output-list)))