The following Emacs Lisp function takes a list of lists and returns a list in which the items of the inner lists have been concatenated to one big list. It is pretty straight-forward and I am convinced something like this must already be part of the standard function library.
(defun flatten (LIST)
(if LIST
(append (car LIST) (flatten (cdr LIST)))
nil))
I am looking for a function that will take a single list of lists as its argument and then append all the inner lists.
(flatten '((a b) (c d)))
will give
(a b c d)
Does anyone know whether this function is already built in, and if so, under which name?
Thanks!
You're either looking for append:
(defun flatten (list-of-lists)
(apply #'append list-of-lists))
If (and only if) you know that you'll always have a list of lists.
Otherwise:
(defun flatten (list)
(mapcan (lambda (x) (if (listp x) x nil)) list))
Emacs 27.1 has flatten-tree:
(flatten-tree '((a b) (c d)))
(a b c d)
See: https://www.gnu.org/software/emacs/manual/html_node/elisp/Building-Lists.html
I stepped into this only recently whilst looking for something different; there is something that might not have been put into evidence by the test data utilized to check the function, depending on whether the original question was meant to refer to generic lists (i.e.: list of list of list of list of...) or just to two-level lists (as in the example).
The solution based on append works fine only with two-level lists, and there is a further issue with the solution based on mapcan.
Basically, the general solution has to be recursive both on car and cdr, as in the flatten defun below.
(setq l '((((1 2) 3) 4) (5 6 7)))
(defun flatten(x)
(cond ((null x) nil)
((listp x) (append (flatten (car x)) (flatten (cdr x))))
(t (list x))))
(defun flatten2(l)
(if l (append (car l) (flatten2 (cdr l))) nil))
(defun flatten3(l)
(mapcan (lambda(x) (if (listp x) x nil)) l))
(flatten l)
(1 2 3 4 5 6 7)
(apply #'append l)
(((1 2) 3) 4 5 6 7)
(flatten2 l)
(((1 2) 3) 4 5 6 7)
The further issue is with the usage of mapcan in flatten3: as mapcan hides an nconc inside, the user must remember that it alters its argument.
l
((((1 2) 3) 4) (5 6 7))
(flatten3 l)
(((1 2) 3) 4 5 6 7)
l
((((1 2) 3) 4 5 6 7) (5 6 7))
Dash is a modern list library for Emacs, and has flatten. It's the second most downloaded package on Melpa, after magit. From the readme:
-flatten (l): Takes a nested list l and returns its contents as a single, flat list.
(-flatten '((1))) ;; => '(1)
(-flatten '((1 (2 3) (((4 (5))))))) ;; => '(1 2 3 4 5)
(-flatten '(1 2 (3 . 4))) ;; => '(1 2 (3 . 4))
-flatten-n (num list): Flatten num levels of a nested list.
(-flatten-n 1 '((1 2) ((3 4) ((5 6))))) ;; => '(1 2 (3 4) ((5 6)))
(-flatten-n 2 '((1 2) ((3 4) ((5 6))))) ;; => '(1 2 3 4 (5 6))
(-flatten-n 3 '((1 2) ((3 4) ((5 6))))) ;; => '(1 2 3 4 5 6)
This package was started 2012-09.
I realize that the original question was "what is the built in function". It appears that there is none. The other solutions do not actually flatten all lists that I tested. This function appears to work. I'm posting it here because this was the first place Google hit when I did my search.
(defun flatten (LIST)
"flattens LIST"
(cond
((atom LIST) (list LIST))
((null (cdr LIST)) (flatten (car LIST)))
(t (append (flatten (car LIST)) (flatten (cdr LIST))))))
e.g.
(flatten (list "a" (list "b" "c" nil) (list (list "d" "e") "f")))
("a" "b" "c" nil "d" "e" "f")
Have a look at nconc
Related
I set myself to the task to write a Common Lisp function that concatenates two lists without using append.
Common Lisp input (concat-lists '(1 2 3) '(4 5 6)) should return (1 2 3 4 5 6)
Even though my solution seems to work it looks overtly complicated
(defun concat-lists(seq1 seq2)
(cond ((not (null seq1)) (cons (car seq1) (concat-lists (cdr seq1) seq2)))
(T (cond ((not (null seq2)) (cons (car seq2) (concat-lists seq1 (cdr seq2))))
(T nil)))))
What I'm looking for is a more elegant solution using reduce where I use seq1 as initial value and then apply a function to each element of seq2, thereby appending each value of the list to seq1. Somehow I always get stuck when trying....
Any help or input is much appreciated. Thanks!
CL-USER 39 > (reduce #'cons
'(1 2 3 4 5)
:initial-value '(a b c d e)
:from-end t)
(1 2 3 4 5 A B C D E)
The solution of Rainer Joswig is really elegant and simple, and respects your request of using reduce.
If you want to see also a recursive, simple solution, then here is the classical one:
(defun concat-lists (seq1 seq2)
(if (null seq1)
seq2
(cons (car seq1) (concat-lists (cdr seq1) seq2))))
(concat-lists '(1 2 3) '(4 5 6))
(1 2 3 4 5 6)
I do understand your requirement for 'reduce'. and here other options:
CL also has 'concatenante'
(concatenate 'list '(1 2 3) '(4 5 6))
There is also the other less complicated (IMHO), and not as elegant.
(defun concat-lists (list1 list2)
(let ((a (copy-list list1))
(b (copy-list list2)))
(rplacd (last a) b)
a))
or
(defun concat-lists (list1 list2)
(let ((a (copy-list list1))
(b (copy-list list2)))
(nconc a b)))
I have a list who's length is divisible by two, and I'm looking for something similar to the answer to this question:
(loop for (a b) on lst while b
collect (+ a b))
However there is overlap between elements:
(1 2 3 4 5) -> (3 5 7 9)
adding 1 and 2 and then 2 and 3 etc.
Where as I have a list like (1 2 3 4) and am looking for something like
((1 2) (3 4))
as output. Is there a way to make loop step correctly over the list?
Another solution.
Something like this should work:
(let ((list '(1 2 3 4)))
(loop :for (a b) :on list :by #'cddr :while b
:collect (cons a b)))
Also a more verbose variant:
(let ((list '(1 2 3 4)))
(loop :for a :in list :by #'cddr
:for b :in (cdr list) :by #'cddr
:collect (cons a b)))
Another approach using the SERIES package.
See also the user manual from Richard C. Waters.
Setup
(ql:quickload :series)
(defpackage :stackoverflow (:use :series :cl))
(in-package :stackoverflow)
Code
(defun pairs (list)
(collect 'list
(mapping (((odd even) (chunk 2 2 (scan 'list list))))
(list odd even))))
scan the content of list as a "serie"
chunk it with M=2 and N=2:
This function has the effect of breaking up the input series items
into (possibly overlapping) chunks of length m. The starting positions
of successive chunks differ by n. The inputs m and n must both be
positive integers.
More precisely, (chunk 2 2 (scan '(1 2 3 4))) produces #Z(1 3) and #Z(2 4)
mapping in parallel over each odd and even element of those series, produce a series of couples, as done by (list odd even).
finally, collect the result, as a list.
Compilation
All the intermediate "series" are compiled away thanks to a stream-fusion mechanism. Here is the macro expansion when pointing at collect:
(LET* ((#:OUT-1120 LIST))
(LET (#:ELEMENTS-1117
(#:LISTPTR-1118 #:OUT-1120)
(#:COUNT-1113 0)
#:CHUNK-1114
#:CHUNK-1115
#:ITEMS-1123
(#:LASTCONS-1106 (LIST NIL))
#:LST-1107)
(DECLARE (TYPE LIST #:LISTPTR-1118)
(TYPE FIXNUM #:COUNT-1113)
(TYPE CONS #:LASTCONS-1106)
(TYPE LIST #:LST-1107))
(SETQ #:COUNT-1113 1)
(SETQ #:LST-1107 #:LASTCONS-1106)
(TAGBODY
#:LL-1124
(IF (ENDP #:LISTPTR-1118)
(GO SERIES::END))
(SETQ #:ELEMENTS-1117 (CAR #:LISTPTR-1118))
(SETQ #:LISTPTR-1118 (CDR #:LISTPTR-1118))
(SETQ #:CHUNK-1114 #:CHUNK-1115)
(SETQ #:CHUNK-1115 #:ELEMENTS-1117)
(COND ((PLUSP #:COUNT-1113) (DECF #:COUNT-1113) (GO #:LL-1124))
(T (SETQ #:COUNT-1113 1)))
(SETQ #:ITEMS-1123
((LAMBDA (ODD EVEN) (LIST ODD EVEN)) #:CHUNK-1114 #:CHUNK-1115))
(SETQ #:LASTCONS-1106
(SETF (CDR #:LASTCONS-1106) (CONS #:ITEMS-1123 NIL)))
(GO #:LL-1124)
SERIES::END)
(CDR #:LST-1107)))
CL-USER 156 > (loop with list = '(1 2 3 4)
while list
collect (loop repeat 2
while list
collect (pop list)))
((1 2) (3 4))
or
CL-USER 166 > (loop with list = '(1 2 3 4 5 6)
while (and list (cdr list))
collect (loop repeat 2 collect (pop list)))
((1 2) (3 4) (5 6))
CL-USER 167 > (loop with list = '(1 2 3 4 5 6 7)
while (and list (cdr list))
collect (loop repeat 2 collect (pop list)))
((1 2) (3 4) (5 6))
How to convert a flat list into an arbitrarily complex tree-like structure? First, a simple example, convert '(1 2 3 4) into '(1 (2 (3 (4)))). I know how to do it with classical recursion:
(defun nestify (xs)
(if (null xs)
(list)
(list (car xs) (nestify (cdr xs)))))
Now, what if the nested structure is arbitrarily complex? For example, I want to convert '(1 2 3 4 5 6 7 8) into '(1 (2 3) (4 (5 6) 7) 8). How can I write a general function that is able to convert a flat list in any such nested structure? I can think of giving a template with dummy values. For example:
* (nestify '(1 2 3 4 5 6 7 8) '(t (t t) (t (t t) t) t))
'(1 (2 3) (4 (5 6) 7) 8)
My first attempt using recursion and custom tree size finding function:
(defun length* (tr)
"Count number of elements in a tree."
(cond ((null tr) 0)
((atom tr) 1)
(t (+ (length* (car tr))
(length* (cdr tr))))))
(defun tree-substitute (xs tpl)
"(tree-substitute '(1 2 3) '(t (t) t)) -> '(1 (2) 3)"
(cond ((null tpl) nil)
((atom (car tpl))
(cons (car xs) (tree (cdr xs) (cdr tpl))))
(t (cons (tree xs (car tpl))
(tree (nthcdr (length* (car tpl)) xs) (cdr tpl))))))
Is there any way to do this better, in a more elegant and concise way? For example, the function converting a list into a tree might not use the template, although I can't think of the method. Can I abstract away recursion and other details and have a neat reduce or some other high-level function?
Turning (1 2 3 4) into (1 (2 (3 (4)))) actually isn't quite as simple as you might hope, if you're using reduce. You need to specify :from-end t if you want to process 4 first, and the reduction function is either called with 3 and 4, if no :initial-value is specified, or with 4 and the initial value, if one is. That means you can use something like this, where the function checks for the special initial case:
(reduce (lambda (x y)
(if y
(list x y)
(list x)))
'(1 2 3 4)
:from-end t
:initial-value nil)
;=> (1 (2 (3 (4))))
A solution that involves a template is much more interesting, in my opinion. It's easy enough to define a maptree function that maps a function over a tree and returns a new tree with the function results:
(defun maptree (function tree)
"Return a tree with the same structure as TREE, but
whose elements are the result of calling FUNCTION with
the element from TREE. Because TREE is treated as an
arbitrarily nested structure, any occurrence of NIL is
treated as an empty tree."
(cond
((null tree) tree)
((atom tree) (funcall function tree))
((cons (maptree function (car tree))
(maptree function (cdr tree))))))
(maptree '1+ '(1 2 (3 (4 5)) (6 7)))
;=> (2 3 (4 (5 6)) (7 8))
Given the maptree function, it's not hard to call it with a function that provides an element from a list of elements, until that list of element is exhausted. This provides a definition of substitute-into:
(defun substitute-into (items tree)
"Return a tree like TREE, but in which the elements
of TREE are replaced with elements drawn from ITEMS.
If there are more elements in TREE than there are in
ITEMS, the original elements of TREE remain in the result,
but a new tree structure is still constructed."
(maptree #'(lambda (x)
(if (endp items) x
(pop items)))
tree))
(substitute-into '(1 2 3 4 5) '(t (u (v)) (w x)))
;=> (1 (2 (3)) (4 5))
(substitute-into '(1 2 3 4 5) '(t u (v w x) y z))
;=> (1 2 (3 4 5) Y Z)
See Also
The maptree above is actually just a special case of a more general reduce, or fold, function for trees. Have a look at Using reduce over a tree in Lisp for some more information about how you can fold over trees. In this case, you could use my tree-reduce function from my answer to that question:
(defun tree-reduce (node-fn leaf-fn tree)
(if (consp tree)
(funcall node-fn
(tree-reduce node-fn leaf-fn (car tree))
(tree-reduce node-fn leaf-fn (cdr tree)))
(funcall leaf-fn
tree)))
and define maptree in terms of it:
(defun maptree (function tree)
(tree-reduce 'cons function tree))
My attempt:
(defun mimicry (source pattern)
(labels ((rec (pattern)
(mapcar (lambda (x)
(if (atom x)
(pop source)
(rec x)))
pattern)))
(rec pattern)))
Test:
CL-USER> (mimicry '(1 2 3 4 5) '(t (u (v)) (w x)))
(1 (2 (3)) (4 5))
I have a sorted list of integers, (1 2 4 5 6 6 7 8 10 10 10). I want to group them all, so that I get ((1) (2) (4) (5) (6 6) (7) (8) (10 10 10)).
So far I have this, which works:
(let ((current-group (list)) (groups (list)))
(dolist (n *sorted*)
(when (and (not (null current-group)) (not (eql (first current-group) n)))
(push current-group groups)
(setf current-group (list)))
(push n current-group))
(push current-group groups)
(nreverse groups))
But I'm sure there must be a much more LISPy way to do this. Any ideas?
Not that bad. I would write it this way:
(defun group (list)
(flet ((take-same (item)
(loop while (and list (eql (first list) item))
collect (pop list))))
(loop while list
collect (take-same (first list)))))
CL-USER 1 > (group '(1 2 4 5 6 6 7 8 10 10 10))
((1) (2) (4) (5) (6 6) (7) (8) (10 10 10))
There's already an accepted answer, but I think it's worth looking at another way of decomposing this problem, although the approach here is essentially the same). First, let's define cut that takes a list and a predicate, and returns the prefix and suffix of the list, where the suffix begins with the first element of the list that satisfies the predicate, and the prefix is everything before that that didn't:
(defun cut (list predicate)
"Returns two values: the prefix of the list
containing elements that do no satisfy predicate,
and the suffix beginning with an element that
satisfies predicate."
(do ((tail list (rest tail))
(prefix '() (list* (first tail) prefix)))
((or (funcall predicate (first tail))
(endp tail))
(values (nreverse prefix) tail))))
(cut '(1 1 1 2 2 3 3 4 5) 'evenp)
;=> (1 1 1) (2 2 3 3 4 5)
(let ((l '(1 1 2 3 4 4 3)))
(cut l (lambda (x) (not (eql x (first l))))))
;=> (1 1), (2 3 4 4 3)
Then, using cut, we can move down the an input list taking prefixes and suffixes with a predicate that's checking whether an element is not eql to the first element of the list. That is, beginning with (1 1 1 2 3 3) you'd cut with the predicate checking for "not eql to 1", to get (1 1 1) and (2 3 3). You'd add the first to the list of groups, and the second becomes the new tail.
(defun group (list)
(do ((group '()) ; group's initial value doesn't get used
(results '() (list* group results))) ; empty, but add a group at each iteration
((endp list) (nreverse results)) ; return (reversed) results when list is gone
(multiple-value-setq (group list) ; update group and list with the prefix
(cut list ; and suffix from cutting list on the
(lambda (x) ; predicate "not eql to (first list)".
(not (eql x (first list))))))))
(group '(1 1 2 3 3 3))
;=> ((1 1) (2) (3 3 3))
On implementing cut
I tried to make that cut relatively efficient, insofar as it only makes one pass through the list. Since member returns the entire tail of the list that begins with the found element, you can actually use member with :test-not to get the tail that you want:
(let ((list '(1 1 1 2 2 3)))
(member (first list) list :test-not 'eql))
;=> (2 2 3)
Then, you can use ldiff to return the prefix that comes before that tail:
(let* ((list '(1 1 1 2 2 3))
(tail (member (first list) list :test-not 'eql)))
(ldiff list tail))
;=> (1 1 1)
It's a simple matter, then, to combine the approaches and to return the tail and the prefix as multiples values. This gives a version of cut that takes only the list as an argument, and might be easier to understand (but it's a bit less efficient).
(defun cut (list)
(let ((tail (member (first list) list :test-not 'eql)))
(values (ldiff list tail) tail)))
(cut '(1 1 2 2 2 3 3 3))
;=> (1 1), (2 2 2 3 3)
I like to use reduce:
(defun group (lst)
(nreverse
(reduce (lambda (r e) (if (and (not (null r)) (eql e (caar r)))
(cons (cons e (car r)) (cdr r))
(cons (list e) r)))
lst
:initial-value nil)))
or using push:
(defun group (lst)
(nreverse
(reduce (lambda (r e)
(cond
((and (not (null r)) (eql e (caar r))) (push e (car r)) r)
(t (push (list e) r))))
lst
:initial-value nil)))
How to calculate the difference between two sets in Emacs Lisp? The sets should be lists.
The programm should be very simple and short, or else I won't understand it. I'm a newbee.
Thx
There is a set-difference function in the Common Lisp extensions:
elisp> (require 'cl-lib)
cl-lib
elisp> (cl-set-difference '(1 2 3) '(2 3 4))
(1)
When I write Elisp code that has lots of list data transformations, I use dash library, because it has loads of functions to work with lists. Set difference can be done with -difference:
(require 'dash)
(-difference '(1 2 3 4) '(3 4 5 6)) ;; => '(1 2)
Disclaimer: this is not an efficient way to do it in eLisp. An efficient way is through a hash-table with a hash function, but since you asked about lists, then here it is:
(defun custom-set-difference (a b)
(remove-if
#'(lambda (x) (and (member x a) (member x b)))
(append a b)))
(custom-set-difference '(1 2 3 4 5) '(2 4 6))
(1 3 5 6)
(defun another-set-difference (a b)
(if (null a) b
(let (removed)
(labels ((find-and-remove
(c)
(cond
((null c) nil)
((equal (car c) (car a))
(setq removed t) (cdr c))
(t (cons (car c) (find-and-remove (cdr c)))))))
(setf b (find-and-remove b))
(if removed
(another-set-difference (cdr a) b)
(cons (car a) (another-set-difference (cdr a) b)))))))
(another-set-difference '(1 2 3 4 5) '(2 4 6))
(1 3 5 6)
The second is slightly more efficient, because it will remove the elements as it makes consequent checks, but the first is shorter and more straight-forward.
Also note that lists are not good representation of sets because they naturally allow repetition. Hash maps are better for that purpose.
Here is a simple & short definition, which should be easy to understand. It is essentially the same as the set-difference function in the Common Lisp library for Emacs, but without any treatment of a TEST argument.
(defun set-diff (list1 list2 &optional key)
"Combine LIST1 and LIST2 using a set-difference operation.
Optional arg KEY is a function used to extract the part of each list
item to compare.
The result list contains all items that appear in LIST1 but not LIST2.
This is non-destructive; it makes a copy of the data if necessary, to
avoid corrupting the original LIST1 and LIST2."
(if (or (null list1) (null list2))
list1
(let ((keyed-list2 (and key (mapcar key list2)))
(result ()))
(while list1
(unless (if key
(member (funcall key (car list1)) keyed-list2)
(member (car list1) list2))
(setq result (cons (car list1) result)))
(setq list1 (cdr list1)))
result)))
GNU Emacs Lisp Reference Manual, Sets and Lists suggests using cl-lib's
cl-set-difference LIST1 LIST2 &key :test :test-not :key
(require 'cl-lib)
(cl-set-difference '(1 2 3) '(2 3 4))
(1)