Extracting field values from emacs org-mode tables - emacs

Delimiter: I am new to emacs.
I am trying to make a function that goes through an org-table and extracts particular values and does some arithmetic on them. The arithmetic is irrelevant to my question but I am having trouble getting field values into a list of integers or floats.
For example, the function
(defun print-field-value()
(interactive)
(setq list '())
(unless (org-table-p) (error "Not in an org-table"))
(goto-char (org-table-begin))
(org-table-next-field)
(while (org-table-p)
(add-to-list 'list (save-excursion (org-table-get-field 1)))
(forward-line))
(print list)
)
when executed on the table
| 7 | 8 | 9 | 10 | 11 | -12 |
| 78 | 46 | 00 | 36 | 64 | 98 |
produces the output
(#(" 78 " 0 4 (fontified t face org-table)) #(" 7 " 0 4 (fontified t face org-table)))
I have two questions for the community:
1. What are these elements of my list? (what type of objects or those)
2. Is there a way I can just get the integers into a list so I can easily perform some arithmetic on the values?
I am ideally looking for a simple way to do this without making another function that parces what the above function returns and extracts the integers. My intuition (maybe flawed) is that there should be an easier way to do this. Am I correct?

#(" 78 " 0 4 (fontified t face org-table)) is a string, in C it would just be noted " 78 ". Next, you don't need to print to return a result : putting list instead of print list was enough in your code.
If you want to discover any mode in emacs, use the fact that all its functions have the same prefix. Here, run describe-function org-table- and press TAB twice to get the list of all functions that work on org tables. You will then see the promising function org-table-analyze, and read its documentation. This way you would probably have come up with the following :
(defun get-field-values ()
(unless (org-table-p) (error "Not in an org-table"))
(org-table-analyze)
(mapcar
(lambda (i)
(mapcar (lambda (j) (string-to-number (org-table-get i j)))
(number-sequence 1 org-table-current-ncol)))
(number-sequence 1 (1- (length org-table-dlines)))))
It will give you the matrix of fields, as a list of list of numbers. Mind this function is not interactive, you run it with M-:

Related

How to access "two" in (("one" . "two")) in elisp

I'm very new to elisp and I'm trying to adapt some existing code.
While looping over a table (generated by the orgmode function org-clock-get-table-data) I try the following:
((equal column "Project") (insert (cdr row)))
which yeilds the following in the Messages buffer:
cond: Wrong type argument: char-or-string-p, (#("Verify CalTime accruals for vacation/sick" 0 41
(fontified t org-category #("Admin" 0 5 (fontified t org-category "Admin" org-category-position 32
line-prefix nil wrap-prefix nil ...)) org-category-position 32 line-prefix #("*" 0 1 (face org-
hide)) wrap-prefix #(" " 0 4 (face org-indent)) ...)) nil 30 (("wps" . "Administration")))
The value that I want to insert is "Administration" so I try this
((equal column "Project") (insert (nth 4 row)))
which yeilds the following in the minibuffer
Wrong type argument, char-or-string-p, (("wps" . "Administration"))
Can someone tell me how I can insert the string "Administration"?
EDIT
Thanks Wes and Drew:
(cdar row)
Doing the above yields the first element of the complex list that I saw in the Messages buffer earlier:
Verify CalTime accruals for vaction/sick
row seems to be a complex list of lists:
((equal column "Project") (insert (car row))); yeilds ^B
((equal column "Project") (insert (cdr row)))
yields:
cond: Wrong type argument: char-or-string-p, (#("Verify CalTime accruals for vacation/sick" 0 41
(fontified t org-category #("Admin" 0 5 (fontified t org-category "Admin" org-category-position 32
line-prefix nil wrap-prefix nil ...)) org-category-position 32 line-prefix #("*" 0 1 (face org-
hide)) wrap-prefix #(" " 0 4 (face org-indent)) ...)) nil 30 (("wps" . "Administration")))
I think each "#" in the output above represents a list item. The fact that (nth 4 row) gives the error output shown above supports this and suggests that something like this might work:
((equal column "Project") (insert (nth 2 (nth 4 row))))
;yeilds wrong type argument, char-or-string-p, nil
So there is probably some function that I need to use to decode that 4th list item....
That 4th element is the orgmode property that has been assigned to the clocktable entry on the row we are parsing. This list of properties is defined in my .emacs:
(setq org-global-properties
;; WPS = Web Platform Services. Time tracking for Google Sheet begun with CalTime Migration
'(("wps_ALL".
"Administration
ASG-Consulting
Chanc-Office-Website
")))
Instead of the assoc + cdr combination, you can also use assoc-default:
ELISP> (assoc-default "one" '(("one" . "two")))
=> "two"
Note that lists of the form (("wps" . "Administration") ("foo" . "bar")) are typically "alists", and so you may need to handle the situation where there is more than one item in your list.
Read: C-hig (elisp) Association Lists RET
and also: (elisp) Dotted Pair Notation
You may obtain a keyed item from an alist with (assoc) or (assq) depending on the form of equality needed for the test. Equivalent strings are not equal objects in elisp, so in this case you want assoc rather than assq.
That gives you the entire (KEY . VALUE) form as a result, and you obtain the cdr of that as usual; hence:
(let ((my-alist '(("wps" . "Administration") ("foo" . "bar"))))
(cdr (assoc "wps" my-alist)))
yields: "Administration"
Lists are complex to think about in elisp. Until you've done enough of them that you stop thinking about them. Ha ha. But read up on the functions like car (picks the first element in a list) and cdr (picks the rest of the rest) to get started. And then you can chain things by combining letters in short sequences at least. So the answer you want is:
(cdar row)

Members in a list - LISP [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 8 years ago.
This question appears to be off-topic because it lacks sufficient information to diagnose the problem. Describe your problem in more detail or include a minimal example in the question itself.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Improve this question
I need to write a program in Lisp to see the number of occurrences of a specific character in a list. For example the occurrences of 1 in the following list [1, 2, 3, 1,1]
A list in Lisp is a sequence of cons nodes: pairs of pointers - the first to the payload datum, and the second to the rest of the list. E.g. for [1,2,3,1,1],
.
/ \
1 .
/ \
2 .
/ \
3 ...... .
/ \
1 NIL
NIL is a special value signaling the empty list, such that the system knows not to try to explore it any further. In Scheme,
(define NIL '())
Recursive list processing paradigm is captured by the notion of fold, where each node . is "replaced" with a binary function f, and the special node NIL is replaced with some special "zero" value z, to create an application chain (f 1 (f 2 (f 3 (... (f 1 z) ...)))). In Scheme,
(define (myfold f z list)
(cond
((null? list) z) ; replace NIL with the initial ("zero") value
(else
(f ; combine
(car list) ; the payload datum, and the delayed,
(lambda () ; by creating a function to calculate it,
(myfold f z ; result of recursively folding
(cdr list))))))) ; the rest of list
That way, the combining function f must process two values: one is a node's payload datum, the other is the (delayed) result of recursively folding, with the same f and z, the rest of the list after that node.
(define (keep-equals v list)
(myfold
(lambda (a r) ; combine ...
(if (equal? v a)
(cons a ... ) ; the same thing goes over the dots, here
... )) ; and here
'() ; replace the NIL of the argument list with this
list))
Since the recursive folding results' calculation is delayed by creating a function to-be-called when the results are needed, we need to "force" that calculation to be performed, when we indeed need those results, by calling that function.
And if you want to count the number of occurrences instead of collecting them in a list, you just need to use a different combining function with a different initial ("zero") value.
In particular, we build a list by consing a value onto the rest of list (with NIL as the initial value, the empty list); whereas we count by incrementing a counter (with 0 as the initial value of that counter).
Calculating e.g. a list's length by folding, we essentially turn its elements each into 1: length [a,b,c,d,e] == 1 + (1 + (1 + (1 + (1 + 0)))). Here, the combining function will need to increment the counter conditionally, only when the payload data are such that we want to count them.
I like pretty well the answers already posted to this question. But it seems like they both involve a fair bit more than the necessary amount of work. On the other hand, given all the thought everyone's put into this, I'm almost embarrassed of how simple my answer is. Anyway, here's what I did:
(defun count-things-in (needle haystack)
"Count the number of NEEDLEs in HAYSTACK."
(reduce '+
(map 'list
#'(lambda (straw)
(if (equalp straw needle) 1 0))
haystack)))
(count-things-in 1 '(1 2 3 1 1))
;; => 3
It's pretty straightforward: you just map over HAYSTACK a function which returns 1 for an element which is EQUALP to NEEDLE or 0 for an element which isn't, and then reduce the resulting list by +. For the given example list, the map operation results in a list (1 0 0 1 1), which the reduce operation then treats as (1 + (0 + (0 + (1 + 1)))), which evaluates to 3.
Benefits of this approach include the use of an equality predicate loose enough to work with strings as well as numbers, and with numbers of different types but the same value -- that is, (equalp 1 1.0) => t; if you desire different behavior, use another equality predicate instead. Using the standard MAP and REDUCE functions, rather than implementing your own, also gives you the benefit of whatever optimizations your Lisp system may be able to apply.
Drawbacks include being not nearly as impressive as anyone else's implementation, and being probably not low-level enough to satisfy the requirements of the asker's homework problem -- not that that latter especially dismays me, given that this answer does satisfy the stated requirement.
I'm new to lisp myself but here is how I would do it. I haven't looked at the other answer yet from Will so I'll check that out after I post this. The member function has the utility of both telling you if it found something in a list, and also returning the rest of that list starting from where it found it:
CL-USER> (member '1 '(0 1 2 3))
(1 2 3)
You could then recursively call a function that uses member and increment a counter from returned values in a variable from a let:
(defun find1 (alist)
(let ((count 0))
(labels ((findit (list)
(let ((part (member '1 list)))
(if part
(progn (incf count)
(findit (rest part)))
0))
count))
(findit alist))))
Here is the result:
CL-USER> (find1 '(1 2 3 4 5))
1
CL-USER> (find1 '(1 1 2 3 4 5))
2
CL-USER> (find1 '(1 1 1 2 3 1 4 5 1 1))
6
You could get rid of that unattractive progn by using cond instead of if
UPDATE: Here is an updated and more elegant version of the above, based on the comments, that I think would qualify as tail recursive as well:
(defun find1 (alist &optional (accum 0))
(let ((part (member '1 alist)))
(if part
(find1 (rest part) (+ accum 1))
accum)))
Here it is in action:
CL-USER> (find1 '(1 2 3 4))
1
CL-USER> (find1 '(1 1 1 1))
4
CL-USER> (find1 '(1 1 0 1 1))
4
CL-USER> (find1 '(0 2 1 0 1 1 0 1 1))
5

How to define symbols that will work like ( and ) by symbol macro?

I am trying define symbols a and b in following way
a + 1 1 b
2
I am trying to do this by using define-symbol-macro
(define-symbol-macro a '( )
(define-symbol-macro b ') )
but this way is not working.
What Lisp does with source code
Common Lisp is an incredibly flexible language, in part because its source code can be easily represented using the same data structures that are used in the language. The most common form of macro expansion transforms the these structures into other structures. These are the kind of macros that you can define with define-symbol-macro, define-compiler-macro, defmacro, and macrolet. Before any of those kind of macroexpansions can be performed, however, the system first needs to read the source from an input stream (typically a file, or an interactive prompt). That's the reader's responsibility. The reader also is capable of executing some special actions when it encounters certain characters, such ( and '. What you're trying to do probably needs to be happening down at the reader level, if you want to have, e.g., (read-from-string "a + 1 1 b") return the list (+ 1 1), which is what you want if you want (eval (read-from-string "a + 1 1 b")) to return 2. That said, you could also define a special custom language (like loop does) where a and b are treated specially.
Use set-macro-character, not define-symbol-macro
This isn't something that you would do using symbol-macros, but rather with macro characters. You can set macro characters using the aptly named set-macro-character. For instance, in the following, I set the macro character for % to be a function that reads a list, using read-delimited-list that should be terminated by ^. (Using the characters a and b here will prove very difficult, because you won't be able to write things like (set-macro-character ...) afterwards; it would be like writing (set-m(cro-ch(r(cter ...), which is not good.)
CL-USER> (set-macro-character #\% (lambda (stream ignore)
(declare (ignore ignore))
(read-delimited-list #\^ stream)))
T
CL-USER> % + 1 1 ^
2
The related set-syntax-from-char
There's a related function that almost does what you want here, set-syntax-from-char. You can use it to make one character behave like another. For instance, you can make % behave like (
CL-USER> (set-syntax-from-char #\% #\()
T
CL-USER> % + 1 1 )
2
However, since the macro character associated with ( isn't looking for a character that has the same syntax as ), but an actual ) character, you can't simply replace ) with ^ in the same way:
CL-USER> (set-syntax-from-char #\^ #\))
T
CL-USER> % + 1 1 ^
; Evaluation aborted on #<SB-INT:SIMPLE-READER-ERROR "unmatched close parenthesis" {1002C66031}>.
set-syntax-from-char is more useful when there's an existing character that, by itself does something that you want to imitate. For instance, if you wanted to make ! an additional quotation character:
CL-USER> (set-syntax-from-char #\! #\')
T
CL-USER> (list !a !(1 2 3))
(A (1 2 3))
or make % be a comment character, like it is in LaTeX:
CL-USER> (set-syntax-from-char #\% #\;)
T
CL-USER> (list 1 2 % 3 4
5 6)
(1 2 5 6)
But consider why you're doing this at allā€¦
Now, even though you can do all of this, it seems like something that would be utterly surprising to anyone who ran into it. (Perhaps you're entering an obfuscated coding competition? ;)) For the reasons shown above, doing this with commonly used characters such as a and b will also make it very difficult to write any more source code. It's probably a better bet to define an entirely new readtable that does what you want, or even write a new parser. even though (Common) Lisp lets you redefine the language, there are still things that it probably makes sense to leaveĀ alone.
A symbol-macro is a symbol that stands for another form. Seems like you want to look at reader macros.
http://clhs.lisp.se/Body/f_set__1.htm
http://dorophone.blogspot.no/2008/03/common-lisp-reader-macros-simple.html
I would second Rainer's comment though, what are you trying to make?
Ok so I love your comment on the reason for this and now I know this is for 'Just because it's lisp' then I am totally on board!
Ok so you are right about lisp being great to use to make new languages because we only have to 'compile' to valid lisp code and it will run. So while we cant use the normal compiler to do the transformation of the symbols 'a and 'b to brackets we can write this ourselves.
Ok so lets get started!
(defun symbol-name-equal (a b)
(and (symbolp a) (symbolp b) (equal (symbol-name a) (symbol-name b))))
(defun find-matching-weird (start-pos open-symbol close-symbol code)
(unless (symbol-name-equal open-symbol (nth start-pos code))
(error "start-pos does not point to a weird open-symbol"))
(let ((nest-index 0))
(loop :for item :in (nthcdr start-pos code)
:for i :from start-pos :do
(cond ((symbol-name-equal item open-symbol) (incf nest-index 1))
((symbol-name-equal item close-symbol) (incf nest-index -1)))
(when (eql nest-index 0)
(return i))
:finally (return nil))))
(defun weird-forms (open-symbol close-symbol body)
(cond ((null body) nil)
((listp body)
(let ((open-pos (position open-symbol body :test #'symbol-name-equal)))
(if open-pos
(let ((close-pos (find-matching-weird open-pos open-symbol close-symbol body)))
(if close-pos
(weird-forms open-symbol close-symbol
`(,#(subseq body 0 open-pos)
(,#(subseq body (1+ open-pos) close-pos))
,#(subseq body (1+ close-pos))))
(error "unmatched weird brackets")))
(if (find close-symbol body :test #'symbol-name-equal)
(error "unmatched weird brackets")
(loop for item in body collect
(weird-forms open-symbol close-symbol item))))))
(t body)))
(defmacro with-weird-forms ((open-symbol close-symbol) &body body)
`(progn
,#(weird-forms open-symbol close-symbol body)))
So there are a few parts to this.
First we have (symbol-name-equal), this is a helper function because we are now using symbols and symbols belong to packages. symbol-name-equal gives us a way of checking if the symbols have the same name ignoring what package they reside in.
Second we have (find-matching-weird). This is a function that takes a list and and index to an opening weird bracket and returns the index to the closing weird bracket. This makes sure we get the correct bracket even with nesting
Next we have (weird-forms). This is the juicy bit and what it does is to recursively walk through the list passed as the 'body' argument and do the following:
If body is an empty list just return it
if body is a list then
find the positions of our open and close symbols.
if only one of them is found then we have unmatched brackets.
if we find both symbols then make a new list with the bit between the start and end positions inside a nested list.
we then call weird forms on this result in case there are more weird-symbol-forms inside.
there are no weird symbols then just loop over the items in the list and call weird-form on them to keep the search going.
OK so that function transforms a list. For example try:
(weird-forms 'a 'b '(1 2 3 a 4 5 b 6 7))
But we want this to be proper lisp code that executes so we need to use a simple macro.
(with-weird-forms) is a macro that takes calls the weird-forms function and puts the result into our source code to be compiled by lisp. So if we have this:
(with-weird-forms (a b)
(+ 1 2 3 a - a + 1 2 3 b 10 5 b 11 23))
Then it macroexpands into:
(PROGN (+ 1 2 3 (- (+ 1 2 3) 10 5) 11 23))
Which is totally valid lisp code, so it will run!
CL-USER> (with-weird-forms (a b)
(+ 1 2 3 a - a + 1 2 3 b 10 5 b 11 23))
31
Finally if you have settled on the 'a' and 'b' brackets you could write another little macro:
(defmacro ab-lang (&rest code)
`(with-weird-forms (a b) ,#code))
Now try this:
(ab-lang a let* a a d 1 b a e a * d 5 b b b a format t "this stupid test gives: ~a" e b b)
Cheers mate, this was great fun to write. Sorry for dismissing the problem earlier on.
This kind of coding is very important as ultimately this is a tiny compiler for our weird language where symbols can be punctuation. Compilers are awesome and no language makes it as effortless to write them as lisp does.
Peace!

Org Table Formatting with Dollar Sign Columns

I'm using this method of creating a column:
http://orgmode.org/worg/org-tutorials/org-spreadsheet-lisp-formulas.html
I have a spreadsheet with 9 columns and column 8 contains filenames which have dollar signs in them, such as:
product_name $500.00.jpg
So I want column 9 to state whether or not the file actually exists, so my org TBLFM is as follows:
#+TBLFM: $9='(file-exists-p (concat "/import/" $8))
So the issue is, whether or not I use the ;L flag at the end of the TBLFM, when applying the formula I immediately get "Invalid Field Specifier: "$500"" because there is no column numbered 500.
Any thoughts as to how I can get this working? I've tried $8 in quotes and not in quotes, with and without the literal flag, and I've tried escaping the dollar sign in the actual columns all with no luck.
Edit: It's important to note that changing my column values to something like: product_name $\ 500.00.jpg does in fact work, however the file-exists-p value returned is obviously incorrect.
Edit: An example org-mode table triggering the error:
| foo | bar | /some/file with $500 in it.jpg | | baz |
| | | | | |
#+TBLFM: $4='(or (file-exists-p $3) "f")
This is the bit of Org code, which handles this case of the formula:
;; Insert the references to fields in same row
(while (string-match "\\$\\(\\([-+]\\)?[0-9]+\\)" form)
(setq n (+ (string-to-number (match-string 1 form))
(if (match-end 2) n0 0))
x (nth (1- (if (= n 0) n0 (max n 1))) fields))
(unless x (error "Invalid field specifier \"%s\""
(match-string 0 form)))
(setq form (replace-match
(save-match-data
(org-table-make-reference x nil numbers lispp))
t t form)))
As you see, there's no way for you to sidestep this bit, however, you could've patched the org-table-eval-formula near that place to allow you some kind of escape sequence to insert the literal sigil, I could get it this far:
;; Insert the references to fields in same row
(while (string-match "\\(^\\|[^\\$]\\)\\$\\(\\([-+]\\)?[0-9]+\\)" form)
(setq n (+ (string-to-number (match-string 2 form))
(if (match-end 3) n0 0))
x (nth (1- (if (= n 0) n0 (max n 1))) fields))
(unless x (error "Invalid field specifier \"%s\""
(match-string 0 form)))
(setq form (replace-match
(save-match-data
(org-table-make-reference x nil numbers lispp))
t t form)))
This will skip $$, unfortunately, this function calls itself recursively until all $ are replaced (I didn't know that). What you could do later: in eLisp code replace double sigils with singles... This worked in my case:
| foo | bar | /some/file with $$500 in it.jpg | f | /some/file with $500 in it.jpg |
| | | | t | |
#+TBLFM: $4='(or (file-exists-p $3) "f")::$5='(format "%s" (replace-regexp-in-string "\\$\\$" "$" $3))
Try escaping the $8 with a dollar char, that is as $$8.

something confusing about define-key (and the issue of when to quote an argument)

It seems one is not supposed to quote KEYMAP when using define-key.
(define-key org-remember-mode-map "\C-c\C-r" 'org-remember-kill)
I'm confused because I think that all arguments of a function that is not quoted are evaluated, and according to the help, define-key is a function, not a macro. I don't see why the value of KEYMAP can be modified after a call of define-key.
(defun increment-value (a)
(setq a (+ 1 a)))
(setq my-num 10)
(increment-value my-num)
my-num ; ===> 10
Update: The answers explain everything, but for those still confused, let me clear up with more examples.
My increment-value example above is equivalent to this:
(let ((n 0))
(print n) ; prints 0
(let ((a n))
(setq a (+ 1 a))
(print a) ; prints 1
)
(print n) ; prints 0
)
What's going on above is, I think, similar to what's going on in this some-map example:
(let ((some-map '(1 2)))
(print some-map) ; prints (1 2)
(let ((a some-map))
(setq a (list 4 (second a)))
(print a) ; prints (4 2)
)
(print some-map) ; prints (1 2)
)
What's going on in define-key is similar to this second some-map example:
(let ((some-map '(1 2)))
(print some-map) ; prints (1 2)
(let ((a some-map))
(setcar a 4)
(print a) ; prints (4 2)
)
(print some-map) ; prints (4 2)
)
Now read the answers again with these three examples in mind and you will get it. Read also http://www.emacswiki.org/emacs/ListModification
You aren't actually changing what 'org-remember-map is (a pointer to a particular list structure), you are modifying the actual structure. Read this info page for details on modifying lists.
Specificially, if you take a look at the documentation for 'make-keymap:
(make-keymap &optional string)
Construct and return a new keymap, of
the form (keymap CHARTABLE . ALIST).
CHARTABLE is a char-table that holds
the bindings for all characters
without modifiers. All entries in it
are initially nil, meaning "command
undefined". ALIST is an assoc-list
which holds bindings for function
keys, mouse events, and any other
things that appear in the input
stream. Initially, ALIST is nil.
You'll see that keymap is a list with three elements. Let me draw that for you (yay M-x artist-mode):
org-remember-map
|
|
v
+----+----+ +----+----+
| | | --+--->+ / | \ |
+-+--+----+ +-/--+--\-+
| | |
v v v
keymap CHARTABLE ALIST
So, the value of the 'org-remember-map is something like the above structure, and when you define a key, what you are doing is changing what is pointed to in the ALIST blob part of the structure.
you're confusing value and name-value mapping.
in your increment-value function, you're not changing a's value as much as changing the mapping of the name a to a new value.
Fundamentally, there is no way to change the value of 10. 10 is 10!
But in the first case, you can either modify the mapping of the name org-remember-mode-map to a fully different map (set a new value), or you can alter the map pointed by that name (its current value). This is what define-key does.
Illustration:
(setq a '(1 2)) -> (1 2)
(setcar a 4) -> 4
a -> (4 2)
Everything you write is completely correct. The thing you are missing is that lists (keymaps are represented as lists) are not values themselves, but containers of values. Thus, you can pass a list to a function and have that function change the values of the list, but the list you have is still the same list.
All the details are in the Cons Cell Type section of the elisp manual.