Org-mode tables: Exclude columns from export - emacs

I have a set of tables in org-mode that I am exporting, but I'd like certain columns used for calculations and consumption by code blocks to be excluded from LaTeX export.
I'm sure I saw a way to do this by specifying a range of columns to export below the table, but I can't find reference to it anywhere on the web so there's a good chance I dreamt it.

Another way to achieve this is by defining a "hidden" column type H in the LaTeX header options of the org file and then use #+ATTR_LATEX: :align llH to indicate that the third column is to be hidden on export (source):
#+LATEX_HEADER: \usepackage{array}
#+LATEX_HEADER: \newcolumntype{H}{>{\setbox0=\hbox\bgroup}c<{\egroup}#{}}
#+ATTR_LATEX: :align llH
|-----+-------+------|
| 2 | 1/2 | junk |
| 4 | 1/4 | junk |
| 8 | 1/2 | junk |

If you are using "Radio Tables" you can do something like
#+ORGTBL: SEND some-name orgtbl-to-latex :skipcols (3)
|-----+-------+------|
| 2 | 1/2 | junk |
| 4 | 1/4 | junk |
| 8 | 1/2 | junk |
See http://www.gnu.org/software/emacs/manual/html_mono/org.html#Radio-tables for all the details.
I believe it may not be possible directly with export via C-c C-e since they offer the same answer at http://comments.gmane.org/gmane.emacs.orgmode/33946 from November 2010.

I use Michael Brand's solution proposed here and cataloged by Derek Feichtinger here (make sure to view the file in raw mode, otherwise the source is hidden by GitHub).
For convenience, I reproduce the code below:
* Exporting tables with some columns hidden
It is desirable to be able and hide columns in exported output. This is often the
case in tables where a lot of computations are done, and where intermediate
results end up in columns that one does not want to end up in the exported document.
This functionality is currently not available by standard org, but since this is Emacs, a simple function
implementing this functionality was published by [[https://github.com/brandm][Michael Brand]] within this [[http://lists.gnu.org/archive/html/emacs-orgmode/2016-05/msg00027.html][emacs-orgmode thread]].
#+BEGIN_SRC emacs-lisp :results silent :exports source
(defun dfeich/org-export-delete-commented-cols (back-end)
"Delete columns $2 to $> marked as `<#>' on a row with `/' in $1.
If you want a non-empty column $1 to be deleted make it $2 by
inserting an empty column before and adding `/' in $1."
(while (re-search-forward "^[ \t]*| +/ +|\\(.*|\\)? +\\(<#>\\) *|" nil t)
(goto-char (match-beginning 2))
(org-table-delete-column)
(beginning-of-line)))
(add-hook 'org-export-before-processing-hook #'dfeich/org-export-delete-commented-cols)
;; (remove-hook 'org-export-before-processing-hook #'dfeich/org-export-delete-commented-cols)
#+END_SRC
The exported table will have col2 removed.
| | col1 | col2 | col3 |
| / | <r> | <#> | |
| | a1 | a2 | a3 |
| | b1 | b2 | b3 |

http://www.gnu.org/software/emacs/manual/html_mono/org.html#The-spreadsheet
3.5.6 Editing and debugging formulas
Use ‘/’ for: Do not export this line. Useful for lines that contain the narrowing ‘’ markers or column group markers.
Note to do this you must use the first column for extra info to do this.

I've found that the most convenient way is to:
make the table not exportable with :noexport: tag,
select the columns I want to be exported with source block, using elisp, and make the result exportable.
Here is an example with elisp.
* Hidden :noexport:
#+NAME: google
| file | total | other | p |
|-----------+-------+-------+----|
| de-01.pdf | 312 | 76 | 76 |
| de-02.pdf | 428 | 101 | 77 |
| de-03.pdf | 1069 | 217 | 80 |
* Exported
Here it comes.
#+begin_src elisp :var data=google :colnames yes
;; select 0th and 3rd column from a table accessible with 'google' name
;; and do some math on it
(mapcar (lambda (e) (list (nth 0 e) (nth 3 e))) data)
#+end_src
#+RESULTS:
| file | p |
|-----------+----|
| de-01.pdf | 76 |
| de-02.pdf | 77 |
| de-03.pdf | 80 |
You can also use any other language that is supported by the source blocks to loop over the results and produce desired output.

Related

How to format table fields as currency in org-mode

I'd like to format fields in an org-mode table as currency
- meaning with currency symbol ($) and commas as thousands separators. I've been using $%.2f to get e.g. $1000.00 but how to get the comma separators e.g. $1,000.00 ? I've RTFM but perhaps I am too dense to get it. Either calc or elisp formula is fine. See sample table below:
| Item | Quantity | Price | Ext |
|----------+----------+--------+----------|
| Widget 1 | 10 | 100.00 | 1000.00 |
| Widget 2 | 5 | 50.00 | 250.00 |
| Widget 3 | 1 | 5.00 | 5.00 |
|----------+----------+--------+----------|
| | | Total | $1255.00 |
#+TBLFM: $4=($2*$3);%.2f::#5$4=vsum(#2..#4);$%.2f
I found no way of doing it consistently, such that you get numbers with thousands separators, and these numbers instead are correctly interpreted for further calculations. So this is not an answer, just to record my research so far.
The following example steals code to format numbers with thousands separators. C-c C-c on the code to define the function, or add to your init file.
Then, the grand total is calculated using elisp, and transformed with the new formatting function.
#+begin_src elisp :results none
(defun my-thousands-separate (num)
"Formats the (possibly floating point) number with a thousands
separator."
(let* ((nstr (number-to-string num))
(dot-ind (string-match "\\." nstr))
(nstr-no-decimal (if dot-ind
(substring nstr 0 dot-ind)
nstr))
(nrest (if dot-ind
(substring nstr dot-ind)
nil))
(pretty nil)
(cnt 0))
(dolist (c (reverse (append nstr-no-decimal nil)))
(if (and (zerop (% cnt 3)) (> cnt 0))
(setq pretty (cons ?, pretty)))
(setq pretty (cons c pretty))
(setq cnt (1+ cnt)))
(concat pretty nrest)))
#+end_src
| Item | Quantity | Price | Ext |
|----------+----------+------------+--------------|
| Widget 1 | 10 | 1001001.00 | 10010010.00 |
| Widget 2 | 5 | 501001.00 | 2505005.00 |
| Widget 3 | 1 | 51001.00 | 51001.00 |
|----------+----------+------------+--------------|
| | | Total | 12,566,016.0 |
#+TBLFM: $4=($2*$3);%.2f::#5$4='(my-thousands-separate (apply '+ '(#2..#4)));N
Note that if you do the same for the row totals, then the comma-separated numbers will not be interpreted correctly for the grand total.
The correct way should be to set the numeric locale and let printf do the trick, but I don't know how to set this for emacs.

org-babel not concatenating strings before sending to code block variable.

I have just started using org-mode and org-babel as a lab notebook. I am trying to use a code block to fill in two columns of a table. The code block seems to work for the first column because those are the right numbers. However, when trying to concatenate a string to the file name in column three so the code blocks works on a different set of files it seems to just run the code block on the original files instead, which produces the same output as column one.
#+name: tRNAs
#+begin_src sh :var filename="" :results silent
cd Data/tRNA
grep -c ">" $filename
#+end_src
#+tblname: sequences
| # of Sequences before QC | # after QC | Original File name|
|--------------------------+------------+------------------|
| 681865 | 681865 | read1 |
| 324223 | 324223 | read2 |
| 1014578 | 1014578 | read3 |
| 971965 | 971965 | read4 |
| 931777 | 931777 | read5 |
| 810798 | 810798 | read6 |
| 965134 | 965134 | read7 |
| 718474 | 718474 | read8 |
|--------------------------+------------+------------------|
#+TBLFM: $1='(org-sbe tRNAs (filename (concat "\"" $3 "\"")))
#+TBLFM: $2='(org-sbe tRNAs (filename (concat "\"final_" $3 "\"")))

How do you copy a formula down a column with the org-mode spreadsheet

After using C-c = to enter a formula, I want that formula to apply to every row in that column. How can I achieve this?
My personal preference is to add a column formula directly into the cell, instead of having to remember so many keyboard shortcuts.
Just start typing into the cell, starting with a = sign. For example, in column 3 type =$1+$2 and that formula will apply to all of the column directly.
You need to C-c C-c to actually apply the formula and calculate results.
For a single cell (non-column) formula, start with := instead.
What you are looking for is to use C-3 C-c = where 3 is the number of rows you want the formula to apply to.
| 3 | 4 | |
| 2 | 8 | |
If I C-2 C-c = in the column after the 4. I can input my formula of $1+$2 which will cascade down to each row, giving me the result of
| 4 | 3 | 7 |
| 2 | 8 | 10 |
#+TBLFM: $3=$1+$2
You are also free to just add #+TBLFM: $3=$1+$2 to the bottom of your table. You can force the formula to revaluate its values with C-c C-c when your point is anywhere over the formula.
C-u C-c C-c re-applies all formulas.
For example, given
| 3 | 4 | _ |
| 2 | 8 | |
| 1 | 12 | |
(where _ marks the location of the cursor) you could type C-c =1, then enter a formula such as
$3=$1+$2
So the table looks like:
| 3 | 4 | 7 |
| 2 | 8 | |
| 1 | 12 | |
#+TBLFM: $3=$1+$2
Now typing C-u C-c C-c yields
| 3 | 4 | 7 |
| 2 | 8 | 10 |
| 1 | 12 | 13 |
1Alternatively, use C-c ' to open a formula editing buffer, write and/or edit the formulas, then close the buffer (with C-x C-s) instead. I find this more convenient since you can use all emacs commands in the buffer, instead of just the limited set you can use in the minibuffer.
Put the point on the cell with the single formula result and then hit C-u C-c *

Convert between an org-mode table and a table.el table without user interaction

I convert org-mode table to table.el table. For that I select the table:
| Option | Type | Value | Descr |
| -[no]h | bool | yes | Print |
| -[no]versio | bool | no | Print |
| -nice | int | 0 | Set t |
| -[no]v | bool | no | Be lo |
| -time | real | -1 | Take |
| -[no]rmvsbd | bool | yes | Removvirtual |
| sites | | | |
| -maxwarn | int | 0 | Numbe |
| procenerate | | | |
| unsta | | | |
| -[no]zero | bool | no | Set pthout |
| defau error | | | |
| -[no]renum | bool | yes | Renum |
| atomty | | | |
and press C-c ~. org-mode then asks me
Convert table to table.el table? (y or n)
How do I answer y programmically? I read the docs of that defun -- there's not way to do it with prefix arg.
Similar functionality in bash:
echo y | script-which-asks-y-or-n
C-c ~ calls the command org-table-create-with-table.el, which provides a bunch of wrappers around calling org-table-convert. If you want to use this function when you know you are already in an org-mode table, you don't need the wrappers, you just need the two (and probably only one) commands: org-table-align and org-table-convert.
So if you're doing this interactively, you can just call M-x org-table-convert and you're done. This assumes the table is already aligned. You can do this by hand by tabbing from one cell to the next, which triggers table alignment. Or you can do it with a small function:
(defun my-convert-tables ()
"No questions asked, just convert the table"
(interactive)
(org-table-align)
(org-table-convert))
You can do this programmatically as follows. You would replace the function name test1 with org-table-create-with-table.el in your defadvice functions that would be otherwise the same as those below.
Using defadvice to run some code before and after the function, we can save the function bound to the symbol y-or-n-p to a global variable and rebind it to a function that simply returns true. After the function we then restore the original functionality.
(setq save-y-or-n-p nil)
(defadvice test1 (around always-yes)
(fset 'save-y-or-n-p (symbol-function 'y-or-n-p))
(fset 'y-or-n-p (lambda (s) t))
ad-do-it
(fset 'y-or-n-p (symbol-function 'save-y-or-n-p)))
(defun test1 ()
(interactive)
(if (y-or-n-p "Happy? ")
(insert "Happy day")
(insert "Unhappy day")))

Copying a range remotely

I've got a table named FOO with the column ("Porc" |- 3 7 15 50 15 7 3) and I'm copying the numbers to another table, shown below. I'm doing it the hard way, cell for cell, but I was wondering if there is a way to copy that range of the remote table (A2 to the bottom) in a single command.
| Pr (%) | ROE de A | ROE de B |
|--------+----------+----------|
| 3 | -11.43 | -34.29 |
| 7 | 0. | -11.43 |
| 15 | 3.43 | 0. |
| 50 | 12. | 17.14 |
| 15 | 20.57 | 34.29 |
| 7 | 24. | 41.14 |
| 3 | 30.86 | 54.86 |
|--------+----------+----------|
| Média | 11.86 | 16.41 |
| Desvio | 8.37 | 17.61 |
#+TBLFM: #2$1=remote(FOO, A2)::#3$1=remote(FOO, A3)::#4$1=remote(FOO, A4)::etc
Thanks
It seems your answer is in the org-mode manual:
$3 = remote(FOO, ###$2)
copy column 2 from table FOO into
column 3 of the current table For the
second example, table FOO must have at
least as many rows as the current
table. Inefficient for large number of
rows.
A Kind of Corollary: Copying all fields in a given row
So just as:
$3 = remote(FOO, ###$2)
copies all the fields from a given column (col2) into column three of the new table, then:
#3 = remote(FOO, #1$$#)
copies all the fields from a given row (row1) into row 3.
There's something about how this standard reference form #r$c interacts with the ## and $# notation that makes this seem a bit abstruse. e.g. this is all the org manual has to say about this remote reference syntax:
## and $# can be used to get the row or column number of the field where the formula result goes.
Umm…?
Posting this example here because I found it all a bit mystifying and hope this helps some else save a few minutes when dealing with rows and tables in the awesome org-mode