Batch export of org-mode files from the command line - emacs

Assume that I have in a certain directory several org-mode files: foo1.org, foo2.org, etc. I would like to have a script (maybe a makefile) that I could invoke something like
$ generate-pdfs
and foo1.pdf, foo2.pdf, etc. will be generated.
I thought that something like emacs --batch --eval <MAGIC> is a good start, but I don't know the magic.
A solution that is solely inside emacs could be of interest as well.

As you said, Emacs has the --batch option to perform operations with Emacs from the shell. In addition to that, you can use the -l flag to load Emacs Lisp code from a file and execute it, and the -f flag to execute a single Lisp function.
Here is a basic example, which exports a single org-mode file to HTML:
emacs myorgfile.org --batch -f org-html-export-to-html --kill
Perhaps you want something more advanced like exporting/publishing a full org-mode project. I do not have sample code for that, but it should not be too complicated.
I also have a sample Makefile I wrote some time ago to export all org-mode files in the directory to HTML (and also copy the HTML files to another directory):
OUT_DIR=/some/output/dir/html
# Using GNU Make-specific functions here
FILES=$(patsubst %.org,$(OUT_DIR)/%.html,$(wildcard *.org))
.PHONY: all clean install-doc
all: install-doc
install-doc: $(OUT_DIR) $(FILES)
$(OUT_DIR):
mkdir -v -p $(OUT_DIR)
%.html: %.org
emacs $< --batch -f org-html-export-to-html--kill
$(OUT_DIR)/%.html: %.html
install -v -m 644 -t $(OUT_DIR) $<
rm $<
clean:
rm *.html
EDIT:
With Org-mode 8 and the new export engine the function for HTML export has changed.
To make the previous examples work with Org 7 or older, replace org-html-export-to-html with org-export-as-html.

I expect to publish (by the end of this week-end) OrgMk, a suite of Makefile and standalone Bash scripts (usable as well under Cygwin) just to do that! Even more: generation of HTML, Ascii, Beamer, etc.
You'll find it on my GitHub account: https://github.com/fniessen/ (where I already have Emacs configuration files, color themes and other stuff such as an Org Babel refcard -- in progress).

Mark a few org files in dired and call this:
(defun dired-org-to-pdf ()
(interactive)
(mapc
(lambda (f)
(with-current-buffer
(find-file-noselect f)
(org-latex-export-to-pdf)))
(dired-get-marked-files)))
If you know what async is, wrap the call as it can take a while.
update:
Here's a version that combines the awesome dired approach with the lame
other one:)
(defun dired-org-to-pdf ()
(interactive)
(let ((files
(if (eq major-mode 'dired-mode)
(dired-get-marked-files)
(let ((default-directory (read-directory-name "dir: ")))
(mapcar #'expand-file-name
(file-expand-wildcards "*.org"))))))
(mapc
(lambda (f)
(with-current-buffer
(find-file-noselect f)
(org-latex-export-to-pdf)))
files)))

Related

emacs cd for all splits

In emacs you can do M-x cd to change the default directory.
I usually have 5 splits/windows, so the cd I do in the first split does't affect the others. What If I want the cd to affect all my splits/buffers.
Is there an alternative command I can use?
There's nothing built-in but it's not too hard to write the function by hand:
(defun cd-all-windows (dir)
(interactive "Ddirectory: ")
(dolist (window (window-list))
(with-current-buffer (window-buffer window)
(cd dir))))
Put that in your .emacs and you should be able to run M-x cd-all-windows to get the desired effect.

Honor in-file settings for batch export of org files as HTML

I would like to use Emacs in batch mode to export a number of org files to HTML from the command-line. And I would like to get the same result than interactively using C-cC-eh, in particular:
honor file-local variables (such as org-export-publishing-directory)
honor all options specified through #+KEYWORD: headlines
Starting from the example given in org-export-as-html-batch, I got to this point:
emacs --batch \
--visit=/tmp/foo.org \
--eval "(defun safe-local-variable-p (sym val) t)" \
--funcall hack-local-variables \
--eval "(setq org-export-headline-levels 4)" \
--funcall org-export-as-html-batch
However, some problems remain:
I need to explicitly specify the headline level and I fail to see why all other #+OPTIONS are honored (like toc:nil) but not this one
I had to manually trigger file-local variables parsing using hack-local-variables (I guess it is not automatically done in batch mode) but more importantly I had to resort to a hack to mark all local variables as safe (I'm sure there is much space for improvement here).
NB:
In case it matters, I'm using emacs 23.2.1 (Debian Squeeze flavour)
Here is a sample org file on which I tested this:
#+TITLE: Foo
#+OPTIONS: H:4 toc:nil author:nil
* 1
** 2
*** 3
**** 4
# Local Variables:
# org-export-publishing-directory: "/some/where";
# End:
I eventually got the following script, which seems to fulfill all my requirements:
#!/bin/sh
":"; exec emacs --script "$0" -- "$#" # -*-emacs-lisp-*-
;;
;; Usage:
;; org2html FILE1 [FILE2 ...]
;; Mark org-related variables as safe local variables,
;; regardless of their value.
(defun my/always-safe-local-variable (val) t)
(dolist (sym '(org-export-publishing-directory
org-export-html-preamble
org-export-html-postamble))
(put sym 'safe-local-variable 'my/always-safe-local-variable))
(defun my/org-export-as-html (filename)
"Export FILENAME as html, as if `org-export-to-html' had been called
interactively.
This ensures that `org-export-headline-levels' is correctly read from
the #+OPTIONS: headline."
(save-excursion
(find-file filename)
(message "Exporting file `%s' to HTML" filename)
(call-interactively 'org-export-as-html)))
(mapcar 'my/org-export-as-html
(cdr argv)) ;; "--" is the first element of argv
A few notes on this script:
The executable emacs-lisp script trick comes from this question.
The only way I found to use the org-export-headline-levels value from the #+OPTIONS: headline is to call org-export-as-html interactively, instead of org-export-as-html-batch.
hack-local-variables does not need to be explicitly called, provided that local variables are marked as safe before the file is opened.
I think it is better to only mark org-related variables as safe, using the safe-local-variable symbol property.

P4CONFIG with emacs

I would like to see examples of how to setup perforce, using the config file functionality where emacs is used as the diff and merge programs (P4DIFF and P4MERGE settings). Even better if this is on Windows.
I'm also struggling with getting the P4EDITOR to work correctly when using emacsclientw, specifically specifying the alternate-editor functionality.
Any tips, suggestions, example configs are very welcome.
Here's a different trick I used to use. It adds a few command line options to emacs so that you can do diffs and merges in a new emacs instance (again using ediff).
;; -diff
(defun command-line-diff (switch)
(let ((file1 (pop command-line-args-left))
(file2 (pop command-line-args-left)))
(ediff file1 file2)))
(add-to-list 'command-switch-alist '("-diff" . command-line-diff))
;; -merge
(defun command-line-merge (switch)
(let ((base (pop command-line-args-left))
(sccs (pop command-line-args-left))
(mine (pop command-line-args-left))
(merg (pop command-line-args-left)))
(ediff-merge-with-ancestor sccs mine base () merg)))
(add-to-list 'command-switch-alist '("-merge" . command-line-merge))
Just put that in your .emacs file. Then you can set your P4DIFF program to be emacs -diff and your P4MERGE program to be emacs -merge.
I'm assuming you're already using p4.el.
Here's a function that will allow you to set your p4-client-config easily:
(defun p4-go (config)
(interactive
(list (read-file-name "P4 Config file: "
(concat (getenv "HOME") "/etc/perforce/")
""
t)))
(p4-set-client-config (expand-file-name config))
t)
Then I just run M-x p4-go <RET> conf <RET>.
My ~/etc/perforce/conf file looks like:
P4CLIENT=ewarmenhoven-ppd
P4PORT=perforce.netflix.com:1666
P4USER=ewarmenhoven
P4EDITOR=emacsclient
P4DIFF=diff -dupU8
P4MERGE=~/bin/emerge
The emerge merge program is just a short little shell script that calls emacsclient appropriately:
#!/bin/bash
base=$1
sccs=$2
mine=$3
merg=$4
emacsclient -e "(ediff-merge-files-with-ancestor \"$base\" \"$sccs\" \"$mine\" () \"$merg\")"
emacsclient "$merg"
If you're using cygwin it should work just fine.
For doing diffs, if it's running from the shell then I want the output in the shell, hence just using normal diff. If it's not, I use p4-ediff, which is bound to C-x p - by default.
The awesome answer by Eric doesn't work properly in latest emacs because of welcome screen. In order to hide the welcome screen (so that you may get the diff properly) please refer Unable to hide welcome screen in Emacs.
Another nifty setting which opens the diff in regular vertical mode is setting the below config variable
(custom-set-variables
;; custom-set-variables was added by Custom -- don't edit or cut/paste it!
;; Your init file should contain only one such instance.
'(ediff-split-window-function (quote split-window-horizontally)))

Dired copy asynchronously

Is there a way to modify/tell dired to copy files asynchronously? If you mark multiple files in dired and then use 'C' to copy them, emacs locks up until every file is copied. I instead want this copy to be started, and for me to continue editing as it goes on in the background. Is there a way to get this behaviour?
EDIT: Actually, C calls 'dired-do-copy' in dired-aux, not in dired itself. Sorry for any confusion.
I think emacs is mostly limited to a single thread - so this may not be directly possible through standard dired commands such as 'C' copy.
However, there is a dired command "dired-do-shell-command" which calls out to a shell to do the work in the background. If you select the files you want to copy and then use key '!' (this runs dired-do-shell-command) then type 'cp ? [destination]' (possibly can use 'copy' if you are on windows). I haven't tested this - so see help on "dired-do-shell-command" for full details.
See also the Emacs function dired-do-async-shell-command.
For an even more generic solution see https://github.com/jwiegley/emacs-async with which you also can evaluate arbitrary Emacs Lisp code through call to a separate Emacs process (which of course incurs a bit of extra latency). More specifically regard file operations see the file dired-async.el in this repo.
Also note that there is work on threading in Emacs under the working name Concurrent Emacs but it's not there yet. See http://www.emacswiki.org/emacs/ConcurrentEmacs for details.
I found this answer quite helpful: https://emacs.stackexchange.com/a/13802/10761. Reading that answer shows that you can make it so that dired will copy with the scp method instead of the ssh method (the latter initially encodes the file with gzip and that can be quite slow). The scp method will only copy with the scp program when the file is larger than tramp-copy-size-limit (which is 10240 by default). Using this scp method in conjunction with dired-async-mode is very nice, as it will not only copy quickly with scp, but it will also do it asynchronously and out of your way.
Also, I think this is useful: https://oremacs.com/2016/02/24/dired-rsync/. It provides this snippet of code to use rsync to copy files in dired:
;;;###autoload
(defun ora-dired-rsync (dest)
(interactive
(list
(expand-file-name
(read-file-name
"Rsync to:"
(dired-dwim-target-directory)))))
;; store all selected files into "files" list
(let ((files (dired-get-marked-files
nil current-prefix-arg))
;; the rsync command
(tmtxt/rsync-command
"rsync -arvz --progress "))
;; add all selected file names as arguments
;; to the rsync command
(dolist (file files)
(setq tmtxt/rsync-command
(concat tmtxt/rsync-command
(shell-quote-argument file)
" ")))
;; append the destination
(setq tmtxt/rsync-command
(concat tmtxt/rsync-command
(shell-quote-argument dest)))
;; run the async shell command
(async-shell-command tmtxt/rsync-command "*rsync*")
;; finally, switch to that window
(other-window 1)))
(define-key dired-mode-map "Y" 'ora-dired-rsync)

Ignore Emacs auto-generated files in a diff

How do I make diff ignore temporary files like foo.c~? Is there a configuration file that will make ignoring temporaries the default?
More generally: what's the best way to generate a "clean" patch off a tarball? I do this rarely enough (submitting a bug fix to an OSS project by email) that I always struggle with it...
EDIT: OK, the short answer is
diff -ruN -x *~ ...
Is there a better answer? E.g., can this go in a configuration file?
This doesn't strictly answer your question, but you can avoid the problem by configuring Emacs to use a specific directory to keep the backup files in. There are different implementations for Emacs or XEmacs.
In GNU Emacs
(defvar user-temporary-file-directory
(concat temporary-file-directory user-login-name "/"))
(make-directory user-temporary-file-directory t)
(setq backup-by-copying t)
(setq backup-directory-alist
`(("." . ,user-temporary-file-directory)
(,tramp-file-name-regexp nil)))
(setq auto-save-list-file-prefix
(concat user-temporary-file-directory ".auto-saves-"))
(setq auto-save-file-name-transforms
`((".*" ,user-temporary-file-directory t)))
In XEmacs
(require 'auto-save)
(require 'backup-dir)
(defvar user-temporary-file-directory
(concat (temp-directory) "/" (user-login-name)))
(make-directory user-temporary-file-directory t)
(setq backup-by-copying t)
(setq auto-save-directory user-temporary-file-directory)
(setq auto-save-list-file-prefix
(concat user-temporary-file-directory ".auto-saves-"))
(setq bkup-backup-directory-info
`((t ,user-temporary-file-directory full-path)))
You can also remove them all with a simple find command
find . -name “*~” -delete
Note that the asterisk and tilde are in double quotes to stop the shell expanding them.
By the way, these aren't strictly temporary files. They are a backup of the previous version of the file, so you can manually "undo" your last edit at any time in the future.
You can create an ignore file, like this:
core.*
*~
*.o
*.a
*.so
<more file patterns you want to skip>
and then run diff with -X option, like this:
diff -X ignore-file <other diff options you use/need> path1 path2
There used to be a .diffignore file "close" to the Linux kernel (maybe an informal file), but I couldn't find it anymore. Usually you keep using this ignore-file, just adding new patterns you want to ignore.
You can create a small sunction/script to it, like:
#!/bin/bash
olddir="/tmp/old"
newdir="/tmp/new"
pushd $newdir
for files in $(find . -name \*.c)
do
diff $olddir/$file $newdir/$file
done
popd
This is only one way to script this. The simple way. But I think you got the idea.
Other suggestion is configuring in emacs a backup dir, so your backup files go always to the same place, outside your work dir!
The poster has listed this as the 'short answer':
diff -ruN -x *~ ...
but feeding this to shell will cause the * to be globbed before diff is invoked.
This is better:
diff -r -x '*~' dir1 dir2
I omit the -u and -N flags as those are matters of taste and not relevant to the question at hand.