Setup a personal wiki in Emacs Org-mode - emacs

I would like to setup a personal wiki in org-mode.
So far, I have tried this two ways. My first attempt was using a single "Scientific Notebook.org" file. In this went all my notes - each with its own headline, date and tags (same format as a blog post). I then turned on org-velocity to quickly navigate the file.
But this created two problems: first, I use a lot of math in my notes (LaTeX previews are one of the reasons I want to us org). But these take sooooo long to load, I can't images trying to open a file with several thousand entries (all filled with math!!)
The other problem I have is with tags. I like to use a lot of multi-word tags to cross-reference my notes. But the way org-mode wraps these in the buffer, makes my headings completely unintelligible. Also (maybe it's just me but) I find CamelCase really hard to read, especially when faced with something like:
:monotonicTransformations:homogeneousFunctions:orderedSets:proofs:
Now my second attempt is with Deft. Here, I have broken up each note into its own .org file and creating a dedicated ~/org/ folder to hold my wiki. But this is where I am stuck:
1) How do you setup auto linking, so that typing say "foo bar" in one note, creates a link to "foo bar.org"? Can this be done with radio target? If not, can this syntax [[foo bar]] be overridden to search for headlines in all files in the ~/org/ directory? I tried adding Wiki.el as a minor mode but no dice...
2) How do you tag individual files? And how can you then pull up a list of all tags and use this to filter your list of notes? I have read that bookmark+ lets you do file tagging. But I got so lost in the online docs...
I would love to hear how others have solved these problem, what minor modes you are using, workflows and keyboard shortcuts or other mods!
Thanks!
-Adam

I'm using a simplistic code for a wiki.
Because that's what a wiki is for me: a quick way to categorize things.
The structure is a following:
each subject has its own org file
each topic of subject has its own heading
all org files are in single directory
you can jump to file, or create a new file with helm
That's it. I've found to need to link anything to anything, jump-to-subject functionality
is more than enough. Here's how this looks:
And once within a subject, I can jump across topics with worf.
Here's how this looks:

Having only one note file is, in my opinion, more flexible and compatible. It tends, however, to get slow for even moderatly sized files. Multiple small files are quick, but they need more effort to set up, and they only work within that setup.
Single File Solution
To speed things up, consider setting org-startup-with-latex-preview to nil (or add #+STARTUP: nolatexpreview to your file).
Tags not only get messy when used for keywords, using them also gets rather slow as your file grows. I've played around with some custom functions, but now avoid tags most of the time. Instead I use flat hierarchies, categories and otherwise rely on occur and org-occur (e.g. M-x org-occur begin_proof).
Multiple Files
The org syntax for linking to other files is rather simple: [[./this idea.org][this idea]]. If that is too much hassle, it should be easy to write a function that replaces the active region with an appropriate link.
If you want to link [[this idea]] to a file "this idea.org", you could add a function to org-open-at-point-functions and handle it yourself.
As for tags, you don't tag a file itself, but rather a single top level headline. This of course means that all your troubles with tags a back as well.
Again, I would recommend not using tags. Just make sure the file contains the right keywords at the right places and use occur and friends.
Edit: An Example `org-open-at-point-function'
If you want to search for a fuzzy link in all files in a directory instead of only the current buffer, you can do this by using the org-open-at-point-functions hook. Here is an example:
(defvar my-link-search-directory "/my/notes/directory/")
(defun my-open-link-function ()
"Open link, interpreting it a the name of a headline."
(let* ((el (org-element-context))
(type (first el))
(link-type (plist-get (cadr el) :type))
(path (let ((path-1 (plist-get (cadr el) :path)))
(when (stringp path-1)
(org-link-unescape path-1)))))
(when (and (eql type 'link)
path
(string= link-type "fuzzy"))
(let* ((path (regexp-quote path))
(result
(delq nil
(org-map-entries
(lambda ()
(when (string-match
path
(org-get-heading))
(list (buffer-file-name) (point))))
nil
;; Here we set the scope.
;; 'agenda would search in all agenda files.
;; We want a list of all org files in `my-link-search-directory'.
(directory-files
my-link-search-directory
t "[.]org\\'")))))
(when result
(when (> (length result) 1)
(message "Warning: multiple search results for %s" path))
(let ((file (caar result))
(pos (cadar result)))
(find-file file)
(goto-char pos)))))))
(add-hook
'org-open-at-point-functions
'my-open-link-function)
Note that I haven't tested this much.
Actually, I would recommend against using this unless you really need it. While making fancy extensions is tempting, keeping your notes as simple as possible is preferably. If you have everything in one file, you could edit your notes with notepad or google docs or whatever, should you ever need to.

Related

Does org-mode solve these 5 `hi-lock` problems?

For very many reference files, I am rather addicted to the combination of hi-lock, text-mode, and the very plain utility of TABs.
Here is a simple example. The file
hi-lock: (("===^^^===" (0 (quote compilation-line-number) t)))
hi-lock: (("http.?://.*" (0 (quote browse-url-button) t)))
===^^^===
===^^^===
Flutter Navigation
Routes and Navigation
Navigator https://api.flutter.dev/flutter/widgets/Navigator-class.html
Named Routes https://docs.flutter.dev/cookbook/navigation/named-routes
===^^^===
etc..
===^^^===
renders as follows.
This works nicely.
I can insert lines to highlight the divisions between sections.
I see HTTP links clearly highlighted (even if I still need to cut-and-paste to access them).
The TABs, as mentioned, provide adequate "section" and "subsection" semantics.
It's just text. This is not a trivial advantage, because my OS (macOS) indexes .txt files. There might be a way to nudge it to index other extensions. Still, the commands of any other file formats (as with, notably, LaTeX) will get in the way.
But there are also several problems.
I need to manually maintain (an otherwise unnecessary) hi-lock header for each file.
I need to maintain regular expressions (though this can be fun for tricky cases).
The semantics of hi-lock are dubious. (Why is compilation-line-number chosen for a separating line?)
Maintaining a reasonable color scheme becomes difficult after a few colors.
Maintaining indentation with TABs is laborious (as you can see from the misalignment of the URLs).
Does org-mode solve these problems while maintaining the advantages of text-mode and hi-lock?
If you're comfortable with ORG, I would particularly like to see how the snippet above can be rendered in an equally—or better—looking way using ORG.
Related:
Compare markdown or org mode
The spirit of the question is:
I couldn't be bothered to write configurations so frequently.
I'm seeking the largest signal-to-noise ratio.
The following is a remedy (working) solution, on the path to migrating to org-mode from a combination of text-mode and hi-lock.
Using the following .emacs file.
(add-to-list 'default-frame-alist '(background-color . "black"))
(add-to-list 'default-frame-alist '(foreground-color . "white"))
(set-frame-font "-*-*-medium-r-normal--24-*-*-*-*-*-*-*")
(setq org-startup-indented t)
(require 'org-superstar)
(add-hook 'org-mode-hook (lambda () (org-superstar-mode 1)))
The following lines
* Flutter Navigation
** Routes and Navigation
- Navigator https://api.flutter.dev/flutter/widgets/Navigator-class.html
- Named Routes https://docs.flutter.dev/cookbook/navigation/named-routes
* etc..
render as follows.
Though this addresses neither the separating lines nor the tabs.
Separating lines are easy, if they're meant for the output (PDF, etc) rather for the Emacs display itself.
Also...
Open URLs with the system-wide browser (
1,
2).

org-mode export to Latex — suppress generation of labels

I'm using org-mode to write a report which I then export to LaTeX.
I have a few different .org files (one per chapter), which I export as "headless" LaTeX and then combine in a master .tex file.
This works nicely, except that the generated .tex files contain labels with conflicting numbers. So both a.tex and b.tex contain \label{sec-1}, for example.
As long as I never actually use these references then it's not much of a problem I think, although the warnings do annoy me. Is there any way to turn off the generation of these labels? It should be simple but I cannot find anything about this in the documentation.
I have written a bit of Lisp which will remove said labels after the export to LaTeX, which looks like this:
(defun remove-orgmode-latex-labels ()
"Remove labels generated by org-mode"
(interactive)
(let ((case-fold-search nil))
(goto-char 1)
(replace-regexp "\\\\label{sec-[0-9][^}]*}" "")
)
)
(add-hook 'org-export-latex-final-hook 'remove-orgmode-latex-labels)
This seems to do the job without removing my own custom labels.
Why not writing your full report as one big Org file?
Anyway, if your prefer having multiple smaller files, I would advice "including" them in one Org master file, as this:
* Chapter 1
#+INCLUDE: "chapter1.org"
* Chapter 2
#+INCLUDE: "chapter2.org"
That way, Org sees only one file (then, I guess that your problem simply disappears), while you edit them as you wish.
As of now (18. 05. 2021) correct solution is this:
(defun my-latex-filter-removeOrgAutoLabels (text backend info)
"Org-mode automatically generates labels for headings despite explicit use of `#+LABEL`. This filter forcibly removes all automatically generated org-labels in headings."
(when (org-export-derived-backend-p backend 'latex)
(replace-regexp-in-string "\\\\label{sec:org[a-f0-9]+}\n" "" text)))
(add-to-list 'org-export-filter-headline-functions
'my-latex-filter-removeOrgAutoLabels)
Only slight modification of previous answer.
This is what worked for me with recent (2020) Org-Mode:
(defun rm-org-latex-labels (text backend _info)
"Remove labels auto-generated by `org-mode' export to LaTeX."
(when (eq backend 'latex)
(replace-regexp-in-string "\\\\label{sec:org[a-f0-9]+}\n" "" text)))
(add-to-list #'org-export-filter-headline-functions
#'rm-org-latex-labels)

Emacs: what are good tactics for navigating directories and opening files?

Last week, infuriated (again) by having to cope with different IDEs for R and Perl, neither of which I like or use enough to get really comfortable in, I decided to try Emacs. This decision was not made without a certain trepidation on my part. My main use is for Perl with cperl and for R with ESS. My environment is Windows 7 Ultimate 64-bit and I am running v23.4.1, which I think is what the ESS package installed on my behalf.
Nearly a week in and so far it has been surprisingly painless, no more involved than any other significant piece of software. I have remapped my ctrl key to caps-lock, changed default folders, messed around with .emacs and made some additions such as auto-install, yasnippet, color-theme, icicles and a few others. Clearly there is some very sophisticated stuff out there. In addition of course certain features of base Emacs are immediately very powerful and useful, such as isearching up and down. On the whole I have been pleasantly surprised and reassured.
One thing that is cruder than I was expecting is the process of finding and opening files. After a cursory read of various tutorials I had this image of quasi-magical file location and filename auto-completion. The main directories in my setup have paths like g:/roaming/code/perl or g:/roaming/code/R/ but I often need to branch into completely different paths like g:/pricedata/support files/sector/project01/ and so on.
Currently I laboriously delete the file path when I need to take a different fork, then use auto-complete to move deeper into that branch of the filesystem. It kinda takes me back to running a bash shell on the Amiga twenty years ago.
What had I expected? Something like (using the above example) typing 'project01' to skip immediately into the folder at the bottom of the path. For some reason I had got the idea in my head that Emacs would preload directories. So maybe this wasn't realistic.
My guess is that my difficulties probably stem from my own lack of familiarity rather than a structural shortcoming in Emacs and leads on to my questions. I can't complain that there is not enough documentation; rather there is abundant information that it is scattered around rather haphazardly. Icicles has a similar problem - if anything there's too much.
1) What is the best tactic for moving around different branches of the file tree when trying to open files in the minibuffer or using some other method? Are there aliases that can be used to shortcut from one place to another or can one specify directories to be preloaded? Do people just cd around a lot? Or am I coming at this from completely the wrong angle and need to adopt a different strategy?
2) With additional setup, can auto-complete be used to find files in (say) the project01 directly above by prefixing with wildcards etc? What should I focus on to become more efficient here? Am I not tapping the power of add-ons like icicles, anything etc?
I realise these questions veer dangerously close the deprecated category of not having clear answers. My defence is that some tips/guidance at this stage before I commit myself to bad habits or a poor long-term solution would be welcome and I suspect the answers will benefit others who might be considering the switch. I am happy to withdraw or rephrase if there are problems.
helm (formerly known as anything) might suit you. You can install it manually, or using a auto-install extension.
For the auto-install way: download it, put in Your load-path, then add to .emacs:
(add-to-list 'load-path "~/.emacs.d/site-lisp/auto-install")
(require 'auto-install)
(setq auto-install-directory "~/.emacs.d/site-lisp/auto-install/")
Then do M-x auto-install-batch anything. After that is done, put in .emacs:
(require 'anything)
(require 'anything-match-plugin)
(require 'anything-config)
(require 'anything-show-completion)
Then do M-x anything for anything.
Also give a try to ECB (stands for Emacs Code Browser). If you're on Linux, you probably have it available in a standard repository.
You could benefit from using ido-mode, which greatly enhances autocompletion nearly everywhere in emacs (especially when finding files or buffers)
(setq ido-enable-flex-matching t
ido-auto-merge-work-directories-length -1
ido-create-new-buffer 'always
ido-use-filename-at-point 'guess
ido-everywhere t
ido-default-buffer-method 'selected-window)
(ido-mode 1)
(put 'ido-exit-minibuffer 'disabled nil)
(when (require 'ido-ubiquitous nil t)
(ido-ubiquitous-mode 1))
While this might help you quickly finding files "not far away", it probably won't help you much finding files in entirely different locations.
If find that one way to begin tackling this problem is using recentf-mode to quickly jump to recent locations (if you have only a small number of usual project directories, this might do the trick). recentf can be coupled with ido using something like this (I'm not sure where I got this snippet from):
(recentf-mode 1)
(setq recentf-max-saved-items 50)
(defun ido-recentf-open ()
"Use `ido-completing-read' to \\[find-file] a recent file"
(interactive)
(if (find-file (ido-completing-read "Find recent file: " recentf-list))
(message "Opening file...")
(message "Aborting")))
(global-set-key (kbd "C-x C-r") 'ido-recentf-open)
With this, you can use C-x C-f (ido-find-file) to look for files near you current location, and C-x C-r (ido-recentf-open) to look for recently opened files (hoping one of them is not too far away from where you want to go).
There are various approaches. File name cache is a built in solution. Opening files from recent files is also very effective, especially if you set the stored files to a high number (e.g. 1000). There are also packages for opening files from anywhere on the file system. And there are meta solutions like anything.el which can show you file completions from multiple sources (history, project files, etc.) in a common interface.
You should investigate the different options to find out which is the most suitable for you.
In addition to IDO and recetf you can use the following methods:
Open the directory recursively.
(defun op-i:dired (rec)
"customized dired: will display directory recursively when called with an argument"
(interactive "P")
(let ((dir (car (find-file-read-args "Dired: " nil)))
(opts (if rec (read-string "options: " "-lhAR") "-lhA")))
(if (file-directory-p dir) (dired dir opts))))
(define-key (current-global-map) (kbd "C-x C-d") 'op-i:dired)
Then you can navigate in the directory buffer as in any other buffer using a (wildcard) search.
Use Emacs registers to point to often used directories. You can initialize them in your .emacs
(set-register ?0 (cons 'file "c:/path/to/my-project0"))
(set-register ?1 (cons 'file "c:/path/to/my-project1"))
Then jump to register using C-x r j
Other people have already mentioned anything, suffice it to say, I believe it solves all your problems. :)
Calling out one specific feature of anything is anything-locate. On *nix systems, this uses the DB created by the locate command to quickly find files. On Windows, I believe it works with "Everything", which should give you near instantaneous search results across all your files. I mean all.
Tom has mentioned file-cache. Anything works with that.
Just as a side note, Dired is an absolute monster once you get used to it. You should read the emacswiki pages related to it. In particular dired-x, which comes with Emacs binds C-x C-j, jumps to current buffer in Dired was a revelation for me. Also dired-find, wdired.
Edit: some details on why C-x C-j is awesome.
When you pop the current buffer in a Dired buffer, you can modify it, i.e. rename, delete, etc. And Emacs is aware of what you've done, and adjusts accordingly.
Other possibilities:
Bookmarks. If you use Bookmark+ then you can also:
Bookmark Dired buffers, which remembers their inserted subdirs, their markings, and their file omissions. You can have a bookmark that opens Dired to a specific set of files that need not be in the same directory or even the same directory tree.
From Dired you can hit a key to create a separate bookmark to each marked file or subdir.
Bookmark files automatically (autofiles).
Tag bookmarks and files (autofile bookmarks) with arbitrary-text tags. (Tags can even be non-text Lisp objects). Use tags to organize, search, etc. different categories of bookmarks.
Bookmark sets of bookmarks or bookmark files or Emacs desktops. Jump to such a bookmark to restore a given project state/context.
Icicles.
Define temporary or persistent sets of files on the fly, even from different directories. Operate on the files in a set in various ways. (Use Emacs filesets, less powerful, the same way.)
Complete against file names from such saved sets during any completion command, not just file-finding.
Act in different ways on candidate files during completion.
Search bookmarked objects or saved sets of files or buffers.
Dired+.
Additional navigation, including i bouncing between a subdirectory listing and the subdirectory's line in the parent listing.
Refs:
http://www.emacswiki.org/cgi-bin/wiki/BookmarkPlus
http://www.emacswiki.org/emacs/Icicles_-_Support_for_Projects
http://www.emacswiki.org/emacs/Icicles_-_File-Name_Input
http://www.emacswiki.org/cgi-bin/wiki/DiredPlus

How do I make an emacs shortcut to switch between related files?

For example, I have the file model/user.py open and I want to have a shortcut that opens controller/user.py. Or I want to switch to test/model/testUser.py (contrived example)
I'd like to make an emacs shortcut which given a file currently open, opens files related in various ways.
If the "related files" follow some kind of pattern, I think it's trivial to write some elisp functions to do the task. Let's say you have a model and need to open his associated controller, you will need to do something like this:
(defun my-open-related-controller ()
(interactive)
(let* ((name (buffer-file-name))) ;gets the filename of the current buffer
;; Of course, this is only an example. The point here is that you need
;; to "discover" the name of the related file based on the current one.
(setf name (replace-regexp-in-string "model" "controller" name))
;; Now you will open the file(if it isn't open already) and switch to it
(find-file name)))
Then you can bind the function to, say, F5:
(define-key name-of-the-mode-map [f5] 'my-open-related-controller)
If you want to crate this binding globally, use:
(global-set-key [f5] 'my-open-related-controller)
Of course, this is just a crude example(since you didn't give many specific details), but should be enough to get you started. Hope it helps!
If you don't fancy writing this yourself and would rather customize an exisiting library, you may like to look at toggle.el. It's designed to do what you're asking for.
There is also the jump.el that rinari uses for this purpose (except for Ruby on Rails projects). I gave the second link, because rinari.el in this project contains settings that manage jumps from one place to another (controller to view, model, migrations, etc.).
It looks like you can get jump.el to jump to a particular method in a file - but that may take a bit of effort.

Stack overflow while generating tags completion table in emacs

I'm using GNU Emacs 23.3 on Windows. I work in a very large codebase for which I generate a TAGS file (using the etags binary supplied with Emacs). The TAGS file is quite large (usually hovers around 100MB). I rarely need to use any functionality beyond find-tag, but there are times when I wish I could do completion out of the TAGS table.
Calling complete-tag causes Emacs to make a completion table automatically. The process takes quite a bit of time, but my problem isn't in the amount of time it takes, but rather the fact that right at the end (around 100% completion), I get a stack overflow (sorry about the unprintable chars):
Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
re-search-forward("^\\(\\([^]+[^-a-zA-Z0-9_+*$:]+\\)?\\([-a-zA-Z0-9_+*$?:]+\\)[^-a-zA-Z0-9_+*$?:]*\\)\\(\\([^\n]+\\)\\)?\\([0-9]+\\)?,\\([0-9]+\\)?\n" nil t)
etags-tags-completion-table()
byte-code(...)
tags-completion-table()
Has anyone else run into this? Know of a way to work around it?
EDIT: Stack output after turning on debug-on-error
EDIT: Removed stack, since I now know what the failing entries look like:
^L
c:\path\to\some\header.h,0
^L
c:\path\to\some\otherheader.h,0
My tags file contains quite a few entries in this format. Looking at the headers involved, it's clear that they couldn't be correctly parsed by etags. This is fine, but I'm surprised that tags-completion-table doesn't account for this format in its regex. For reference, here's what a real entry looks like:
^L
c:\path\to\some\validheader.h,115
class CSomeClass ^?12,345
bool SomeMethod(^?CSomeClass::SomeMethod^A67,890
The regexp in question is used to match a tag entry inside the TAGS file. I guess that the error can occur if the file is incorrectly formatted (e.g. using non-native line-endings), or if an entry simply is really, really large. (An entry is typically a line or two, which should not be a problem for the regexp matcher.)
One way of tracking down the problem is go to the TAGS buffer and see where the point (cursor) is, after the error has occurred. Once you know which function it is, and you could live without tags for it, you could simply avoid generating TAGS entries for it.
If the problem is due to too complex entry, I would suggest that you should send bug report to the Emacs team.
If you load the tags table (open the TAGS table with Emacs, then bury-buffer), try M-x dabbrev-expand (bound to M-/). If the present prefix is very common, you might end up running through many possible completions before reaching the desired one.
I don't use Windows, but on the Mac and Linux machines I use, I have not faced this issue.
This looks like a bug in Emacs, see:
https://groups.google.com/d/msg/gnu.emacs.help/Ew0sTxk0C-g/YsTPVEKTBAAJ
https://debbugs.gnu.org/db/20/20703.html
I have applied the suggested patch to etags-tags-completion-table (copied below in completeness for your convenience) and trapped an error case.
I'm triggering the error in an extremely long line of code (46,000 characters!). I presume somebody programmatically generated the line and pasted it into the source. A workaround could be to simply filter such lines at the ctag building or loading stage, just something that deletes "long" lines, whatever that may mean. Probably 500 characters is long enough!
I could also look at adding maximum sizes to my regexes in ctags, but that really isn't a general solution because many ctags patterns do not have such limits.
(defun etags-tags-completion-table () ; Doc string?
(let ((table (make-vector 511 0))
(progress-reporter
(make-progress-reporter
(format "Making tags completion table for %s..." buffer-file-name)
(point-min) (point-max))))
(save-excursion
(goto-char (point-min))
;; This monster regexp matches an etags tag line.
;; \1 is the string to match;
;; \2 is not interesting;
;; \3 is the guessed tag name; XXX guess should be better eg DEFUN
;; \4 is not interesting;
;; \5 is the explicitly-specified tag name.
;; \6 is the line to start searching at;
;; \7 is the char to start searching at.
(condition-case err
(while (re-search-forward
"^\\(\\([^\177]+[^-a-zA-Z0-9_+*$:\177]+\\)?\
\\([-a-zA-Z0-9_+*$?:]+\\)[^-a-zA-Z0-9_+*$?:\177]*\\)\177\
\\(\\([^\n\001]+\\)\001\\)?\\([0-9]+\\)?,\\([0-9]+\\)?\n"
nil t)
(intern (prog1 (if (match-beginning 5)
;; There is an explicit tag name.
(buffer-substring (match-beginning 5) (match-end 5))
;; No explicit tag name. Best guess.
(buffer-substring (match-beginning 3) (match-end 3)))
(progress-reporter-update progress-reporter (point)))
table))
(error
(message "error happened near %d" (point))
(error (error-message-string err)))))
table))