Organizing my papers in org-mode - emacs

I've just started using org-mode, and so far find it quite useful. I have a very large collection of technical papers in a directory tree, and I'd like to go through them and index them through org-mode. What I'd like is to have a way of going through them and look at the unannotated ones, and annotate them one by one. I imagine doing this by first making up a file of links like [[xxx.pdf][not done yet]], and then being presented with the not done ones, glancing at them and deciding how what annotations to put in. In addition I'd like to add tags. What I'd really like is to be able to make up new tags on the fly. Has anyone done anything like this in org-mode?
Victor

If you have your papers organized like this,
% ls -1 ~/References/functional-programming
The Lambda Calculus.pdf
Recursive Functions of Symbolic Expressions and Their.pdf
you can run a quick script to build an org-file. Save the following as make-org and run it from your directory of papers (sh make-org > papers.org).
#! /bin/sh
#
# make-org -- generates an org-mode file from a directory of PDFs
#
# AUTHOR:
# Jon-Michael Deldin
# USAGE:
# cd ~/path/to/papers && make-org > papers.org
#
echo "#+TITLE: Research Papers"
echo "#+STARTUP: align hidestars indent lognotedone"
echo
for f in *.pdf; do
name=${f%.*} # strip extension
path=$(echo $f | sed 's/ /%20/') # encode spaces as %20
echo "* TODO $name :unread:"
echo
echo "[[file:$path][$name]]"
echo
done
Open papers.org in Emacs, run C-u C-c C-q to realign the tags. Your file should now look like this:
In addition I'd like to add tags. What I'd really like is to be able to make up new tags on the fly.
Once you have a headline (thing with * at the beginning, you can hit C-c C-c and add any tag you want.
You may find this detailed write-up of using org-mode and RefTeX or this alternate approach useful, especially if you use LaTeX.

Heres is the modified version that works on a directory.
#! /bin/sh
#
# make-org -- generates an org-mode file from a directory of PDFs
#
# AUTHOR:
# Jon-Michael Deldin
# USAGE:
# cd ~/path/to/papers && make-org > papers.org
#
echo "#+TITLE: Research Papers"
echo "#+STARTUP: align hidestars indent lognotedone"
echo
for f in $(find . -name '*.pdf'); do
name=${f%.*} # strip extension
path=$(echo $f | sed 's/ /%20/') # encode spaces as %20
echo "* TODO $name :unread:"
echo
echo "[[file:$path][$name]]"
echo
done

Related

Autocomplete directories in a subfolder with the Fish shell

I'm having trouble getting the 'complete' function in the fish shell to behave as I would like and I've been searching for an answer for days now.
Summary
Essentially I need to provide tab directory auto-completion as if I was in a different directory to the one I am currently in. It should behave exactly as 'cd' and 'ls' do, but with the starting point in another directory. It seems like such a trivial thing to be able to do but I can't find a way to make it work.
Explanation
Example folder structure below
- root
- foo
- a
- dir1
- subdir1
- dir2
- subdir2
- b
- dir3
- subdir3
- dir4
- subdir4
I am running these scripts whilst in the 'root' directory, but I need tab auto-complete to behave as if I was in the 'foo' directory.
testfunc -d a/dir2/subdir2
Instead of
testfunc -d foo/a/dir2/subdir2
There are a lot of directories inside 'foo' and a lot of sub-directories within them, and this auto-complete behaviour is necessary to speed our process (this script is used extensively throughout the day).
Attempted Solution
I've tried using the 'complete' builtin to get this working by specifying the directory to use, but all this managed to do was auto-complete the first level of directories with a space after the argument instead of continuing to auto-complete like 'cd' would.
complete -x -c testfunc -a "(__fish_complete_directories ./foo/)"
Working bash version
I have already got this working in Bash and I am trying to port it over to fish. See below for the Bash version.
_testfunc()
{
local cur prev words cword
_init_completion || return
compopt +o default
case $prev in
testfunc)
COMPREPLY=( $( compgen -W '-d' -- "$cur" ) )
compopt +o nospace
return
;;
-d)
curdir=$(pwd)
cd foo/ 2>/dev/null && _filedir -d
COMPREPLY=( $( compgen -d -S / -- "$cur" ) )
cd $curdir
return
;;
esac
} &&
complete -o nospace -F _testfunc testfunc
This is essentially stepping into the folder that I want, doing the autocompletion, then stepping back into the original folder that the script was run in. I was hoping this would be easier in Fish after getting it working in Bash (I need to support these two shells), but I'm just pulling my hair out.
Any help would be really appreciated!
I am not a bash completions expert, but it looks like the bash completions are implemented by changing directories, running completions, and then changing back. You can do the same in fish:
function complete_testfunc
set prevdir $PWD
cd foo
__fish_complete_directories
cd $prevdir
end
complete -x -c testfunc -a "(complete_testfunc)"
does that work for you?

Add text to the top of multiple files in emacs

I'd like to add a couple lines of text (copyright) to the top of all text files in a directory. Can I do this in emacs without copy/pasting for each file?
This is copied from Chris Conway's answer to a different question: Using Emacs to recursively find and replace in text files not already open
M-x find-name-dired: you will be prompted for a root directory and a filename pattern.
Press t to "toggle mark" for all files found.
Press Q for "Query-Replace in Files...": you will be prompted for query/substitution regexps.
Proceed as with query-replace-regexp: SPACE to replace and move to next match, n to skip a match, etc.
You can use it the same way
Yes, with
find . -type f -exec emacs -batch '{}' --eval '(insert-string "foo\nbar\nbaz\n")' -f save-buffer \;
or something to that effect. The emacs bit is
emacs -batch filename --eval '(insert-string "foo\nbar\nbaz\n")' -f save-buffer
replace "foo\nbar\nbaz" with your message. However, using emacs for this is really a lot of overkill. You could just put your copyright header into a file and use cat header file > tempfile; mv tempfile file.

Why vim doesn't insert cyrillic symbols properly in Ex mode?

To make changes in several files I use the following script:
echo search('publications.html"') | normal o<li>Книжные серии</li>
echo search('collections.html"') | d
echo search('photo.html"') | d
wq
Then I do for file in *.html do; vim -e $file < script; done
As a result a string "^Z=86=K5 A5#88" is inserted instead of "Книжные серии".
All html files as well as the script itself are utf-8 encoded, and no other problems with Cyrillic revealed.
What's going on?
Thanks in advance for any comment!
According to the vim_use mailing list response to this same question, multibyte characters are not handled in Vim's ex mode.
Thus, the solution is to not use ex mode at all.
Instead of using:
vim -e $file < script
Use the -S flag to run the script outside of ex mode:
vim -S script $file
You probably need to tell Vim to interpret your script as utf-8, even if all the encodings are correct.
Try inserting this line at the top of your script:
scriptencoding utf-8

Search for files & file names using silver searcher

Using Silver Searcher, how can I search for:
(non-binary) files with a word or pattern AND
all filenames, with a word or pattern including filenames of binary files.
Other preferences: would like to have case insensitive search and search through dotfiles.
Tried to alias using this without much luck:
alias search="ag -g $1 --smart-case --hidden && ag --smart-case --hidden $1"
According to the man page of ag
-G --file-search-regex PATTERN
Only search files whose names match PATTERN.
You can use the -G option to perform searches on files matching a pattern.
So, to answer your question:
root#apache107:~/rpm-4.12.0.1# ag -G cpio.c size
rpm2cpio.c
21: off_t payload_size;
73: /* Retrieve payload size and compression type. */
76: payload_size = headerGetNumber(h, RPMTAG_LONGARCHIVESIZE);
the above command searches for the word size in all files that matches the pattern cpio.c
Reference:
man page of ag version 0.28.0
Note 1:
If you are looking for a string in certain file types, say all C sources code, there is an undocumented feature in ag to help you quickly restrict searches to certain file types.
The commands below both look for foo in all php files:
find . -name \*.php -exec grep foo {}
ag --php foo
While find + grep looks for all .php files, the --php switch in the ag command actually looks for the following file extensions:
.php .phpt .php3 .php4 .php5 .phtml
You can use --cpp for C++ source files, --hh for .h files, --js for JavaScript etc etc. A full list can be found here
Try this:
find . | ag "/.*SEARCHTERM[^/]*$"
The command find . will list all files.
We pipe the output of that to the command ag "/.*SEARCHTERM[^/]*$", which matches SEARCHTERM if it's in the filename, and not just the full path.
Try adding this to your aliases file. Tested with zsh but should work with bash. The problem you encountered in your example is that bash aliases can't take parameters, so you have to first define a function to use the parameter(s) and then assign your alias to that function.
searchfunction() {
echo $(ag -g $1 --hidden)
echo $(ag --hidden -l $1)
}
alias search=searchfunction
You could modify this example to suit your purpose in a few ways, eg
add/remove the -l flag depending on whether or not you want text results to show the text match or just the filename
add headers to separate text results and filename results
deduplicate results to account for files that match both on filename and text, etc.
[Edit: removed unnecessary --smart-case flag per Pablo Bianchi's comment]
Found this question looking for the same answer myself. It doesn't seem like ag has any native capability to search file and directory names. The answers above from Zach Fogg and Jikku Jose both work, but piping find . can be very slow if you're working in a big directory.
I'd recommend using find directly, which is much faster than piping it through ag:
Linux (GNU version of find)
find -name [pattern]
OSX (BSD version of find)
find [pattern]
If you need more help with find, this guide from Digital Ocean is pretty good. I include this because the man pages for find are outrageously dense if you just want to figure out basic usage.
To add to the previous answers, you can use an "Or" Regular Expression to search within files matching different file extensions.
For example to just search a string in C++ header files [.hpp] and Makefiles [.mk] ) :
ag -G '.*\.(hpp|mk)' my_string_to_search
After being unsatisfied with mdfind, find, locate, and other attempts, the following worked for me. It uses tree to get the initial list of files, ag to filter out directories, and then awk to print the matching files themselves.
I wound up using tree because it was more (and more easily) configurable than the other solutions I tried and is fast.
This is a fish function:
function ff --description 'Find files matching given string'
tree . --prune --matchdirs -P "*$argv*" -I "webpack" -i -f --ignore-case -p |
ag '\[[^d].*' |
awk '{print $2}'
end
This gives output similar to the following:
~/temp/hello_world $ ff controller
./app/controllers/application_controller.rb
./config/initializers/application_controller_renderer.rb
~/temp/hello_world $

Emacs: How to generate a wordlist for a document?

I would like to generate an index for a LaTex document with RefTex, following this advice from the RefTex manual:
"...you might want to start from a word list of the document and remove all words which should not be indexed." (-> collecting phrases for the index phrases file).
Now I ask myself: how do I generate such a word list for my multi-file LaTex Document?
I don't find the answer in the Emacs manual or on the web. But Emacs must be able to do that, right?
Thanks for any hints.
a quick way to get started (at the command line, not emacs):
sed 's/ */\n/g' < myDocument.txt | sort -f | uniq > wordListToEdit.txt
I found a solution that is independent from Emacs, but it produces a file with all tokens found in the document(s).
I just marked all the .tex files in my LaTeX project in Emacs Dired, and then used
! myshellscript
to run the following script on all of them.
You find more Information about nltk and Python here: http://www.nltk.org/
#!/usr/bin/env bash
echo $0
echo $1
python -c "\
from __future__ import division;\
import nltk, re, pprint;\
f = open('$1');\
raw = f.read();\
print nltk.word_tokenize(raw)\
" >> tok